Senior Data Engineer

About us: Qualgo is an R&D center specializing in cybersecurity products and solutions. We are on a mission to build a trusted cyberspace where individuals and businesses can thrive with confidence. Role Summary: As a Senior Data Engineer, you will play a crucial role in designing, building, and maintaining the core data infrastructure that powers Qualgo's products and data-driven initiatives. You will be responsible for developing scalable data pipelines, managing our data warehouse and data lake environments, and ensuring data quality and reliability. You will work closely with data scientists, analysts, software engineers, and product teams to understand their data needs and provide them with the necessary tools and infrastructure. This is a hands-on role requiring deep technical expertise, strong problem-solving skills, and the ability to learn and apply new technologies. Key Responsibilities: Data Pipeline Development: Design, build, and optimize robust batch and streaming pipelines using tools like Apache Spark, Flink, or Airflow. Data Crawling & Ingestion: Build scalable back-end services to crawl, ingest, and normalize data from multiple sources (Websites, APIs, databases, flat files, ...etc ). Data Modeling & Design: Define scalable data models and schemas using best practices (e.g., Kimball, Data Mesh, Data Fabric). Collaboration: Work with data scientists, analysts, engineers, and product teams to ensure the availability, usability, and performance of data pipelines. Monitoring & Troubleshooting: Proactively monitor data systems and resolve issues swiftly to ensure reliability and performance. Documentation: Create clear, comprehensive documentation for pipelines, tools, and processes. Mentorship: Support and guide junior data engineers through reviews, pairing, and best practices. Technology Evaluation: Stay updated on the latest technologies and recommend tools that improve data engineering workflows and scalability. Perform the duties and tasks assigned by your direct report or as otherwise instructed by the Company. Qualifications and Skills Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field Required: 5+ years of hands-on experience in data engineering roles. Proficiency in Python, Java, or Scala for data pipelines and back-end services. Deep SQL knowledge and experience with RDMS (e.g., PostgreSQL, MySQL). Solid experience building crawling or ingestion systems for structured and unstructured data (Airbyte, Posthog, Apache Nifi, etc...). Experience with streaming or messaging platforms (e.g., Kafka, NATS). Experience with workflow orchestration tools (e.g., Apache Airflow, Flink, etc...). Experience with AWS and cloud services like S3, Redshift, EMR, Glue, etc. Familiarity with data modeling on Row/ Columnar / or Vector DB (e.g., Clickhouse, Citus, Qdrant, ...) Understanding data warehouse, data lake architecture, and modern ETL/ELT best practices. Fluency in English. Nice to Have: Experience with NoSQL and graph databases (e.g., Neo4j, MongoDB, Redis). Experience with Docker and Kubernetes for containerized deployment. Background in Agile development practices. Soft Skills: Strong problem-solving mindset with the ability to work independently. Excellent communication skills—able to work effectively with both technical and non-technical teams. Collaborative team player, mentor, and proactive contributor. What we offer: Competitive salary and benefits package. Opportunity to work on a product that impacts millions of users. A dynamic and supportive work environment. Premium health insurance for you and your family. Professional growth and development opportunities. Annual leave 12 days per year + 1 Birthday Leave + 1 X’Mas Performance review: once per year Internal training/sharing and professional Training courses Team building, company trip, year end party, monthly activities,.... Devices: Macbook and screen (If needed) Free tea and coffee Comfortable working Area Working hour: 9am - 6pm from Monday to Friday