Data Engineer

Company DescriptionUpshifters is a dedicated team of innovative digital marketing professionals focused on elevating brands to new heights. With a passion for creativity and impact, the team at Upshifters works diligently to help businesses achieve measurable growth and success. Their tailored approach ensures unique solutions for each client, emphasizing collaboration and forward-thinking strategies.Role DescriptionThis is a full-time hybrid role for a Data Engineer. The position is based in Qesm El Maadi, with flexibility for partial remote work. The Data Engineer will be responsible for designing, developing, and maintaining reliable data pipelines and architecture. Daily tasks include building and optimizing ETL processes, implementing scalable data models, managing data warehousing solutions, and analyzing data trends to support the organization’s decision-making and business goals.Key ResponsibilitiesData Pipeline Development: Build and maintain ETL (Extract, Transform, Load) pipelines to ingest, clean, and transform data from diverse sources into our data warehouse.Data Quality Assurance: Implement data quality checks and validation processes to ensure the accuracy and integrity of collected data.Implement data models and database schemas that optimize data storage and retrieval.Data Scraping: Develop web scraping scripts and tools to extract data from websites, APIs, and other online sources. Ensure data quality and reliability.QualificationsBachelor's degree in computer science, Information Technology, or a related fieldProven experience as a Data Engineer, with at least 3 years of experienceProficiency in programming languages such as Python, Java, or ScalaExperience using a pipeline orchestrator (e.g., Temporal, Airflow)Strong SQL skills for data manipulation and querying.Extensive experience with data modeling, database design, and performance optimization techniquesFamiliarity with data storage technologies (e.g., SQL databases, NoSQL databases).Knowledge of data warehousing concepts and tools Experience with AWS or similar cloud providersExcellent problem-solving and communication skills Proven experience in web scraping and data engineering is a plus