Data Engineer

hackajob is collaborating with Virgin Media to connect them with exceptional tech professionals for this role.Virgin Media O2 are on a mission to become the most recommended brand by ourcustomers. At the heart of this strategy is digital innovation and our significant dataresources.So that's what you'll get up to, but what about us?Well, we're super proud of our history, helping communities to stay connected withoodles of top-notch products and services. We offer the full kit and caboodle -broadband, TV, mobile and landline - equipping our customers out with the very latesttech.But it's not just what we do, but why we do it that really matters.Our mission is to become the most recommended brand, by our people and ourcustomers. A massive part of that journey is about how we ensure that our brilliantpeople have a working environment in which they can truly belong and thrive. For us, it'scritical that every single person can bring, and be, their whole selves at work, and we'reworking hard every day to achieve this.Your Mission Use innovative GCP technology at scale to develop and optimize batch and realtime data pipelines serving millions of Virgin Media O2 customers Build and maintain internal self-service data tools to empower stakeholders anddecreases manual processes within the team Work closely with stakeholders from various departments to gatherrequirements and ensure the successful delivery of solutions that precisely meettheir needs Collaborate with highly skilled Architects, DevOps Engineers, Data Engineers,Analytic Engineers and Analysts to achieve the team’s Objectives and Key Resultsfor our stakeholders Troubleshoot and resolve issues, providing timely and effective solutions tomaintain system stability and availability, ensuring they adhere to service levelagreements (SLAs) Utilise Infrastructure as Code (IaC) principles to deploy cloud infrastructureeffectively and efficiently using tools like Terraform Stay up to date with the latest industry trends, technologies, and best practices,continuously improving your skills and applying them to enhance the organisation'stechnical capabilities Actively participate in code reviews, promoting a culture of high-quality codeand sharing knowledge with other team members. Play an instrumental role in identifying areas for improvement, proposinginnovative ideas, and contributing to the overall growth and success of the team andthe organisationWe’d love to hear from you, if you… Are business value and delivery focused Have the “I can solve it” attitude Are a team player and have excellent communication skills Have strong proficiency in Python with proven Data Engineering experience Have strong SQL skills including analytic functions and ensuring data quality Understand DevOps best practices including IaC and CI/CD Had production experience working on a cloud environment, preferably GCP Had production experience using a data warehouse engine such as BigQuery orRedshift Had production experience using a key/value store such as Datastore orDynamoDBIt would be nice if you… Have worked in an Agile environment Came across SOLID principles and interested in the concept Have practiced or used TDD Have used Apache Airflow and maintained pipelines on Production Had experience with Apache Beam or Apache Spark Had experience with Kubernetes Had experience using TerraformWhat we use… Apache Airflow on Cloud Composer Cloud Function, Cloud Run, Firestore in Datastore mode, Pubsub, BigQuery,Cloud Storage Terraform, GitLab and others