Cloud Data Engineer

OverviewAt Best Doctors Insurance, the Cloud Data Engineer is responsible for building and maintaining cloud-database data systems that support secure, efficient management of healthcare data. The role ensures scalable infrastructure for analytics and helps deliver innovative, data driven services to our members.Primary ResponsibilitiesCloud Data Pipeline Development: Design and implement robust, scalable data pipelines in cloud environments (e.g., AWS, Azure, Google Cloud) to collect, process, and transform data from a variety of sources.Cloud ETL Processes: Develop and manage cloud-native ETL (Extract, Transform, Load) workflows using services such as Azure Data Factory, Google Cloud Dataflow, or third-party tools like Matillion.Cloud Database Management: Maintain and optimize cloud data warehouses (e.g., Snowflake, Amazon Redshift, Google BigQuery) and databases to ensure security, reliability, and high performance.Data Integration: Integrate data from APIs, on-premises systems, and third-party applications into cloud storage solutions, ensuring seamless and secure data flow across platforms.Data Quality: Implement data quality frameworks and enforce standards to guarantee the accuracy, consistency, and completeness of cloud-based data assets.Collaboration: Partner with data scientists, analysts, and business stakeholders to understand analytic requirements and deliver the necessary cloud infrastructure and data solutions.Documentation: Create and maintain clear documentation for cloud data systems, processes, and workflows to enable organizational learning and continuity.Key QualificationsEducational Background: Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.Experience: Demonstrated experience as a Data Engineer with a focus on cloud environments and large-scale data systems.Technical Skills: Proficiency in SQL, Python, and cloud-native programming tools.SSIS, Snowflake, and Matillion: Strong background with data integration and transformation tools, including expertise in SSIS and Matillion, and hands-on experience designing and optimizing data models in Snowflake or similar cloud warehouses.Cloud Platforms: Deep hands-on experience with at least one major cloud provider (AWS, Azure, or Google Cloud), including use of their data management and analytics services.Security & DevOps: Familiarity with cloud security practices, IAM, and DevOps methodologies for automating deployments and managing infrastructure as code.Problem-Solving: Advanced analytical and troubleshooting skills to resolve technical challenges in cloud-based data pipelines and systems.Communication: Strong interpersonal and communication skills, with the ability to explain cloud technologies to non-technical stakeholders.Team Player: Collaborative approach with experience working on distributed teams and cross-functional projects.Preferred QualificationsCertifications: Relevant cloud certifications such as AWS Certified Data Analytics – Specialty, Google Cloud Professional Data Engineer, or Microsoft Certified: Azure Data Engineer Associate.Advanced Analytics: Familiarity with machine learning pipelines and advanced analytics in the cloud.Data Visualization: Experience connecting cloud data sources to visualization tools like Tableau, Power BI, or Looker.Project Management: Exposure to agile development and project management tools for delivering cloud data solutions efficiently.