Data Engineer, DaaS

PayPay's growth is driving a rapid expansion of PayPay product teams and the need for a robust Data Engineering Platform to support our growing business needs is more critical than ever. The DaaS team’s responsibility is to design, implement, and operate this platform by using cutting edge technologies such as Spark, Hudi, Delta Lake, Scala, and AWS suite of data tools. We are looking for talented Data Engineers to join our team and help us scale our platform across the organizations.Main ResponsibilitiesDesign, develop, and maintain scalable data ingestion pipelines using AWS Glue, Step Functions, Lambda, and TerraformOptimize and manage large scale data pipelines to ensure high performance, reliability, and efficiencyImplement data processing workflows using Hudi, Delta Lake, Spark, and ScalaMaintain and enhance Lakeformation and Glue Data Catalog for effective data management and discoveryCollaborate with cross-functional teams to ensure seamless data flow and integration across the organizationImplement best practices for observability, data governance, security, and complianceQualifications5+ years experience as a Data Engineer or in a similar roleHands-on experience with Apache Hudi, Delta Lake, Spark, and ScalaExperience designing, building, and operating a DataLake or Data WarehouseKnowledge of Data Orchestration tools such as Airflow, Dagster, PrefectStrong expertise in AWS services, including Glue, Step Functions, Lambda, andEMRFamiliarity with change data capture tools like Canal, Debezium, and MaxwellExperience with data warehousing tools like AWS Athena, BigQuery, DatabricksExperience in at least one primary language (e.g. Scala, Python, Java) and SQL (any variant)Experience with data cataloging and metadata management using AWS Glue Data Catalog, Lakeformation, or Unity CatalogProficiency in Terraform for infrastructure as code (IaC)Strong problem-solving skills and ability to troubleshoot complex data issuesExcellent communication and collaboration skillsAbility to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously