Data Engineer

Design and implement enterprise-scale data architectures, including data lakes, warehouses, and real-time streaming platforms.Develop and maintain ETL/ELT pipelines that efficiently process large volumes of structured and unstructured data from diverse sources.Ensure data quality, governance, and security standards are embedded throughout all data engineering processes.Hands-on development of Databricks notebooks using PySpark, Python, and SQL for ETL automation.Create and optimize PySpark scripts for efficient data extraction, transformation, and loading from large datasets.Implement custom data manipulation, validation, and error-handling solutions to enhance ETL robustness.Provide technical mentorship to junior data engineers and analysts.Lead code reviews, establish best practices, and drive the adoption of modern data engineering tools and methodologies.Collaborate with cross-functional teams including data scientists, analysts, and software engineers to deliver integrated solutions.Monitor and optimize data pipeline performance, implementing solutions for scalability and cost-effectiveness.Conduct testing, debugging, and troubleshooting of data transformation processes.Verify data integrity throughout pipeline stages and resolve complex technical issues.Qualifications:Bachelor’s degree in Information Technology, Computer Science, or a related field.Minimum of 4 years of hands-on experience as a Data Engineer.Strong expertise in AWS cloud services, including Redshift, S3, Glue, EMR, Kinesis, and Lambda.Proficiency in the Databricks platform, with solid experience in PySpark, Python, and SQL implementation.Proven experience in data warehouse design and implementation.Hands-on experience in ETL/ELT migration projects and pipeline development.Familiarity with real-time streaming technologies.Experience with data exchange platforms and API integration.Beyondsoft International (Singapore) Pte. Ltd is committed to being an equal opportunity employer and provides equal employment opportunities to all employees and applicants. We strive to cultivate a workplace that celebrates diversity and inclusion, where individuals of all backgrounds—regardless of nationality, ethnicity, religion, age, gender identity, sexual orientation, or any other distinguishing trait—can succeed and thrive. We prohibit discrimination and harassment of any type with regard to race, color, religion, age, national origin, disability status, genetics, sexual orientation, gender identity, or expression. This policy applies to all terms and conditions of employment, including recruiting, hiring, and the entire employee lifecycle. We are focused on creating an environment where everyone can reach their full potential.Employment offers from Beyondsoft International (Singapore) Pte. Ltd. are contingent upon the successful completion of any required pre-employment processes, in line with applicable laws and regulations. Beyondsoft International (Singapore) Pte. Ltd. does not ask for any recruitment fees, nor does it request any unauthorized payments from candidates as part of the hiring process.