Data Engineer (Senior) ID43916

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!ABOUT THE ROLEAs a Data Engineer, you’ll play a key role in ensuring smooth and successful device launches through observability, test automation, and real-time launch support. You’ll transform raw telemetry into actionable insights, build dashboards and anomaly detection systems, and automate data validation workflows. This role offers the opportunity to work on cutting-edge devices, collaborate cross-functionally, and directly impact product reliability and customer experience. WHAT YOU WILL DO Turn raw device telemetry into actionable alerts with strong observability, data quality, and launch support baked in; Work with cross-functional teams to establish device observability and health monitoring; Define logging, metrics, and traces for device and pipeline health, and ship dashboards for latency, uptime, error rates, and crash signals; Add anomaly detection over telemetry and logs and track health KPIs during launches; Develop automated anomaly detection for logging anomalies in device telemetry; Build dashboards to track device health and ensure all data artifacts have proper quality, integrity, and reliability; Participate in remote or on-site go-lives to monitor device health and triage issues in real time using dashboards; Collaborate with QA to ensure access to proper data and pipelines for testing and build automations as needed; Develop tooling to simulate various on-device scenarios for launch-readiness; Support on-site testing and live device launches remotely by monitoring device health based on metrics and dashboards; Triage issues in real time by analyzing logs and identifying failure sources to enable rapid resolution and successful launches. MUST HAVES 5–8+ years of experience in data engineering with Spark (Structured Streaming) or similar tools in production; Proficiency in Python and strong SQL/pandas skills; Solid understanding of streaming fundamentals; Experience building alerting, monitoring, or observability systems; Experience turning manual checks into automated guardrails; Comfortable supporting live device rollouts and post-launch monitoring; Upper-intermediate English level. PERKS AND BENEFITS Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps. Competitive compensation: We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities. A selection of exciting projects: Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands. Flextime: Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.