Data Engineer

Job Role : As a Data Engineer, you'll build and maintain data pipelines and architectures. Responsibilities include optimizing databases and ETL processes, using Python or SQL, and collaborating with data teams for informed decision-making. Why Choose Ideas2IT: Ideas2IT has all the good attributes of a product startup and a services company. Since we launch our products, you will have ample opportunities to learn and contribute. However, single-product companies stagnate in the technologies they use. In our multiple product initiatives and customer-facing projects, you will have the opportunity to work on various technologies.

AGI is going to change the world. Big companies like Microsoft are betting heavily on this (see here and here). We are following suit. As a Data Engineer, exclusively focus on engineering data pipelines for complex products‍What’s in it for you?You will work on diverse technology challenges like: A robust distributed platform to manage a self-healing swarm of bots onunreliable network / computeLarge-scale Cloud-Native applicationsDocument Comprehension Engine leveraging RNN and other latest OCR techniquesCompletely data-driven low-code platformYou will leverage cutting-edge technologies like Blockchain, IoT, and Data Science as you work on projects for leading Silicon Valley startups.

Your role does not start or end with just Java development; you will enjoy the freedom to share your suggestions on the choice of tech stacks across the length of the projectIf there is a certain technology you would like to explore, you can do your Technical PoCsWork in a culture that values capability over experience and continuous learning as a core tenetHere’s what you’ll bringProficiency in SQL and experience with database technologies (e. g. , MySQL, PostgreSQL, SQL Server).

Experience in any one of the cloud environments – AWS, AzureExperience with data modeling, data warehousing, and building ETL pipelines. Experience building large-scale data pipelines and data-centric applications using any distributed storage platformExperience in data processing tools like Pandas, pyspark. Experience in cloud services like S3, Lambda, SQS, Redshift, Azure Data Factory, ADLS, Function Apps, etc.

Expertise in one or more high-level languages (Python/Scala)Ability to handle large-scale structured and unstructured data from internal and third-party sourcesAbility to collaborate with analytics and business teams to improve data models that feed business intelligence tools, increase data accessibility, and foster data-driven decision-making across the organizationExperience with data visualization tools like PowerBI, TableauExperience in containerization technologies like Docker , KubernetesAbout Us: Ideas2IT stands at the intersection of Technology, Business, and Product Engineering, offering high-caliber Product Development services.

Initially conceived as a CTO consulting firm, we've evolved into thought leaders in cutting-edge technologies such as Generative AI, assisting our clients in embracing innovation. Our forte lies in applying technology to address business needs, demonstrated by our track record of developing AI-driven solutions for industry giants like Facebook, Bloomberg, Siemens, Roche, and others. Harnessing our product-centric approach, we've incubated several AI-based startups—including Pipecandy, Element5, IdeaRx, and Carefi. in—that have flourished into successful ventures backed by venture capital. With fourteen years of remarkable growth behind us, we're steadfast in pursuing ambitious objectives. ‍P. S.

We're all about diversity, and our doors are wide open to everyone. Join us in celebrating the awesomeness of differences!