Data Engineer, Data Platforms

Full Time
  • Full Time
  • India

Join a team building next‑gen data pipelines on AWS and Databricks, ensuring reliable, scalable ingestion for financial datasets. You’ll partner with platform and offshore teams to transition, optimize and monitor enterprise‑wide data flows.

What You’ll Do

  • Transition existing pipelines to the enterprise data platform with Databricks.
  • Stabilize and optimize pipelines for performance, scalability, and reliability.
  • Implement repeatable ingestion, transformation, testing, deployment, and monitoring patterns.
  • Apply secure AWS deployment and governed data access controls.
  • Design metadata‑driven pipelines using Delta Lake, Iceberg, Unity Catalog.
  • Establish observability: logging, alerts, dashboards for production pipelines.
  • Support batch and streaming onboarding via AWS Glue, Lambda, Kinesis.

What You Need

  • 5‑7 years data engineering experience building batch or streaming pipelines.
  • Hands‑on with Databricks lakehouse platforms.
  • Proficient in AWS services: S3, Glue, Lambda, Lake Formation, Kinesis, IAM.
  • Experience with Delta Lake, Apache Iceberg, Unity Catalog.
  • Knowledge of data quality, schema management, lineage, encryption, access control.
  • Provide technical guidance through code and design reviews.
  • Use AI‑assisted coding tools like Claude or GitHub CoPilot.

Good to Have

  • Enterprise onboarding frameworks and reusable pipeline template experience.
  • Familiarity with data cataloging, metadata management, and governed data products.
  • Understanding of semantic modeling, such as semantic layers or canonical models.

The Opportunity

You’ll be part of S&P Dow Jones Indices, the world’s largest provider of index‑based market benchmarks. The role contributes to platforms that support over $7 trillion in passively managed assets.

Similar Jobs to Apply
  • CME Group
    India

    Join a fast‑moving team that builds the core infrastructure enabling engineers to ship applications quickly and securely. You’ll work with Kubernetes, GitOps and Google Cloud to create self‑serv
  • thinkbridge
    India

    Join a six‑month contract as a Full Stack Engineer, building and enhancing web applications using C#, .NET, and Claude Code. The role blends front‑end and back‑end work, with three days a week o
  • Walmart
    India

    Join a high‑impact team building Walmart’s next‑generation data platforms that power AI, analytics and real‑time decision making across the enterprise. You’ll work on large‑scale batch and
  • Aristocrat
    India

    Join Aristocrat Gaming’s AI Products & Enablement team to drive cutting‑edge AI and automation initiatives. Collaborate with industry experts and contribute to innovative solutions that enhanc