Data engineering lead

Data Engineering Lead – Bengaluru | Full time Lead a high‑performing data engineering team to design, build, and optimise enterprise‑scale data platforms that power analytics, BI, and AI across Takeda’s global operations. This role sits at the heart of Takeda’s Innovation Capability Center, driving a digital transformation that impacts patients, people, and the planet. What You’ll Do Architect and implement scalable batch, micro‑batch, and streaming data pipelines. Define and enforce data‑engineering standards, design patterns, and governance controls. Optimise Spark/Databricks performance, including cluster sizing, caching, and workload tuning. Mentor engineers, conduct design reviews, and champion agile delivery practices. Collaborate with BI, Data Science, AI, and global product teams to translate requirements into technical roadmaps. Embed data quality, lineage, and cost‑optimisation into platform design. Drive observability, error handling, and SLA compliance for production workloads. What You Need 10+ years in software development, data engineering, and ETL with proven leadership. Strong Python, SQL, and PySpark skills; hands‑on Spark Structured Streaming experience. Deep expertise in AWS services (Lambda, DMS, Step Functions, S3, Aurora, CloudWatch). Extensive experience with Databricks/Delta Lakehouse, Unity Catalog, and data‑mesh concepts. Track record of building enterprise‑scale data solutions and dimensional modelling. Proficiency with Git (GitHub/GitLab) and CI/CD pipelines (GitHub Actions). Ability to optimise cloud costs while maintaining performance and reliability. Good to Have Master’s degree in Computer Science or related field. Experience with CDK, IICS, Airflow/Tidal, NoSQL, and ETL tools such as DataStage or Talend. Databricks Certified Data Engineer Professional or AWS