Databricks + DataStage Engineer – Bangalore
You’ll lead the migration of legacy IBM DataStage workloads to a modern Databricks platform, enabling faster, scalable ETL pipelines. The role is critical for the company’s push to modernize data engineering and offers a chance to work hands‑on with SparkSQL, PySpark and Delta Lake in a fast‑moving team.
What You’ll Do
Analyze existing DataStage Server and Parallel jobs to define migration scope.
Translate ETL logic into SparkSQL or PySpark code on Databricks.
Build metadata‑driven ingestion patterns for reusable pipelines.
Conduct unit, integration and reconciliation testing of migrated jobs.
Optimize performance using Delta Lake, partitioning and Z‑ordering.
Support CI/CD pipelines with Git‑based version control.
Collaborate closely with architects and DataStage subject‑matter experts.
What You Need
7+ years IBM DataStage experience (Server Jobs mandatory).
Hands‑on expertise with Databricks on AWS or Azure.
Strong SparkSQL and PySpark development skills.
Proficiency in SQL (Teradata/Oracle experience a plus).
Experience implementing Delta Lake and performance tuning.
Familiarity with CI/CD tools such as GitLab or Azure DevOps.
Good to Have
Parallel DataStage job experience.
Exposure to cloud‑native data architectures.
Knowledge of ETL/ELT design patterns.
The Opportunity
The team is ramping up immediately, seeking candidates who can join within 15 days. You’ll be at the core of a strategic data‑modernization effort in Bangalore, gaining deep expertise in both legacy and cutting‑edge data platforms.