As a Senior Data Engineer at Morningstar, you will play a crucial role in designing, developing, and optimizing data pipelines to support data integration, transformation, and storage in a modern data warehouse. Your responsibilities will include managing and maintaining cloud-based data infrastructure on AWS, maintaining SQL Server-based data models and Informatica ETL processes, and supporting the migration of data pipelines to a modern data warehouse like Snowflake. You will work closely with data analysts, data scientists, product managers, and other stakeholders to understand business requirements and provide efficient data solutions. Additionally, you will write efficient, reusable, and scalable Python code to automate data workflows, monitor data quality, ensure data integrity, governance, and security, troubleshoot and optimize performance issues across data systems, and mentor and collaborate with junior data engineers to improve overall team capabilities.
Key Responsibilities:
– Design, develop, and optimize data pipelines for data integration, transformation, and storage.
– Manage and maintain cloud-based data infrastructure on AWS.
– Maintain SQL Server-based data models and Informatica ETL processes.
– Support the migration of data pipelines from SQL Server to a modern data warehouse like Snowflake.
– Optimize SQL queries, stored procedures, indexing strategies, and data models for efficiency and scalability.
– Work closely with stakeholders to understand business requirements and provide efficient data solutions.
– Write efficient, reusable, and scalable Python code to automate data workflows.
– Monitor data quality, ensure data integrity, governance, and security in data pipelines.
– Troubleshoot and optimize performance issues across data systems.
– Mentor and collaborate with junior data engineers to improve overall team capabilities.
Qualifications Required:
– Bachelor’s degree in Computer Science or a related field.
– Over eight years of experience in data engineering, building end-to-end data integration and data warehousing solutions for analytics teams.
– Experience in designing and implementing ETL/ELT frameworks for complex data warehouses/marts.
– Proficiency in handling large data sets, with experience in performance tuning and troubleshooting.
– Experience with AWS compute technologies (Lambda, ECS/ECR, EC2, API Gateway, IAM, S3).
– Experience with data visualization tools like Tableau or Power BI.
– Experience with Snowflake, DBT, and DataHub is a plus.
– Proficiency in Python programming for data engineering tasks.
– Expert-level SQL skills.
– Experience with version control.
– Strong understanding of data governance practices.
– Proven experience working within a global agile team.
– A flexible team player who remains organized and productive while managing multiple deliverables.
Morningstar’s hybrid work environment allows you to work remotely and collaborate in-person each week, providing the opportunity for purposeful interactions at least three days every week. Join Morningstar India Private Ltd. in Delhi and be part of a vibrant technical community where you can grow, share your knowledge, and contribute to empowering users with innovative products. As a Senior Data Engineer at Morningstar, you will play a crucial role in designing, developing, and optimizing data pipelines to support data integration, transformation, and storage in a modern data warehouse. Your responsibilities will include managing and maintaining cloud-based data infrastructure on AWS, maintaining SQL Server-based data models and Informatica ETL processes, and supporting the migration of data pipelines to a modern data warehouse like Snowflake. You will work closely with data analysts, data scientists, product managers, and other stakeholders to understand business requirements and provide efficient data solutions. Additionally, you will write efficient, reusable, and scalable Python code to automate data workflows, monitor data quality, ensure data integrity, governance, and security, troubleshoot and optimize performance issues across data systems, and mentor and collaborate with junior data engineers to improve overall team capabilities.
Key Responsibilities:
– Design, develop, and optimize data pipelines for data integration, transformation, and storage.
– Manage and maintain cloud-based data infrastructure on AWS.
– Maintain SQL Server-based data models and Informatica ETL processes.
– Support the migration of data pipelines from SQL Server to a modern data warehouse like Snowflake.
– Optimize SQL queries, stored procedures, indexing strategies, and data models for efficiency and scalability.
– Work closely with stakeholders to understand business requirements and provide efficient data solutions.
– Write efficient, reusable, and scalable Python code to automate data workflows.
– Monitor data quality, ensure data integrity, governance, and security in data pipelines.
– Troubleshoot and optimize