Data Engineer (42726)
I am looking for an experienced Data Engineer to support a Treasury Data Warehouse modernization project focused on migrating legacy MS SQL/T-SQL pipelines to Databricks. You will work on ingestion pipelines, Spark/PySpark transformations, workflow orchestration, and ETL migration while helping deliver a functional MVP/PoC for reporting and data processing. Strong Databricks, Python, Spark, and SQL experience are essential.
🚀 Project
- implementation of a Treasury Data Warehouse modernization project
- development and migration of data pipelines from MS SQL/T-SQL DWH to Databricks
- implementation of ingestion pipelines and data transformations
- development of orchestration workflows
- migration and rewrite of legacy ETL logic into Spark/Databricks framework
- delivery of a functional MVP/PoC for data processing and reporting
- optimization and transformation of data workflows
- collaboration within an existing Databricks platform environment
🎯Skills
- strong experience as a Data Engineer (4–6+ years)
- strong Databricks experience
- strong Python and Spark/PySpark experience
- strong SQL skills
- understanding of T-SQL
- experience with ETL/ELT pipeline development
- experience with legacy DWH migrations
- experience with data transformation and optimization
- workflow orchestration experience using Databricks workflows, ADF, or similar tools
- understanding of medallion architecture (Bronze/Silver/Gold)
- English B2+/C1
💡 Nice to have
- SQL Server or SSIS background
- experience with Azure ecosystem including ADF and Azure Storage
- understanding of OLAP cubes and multidimensional data models
- banking or treasury domain experience
- Tableau or Power BI exposure
- experience validating outputs between legacy and new systems
#LI-MP11