Databricks Engineer (43252)

I'm looking for a Databricks Engineer who will design, optimize, and maintain ELT pipelines while ensuring performance, scalability, and data quality across the platform. I expect hands-on experience with Databricks, Python, SQL, Delta Lake, Azure, and production data engineering environments. If you're interested in this opportunity, feel free to reach out and I'd be happy to share more details.

🚀 Project
- designing and implementing ELT pipelines in Databricks for data ingestion, transformation, and delivery
- building and maintaining scalable data pipelines across the Medallion architecture (Bronze, Silver, Gold)
- developing batch and streaming data processing solutions
- working with the Databricks Lakehouse platform, including Delta Lake, Workflows, and Notebooks
- managing and optimizing Delta tables, orchestration workflows, and pipeline execution
- implementing data quality, monitoring, and observability across data pipelines
- monitoring, troubleshooting, and optimizing pipeline performance, scalability, and reliability
- optimizing compute and storage utilization to improve cost efficiency
- collaborating with Data Architects, Analytics Engineers, AI Engineers, and Analysts to deliver high-quality data products
- translating business requirements into scalable technical data solutions
- ensuring compliance with data governance, security, and cataloging standards
- contributing to CI/CD, DevOps practices, technical documentation, and reusable data engineering assets

🎯 Skills
- 3–6+ years of experience in data engineering
- strong hands-on experience with: Databricks, Python, SQL for transformation and data modeling
- experience with Azure Data Platform (ADLS, ADF, etc.)
- proven experience working with Lakehouse / Delta architecture
- experience building ETL/ELT pipelines in production environments
- understanding of data modeling, partitioning, and performance tuning

💡 Nice to have
- experience with streaming (Spark Structured Streaming)
- knowledge of CI/CD, Git, and DevOps practices
- familiarity with Unity Catalog / data governance frameworks
- exposure to AI/ML pipelines (MLflow, feature engineering)

#LI-MM1