Data Engineer (43168)

I'm looking for a Data Engineer who will drive enterprise data migrations, design scalable cloud architectures, and automate data platform operations. I expect strong experience with Azure, VMware Cloud Foundation, database migration projects, Redis, object storage technologies, Python, and Agile delivery environments.

🚀 Project
- leading the migration of enterprise data services from Azure to VMware Cloud Foundation (VCF) and VMware Data Services Manager (DSM)
- designing and implementing end-to-end data migration strategies, including dual-write, cutover, reconciliation, validation, and rollback approaches
- leading the migration of Cosmos DB workloads and other Azure-hosted data services to VMware DSM
- deploying, configuring, and optimizing Redis Operator as the platform caching layer
- implementing and managing MinIO and Ceph-based object storage solutions
- deploying and managing Qdrant or similar vector databases for AI and semantic search use cases
- defining and executing data validation, reconciliation, and consistency verification processes
- configuring and maintaining backup, disaster recovery, failover, and business continuity mechanisms
- monitoring and optimizing database, storage, and data platform performance, scalability, and reliability
- supporting data ingestion and preparation for AI and vector database workloads
- collaborating with platform engineering, infrastructure, security, and AI teams on scalable data architecture design
- developing automation and operational tooling for migration, monitoring, and platform management
- participating in architecture reviews, sprint planning, backlog refinement, daily standups, and retrospectives
- providing technical leadership on cloud migration and data platform best practices
- ensuring compliance with data governance, security, and operational standards throughout migration activities

🎯 Skills
- Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field
- strong hands-on experience with Microsoft Azure data platforms and cloud services
- strong hands-on experience with VMware Cloud Foundation (VCF), VMware Data Services Manager (DSM), and VMware-based cloud platforms
- experience migrating databases, storage platforms, and data services between cloud environments
- 5+ years of experience in Data Engineering, Database Engineering, Data Platform Engineering, or Infrastructure Engineering roles
- strong experience with distributed database technologies and DBaaS platforms
- hands-on experience with NoSQL databases, including Cosmos DB or equivalent technologies
- proven experience designing and executing large-scale data migrations with minimal downtime
- strong understanding of migration patterns including dual-write, cutover, rollback, and reconciliation strategies
- experience with Redis clusters and enterprise caching architectures
- experience implementing and managing object storage platforms such as MinIO, Ceph, or S3-compatible solutions
- experience with vector databases such as Qdrant, Pinecone, Weaviate, Milvus, or similar technologies
- strong knowledge of data validation, reconciliation, and data quality frameworks
- experience with backup, disaster recovery, and failover strategies for distributed systems
- strong understanding of performance optimisation for large-scale data platforms
- strong scripting and automation skills using Python, Bash, or similar languages
- excellent analytical and problem-solving capabilities
- strong communication, stakeholder management, and collaboration skills
- experience working in Agile/Scrum environments

💡 Nice to have
- experience supporting AI platforms and Retrieval-Augmented Generation (RAG) architectures
- familiarity with Databricks, Spark, Delta Lake, or modern data platform technologies
- experience with AI search solutions and vector search architectures
- knowledge of Azure networking and cloud infrastructure architecture
- knowledge of data governance, security, and compliance frameworks
- experience with observability and monitoring platforms such as Prometheus, Grafana, or ELK.
- relevant Azure, VMware, Kubernetes, or cloud certifications

#LI-AB2