Databricks Data Engineer

Résumé du poste

The role involves developing ETL/ELT pipelines and managing data migration projects using Azure Databricks. You will be responsible for optimizing SQL performance and ensuring data reconciliation across complex data environments.

Détails du poste

Proven experience with Azure Databricks, including Unity Catalog and Data Lakehouse Architecture. Strong proficiency in PySpark and Spark SQL. Expertise in ETL/ELT Development. Experience with Azure Data Lake Storage (ADLS). Familiarity with Git and CI/CD practices. SQL Optimization and Performance Tuning skills. Experience in Data Migration and Reconciliation Frameworks. GCP BigQuery Migration Experience is essential.

Ce que vous ferez

The role involves developing ETL/ELT pipelines and managing data migration projects using Azure Databricks. You will be responsible for optimizing SQL performance and ensuring data reconciliation across complex data environments.

Exigences

Candidates must have proven experience with Azure Databricks, Unity Catalog, and Data Lakehouse architecture. Proficiency in PySpark, Spark SQL, and experience with GCP BigQuery migration are essential requirements.

Compétences indiquées

• CI/CD · Souhaitée
• Git · Souhaitée

Autres compétences pertinentes

Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.

• Azure Databricks
• Unity Catalog
• Data Lakehouse Architecture
• PySpark
• Spark SQL
• ETL/ELT Development
• Azure Data Lake Storage
• Git
• CI/CD
• SQL Optimization
• Performance Tuning
• Data Migration
• Reconciliation Frameworks
• GCP BigQuery

Domaines d’emploi

• Data & Analytics
• Technology
• Software
• Engineering
• Consulting