Azure Databricks
Offre en anglaisDesign and develop ETL pipelines using Azure Databricks and Delta Lake, focusing on batch processing and structured streaming. Manage end-to-end environments using Unity Catalog and implement data governance and security practices.
- Sur place
- Toronto, ON
- Publié 13 août 2026
- Postuler avant le 12 sept. 2026
- 1 poste
D’autres postes auxquels postuler directement
Des possibilités semblables publiées par des employeurs qui recrutent sur Jobs.ca, sans formulaire externe.
Résumé du poste
Azure Databricks Location: Toronto, Canada Job Description: We are seeking a highly skilled Backend/API Developer with strong hands-on experience in Azure Databricks and Apache Spark (PySpark/Scala). The ideal candidate will have a solid background in SQL, data transformation techniques, and cloud platforms (Azure/AWS/GCP). You will be responsible for building and optimizing ETL pipelines, ensuring data integrity, and implementing data governance practices. Responsibilities Design and develop ETL pipelines using Azure Databricks and Delta Lake, focusing on batch processing (autoloader) and Spark structured streaming. Create and manage end-to-end environments, including catalogs, schemas, tables, materialized views, functions, and volumes using Unity Catalog. Implement slowly changing dimensions (SCD1 and SCD2) on dimension tables and build change data capture (CDC) pipelines. Utilize Lakehouse federation to create foreign catalogs for accessing data from external sources. Optimize data processing through effective partitioning and liquid clustering in Databricks. Collaborate with cross-functional teams to ensure data governance and security practices are adhered to. Participate in CI/CD pipeline development and DevOps practices to enhance deployment efficiency. Mandatory Skills Expertise in Azure Databricks Expert-level proficiency in SQL Regular experience with CI/CD practices
Ce que vous ferez
Design and develop ETL pipelines using Azure Databricks and Delta Lake, focusing on batch processing and structured streaming. Manage end-to-end environments using Unity Catalog and implement data governance and security practices.
Exigences
Requires expert-level proficiency in SQL and extensive hands-on experience with Azure Databricks and Apache Spark. Candidates must have regular experience with CI/CD practices and data transformation techniques.
Compétences indiquées
- SQLSouhaitée
- CI/CDSouhaitée
Autres compétences pertinentes
Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.
- Azure Databricks
- Apache Spark
- PySpark
- Scala
- SQL
- ETL Pipelines
- Delta Lake
- Unity Catalog
- CI/CD
- DevOps
- Data Governance
- Lakehouse Federation
- SCD1
- SCD2
- CDC Pipelines
- Liquid Clustering
Domaines d’emploi
- Data & Analytics
- Software
- Technology
- Finance & Accounting
- Healthcare
Renseignements supplémentaires
- Expérience minimale
- 5+ ans
- Postuler avant le
- 12 sept. 2026
- Langue de l’offre
- anglais
- Heures de travail
- 40 heures par semaine
- Niveau d’expérience
- Mid-Senior level
- Mode de candidature
- La candidature directe est offerte