Azure Data engineer

Résumé du poste

Develop and maintain data pipelines using Azure Data Factory and Snowflake to support enterprise data warehousing. Implement change data capture and ETL/ELT processes using Databricks and PySpark.

Détails du poste

Role: Azure Data Engineer Skillset: Snowflake, ADF, Python & SQL are must and Databricks is a good to have. Skills required - Python Modules: • Standard Libraries (required) • Azure SDK (preferred) • Keyring (preferred) • Pandas (preferred) • Snowflake Connector (preferred) ADF Components: • Flow Control Logic • Lookups • Copy Activities • Child pipeline execution • Script calls • Batch service calls Dataflows • EMS , Event subscription, Event hub, Function Apps etc Snowflake Objects: • COPY INTO Scripts/Commands • Tasks • Dynamic Tables Databricks: • Notebooks to perform change data capture • Familiarity with Spark/PySpark ETL & Development Basics - • CI/CD in Azure DevOps • Knowledge of EDW , ETL, ELT processes, Data warehousing • Advanced SQL knowledge • Code deployment process • Troubleshooting pipelines • ADO • Agile methodologies

Ce que vous ferez

Develop and maintain data pipelines using Azure Data Factory and Snowflake to support enterprise data warehousing. Implement change data capture and ETL/ELT processes using Databricks and PySpark.

Exigences

Must be proficient in Python, SQL, Snowflake, and Azure Data Factory. Experience with Azure DevOps, CI/CD, and Agile methodologies is required.

Compétences indiquées

• SQL · Souhaitée
• CI/CD · Souhaitée
• Python · Souhaitée

Autres compétences pertinentes

Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.

• Snowflake
• Azure Data Factory
• Python
• SQL
• Databricks
• PySpark
• Azure DevOps
• ETL
• ELT
• Data Warehousing
• CI/CD
• Agile Methodologies
• Azure SDK
• Pandas
• Event Hub
• Function Apps

Domaines d’emploi

• Data & Analytics
• Technology
• Software
• Engineering
• Consulting