Trouvez votre prochaine possibilité

Détails du poste

We are looking for a GCP Senior Data Engineer with strong experience in GCP, BigQuery, Dataflow, Dataproc, Pub/Sub, Spark, Python, SQL, Terraform & CI/CD. **Required Skills** Technical: Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam). GCP Expertise: Hands-on with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform. DevOps: Familiarity with CI/CD, Git, and infrastructure-as-code. **Nice to Have** GCP Certifications(e.g., Professional Data Engineer). Experience with real-time processing, Vertex AI, or multi-cloud environments. **Key Responsibilities** Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions. Data Infrastructure: Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage. Team Leadership: Mentor junior engineers, review code, and establish data engineering standards, CI/CD, and monitoring (e.g., Cloud Monitoring, Logging). Performance & Cost: Optimize pipeline performance, query efficiency, and GCP cost managemen (e.g., slot reservations, partitioning). Collaboration: Work with data architects, scientists, and analysts to deliver self-service data platforms and support advanced analytics/ML use cases. Data Quality & Governance: Implement data validation, lineage, and metadata management(e.g., Data Catalog, dbt).

Ce que vous ferez

Design and optimize ETL/ELT pipelines using GCP tools like Dataflow and Dataproc while managing BigQuery infrastructure. Lead junior engineers and collaborate with data scientists to deliver self-service data platforms and governance.

Exigences

Requires proficiency in SQL, Python, and distributed systems with hands-on expertise in the GCP data suite and Terraform. Experience with CI/CD and infrastructure-as-code is essential for this mid-senior level role.

Compétences indiquées

• SQL · Souhaitée
• CI/CD · Souhaitée
• Java · Souhaitée
• Google Cloud · Souhaitée
• Terraform · Souhaitée
• Python · Souhaitée

Autres compétences pertinentes

Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.

• GCP
• BigQuery
• Dataflow
• Dataproc
• Pub/Sub
• Spark
• Python
• SQL
• Terraform
• CI/CD
• Java
• Scala
• Apache Beam
• Airflow
• Cloud Functions
• dbt

Domaines d’emploi

• Data & Analytics
• Technology
• Software
• Engineering
• Consulting