Group Manager - Data Engineering

About the Role

We are seeking a highly skilled Data Engineer with expertise in Google Cloud Platform (GCP), Generative AI, Python, SQL, ETL, Dataproc, and Airflow. The ideal candidate will play a key role in designing and implementing scalable cloud-based data platforms, modern ETL pipelines, and AI-enabled data solutions.

Key Responsibilities

Design, build, and maintain scalable ETL/ELT pipelines using Python and SQL.

Develop cloud-native data engineering solutions on GCP.

Implement batch and real-time data processing pipelines using Dataproc and Dataflow.

Build and manage workflow orchestration using Apache Airflow / Cloud Composer.

Work on large-scale structured and unstructured datasets for analytics and AI use cases.

Integrate Generative AI and LLM-based capabilities into enterprise data platforms.

Optimize BigQuery workloads and improve data processing efficiency.

Collaborate with data scientists, analysts, architects, and business teams to deliver data-driven solutions.

Ensure data quality, governance, scalability, reliability, and security.

Participate in architecture reviews and provide technical leadership and mentorship.

Required Skills

Strong experience in Google Cloud Platform (GCP) services:BigQuery

Cloud Storage

Dataflow

Pub/Sub

Dataproc

Cloud Composer / Apache Airflow

Hands-on experience in Generative AI / LLM solutions.

Strong programming expertise in Python.

Advanced SQL development and query optimization skills.

Extensive experience in ETL/ELT pipeline development.

Hands-on experience with workflow orchestration using Apache Airflow.

Experience in distributed data processing using Dataproc / Spark / PySpark.

Good understanding of data warehousing concepts and cloud architecture.

Experience with Git, CI/CD pipelines, and Agile methodologies.

Preferred Skills

Experience with Vertex AI, LangChain, RAG architecture, vector databases, or AI orchestration frameworks.

Exposure to streaming technologies such as Kafka or Pub/Sub.

Knowledge of Infrastructure as Code tools such as Terraform.

Familiarity with MLOps concepts and ML pipeline integration.

GCP certifications are a plus.

Educational Qualification

Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or related discipline.

Desired Candidate Profile

Excellent analytical and problem-solving skills.

Strong communication and stakeholder management abilities.

Ability to work in a fast-paced, collaborative environment.

Passion for cloud technologies, modern data platforms, and AI innovation.

Graduate or above