Principal Engineer, Data Analytics Engineering

Job Description

As a GenAI Solution Architect, you will design and implement enterprise-grade Generative AI solutions that seamlessly integrate with business applications and workflows. This role spans end-to-end architecture—from building & maintaining - GenAI pipelines, prompt engineering strategies, and multi-LLM gateways to managing data lake houses, retrieval & knowledge governance frameworks. You will develop ontologies, taxonomies, and agentic workflows for autonomous reasoning while ensuring compliance, observability, and cost optimization. The ideal candidate combines deep expertise in AI/ML systems, data engineering, and enterprise integration to deliver scalable, secure, and efficient GenAI solutions that transform knowledge management and decision-making across the organization. Preference will be given to candidates with experience in both leveraging industry-leading solutions and building custom GenAI solutions from the ground up

Key Responsibilities:

1.GenAI Development & Integration

• Design and implement GenAI workflows for enterprise use cases.

• Develop prompt engineering strategies and feedback loops for LLM optimization.

• Capture and normalize LLM interactions into reusable Knowledge Artifacts.

• Integrate GenAI systems into enterprise apps (APIs, microservices, workflow engines)

• Programming languages: Python

2. Data Lakehouse & Knowledge Management

• Architect and maintain Lakehouse environments for structured and unstructured data.

• Implement pipelines for document parsing, chunking, and vectorization.

• Maintain knowledge stores, indexing, metadata governance

• Enable semantic search and retrieval using embeddings and vector databases.

3. Ontology & Taxonomy Engineering

• Build and maintain domain-specific ontologies and taxonomies.

• Establish taxonomy governance and versioning.

• Connect semantic registries with LLM learning cycles.

• Enable knowledge distillation from human/LLM feedback.

4. AI Governance & Knowledge Distillation

• Establish frameworks for semantic registry, prompt feedback, and knowledge harvesting.

• Ensure compliance, normalization, and promotion of LLM outputs as enterprise knowledge.

5. Observability & Cost Optimization

• Implement observability frameworks for GenAI systems (performance, latency, drift).

• Monitor and optimize token usage, inference cost, and model efficiency.

• Maintain dashboards for usage analytics & operational metrics.

• Make Build vs. Buy decisions based on cost-benefit analysis

6. Model Gateway & Multi-LLM Strategy

• Architect model gateways to access multiple LLMs (OpenAI, Anthropic, Cohere, etc.).

• Dynamically select models based on accuracy vs. cost trade-offs.

• Benchmark and evaluate models for enterprise-grade performance.

7. Agentic Workflows

• Design and implement agent-based orchestration for multi-step reasoning and autonomous task execution.

• Design and implement agentic workflows using industry-standard frameworks for autonomous task