Data Engineer - Gen AI & LLM
Talentgigs · Bengaluru, Karnataka, India
- Full-time
- 5+ years
- Posted 1 month ago
Job description
About the Role
We are looking for a passionate Data Engineer with 5+ years of experience in designing, developing, and maintaining scalable data platforms and ETL/ELT pipelines. The ideal candidate should possess strong expertise in Python, SQL, cloud data services, and modern data engineering frameworks. You will play a key role in building reliable, high-performance data solutions that support analytics, reporting, and AI/ML initiatives.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
- Build and optimize data ingestion processes from multiple structured and unstructured
data sources.
- Develop robust data models and data warehouses for analytics and reporting.
- Design and optimize SQL queries for high-performance data processing.
- Build and maintain data lakes using cloud storage solutions.
- Implement data validation, cleansing, transformation, and quality checks.
- Integrate data solutions with cloud platforms such as AWS, Azure, or Google Cloud
Platform.
- Develop batch and real-time data processing pipelines using modern data processing
frameworks.
- Collaborate with data scientists, analysts, and application teams to deliver reliable data
solutions.
- Follow software engineering best practices, including Git, CI/CD pipelines, testing,
monitoring, and technical documentation. Required Skills
- 5+ years of experience in Data Engineering.
- Strong programming experience in Python.
- Hands-on experience with PySpark and Apache Spark.
- Advanced SQL skills, including query optimization and performance tuning.
- Experience with AWS services/ Azure/ GCP.
- Experience building scalable ETL/ELT pipelines.
- Strong exp in Generative AI and Large Language Models (LLMs)
- Knowledge of data warehousing concepts and dimensional modeling.
- Experience working with large-scale distributed datasets.
- Familiarity with Git and CI/CD pipelines.
- Understanding Agile/Scrum methodologies.
Preferred Qualifications
- Experience with Apache Airflow or similar workflow orchestration tools.
- Knowledge of Kafka and real-time data streaming.
- Familiarity with Docker, Kubernetes, and Terraform (IaC).
- Exposure to Databricks and modern data lake technologies (Delta Lake, Apache Iceberg,
or Apache Hudi).
- Understanding data governance, metadata management, and CI/CD practices.
- Strong exp in Generative AI and Large Language Models (LLMs
About the company
Talentgigs – Curating Excellence in Digital Recruitment At Talentgigs, we are redefining the future of hiring through a seamless blend of cutting-edge technology and deep human insight. As one of India’s most forward-thinking recruitment partners, we specialize in crafting meaningful connections between exceptional talent and visionary organizations across IT and non-IT sectors. Whether you're a company looking to scale with precision or a candidate seeking your next big opportunity, Talentgigs is your trusted partner in navigating the evolving world of work. ✦ For Employers We understand that the right talent is the cornerstone of every successful business. Through our AI-powered sourcing strategies, targeted digital marketing, and an agile recruitment framework, we deliver high-impact hiring solutions — fast, focused, and future-ready. With over 250+ MNCs relying on us, we are proud to be a preferred hiring ally for some of the world’s most respected brands. ✦ For Professionals We don’t just match resumes — we shape careers. Our approach is personalized, strategic, and designed to align your aspirations with opportunities that truly fit. At Talentgigs, you’re not just a candidate — you’re part of a community powered by growth and guided by care. 🚀 What Distinguishes Us: ✅ Digital Foresight: Leveraging AI and analytics for smarter hiring decisions. ✅ Human-Centric Values: Every interaction is built on empathy, trust, and transparency. ✅ Domain Expertise: Deep understanding across diverse industries and roles. ✅ End-to-End Efficiency: From sourcing to onboarding, we make hiring effortless. Join Talentgigs — where technology meets intuition and careers meet opportunity. Hiring should be more than a process — it should be a partnership.
- Headquarters
- Bengaluru, Karnataka, India
- Company size
- 14
- Industry
- IT Services and IT Consulting
- Website
- talentgigs.in/lets-talk