Search by job, company or skills

Senior AI Engineer

Senior AI Engineer

Flipkart Leap
8-10 Years
Not Disclosed
Early Applicant
  • Posted 2 days ago
  • Be among the first 10 applicants

Job Description

About the Team & Mission

The ML Platform team maximizes the speed, throughput, and hardware efficiency of our production Large Language Model (LLM) inference stack. As models scale and real-time demand grows, post-training optimization and aggressive server-level utilization are our primary levers for delivering low-latency AI at sustainable infrastructure economics.

We operate at the intersection of LLM serving engines, post-training compression, and bare-metal accelerator architecture. Our mission is to squeeze maximum compute and memory bandwidth out of every server node—turning heavy foundation models into ultra-fast, cost-efficient endpoints.

Role Overview

We are hiring a deeply technical Senior / Staff Software Engineer (Individual Contributor) with 8+ years of experience to partner directly with the Leadership in architecting and scaling our LLM inference platform.

In this high-leverage IC role, you will serve as our primary technical anchor for inference performance. You will own end-to-end serving architecture, drive post-training optimization pipelines, eliminate hardware bottlenecks, and guide engineers through complex systems challenges.

Key Responsibilities

  • Post-Training Optimization: Architect and productionize model compression and acceleration pipelines—including quantization (FP8/INT8/INT4), distillation, and speculative decoding—to slash latency (TTFT/ITL) while preserving model quality.
  • Hardware & Server Maximization: Drive bare-metal utilization across multi-GPU and multi-node topologies by optimizing memory bandwidth, tensor core occupancy, interconnects, and parallelism strategies (tensor, pipeline, expert, and disaggregated serving).
  • Inference Engine & Kernel Engineering: Extend modern inference frameworks (such as vLLM or SGLang) and build custom fused kernels and batching schedulers to eliminate compute, memory, and CPU host bottlenecks.
  • Profiling & Automated Diagnostics: Build deep telemetry, roofline modeling, and automated bottleneck-detection workflows to pinpoint hardware stalls and prevent performance regressions in production.
  • Technical Leadership: Partner with Leadership to shape our multi-quarter inference roadmap, author high-clarity RFCs and influence hardware capacity decisions.
  • Mentor Junior Engineers: Level up junior and mid-level engineers through code reviews, pair programming, and hands-on systems coaching.

Minimum Qualifications

  • Education: Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent practical systems experience.
  • Experience:8+ years of hands-on software engineering experience in high-performance computing (HPC), ML inference systems, GPU programming, or low-latency distributed infrastructure.
  • LLM Inference Mastery: Proven track record optimizing large-scale LLM serving in production, with a deep understanding of transformer memory footprints, KV-cache mechanics, continuous batching, and latency/throughput trade-offs.
  • Hardware & Systems Depth: Strong grasp of accelerator microarchitecture (compute vs. memory-bound workloads, roofline analysis, memory hierarchies) and multi-GPU server interconnects.
  • Programming Proficiency: Expert-level coding skills in C++ and Python, with experience building low-overhead, production-critical systems software.

Preferred Qualifications

  • GPU & Compiler Internals: Practical experience writing or profiling custom accelerator kernels and working close to ML compilers or runtime execution graphs.
  • Production Compression & Decoding: Track record deploying quantized models or speculative decoding techniques in high-throughput production environments.
  • Open-Source Inference Familiarity: Experience working inside or contributing to modern open-source LLM serving engines and attention libraries.
  • Complex Workload Scaling: Background optimizing Mixture-of-Experts (MoE) routing overhead or ultra-long-context inference workloads.
  • Automated Performance Tooling: Experience building automated benchmarking harnesses, hardware utilization models, or AI-assisted trace debugging tools.

Why Flipkart

Flipkart is India's leading e-commerce platform, where innovation meets impact at scale. As part of the AI Solutions team, you will shape the foundational AI infrastructure powering experiences for 500M+ users, millions of sellers, and complex supply chains. Flipkart is part of the Walmart-owned Flipkart Group, which includes Flipkart Wholesale, Flipkart Health+, Cleartrip, Shopsy, Flipkart Travels, and Myntra. Join us to build the platforms that make AI at Flipkart possible.

More Info

Key Skills

multi-GPU

multi-node topologies

speculative decoding

SGLang

custom fused kernels

quantization

roofline modeling

GPU programming

INT4

tensor core occupancy

INT8

ML inference systems

model compression

FP8

vLLM

automated bottleneck-detection

low-latency distributed infrastructure

batching schedulers

parallelism strategies

About Company

Similar Jobs

5-8 yrs
Bengaluru, India
Skills:
Databricks, FastAPI, Python, AWS, LangGraph, LangChain, LLMs, Transformers, Azure AI Foundry, Semantic Kernel
6-9 yrs
Bengaluru, India
Skills:
Machine Learning, Artificial Intelligence, Python, Generative AI, Large Language Models, Retrieval-Augmented Generation
6-8 yrs
Bengaluru, India
Skills:
Api Management, AI ML, Tensorflow, Azure ML, Nlp, MLops, Pytorch, Cosmos DB, Python, Azure DevOps, Generative AI, RAG systems, scikit-learn, LLMOps, Key Vault, Container Apps, AKS, Azure OpenAI, Azure AI Foundry, GitHub Actions, Azure AI Search, Application Insights
6-10 yrs
Bengaluru, India
Skills:
cloud security, Azure ML, Azure, Python, LLM evaluation guardrails, AI GenAI development, Azure OpenAI, Semantic Kernel, LangGraph, AutoGen, ML fundamentals, cost management, LlamaIndex
5-8 yrs
Bengaluru, India
Skills:
Tensorflow, Numpy, Pytorch, Docker, Dicom, Python, AWS, Pytest, Gcp, Opencv, Keras, FastAPI, Azure, Kubernetes, scikit-learn, Segmentation, TensorRT, MLflow, CI CD, CNNs, ONNX, Classical ML, Vision Transformers, nnU-Net, Weights Biases, Calibration, NIfTI, Clinical Metrics, MONAI, Detection