Manager, Software Engineering - Production AI Inference

NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a deeply technical software manager to lead production AI inference for NVIDIA Inference Microservices (NIM), the production runtime through which customers deploy optimized, enterprise-supported AI inference across cloud, data center, and edge environments. NIM makes state-of-the-art AI models available as production-ready software stack, combining optimized inference engines, model profiles/recipes, validated runtime configurations, and security hardening. This role leads the team accountable for turning fast-moving model and inference engine work into reliable NIM releases that customers can operate with confidence.

This is a hands-on engineering management role for someone who can run production execution without managing from a distance. You will lead engineers working across model onboarding, serving stack integration, performance profiling/optimization, release quality, security readiness, automation, observability, and operational health. You will partner closely with the product, solution architect, security, research, and other internal engineering teams to make day-0 model launches repeatable and to raise the production bar for every NIM release.

What you'll be doing:

• Lead the team responsible for shipping production-ready LLM NIMs, including planning, new model onboarding, validated serving recipes, release readiness, and post-release follow-through.

• Build a predictable operating model for the team through roadmap planning, a weekly execution rhythm, launch checklists, clear ownership boundaries, collaborator communication, and issue management.

• Own project execution by anticipating schedule, staffing, and dependency risks. Adapt plans under pressure and collaborate with peer managers to dynamically prioritize engineering timelines to remain agile in the fast paced AI industry.

• Drive continuous improvement in production workflows through RCCA and partner feedback, removing unnecessary and redundant work while keeping the team passionate about production outcomes.

• Build and maintain a world-class AI inference engineering team by building an innovative culture, setting clear expectations, maintaining active feedback loops, and mentoring engineers and emerging leaders.

What we need to see:

• 10+ overall years building production software, including 3+ years of managing software engineering teams.

• Experience delivering production software with strong quality, reliability, and release expectations.

• Experience driving process improvements, and improving operational efficiency.

• Excellent communication and collaborator management; ability to influence executive leadership across product, research, security, and operations.

• Deep understanding of AI/ML fundamentals, innovative model architectures, inference engine/kernel, performance optimization strategies, accelerated computing, large-scale distributed systems, and security hardening.

• A degree in Computer Science, Computer Engineering, or a related field (BS or MS) or equivalent experience.

Ways to stand out from the crowd:

• Built and managed globally distributed organizations; established durable engineering processes that significantly improved quality and velocity across multiple teams.

• Recognized industry leader with contributions to open-source ecosystems (i.e vLLM, SGLang, TensorRTLLM, Dynamo, Triton, PyTorch), technical publications, or talks in containers, Kubernetes, GPU, or inference communities.

• Drove measurable performance improvements for large-scale LLM inference systems, including latency, throughput, GPU utilization, cost efficiency, and performance regression prevention across production releases.

• Hands-on experience with core GPU technologies such as CUDA, cuDNN, CUTLASS, cuBLAS, NCCL, NIXL, NVLink, and GPUDirect RDMA.

• Hands-on experience delivering enterprise or government-ready AI software, including FedRAMP, air-gapped deployments, regulated environments, security hardening, compliance evidence, and production support expectations.

With competitive salaries and a generous benefits package, we are widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us and, due to unprecedented growth, our exclusive engineering teams are rapidly growing. If you're a creative and autonomous engineer with a real passion for technology, we want to hear from you. NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most hard-working and talented people in the world working for us. If you're creative and passionate about developing cloud services we want to hear from you!

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD for Level 3, and 272,000 USD - 431,250 USD for Level 4.Applications for this job will be accepted at least until July 6, 2026.

This posting is for an existing vacancy. 

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.