OpenAI Logo

OpenAI

Network Operations Engineer, AI Networking

Posted 2 Days Ago
Be an Early Applicant
In-Office
San Francisco, CA
157K-221K Annually
Senior level
In-Office
San Francisco, CA
157K-221K Annually
Senior level
Operate and improve large-scale Ethernet fabrics that support GPU clusters, storage, and management infrastructure. Monitor and resolve production incidents, execute changes and maintenance, manage hardware lifecycle, perform root-cause analysis, build telemetry and runbooks, and automate operational tasks to improve reliability and reduce toil across global AI data centers.
The summary above was generated by AI

About the Team

OpenAI’s Infrastructure Operations team is responsible for the availability, reliability, and operational excellence of one of the world’s largest AI infrastructure networks. The team owns day-to-day operations of production AI networks across Industrial Compute's data centers, working with colocation providers, deployment teams, and hardware vendors to deliver highly available GPU infrastructure for AI training and inference workloads.

About the Role

We are seeking an Infrastructure Operations Engineer to operate and improve the large-scale Ethernet fabrics that support GPU clusters, storage systems, and management infrastructure. This role combines hands-on production operations with automation, observability, and incident response across a global AI network.

The ideal candidate has experience operating high-availability data center, cloud, AI, or HPC networks and can move comfortably from physical-layer troubleshooting to routing and fabric behavior, change execution, and root-cause analysis. You will partner closely with network architecture, systems engineering, GPU engineering, storage engineering, security, deployment, site operations, service providers, colocation partners, and hardware vendors to raise reliability and reduce operational toil.

Key Responsibilities

  • Own the operational health, availability, and reliability of production AI network infrastructure across Industrial Compute's data centers.

  • Monitor, troubleshoot, and resolve network incidents while meeting service-level objectives (SLOs), reducing Mean Time to Detect (MTTD), and minimizing Mean Time to Recovery (MTTR).

  • Operate and maintain large-scale Ethernet fabrics supporting GPU compute, storage, and management networks.

  • Execute production network changes, maintenance windows, and capacity expansions with minimal customer impact.

  • Manage the hardware lifecycle, including switch and optics replacements, RMA coordination, software upgrades, and preventive maintenance.

  • Support new AI cluster deployments, data center expansions, and infrastructure migrations in partnership with deployment and engineering teams.

  • Partner with cloud service providers (CSPs), colocation providers, Smart Hands teams, and hardware vendors to maintain production infrastructure.

  • Perform root-cause analysis (RCA) for production incidents and drive permanent corrective actions that eliminate recurring issues.

  • Build and maintain monitoring, telemetry, dashboards, and alerting to improve network observability and proactive issue detection.

  • Develop and improve operational runbooks, playbooks, troubleshooting documentation, and standard operating procedures.

  • Automate repetitive operational tasks using Python and infrastructure automation frameworks to reduce toil and improve efficiency.

  • Continuously identify opportunities to improve service reliability, scalability, operational maturity, and engineering efficiency.

Qualifications

  • Bachelor’s degree in Computer Science, Network Engineering, or a related discipline, or equivalent practical experience.

  • 5+ years of experience operating large-scale data center, cloud, AI, or HPC network infrastructure.

  • Experience supporting production network environments with high-availability requirements.

  • Hands-on experience with one or more of the following platforms: Cisco NX-OS, Arista EOS, NVIDIA Spectrum / Cumulus Linux, or Juniper JunOS.

  • Strong knowledge of Layer 2 and Layer 3 networking, BGP, OSPF, ECMP, MLAG, LACP, VRFs, and VLANs.

  • Experience troubleshooting physical infrastructure, including fiber optics, transceivers, DAC/AOC cables, and high-speed Ethernet links.

  • Experience performing software upgrades, hardware maintenance, and production change management.

  • Excellent analytical and troubleshooting skills, with the ability to communicate technical risk clearly across teams.

Preferred Skills

  • Experience operating AI or High-Performance Computing (HPC) network environments.

  • Experience with NVIDIA AI networking technologies and GPU infrastructure.

  • Experience supporting RoCE v2 or RDMA-based Ethernet fabrics, with a strong understanding of Priority Flow Control (PFC), Explicit Congestion Notification (ECN), Data Center Quantized Congestion Notification (DCQCN), Quality of Service (QoS), and lossless Ethernet networking.

  • Experience supporting 100G, 200G, 400G, and 800G Ethernet networks.

  • Experience with GPU platforms including NVIDIA HGX, DGX, GB200, or equivalent AI infrastructure.

  • Experience supporting distributed storage environments such as VAST, DDN, or similar technologies.

  • Experience working with cloud service providers such as AWS, Azure, or Google Cloud, and with third-party colocation providers.

  • Experience with network monitoring and telemetry technologies, including Prometheus, Grafana, gNMI, streaming telemetry, SNMP, or similar tools.

  • Experience developing automation using Python, Git, REST APIs, Terraform, or similar automation frameworks.

Work Environment and On-Call

  • Participate in a 24x7 on-call rotation supporting mission-critical AI infrastructure.

  • Support time-sensitive production incidents, maintenance windows, capacity expansions, and network changes with a focus on service availability and minimal customer impact.

  • This role requires up to 30% travel to data center locations for new turnups and acceptance activities, as needed.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. 

We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.

For additional information, please see OpenAI’s Affirmative Action and Equal Employment Opportunity Policy Statement.

Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.

To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Similar Jobs

9 Minutes Ago
Hybrid
Irvine, CA, USA
174K-231K Annually
Senior level
174K-231K Annually
Senior level
Digital Media • Information Technology • News + Entertainment
Lead analytical and operational agenda for Xumo's FAST business: develop business cases, financial and scenario models, drive cross-functional analyses, inform senior leadership decisions on partnerships, monetization, expansion, and operational improvements, and produce executive-ready materials.
Top Skills: ExcelGoogle SlidesLookerPeriscopePowerPointTableau
9 Minutes Ago
Remote or Hybrid
California, USA
63K-126K Annually
Senior level
63K-126K Annually
Senior level
Digital Media • Information Technology • News + Entertainment
Responsible for driving sales of Comcast Business products in mid-market and enterprise clients, managing relationships, and delivering sales presentations. Requires industry knowledge, strategic partnerships, and excellent customer service skills.
Top Skills: CybersecurityEthernetInternet TechnologiesLanManNetwork DesignNetwork SecurityNetworking ProtocolsSdwanVoipVpnWanWdm
10 Minutes Ago
Remote or Hybrid
California, USA
63K-126K Annually
Senior level
63K-126K Annually
Senior level
Digital Media • Information Technology • News + Entertainment
Drive acquisition and management of mid-market and enterprise multi-location customers by developing territory strategy, prospecting, delivering face-to-face sales presentations, closing complex solutions, building partner relationships, and ensuring customer retention and operational service levels.
Top Skills: Business Continuity/Disaster RecoveryCustomer Premise Equipment (Cpe)CybersecurityEthernetInternet TechnologiesLanManNetwork SecurityNetworking Protocols (Layers 1-3)SdwanVoipVpnWanWdm

What you need to know about the Los Angeles Tech Scene

Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.

Key Facts About Los Angeles Tech

  • Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
  • Key Industries: Artificial intelligence, adtech, media, software, game development
  • Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
  • Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account