OpenAI Logo

OpenAI

Datacenter Incident Program Manager

Reposted 19 Days Ago
Remote
Hiring Remotely in United States
126K-228K Annually
Senior level
Remote
Hiring Remotely in United States
126K-228K Annually
Senior level
The Datacenter Incident Program Manager oversees the incident management lifecycle in data centers, defines response protocols, manages incident tools, and leads post-incident reviews.
The summary above was generated by AI

About the Team:

OpenAI, in close collaboration with our capital partners, is embarking on a journey to build the world’s most advanced AI infrastructure ecosystem. Our Stargate program develops and deploys massive, state-of-the-art data center campuses in partnership with industry leaders such as Oracle today—and through future OpenAI infrastructure projects tomorrow. We design for scale, speed, and reliability, and we need experienced hardware professionals who can help ensure our high-density compute environment operates at peak performance.

About the Role:

The Datacenter Incident Program Manager is responsible for designing, operating, and continuously improving the end-to-end incident management lifecycle across mission-critical data center environments.This role owns the “before, during, and after” mechanics of incidents — establishing standards and playbooks in steady state, serving as (or designating) Incident Commander during active events, and driving structured post-incident review and corrective action to closure.

The ideal candidate brings operational credibility in hyperscale or mission-critical infrastructure, demonstrates calm leadership during high-pressure events, and has a strong bias toward structured documentation, process clarity, and measurable improvement.

Key Responsibilities

  • Define and maintain incident severity levels (SEV definitions), classification criteria, and escalation thresholds.

  • Establish end-to-end incident response standards: protocols, lifecycle stages (declare → stabilize → mitigate → recover → close), and operating cadence.

  • Build and maintain governance artifacts: runbooks, war room formats, reporting templates, and decision/communication standards.

  • Create and operationalize notification trees, stakeholder comms templates (initial, periodic updates, recovery/closure), and executive escalation criteria.

  • Define clear RACI across Facilities, Hardware Ops, Network, Security, and vendor/partner teams, including handoffs and accountability paths.

  • Set and manage SLAs/OLAs for acknowledgment, escalation, containment, mitigation, and reporting.

  • Implement and run incident management tooling (ticketing, paging, logging) and ensure integrations with monitoring and workflow systems.

  • Establish dashboards and program health metrics to track incident performance and readiness.

  • Lead readiness activities: tabletop exercises, cross-functional simulations, IC/Deputy training, and a rotating on-call IC bench with certification standards.

  • Serve as Incident Commander as needed: declare severity, stand up the war room, assign functional leads, and drive structured execution under pressure.

  • Maintain real-time documentation (decisions, timelines, impact scope) and ensure clear restoration objectives and scope control during active events.

  • Run post-incident reviews (PIRs), validate timelines, drive structured RCA (e.g., 5 Whys, Fault Tree), and separate root cause vs contributing factors.

  • Define corrective/preventative actions (CAPAs), assign accountable owners, track to verified closure, and escalate overdue actions.

  • Publish trend reporting (incident taxonomy, counts by severity, MTTA/MTTR, repeat failure domains) and feed systemic gaps back into design and operations teams.

Qualifications

  • 7+ years in mission-critical infrastructure, data center operations, or reliability engineering

  • Direct experience leading major incidents (P1/P0 equivalent)

  • Strong familiarity with facilities systems, hardware operations, or network infrastructure

  • Demonstrated experience running war rooms and executive updates

  • Experience conducting root cause analysis and corrective action tracking

Ability to remain calm and decisive under high-pressure conditions
Preferred Skills

  • Experience in hyperscale or high-density AI compute environments

  • Background in facilities commissioning, facility operations, hardware operations, or network reliability

  • Familiarity with ISO-based quality systems or structured operational documentation frameworks

  • Experience implementing incident tooling (PagerDuty, ServiceNow, Jira, etc.)

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. 

We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.

For additional information, please see OpenAI’s Affirmative Action and Equal Employment Opportunity Policy Statement.

Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.

To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Similar Jobs

44 Minutes Ago
Remote or Hybrid
US
100K-110K Annually
Mid level
100K-110K Annually
Mid level
Agency • Gaming • Marketing Tech • Mobile • Analytics
The Data Security Engineer will develop and implement data protection strategies, ensuring compliance with regulations like GDPR and HIPAA while securing sensitive data and collaborating with various teams.
Top Skills: Analytical ToolsCcpaCismCisspCybersecurityDlp StrategiesGdprHipaaInformation Technology
46 Minutes Ago
Easy Apply
Remote
USA
Easy Apply
150K-180K Annually
Senior level
150K-180K Annually
Senior level
Artificial Intelligence • Healthtech • Information Technology • Software • Conversational AI • Generative AI • Automation
As a Senior Software Engineer, you will design and build key features for Collectly's AI-driven revenue cycle management solutions, collaborating closely with teams to ensure quality and scalability.
Top Skills: FastapiFlaskPythonReact
An Hour Ago
Remote or Hybrid
United States
82K-110K Annually
Mid level
82K-110K Annually
Mid level
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
The Security Operations Analyst reviews security events, investigates malicious activity, leads incident responses, and documents findings while participating in threat hunting activities.
Top Skills: LinuxmacOSSecurity Information And Event Management (Siem)SplunkWindows

What you need to know about the Los Angeles Tech Scene

Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.

Key Facts About Los Angeles Tech

  • Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
  • Key Industries: Artificial intelligence, adtech, media, software, game development
  • Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
  • Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account