fal Logo

fal

Operations Engineer, HPC Networking

Reposted 4 Days Ago
Remote
Hiring Remotely in USA
Mid level
Remote
Hiring Remotely in USA
Mid level
As an Operations Engineer, you will manage HPC networking, monitor and debug InfiniBand and Ethernet fabrics, and support fabric bring-up while enhancing operational tools and runbooks.
The summary above was generated by AI

fal is the generative media ecosystem powering the next generation of AI products. We build the infrastructure, tools, and model access that teams need to move from idea to production, and do it at scale without compromise. For developers and enterprises, fal is the foundation that makes generative media not just possible, but practical: a unified platform where high-performance inference, orchestration, and observability come together to unlock new categories of AI-native products.

As generative media reshapes industries across a market projected to grow by hundreds of billions over the next decade, fal is becoming the ecosystem that ambitious teams build on.

About the role

We're hiring an Operations Engineer for HPC Networking to keep our InfiniBand and Ethernet fabrics healthy as we scale.

This is a hands-on role. You'll bring up new fabrics alongside DC ops, monitor the ones in production, and chase down the weird stuff: link flaps, congestion, NCCL stalls, firmware bugs that only show up at scale. 

You're a fit if you've:
  • Operated InfiniBand fabrics in production: subnet manager, routing, partitioning, monitoring.
  • Debugged the full stack: cables, transceivers, switch firmware, HCAs, drivers, NCCL.
  • Brought up new fabrics from cable pull through validation.
  • Scripted your way through repetitive operational work (bash, python, go, whatever).
  • Nice to have: Ethernet RoCE, Spectrum-X, or large-scale GPU cluster networking.
Who you are:
  • Detail-oriented. Cable plant hygiene is a personality trait.
  • Calm under fire. A fabric incident during a customer training run doesn't rattle you.
  • You read vendor release notes for fun, or at least out of self-defense.
  • You'd rather find the root cause than reboot the switch.
Responsibilities:
  • Monitor health and performance of InfiniBand and Ethernet fabrics: switches, HCAs, transceivers, links.
  • Investigate and resolve fabric issues: connectivity, congestion, performance regressions.
  • Support fabric bring-up alongside DC ops and customer-facing teams.
  • Run maintenance and upgrades on switches and control plane components.
  • Partner with cluster ops on cross-domain incidents where the line between compute and network is blurry.
  • Improve the tooling and runbooks so the next incident resolves faster than the last.

Similar Jobs

2 Hours Ago
Remote or Hybrid
3 Locations
97K-134K Annually
Mid level
97K-134K Annually
Mid level
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Lead shopper and category insights for Meijer, using syndicated, shopper, and retailer data to develop category strategies, planograms, and dashboards that drive assortment, promotion, and merchandising recommendations to grow sales and market share. Collaborate with sales, buyers, and cross-functional teams and support joint business planning and performance optimization.
Top Skills: ApolloDunnhumbyIri/CircanaJdaExcelMicrosoft PowerpointNielseniqNumeratorPower BIPower QueryShelfiqSpectraTableauVendornet
2 Hours Ago
Remote or Hybrid
CA, USA
130K-234K Annually
Senior level
130K-234K Annually
Senior level
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Outbound-focused senior account executive responsible for sourcing and closing new restaurant merchant logos. Duties include prospecting, discovery, demos, consultative selling of Square ecosystem, field relationship building, partnering with BD/Product/Marketing, managing the sales cycle and onboarding, and meeting monthly sales KPIs using Salesforce.
Top Skills: SalesforceSquare
3 Hours Ago
Remote or Hybrid
144K-216K Annually
Senior level
144K-216K Annually
Senior level
Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
Lead and manage audit, review, and compilation engagements for construction clients. Oversee engagement planning, staffing, budgeting, fieldwork, risk assessment, and reporting. Advise on complex GAAP and FASB issues, support clients with accounting compliance and ASC updates, provide training, supervise teams, and participate in business development and community activities.

What you need to know about the Los Angeles Tech Scene

Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.

Key Facts About Los Angeles Tech

  • Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
  • Key Industries: Artificial intelligence, adtech, media, software, game development
  • Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
  • Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account