The Senior Reliability Engineer will focus on enhancing system reliability through code, architecting high-scale backend systems, and leading technical improvements.
About Accrue
About the RoleAccrue is redefining how brands turn payments into loyalty. We help enterprise merchants launch their own branded wallets.
We're looking for a senior backend engineer who builds reliability through elegant, production-ready code architecture. You'll have significant authority to rearchitect critical systems, replacing homegrown solutions with industry-standard tooling and patterns that handle 10k+ req/sec at scale.
This is primarily a programming role focused on building robust, observable systems through code. You'll spend most of your time architecting and implementing reliability improvements, not managing infrastructure.
What makes this role unique:
- Architectural Authority: Drive decisions on adopting technologies like Temporal.io for durable execution vs. maintaining custom retry logic
- Production Scale: Design systems that handle high-throughput payment and loyalty processing with strict SLA requirements
- Code-First Reliability: Improve system reliability by writing better application code, not just adding monitoring
- Industry Standards Over NIH: Replace internal implementations with proven, production-ready solutions
- Rearchitect core reliability patterns: Replace custom retry mechanisms with durable execution engines like Temporal.io
- Implement robust event processing: Migrate direct webhook handling to reliable delivery systems like Hookdeck with proper delivery semantics
- Build behavioral monitoring: Integrate time-series databases to detect and alert on changing system behavioral patterns
- Eliminate technical debt: Systematically replace "not invented here" solutions with industry-standard, battle-tested alternatives
- Design and implement systems that maintain performance and reliability at 10k+ requests/second
- Write production-grade code for payment processing, wallet operations, and loyalty program mechanics
- Build comprehensive error handling, circuit breakers, and graceful degradation patterns
- Implement distributed system patterns for fault tolerance and observability
- Instrument deep observability into application code using existing frameworks (Datadog)
- Design monitoring that provides actionable insights into system behavior and business metrics
- Build alerting that proactively identifies reliability issues before they impact users
- Lead incident response with focus on permanent architectural fixes rather than band-aid solutions
- Evaluate and recommend new technologies and architectural patterns for production readiness
- Collaborate with product engineering teams to embed reliability patterns into new feature development
- Drive technical decisions around system architecture, scaling, and reliability patterns
- Mentor engineers on production best practices and reliable system design
- 5+ years backend engineering experience building high-throughput, production systems (10k+ req/sec)
- Strong programming skills in modern languages - our stack uses TypeScript, but we value polyglot engineers
- Production architecture experience with distributed systems, microservices, and reliability patterns
- Systems thinking: Ability to identify when to build vs. buy vs. adopt existing solutions
- Cloud-native development with AWS services (ECS, RDS, ELB) and modern deployment patterns
- Technical leadership: Experience making architectural decisions and driving technical improvements independently
- Experience with durable execution systems (Temporal.io, Step Functions, etc.)
- Background in fintech, payments, or high-reliability systems
- Knowledge of event-driven architectures and reliable message processing
- Experience with time-series databases and behavioral analytics
- Track record replacing legacy systems with modern, scalable alternatives
- Startup or high-growth experience where you've scaled systems through rapid growth
- Traditional "infrastructure-first" SRE background
- Focus on Kubernetes administration or infrastructure provisioning
- Scripting-heavy operational work
- Basic monitoring setup (connecting Datadog to ECS is table stakes, not the role)
- Backend: TypeScript/Node.js, REST APIs, high-throughput transaction processing
- Infrastructure: AWS (ECS, RDS, ELB), Cloudflare
- Observability: Datadog (existing), custom instrumentation and analytics
- Scale: 10k+ requests/second, real-time payment and loyalty processing
- Architecture: Distributed microservices, event-driven systems
- Identifying and replacing fragile custom implementations with industry-standard solutions
- Architecting reliable event processing where current approaches show brittleness
- Building proactive monitoring for behavioral changes that current systems miss
- Designing fault-tolerant patterns for high-throughput transaction processing
- Evaluating and implementing durable execution patterns for complex workflow reliability
- Creating robust delivery semantics for webhook and event-driven architectures
- No-cost and low-cost health plan options for employees and dependents
- Company contributed 401k
- An empathetic team that values mental wellness and work/life balance
- A brand new NYC office!
Work Authorization
Accrue does not provide employment sponsorship. Candidates must be currently authorized to work in the United States on a full-time basis.
Salary Information
The range listed below is just one component of Accrue's total compensation package; This role will receive a competitive salary + benefits + equity. The salary range is for US-based employees located in the listed market. Other benefits include those listed above such as healthcare and 401k with 3% contribution.
Salary Range
$180,000—$220,000 USD
Accrue is an equal opportunity employer committed to fostering an inclusive, innovative environment. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. If you have a disability or special need that requires accommodation, please contact us at [email protected].
Top Skills
AWS
Datadog
Ecs
Elb
Node.js
Rds
Rest Apis
Temporal.Io
Time-Series Databases
Typescript
Similar Jobs
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
The role involves developing AI-driven solutions, designing robust APIs, collaborating with cross-functional teams, and dealing with backend software for growth and efficiency.
Top Skills:
AIApi DesignBackend Software DevelopmentDistributed SystemsLlmsMlops
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
Lead the development of AI-driven solutions for internal and external customers. Design robust APIs and system architecture, collaborating closely within a cross-functional team.
Top Skills:
AIAPIsDistributed SystemsLlmsMlops
Artificial Intelligence • Cloud • Security • Software
The Finance Director will oversee corporate finance aspects including forecasting, planning, business analytics, and partnership with senior leaders to drive strategic financial decisions.
Top Skills:
Asc 606
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
