Operate and stress-test LLM-based AI tools, reproduce and document edge cases, file and track tickets, validate fixes, provide prioritized product feedback, triage failures with engineers, design lightweight tests, and support the Head of Engineering with task and calendar management to scale experimentation and improve production reliability.
About the Client
The hiring manager is the Head of Engineering, who partners closely with product team and iterate quickly on emerging LLM workflows. The culture values autonomy, candor, and high standards in a startup-paced environment.
The hiring manager is the Head of Engineering, who partners closely with product team and iterate quickly on emerging LLM workflows. The culture values autonomy, candor, and high standards in a startup-paced environment.
Why does this role exist?
This role exists to serve as a highly autonomous right hand to the Head of Engineering, operating and stress-testing experimental AI tools while closing the feedback loop between users and the engineering/product teams. By proactively finding breakages, documenting issues, and prioritizing work across a 30+ person team, this role accelerates engineering velocity and product quality. It ensures the organization can scale experimentation, reduce failures in production, and maintain tight operational rhythms without relying on rigid SOPs.
The Impact you’ll make
AI Tool Operations & Testing
- Operate, stress-test, and quality-check cutting-edge AI tools and LLM-based systems that may frequently break
- Reproduce edge cases, capture logs/screens, and write clear repro steps; file/track tickets in Shortcut (Clubhouse)
- Validate fixes and run smoke/regression tests before and after deployments
Product Feedback & Collaboration
- Provide structured, actionable feedback to the Head of Engineering and product/engineering teams
- Collect and synthesize feedback from other assistants/users; turn insights into prioritized improvements
- Critique internal software and propose data-backed enhancements to workflows and UX
Technical Troubleshooting & QA
- Triage broken workflows, diagnose likely failure points, and collaborate with engineers on root causes
- Consolidate deployment updates and confirm changes behave as expected in production
- Design lightweight test plans for ambiguous features; track outcomes and follow through to resolution
Operational Support to the Head of Engineering
- Manage and organize the Head of Engineering’s to-do list; drive proactive follow-through across a 30+ person team
- Maintain and optimize work queues; ensure the right priorities are surfaced at the right time
- Handle calendar management via AI tools; coordinate with stakeholders as needed
Systems & Process Optimization
- Build simple, scalable processes in ambiguous environments; streamline task flows and handoffs
- Propose and implement improvements to reduce repeated failure modes and increase throughput
- Leverage AI productivity tools to automate routine steps and create operational leverage
Skills, Knowledge and Expertise
Required:
- Demonstrated technical mindset with strong AI tool fluency (e.g., ChatGPT, LLM workflows) or a programming background
- Proven experience in product operations, QA/Testing, technical support, or similar role working closely with engineers
- Exceptional written communication for clear, structured bug reports and product feedback
- Comfortable operating independently in ambiguous, fast-paced environments with minimal SOPs
- Availability to work 40 hours/week, Monday–Friday, 9:00 am–5:00 pm Eastern Time (may shift slightly earlier after 3–4 weeks)
WFH Set-Up:
- Computer with at least 8GB RAM, an Intel i5 core processor/AMD Ryzen 5 Processor and up.
- Internet speed of at least 40MBPS
- Headset with an extended mic that has noise cancellation and a webcam
- Back-up computer and internet connection
- Quiet, dedicated workspace at home
Your Superpowers:
- Technical: AI tool proficiency; systems thinking; test design and execution; QA methodologies; debugging mindset; clear documentation; comfort with Shortcut (Clubhouse), GSuite, Slack, and experimental internal tools (e.g., Magic AI); familiarity with messaging tools (e.g., Messenger). Bonus: basic scripting/automation, product analytics literacy.
- Operational: Task prioritization; work queue management; lightweight process design; deployment update consolidation; calendar management via AI.
- Behavioral: Proactive ownership; “figure it out” mentality; high attention to detail; resilience when tools break; curiosity and growth mindset; comfort giving candid, constructive feedback; ability to work independently and collaboratively with engineers.
You should apply if...
- You love being an early power user of new AI tools, enjoy breaking things to make them better, and can translate ambiguous symptoms into actionable engineering feedback.
- You thrive in uncertainty, don’t need handholding, and naturally create order and leverage through systems and process improvements.
- You communicate crisply in writing, challenge assumptions respectfully, and partner well with engineers and product managers.
- You want to grow at the intersection of AI operations, product testing, and systems thinking. Bonus: You’re in a LATAM timezone for strong overlap with EST, and/or you have experience in programming, product, or data analytics.
What to expect...
Work Setup:
- Remote position
- Must have a reliable internet connection and a quiet workspace
- Required to provide own computer with Intel Core i5 or something similar or higher operating system
Working Hours:
- 40 hours per week
- M-F 9am to 12 noon, 1 pm to 4 pm US Timezone
Compensation:
- $7 per hour
- No benefits package included
Benefits
About
Magic has connected top remote talent with fast-growing businesses for over 10 years.Founded in San Francisco in 2015, we now have thousands of remote workers around the world. Magic is backed by Sequoia Capital and Y Combinator.
Similar Jobs
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
As a Player Development Executive, you will enhance VIP player relationships, drive sales metrics, analyze performance data, and ensure adherence to VIP policies.
Top Skills:
Crm SoftwareData Analysis
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
As a VIP Host, you will manage player relationships, execute VIP engagement strategies, lead reactivation efforts, and ensure a world-class customer experience.
Cloud • Fintech • Food • Information Technology • Software • Hospitality
The Staff Software Engineer will develop full-stack applications, lead projects, mentor teammates, and drive architectural discussions to enhance Toast's restaurant platform.
Top Skills:
AIDistributed SystemsDynamoDBGraphQLJavaKotlinMachine LearningMicroservicesReactRest
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
