The QA Engineer will lead testing for the AI Forms Platform, focusing on validating AI auto-mapping models, automating tests, and ensuring quality during legacy system migration.
Filevine is forging the future of legal work with cloud-based workflow tools. We have a reputation for intuitive, streamlined technology that helps professionals manage their organization and serve their clients better. We’re also known for our team of extraordinary and passionate professionals who love working together to help organizations thrive. Our success has catapulted Filevine to the forefront of our field—we are ranked as one of the most innovative and fastest-growing technology companies in the country by both Deloitte and Inc.
Our Mission
Filevine is building the seamless intersection between legal and business by creating a world- class platform to help professionals scale.
Job Summary
We are looking for a forward-thinking QA Engineer to lead the testing strategy for our new AI-first Forms Platform. In this role, you will move beyond traditional functional testing to build "evaluation harnesses" that measure the accuracy of our auto-mapping AI models against golden datasets. You will be responsible for minimizing defect escape rates and ensuring that complex government forms (for Immigration, Family Law, etc.) are rendered perfectly. You will use AI tools daily to generate test cases, write automation scripts, and analyze regression data to ensure our migration from legacy systems is seamless.
Responsibilities
- AI Model Validation: Design and maintain evaluation harnesses (evals) to measure the accuracy of AI auto-mapping models, ensuring they meet strict quality thresholds before release.
- Test Automation: Build robust, automated test suites (end-to-end and integration) that run within our CI/CD pipelines to catch regressions immediately.
- Golden Datasets: Maintain and expand "golden datasets" of perfectly mapped forms to serve as the ground truth for regression testing.
- Defect Prevention: Collaborate with developers to identify "model quality variance" and implement gates that prevent low-confidence predictions from reaching production.
- Migration Quality: Verify data integrity during the migration of pilot customers from the legacy platform to the new architecture.
- Efficiency: Leverage LLMs and AI coding assistants to rapidly write test scripts and generate edge-case scenarios for complex legal forms.
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or related field.
- 3+ years of experience in QA Automation, with proficiency in tools like Cypress, Selenium, or Playwright.
- Coding Proficiency: Strong ability to write test scripts in JavaScript, TypeScript, or Python (Java/C++ proficiency also accepted).
- AI/Data Testing: Experience testing non-deterministic systems (AI/ML models) or complex data migration projects is highly preferred.
- Agile/Scrum: 1 to 3 years experience working in agile environments with frequent release cycles.
- Problem Solving: Demonstrated ability to use AI tools to solve coding roadblocks and optimize testing workflows.
- Communication: Excellent verbal and written communication skills to articulate quality risks to the product team.
Filevine is an Equal Opportunity Employer. Qualifications for employment, promotion and other terms and conditions of employment are based upon the ability to perform the job. Equal-employment opportunities are provided to all applicants and employees without regard to race, creed, religion, color, age, national origin, sex, disability, veteran status, or other legally protected class. Filevine is committed to providing reasonable accommodations for qualified individuals with disabilities. If you need assistance or accommodation due to disability, or if you have concerns related to Filevine's equal employment opportunities, you may contact us at [email protected]
Cool Company Benefits:
- A dynamic, rapidly growing company, focused on helping organizations thrive
- Medical, Dental, & Vision Insurance (for full-time employees)
- Competitive & Fair Pay
- Maternity & paternity leave (for full-time employees)
- Short & long-term disability
- Opportunity to learn from a dedicated leadership team
- Centrally located open office building in Sugar House (onsite employees)
- Top-of-the-line company swag
Privacy Policy Notice
Filevine will handle your personal information according to what’s outlined in our Privacy Policy.
Communication about this opportunity, or any open role at Filevine, will only come from representatives with email addresses using "filevine.com". Other addresses reaching out are not affiliated with Filevine and should not be responded to.
Top Skills
Cypress
JavaScript
Playwright
Python
Selenium
Typescript
Similar Jobs
Computer Vision • Healthtech • Information Technology • Logistics • Machine Learning • Software • Manufacturing
Lead the go-to-market strategy for Dandy's hardware products, focusing on customer needs, positioning, and cross-functional launch execution.
Top Skills:
HardwareProduct MarketingSoftware
Artificial Intelligence • eCommerce • Information Technology • Internet of Things • Automation
The Technical Architect leads the design of technical services at CDW, ensuring alignment between technology and business goals while overseeing project lifecycles and vendor relationships.
Top Skills:
Artificial IntelligenceGoogle Apps ManagerGoogle WorkspacePython
Artificial Intelligence • eCommerce • Information Technology • Internet of Things • Automation
As a Senior Software Engineer, you will manage and enhance digital tools, streamline operations, support the service team, and ensure compliance with security policies.
Top Skills:
Google Cloud PlatformJavaScriptJIRAMondayPythonZapier
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
