AI QA Specialist

We're on a mission to make it possible for every person, team, and company to be able to tailor their software to solve any problem and take on any challenge. Computers may be our most powerful tools, but most of us can't build or modify the software we use on them every day. At Notion, we want to change this with focus, design, and craft.

We've been working on this together since 2016, and have customers like OpenAI, Toyota, Figma, Ramp, and thousands more on this journey with us. Today, we're growing fast and excited for new teammates to join us who are the best at what they do. We're passionate about building a company as diverse and creative as the millions of people Notion reaches world

We are seeking an AI Support QA Specialist to raise the quality bar for Notion’s AI-powered support. You’ll run systematic QA on our AI chat and email experiences and accelerate validation for our AutoQA system so quality metrics remain accurate before and after launch. You’ll turn insights into action by updating conversational design and our knowledge base, and by partnering closely with User Operations and our QA Ambassadors.

• Perform routine, structured QA reviews of AI support interactions across chat and email for qualitative and quantitative accuracy

• Validate and continuously improve AutoQA scoring and metrics prior to deployment and in ongoing operation

• Translate QA findings into recommendations and updates to conversational design and the CX knowledge base

• Partner with User Operations and CX leadership to ensure QA efforts align with strategic initiatives

• Monitor and report on trends, gaps, and opportunities for automation or targeted reviews

This is a 6 month remote-friendly contractor role with potential to extend, requiring [30-40 hours/week].

• Weekly QA reviews and reporting on AI chat and email interactions

• AutoQA validation reports before and after feature launches

• Clear documentation of QA findings and recommendations for conversational and knowledge base improvements

• Support for an internal QA Ambassador program including training and documentation

• Proactive identification of process gaps and opportunities for automation or targeted review

• You have experience in Customer Experience as a Product specialist or QA Specialist, analyzing support responses (both agent and AI) at scale

• You excel at analyzing support interactions with a critical eye, identifying improvements while balancing empathy, clarity, and solution-focused communication

• You have strong written communication and the ability to turn insights into clear actions

• You are organized, effectively manage your time and can work independently

• You’re a collaborative partner who can work across CX, Ops, and vendor teams
  

• Deep knowledge of Notion's product and feature set

• Experience with MaestroQA or other QA platforms with LLM-powered systems

• Familiarity with knowledge base management and conversational design best practices

• Experience with AI prompt engineering and understanding prompt design principles to optimize AI-generated responses.

We hire talented and passionate people from a variety of backgrounds because we want our global employee base to represent the wide diversity of our customers. If you’re excited about a role but your past experience doesn’t align perfectly with every bullet point listed in the job description, we still encourage you to apply. If you’re a builder at heart, share our company values, and enthusiastic about making software toolmaking ubiquitous, we want to hear from you.

Notion is proud to be an equal opportunity employer. We do not discriminate in hiring or any employment decision based on race, color, religion, national origin, age, sex (including pregnancy, childbirth, or related medical conditions), marital status, ancestry, physical or mental disability, genetic information, veteran status, gender identity or expression, sexual orientation, or other applicable legally protected characteristic. Notion considers qualified applicants with criminal histories, consistent with applicable federal, state and local law. Notion is also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, please let your recruiter know.
By clicking “Submit Application”, I understand and agree that Notion and its affiliates and subsidiaries will collect and process my information in accordance with Notion’s Global Recruiting Privacy Policy.

#LI-Onsite