AI QA Specialist

About Us:

Notion helps you build beautiful tools for your life’s work. In today's world of endless apps and tabs, Notion provides one place for teams to get everything done, seamlessly connecting docs, notes, projects, calendar, and email—with AI built in to find answers and automate work. Millions of users, from individuals to large organizations like Toyota, Figma, and OpenAI, love Notion for its flexibility and choose it because it helps them save time and money.

In-person collaboration is essential to Notion's culture. We require all team members to work from our offices on Mondays and Thursdays, our designated Anchor Days. Certain teams or positions may require additional in-office workdays.

About the Role:

We are seeking an AI Support QA Specialist to raise the quality bar for Notion’s AI-powered support. You’ll run systematic QA on our AI chat and email experiences and accelerate validation for our AutoQA system so quality metrics remain accurate before and after launch. You’ll turn insights into action by updating conversational design and our knowledge base, and by partnering closely with User Operations and our QA Ambassadors.

• Perform routine, structured QA reviews of AI support interactions across chat and email for qualitative and quantitative accuracy

• Validate and continuously improve AutoQA scoring and metrics prior to deployment and in ongoing operation

• Translate QA findings into recommendations and updates to conversational design and the CX knowledge base

• Partner with User Operations and CX leadership to ensure QA efforts align with strategic initiatives

• Monitor and report on trends, gaps, and opportunities for automation or targeted reviews

This is a 6 month remote-friendly contractor role with potential to extend, requiring [30-40 hours/week].

What You'll Do:

• Weekly QA reviews and reporting on AI chat and email interactions

• AutoQA validation reports before and after feature launches

• Clear documentation of QA findings and recommendations for conversational and knowledge base improvements

• Support for an internal QA Ambassador program including training and documentation

• Proactive identification of process gaps and opportunities for automation or targeted review

Skills You'll Need to Bring:

• You have experience in Customer Experience as a Product specialist or QA Specialist, analyzing support responses (both agent and AI) at scale

• You excel at analyzing support interactions with a critical eye, identifying improvements while balancing empathy, clarity, and solution-focused communication

• You have strong written communication and the ability to turn insights into clear actions

• You are organized, effectively manage your time and can work independently

• You’re a collaborative partner who can work across CX, Ops, and vendor teams

Nice to Haves:

• Deep knowledge of Notion's product and feature set

• Experience with MaestroQA or other QA platforms with LLM-powered systems

• Familiarity with knowledge base management and conversational design best practices

• Experience with AI prompt engineering and understanding prompt design principles to optimize AI-generated responses.

We hire talented and passionate people from a variety of backgrounds because we want our global employee base to represent the wide diversity of our customers. If you’re excited about a role but your past experience doesn’t align perfectly with every bullet point listed in the job description, we still encourage you to apply. If you’re a builder at heart, share our company values, and enthusiastic about making software toolmaking ubiquitous, we want to hear from you.

Notion is proud to be an equal opportunity employer. We do not discriminate in hiring or any employment decision based on race, color, religion, national origin, age, sex (including pregnancy, childbirth, or related medical conditions), marital status, ancestry, physical or mental disability, genetic information, veteran status, gender identity or expression, sexual orientation, or other applicable legally protected characteristic. Notion considers qualified applicants with criminal histories, consistent with applicable federal, state and local law. Notion is also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, please let your recruiter know. By clicking “Submit Application”, I understand and agree that Notion and its affiliates and subsidiaries will collect and process my information in accordance with Notion’s Global Recruiting Privacy Policy.

#LI-Onsite