As a Search Relevance SME, you will evaluate data submissions, monitor quality, provide feedback to raters, and recommend training sessions based on error trends.
Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant, diverse, and scalable to supercharge their AI models. As a Welocalize brand, WeloData leverages over 25 years of experience in partnering with the world’s most innovative companies and brings together a curated global community of over 500,000 AI training and domain experts to offer services that span:
ANNOTATION & LABELLING: Transcription, summarization, image and video classification and labeling.
ENHANCING LLMs: Prompt engineering, SFT, RLHF, red teaming and adversarial model training, model output ranking.
DATA COLLECTION & GENERATION: From institutional languages to remote field audio collection.
RELEVANCE & INTENT: Culturally nuanced and aware, ranking, relevance, and evaluation to train models for search, ads, and LLM output.
Want to join our Welo Data team? We bring practical, applied AI expertise to projects. We have both strong academic experience and a deep working knowledge of state-of-the-art AI tools, frameworks, and best practices. Help us elevate our clients' Data at Welo Data.
Role Overview
We are seeking a Search Relevance Subject Matter Expert (SME) to join our team. In this role, you will act as a Subject Matter Expert for a high-profile streaming platform. Working closely with our raters, you will evaluate submissions in Label Studio, monitoring the data quality. By identifying recurring errors and offering actionable feedback, you will ensure our team consistently meets and exceeds client expectations.
Key Responsibilities
- Quality Assurance & Guideline Adherence
- Evaluate/correct data annotations for accuracy and compliance with the client’s rating guidelines.
- Collaborate with team members to clarify and update guidelines as needed.
Qualifications
- Previous QA experience: Prior exposure to any kind of QA is highly desirable.
- Strong Analytical & Communication Skills: Ability to synthesize error trends and present clear, actionable insights.
- Multicultural & Pop Culture Awareness: Deep understanding of multicultural perspectives and global pop culture trends is essential for enhancing search relevance.
- Strong English Proficiency: Good command of written and spoken English, as all communication, documentation, and task guidelines are in English.
- Availability for 10–20 hours per week: Flexible schedule with some overlap in working hours for communication with global teams.
- Self-Motivated & Detail-Oriented: Capable of working independently, meeting deadlines, and maintaining strict quality standards in a remote setting.
Top Skills
Label Studio
Similar Jobs
.png)
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
The Bilingual SMB Account Manager will nurture customer relationships, understand needs, cross-sell products, and collaborate with internal teams to enhance customer experiences.
Top Skills:
CRMSalesforce
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
As a Bilingual SMB Account Manager, you will manage relationships with customers, helping them use Square products effectively while achieving their business goals and driving revenue through upselling and cross-selling.
Top Skills:
CRMSalesforce
Cloud • Security • Software • Cybersecurity • Automation
Develop and improve the GitLab product with focus on AI capabilities and integration of technologies while collaborating in a remote team environment.
Top Skills:
Ai TechnologiesGraphQLPythonRspecRuby On Rails
What you need to know about the Los Angeles Tech Scene
Los Angeles is a global leader in entertainment, so it’s no surprise that many of the biggest players in streaming, digital media and game development call the city home. But the city boasts plenty of non-entertainment innovation as well, with tech companies spanning verticals like AI, fintech, e-commerce and biotech. With major universities like Caltech, UCLA, USC and the nearby UC Irvine, the city has a steady supply of top-flight tech and engineering talent — not counting the graduates flocking to Los Angeles from across the world to enjoy its beaches, culture and year-round temperate climate.
Key Facts About Los Angeles Tech
- Number of Tech Workers: 375,800; 5.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Snap, Netflix, SpaceX, Disney, Google
- Key Industries: Artificial intelligence, adtech, media, software, game development
- Funding Landscape: $11.6 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Strong Ventures, Fifth Wall, Upfront Ventures, Mucker Capital, Kittyhawk Ventures
- Research Centers and Universities: California Institute of Technology, UCLA, University of Southern California, UC Irvine, Pepperdine, California Institute for Immunology and Immunotherapy, Center for Quantum Science and Engineering
