LLM AI Quality Analyst (Personalization) Chinese

Hiring: LLM – AI Quality Analyst (Personalization) – Chinese

Role Details

Role: LLM – AI Quality Analyst (Personalization) – Chinese

Work Mode: Remote

Location: Must be located in Malaysia, Thailand, Singapore, or the USA

Experience: 1+ year

We're looking for Chinese-speaking AI Quality Analysts to join a global team evaluating a new personalization feature for Gemini.

This is an exciting opportunity for candidates with experience in AI evaluation, data annotation, content moderation, prompt evaluation, or analytical quality roles who want to work on advanced AI technology.

What You'll Do

You will evaluate how effectively an AI model uses personal context from conversations and connected data sources to provide more relevant and helpful responses.

Your work will include:

• Designing and executing multi-turn prompts based on personal experiences and context.
• Evaluating AI responses for Grounding, Integration and Helpfulness.
• Performing Side-by-Side (SxS) comparisons and ranking model responses.
• Identifying incorrect personalization, hallucinations, poor inferences and forced connections.
• Reviewing responses for subtle differences in naturalness, relevance and quality.
• Writing clear, structured rationales explaining your evaluations.
• Verifying relevant Debug Info and data-source usage.
• Providing detailed feedback and annotations to improve AI quality.
• Maintaining strict data hygiene, including deleting evaluation conversations when required.

What We're Looking For

• Strong Chinese reading and writing proficiency is mandatory.
• 1+ year of relevant experience in AI evaluation, data annotation, content moderation, LLM evaluation, quality analysis or a related analytical role is preferred.
• Excellent analytical and critical-thinking skills.
• Strong attention to detail and ability to evaluate nuanced AI responses.
• Ability to design creative, multi-turn prompts.
• Strong written communication and ability to provide evidence-based evaluation rationales.
• Comfortable working independently in a remote environment.
• Laptop/desktop with a reliable internet connection.

Important Project Requirements

Please review these carefully before applying:

Location: Candidates must be based in Malaysia, Thailand, Singapore or USA, subject to country-specific eligibility guidelines.

Chinese: Strong Chinese reading and writing proficiency is mandatory.

Personal Google Account: Candidates must be willing to use their primary personal Google account, not a testing account, and enable relevant personal data sources required for genuine evaluation.

Availability:

• 30 or 40 hours/week
• Full-time availability in your local timezone
• Minimum 4 hours of PST overlap
• Approximately 8 hours/day for the full-time schedule
• Must be available to start immediately for a 4-month contract.

Preferred Background

A BS/BA degree or equivalent experience in areas such as:

Linguistics | Journalism | Computer Science | Policy | Law | Ethics | Communications | Other analytical fields

Experience with LLM evaluation, RLHF, prompt engineering, AI quality, personalization, grounding or hallucination detection is a strong advantage.