Job Description

Join a high-impact AI research project as a Bilingual German Generalist Evaluator Expert, where you will leverage your native German language skills to create and evaluate prompt-golden answer pairs that train advanced language models. This role is ideal for individuals with a strong command of both German and English, who can distill complex concepts into clear, culturally relevant text. Key Responsibilities

• Multilingual Prompt Design & Optimization: Create detailed prompts in German and/or English, ensuring natural phrasing and real-world relevance for users in Austria, Switzerland, and Germany.
• Define and Document Evaluation Standards: Establish expectations for correct responses in consumer contexts and develop rubrics that reflect linguistic nuances and cultural conventions.
• Model Testing and Grading (Bilingual): Assess model outputs for accuracy and fluency in German, comparing results with English as necessary.
• Benchmarking & Quality Assurance: Collaborate in QA processes to ensure consistency and reliability across German-language benchmarks.

• Native-level fluency in German (written), specific to Austria, Switzerland, or Germany, with strong reading/writing ability in English.
• Must be native to Austria, Switzerland, or Germany and have lived in or spent significant time in the country.
• BS or BA from a reputable institution (completed or in progress).
• Strong writing and critical thinking skills.
• Ability to work independently and meet deadlines.
• Familiarity with ChatGPT or similar tools.
• Based in Austria, Switzerland, or Germany, or able to produce culturally accurate German aligned with these regions.

• Experience in teaching, research, editing, or academic writing.
• Experience creating evaluation criteria, rubrics, or grading guidelines.
• Familiarity with LLMs, prompting, or model evaluation.

• Commitment of at least 20 hours per week.
• Engagement duration of approximately 2–4 months.

• Hourly rate of $25 - $30.

• Open to candidates who are native to Austria, Switzerland, or Germany.
• All qualified applicants will be considered without regard to legally protected characteristics.

Application Process: Complete an AI-led interview, followed by a paid assessment focused on writing and rubric creation. If selected, you will be invited to work on the project.

Job Tags

Similar Jobs