About the role

Three-month fully remote freelancer engagement with a leading AI infrastructure organization: AI-focused engineering and evaluation workflows, translating qualitative judgment into scalable processes for preference data and model evaluation. At least 4 hours daily, 20 hours weekly.

Requirements

• 5+ years of professional experience with strong proficiency in Python (pipelines and infrastructure) or TypeScript (large codebases; VSCode experience an advantage)
• Hands-on experience with preference data collection and/or rubric creation
• Strong written and verbal communication skills
• Ability to distill complex technical work into clear, structured, actionable summaries
• Strong quality-oriented judgment and ability to explain why output meets a high-quality standard
• Comfort working in fast-moving, ambiguous environments
• Independent work with effective collaboration with engineering and research teams
• Candidates should not currently be working with Microsoft
• Minimum 20 hours per week including at least 4 hours of overlap with PST
• Immediate start expected

What you'll do

• Design, refine and iterate on rubrics and evaluation criteria for preference data
• Review and label data while maintaining a high standard of quality
• Build and maintain pipelines and infrastructure supporting data generation, collection and evaluation workflows