What you'll do
- Design evals (unit/integration) for task families
- Create failure taxonomies; analyze false+/false‑
- Draft guardrails; collaborate on PRs to filters/tests
What we're looking for
- Python + eval frameworks (custom/Ragas/etc.)
- Red‑team exposure a plus
- Clear writing and taxonomy design
Where this role fits
This role sits in the AI Safety & Eval TalentZone and commonly supports packs like Cost & Quality Sprint and Guardrail & Reg Audit. Work is routed under QA; this is a managed bench, not a marketplace.