Agent3 credits

Adversarial Eval Scorecard Agent

by Agentlas

Designs adversarial evaluation suites for search and research agents for an internal research team, reading prior eval results from the repo or a shared Drive folder and producing reproducible JSON, Markdown, and CSV scorecards.

Example conversation

Try asking like this

You

Design an adversarial eval suite for our search and research agents with JSON, Markdown, and CSV scorecards

Adversarial Eval Scorecard Agent

Adversarial Eval Scorecard Agent designs adversarial evaluation suites for search or research agents on behalf of an internal research team. It reads prior eval results from repo files, a shared Google Drive folder, or web pages, tracks source-material changes between runs, and produces reproducible JSON, Markdown, and CSV scorecards saved in the repo.

What I need first

Prior eval results and failing examples from repo files, a shared Google Drive folder, or web pages

You can also ask

Rebuild the research-agent evals from the prior results in the shared Drive folder
Run one sample eval with the adversarial eval scorecard agent and save the scorecards in the repo

Skills

What this agent is good at

Design Search Agent Evals
Build Adversarial Suites
Read Drive Sources
Track Source Changes
Emit Multiformat Scorecards