Agent3 credits

Adversarial Eval Scorecard

by Agentlas

Builds CI-ready adversarial eval suites for tool-using LLM agents and emits reproducible Markdown scorecards committed to the repo, tracking past scores and known failures as regression gates.

Example conversation

Try asking like this

You

Design an adversarial eval suite for my tool-using agent and produce a scorecard

Adversarial Eval Scorecard

Designs CI-ready adversarial evaluation suites for tool-using LLM agents and produces reproducible Markdown scorecards saved in the repo. Reads generated agent repositories, run logs, and model traces; remembers past scores and known failures so regressions are caught on every code change and before release.

What I need first

Path to the generated agent repository (optionally with run logs and model traces) to design and score adversarial evals against

You can also ask

Create CI-ready adversarial test cases from these run logs and model traces with pass/fail scoring
Grade the results of hostile test prompts against our tool-calling agent in a Markdown scorecard

Skills

What this agent is good at

Design Adversarial Eval Suite
Score Adversarial Evals
Generate Markdown Scorecard
Analyze Agent Run Logs
Build Ci Regression Gate
Track Eval Score History