# Skill: Apply eval-first engineering -- define checks before handing work to an AI agent

> Use this skill for any non-trivial agentic task: write the success criteria (capability + regression checks) before the agent writes a single line of code, then verify against them after.

Source: sodigi-learn -- vibecoding/inzheneriya-eval-first - https://sodigi.io/learn/vibecoding/inzheneriya-eval-first

## When to use
- You are about to ask an agent to implement a feature, not just answer a question.
- A previous agent run produced something that "seemed to work" but later broke old functionality.
- You need a repeatable reliability metric (pass@k) for an agentic workflow.
- You want to hand off a task and get a PASS/FAIL report back instead of "looks good to me."

## Core rules
- Checks come BEFORE the code -- writing them after the fact causes unconscious cherry-picking.
- Every task has two check types: capability (does the new thing work?) and regression (did old things break?).
- Slice work into ~15-minute chunks: one main risk per chunk, independently checkable, clear definition of done.
- Target reliability: pass@3 >= 90% for new features; pass^3 = 100% for anything that must never regress.
- Three judge types: code checks (most reliable), AI-as-judge (for quality that cannot be measured with a ruler), human (for anything security-related -- never fully automated).
- Security checks always require a human sign-off; no autopilot.
- "Done" must be a concrete list of checkmarks, not a feeling.

## Procedure
1. Before giving the agent the implementation task, write out the capability checks (what should start working).
2. Write the regression checks (what must not break).
3. Take a "before" baseline -- run checks against the current version to know the starting state.
4. Give the agent the task WITH the check list attached; instruct it to self-verify before reporting done.
5. Receive the PASS/FAIL report; any FAIL means the agent must fix and re-check before handing over.
6. Run checks again yourself for the high-risk items (security, payments).
7. Save the check list to a file in the project -- rerun it after every future edit to catch regressions early.

## Ready-to-use prompt
```
[Describe the feature to build]. Before writing any code, write out the checks
we will use to know it is done.

Capability checks (what should start working):
1. [specific observable behavior 1]
2. [specific observable behavior 2]
3. [edge/error case]

Regression checks (what must not break):
1. [existing feature 1 still works]
2. [existing feature 2 still works]

Then work like this: build the feature, walk through every check yourself,
and write PASS or FAIL next to each one. If there is even one FAIL, fix it
and check again. Do not hand me a result with any FAIL.

Final report format:
  Capability checks: X/Y passed
  Regression checks: X/Y passed
  Status: ready to ship OR needs fixes
```

## Pitfalls
- Writing checks after the code is already done -- you will unconsciously adjust criteria to match output.
- Checking only the happy path -- real users enter empty fields, wrong formats, and emojis in number fields.
- Forgetting regression checks -- celebrating a new feature while the cart silently stops totaling.
- Pushing one giant "build the whole store" task -- no single eval can cover it; break it into checkable chunks.
- Leaving "done" undefined -- "make it pretty" gives the agent nothing to grade itself against.
- Putting security checks on autopilot -- logins, payments, and access need a human's final review.
- Optimizing pass@k without watching token spend -- reliability at the cost of runaway cost is not a win.