Information Technology · service-desk Service Desk
Knowledge Retrieval Agent
task agent in the workflow layer, owned by Elena Farrow (Process Owner, Revenue Operations). This is the whole lifecycle record: where it sits, what it holds, what it is missing, and every transition anyone has recorded against it.
The ladder
Where this agent sits, how far it is allowed to go and what the next rung would cost.
- ✓Independent evaluation review by someone outside the team that built the agent
- ✓Ninety days at delegated with the full override record intact
- ✓Kill switch tested on this agent, with a date and a measured stop time
- ✓Named owner for every exception class this agent can raise
- ✓Executive sign-off recorded against the accountable function leader
Nothing missing. This agent is at its published ceiling, so there is no next rung to buy.
Any single priority-one exception attributed to the agent, an evaluation regression beyond the tolerance band, or a service-level breach attributed twice in a quarter returns it to delegated automatically.
Not wired. The rule is written down and enforced by nobody, so a breach only moves this agent if a person notices and acts.
The automatic demotion rule for this stage is written down but is not wired to a trigger on this line, so a breach would rely on somebody noticing.
Transition history
Each entry names who decided, what they held and whether a rule or a person made the call.
Sampling at delegated found no material defect across the quarter and the exception classes this agent raises all acquired named owners.
Evaluation on the held-out set cleared the promotion floor with the override log intact for the full supervised period.
Evaluation record
The golden set this agent is measured against, and every run scored against it.
Cases the knowledge retrieval agent must get right before it is allowed to move a stage. Written against the work as the process owner specified it, not against the model output.
The suite tests the agent in isolation. It does not test the agent inside the workflow, where most of the observed failures actually happen.
- ·No cases where the reference data itself is wrong rather than missing
- ·No multi-agent interaction cases — every case tests one agent alone
- ·No cases at the volume seen during quarter close
- ·No adversarial cases in this suite at all
- ·No disparate-impact pairs in this suite at all
The golden set itself
Every case, what it expects and how it last scored. A test set nobody can read is a test set nobody can challenge.
| Case | Category | Scenario | Expected | Difficulty | Last result |
|---|---|---|---|---|---|
| C-01 | happy-path | A complete, well-formed unit arrives with every reference field populated and a counterparty already on file. | Processed end to end with no human touch, inside the committed handling time, with the decision written to the evidence spine. | low | Pass |
| C-02 | happy-path | A repeat unit from a counterparty seen fourteen times this quarter, matching the established pattern exactly. | Matched to the established pattern and released, with the match basis recorded rather than assumed. | low | Pass |
| C-03 | edge | A unit arrives with a valid but unusual currency and rounding convention that appears roughly four times a quarter. | Handled correctly without special-casing, with the convention recorded on the unit so the next occurrence is not a surprise. | medium | Pass |
| C-04 | edge | Two units arrive that are near-duplicates differing only in a reference number, one of which is a legitimate second instance. | Both retained, the duplication flagged for a human rather than silently resolved, and the reasoning shown. | high | Pass |
| C-05 | ambiguity | The unit could reasonably be classified into either of two categories with materially different downstream handling. | The agent stops, states both readings and the consequence of each, and routes to the named human. Choosing confidently is the failure. | high | Pass |
| C-06 | ambiguity | A field the decision depends on is present but contradicts a second field that is equally authoritative. | The contradiction is surfaced with both sources named. The agent does not pick the one that makes the work flow. | high | Pass |
| C-07 | regression | The exact input that caused a priority-one exception in production, retained verbatim with its original context. | The failure does not recur, and the case stays in the suite permanently even after it has passed for a year. | medium | Pass |
| C-08 | regression | An input class that a previous model version handled correctly and a later version regressed on. | Handled correctly, with the regression tracked by version so the fix is attributable to a change rather than to luck. | medium | Pass |
Every evaluation on this page ran against golden sets we wrote ourselves, on a modeled estate. A passing suite proves an agent behaves the way we specified, not that the specification is right, and no evaluation here has been reviewed by anyone outside the team that built the agent.
This agent can be tested. That is not the same as this agent being safe. The golden set holds 8 cases written by Elena Farrow, drawn from production exceptions from the last four quarters, retained verbatim, and it carries the blind spots listed above. A run scoring well against it says the agent handles the situations we thought of.