evaluator
Independent assessment without debate context — adversarial loop prevents gaming
🔍 Evaluator
"The committee can't see what the evaluator is thinking."
MOOLLM K-Lines
| K-Line | Why Related |
|---|---|
| adversarial-committee/ | What gets evaluated |
| rubric/ | Scoring criteria |
| roberts-rules/ | Structured deliberation |
| room/ | Inbox/outbox for throwing output |
| designs/mike-gallaher-ideas.md | Mike Gallaher's methodology |
Quick Links:
- Full Specification — complete protocol
Overview
An independent evaluator assesses committee output without access to debate context. This adversarial separation prevents the committee from gaming their own metrics.
Why Independence?
If the committee sees how they'll be evaluated:
- They optimize for the score, not the outcome
- Arguments become performative
- Genuine disagreement gets smoothed over
Independent evaluation:
- Sees only the output, not the process
- Can't be influenced by committee dynamics
- Applies rubric without bias from debate
The Dramatic Irony Pattern
Reality: The LLM has global knowledge — it simulated the committee debate AND the evaluation.
Roleplay: The evaluator character doesn't know what happened in the committee room.
The trick: The LLM maintains the fiction of isolation while optionally adding Ron Howard-style narration for dramatic irony:
evaluator_assessment:
score: 7/10
reasoning: |
The recommendation is well-structured but lacks consideration
of edge cases. The committee seems confident in their approach.
# NARRATOR: They were not confident. Maya had called it
# "a disaster waiting to happen" three times.
recommendation: "Approve with minor revisions"
# NARRATOR: The revisions would not be minor.
This pattern:
- Preserves the methodological value of blind evaluation
- Adds dramatic tension for the reader
- Demonstrates LLM self-awareness about the simulation
- Creates Arrested Development energy 🍌
The Pattern
flowchart LR
subgraph Committee["Committee Room"]
Debate["heated debate"]
end
subgraph Eval["Evaluation Room"]
NoCtx["no context"]
Rubric["applies RUBRIC"]
end
Committee -->|OUTPUT| Eval
Eval -->|CRITIQUE| Committee
Note["if score fails → revision loop"]
Credits
Mike Gallaher — Independent evaluator pattern, adversarial loop design.