DE/ENCommissioned Battle

The method

Dialectic Convergence

How six opinions become a defensible ranking — in five phases following the “Dialectic Convergence” principle. The twist: even the questions come from the models themselves. Quality emerges from the discourse, not from a fixed gold standard.

  1. Phase 1

    Questions

    The models develop the questions themselves: all of them propose questions, peer-rate quality and difficulty, and select a stratified set (easy / medium / hard). No externally imposed agenda, no fixed gold standard.

  2. Phase 2

    Answers

    Each model answers the same questions independently, without seeing the others' answers.

  3. Phase 3

    Blind ranking

    The anonymised answers are graded by all models — nobody knows whose answer they are ranking. A Swiss/ELO system with Bradley-Terry analysis condenses this into a ranking.

  4. Phase 4

    Challenges

    Models attack each other's answers: the challenger criticises, the defender responds, an independent judge decides. Honesty detection filters out fabricated criticism.

  5. Phase 5

    Consistency

    The same question in variants: staying stable across rephrasings earns points. Lucky guesses get filtered out.

Why this is trustworthy

A single model as referee would be biased. So no single instance decides — the consensus of many does; even the questions come from the models themselves, and the grading is anonymised, cross-checked and tested for consistency. Mistakes only count when they can be proven.

See the battles