RealCompanion
RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations
A benchmark evaluating AI companions on understanding long-term relationships using 10 generated personas with ~27k messages and derived profiles/personas/ground truths.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info realcompanion
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Antigravity Gemini 3.8 Flash | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Codex GPT-5.6-sol | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Claude Opus 5.5 | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Content & Construction: The dataset consists of ten synthetic longitudinal conversations between users and AI companions, totaling over 27,000 messages across up to 120 days. It includes four derived files per persona: a conversation log, a profile (stated facts), a persona (inferred traits/feelings), chat ground truth with reasoning traces, and a question set.
Scoring & Key Findings: The benchmark measures three tracks: reconstruction of the user's state/persona, quality of conversational replies, and answers to written questions. It finds that recent messages suffice for 95.9% of probes, memory detectors fail on real data, and agent systems show a massive cost disparity in reconstructing personas.