Development

WasItTheModel

Tells whether a stretch of frustration was caused by the model or by our own harness and prompting, with the evidence that lines up in time.

03
Workflows
05
Tools
00
References
12
Triggers

Key Points

  • It works for any window back to the start of the transcript record: it cuts the principal's frustrated prompts into episodes, checks each against signals that overlap it in time (the status incidents of whichever provider served the model, provider-side API errors, X complaint volume that hour, harness changes in the 12 hours before, tool-error spikes), and returns sentences with the evidence, an evidence table, and a share link to a page with charts
  • Detects which agent harnesses (Claude Code, Codex) and model providers are in use
Not for filing a complaint (Errata), improving a target by experiment (HillClimb), live outage alerts (Vigil)

Inside the Skill

The thinking, frameworks, and architecture that distinguish this skill from a generic version of the same task.

Ideal State

  • Every answer reads like evidence, never a bare verdict: "your frustration yesterday from 20:12 to 20:44 (about 32 minutes) was most likely X; the evidence: …", with what was checked and found normal on the other side.
  • A side is blamed only on two independent signals that overlap the episode in time, and the headline counts model, harness, both and unclear, so a window where nothing lines up says so.
  • /witm <window> returns the headline, the sentences, the evidence table and a probed share link to the page.

Intent Routing

Intent Trigger File
Report "/witm 1d", "was it the model", "was it us or the model lately", "model or harness this week" Intents/Report.md
Day "was it the model on Tuesday", "what happened on October 7" Intents/Day.md
Refresh "refresh WasItTheModel", "backfill", "sample X counts" Intents/Refresh.md
File Read when
Tools/WasItTheModel.help.md the CLI's commands, flags and exit codes

Data & Config

The skill reads, never writes: the agent transcripts it finds, the git history of each harness config directory (and a LifeOS install's USER repo), the status feeds of the providers in use and the X counts API. It writes counts and timestamps only (never prompt or message text) to LIFEOS/MEMORY/STATE/wasitthemodel/: ledger.json (per-file counts per UTC hour, plus prompt, frustration and API-error timestamps), incidents.json (provider incident history, only ever grows), x-hours.json (hourly complaint counts per model family), and pages/ (rendered reports). X counts use the X skill's bearer credentials. Per-install settings live in LIFEOS/USER/CUSTOMIZATIONS/SKILLS/WasItTheModel/config.json, every key optional: providers (add or override by id: label, model regex, products, Statuspage-compatible history/incidents/page URLs), repos (extra git repos with heavy/core path regexes), extraPatterns (more regexes for a detected repo by id), and shareCommand (a CLI that takes a file and prints a link; without it, the page path is given instead). The wasitthemodel-daily workflow (LIFEOS/USER/WORKFLOWS/) runs daily each morning so the X series keeps growing, and writes one Vigil event per day (medium when an episode was pinned, info otherwise). WITM_STATE_DIR, WITM_TRANSCRIPTS and WITM_CONFIG override the paths for tests.

Gotchas

  • Day-level coincidence is not evidence (2026-10-12). The first version blamed a day on any status incident within 48 hours and reported "mostly the model"; judged by time-aligned episodes, none of the 145 episodes since mid-July could be pinned on the model, and a status incident overlapped one of them. Signals must overlap the episode.
  • Headless runs are not the principal. On 2026-09-26, claude -p jobs replayed a frustration corpus: 218 prompts, 180 "frustrated", against 93 typed prompts at 7.5%. Only Claude Code prompts with entrypoint: cli and Codex sessions from the TUI or a desktop app count; codex_exec is a job. Headless runs still count for API and tool errors.
  • Codex writes its own text as user messages. Its AGENTS.md instructions and its history summaries ("The following is the Codex agent history…") arrive with role user; on 2026-10-12, 694 of 738 "frustrated" Codex prompts were those summaries. Both are dropped.
  • The busiest model in an hour is not the model you were fighting. Each frustrated prompt records the model its session last answered with, and the episode is judged against that model's provider, so frustration in a Codex session is not checked against Anthropic because Claude agents were busier that hour.
  • Anthropic posts about one incident a day (81 in the 90 days to 2026-10-12), and some are about sign-in, connectors, billing or compliance data. Only incidents about models, the API, the provider's own products or a service-wide disruption count (incidentRelevant), only for the provider in use, and not when they name a different model family seen in the transcripts. A provider with no readable feed is reported as such, never as "no incident".
  • Core harness files change almost every day. Twice the median 12-hour churn fired on nearly every episode in the week to 2026-10-12, so a change counts only in the top tenth of 12-hour windows of the two weeks before (git log --numstat lines changed). Instruction and settings files (CLAUDE.md, AGENTS.md, GEMINI.md, settings.json, config.toml, system prompts) count as heavy; hooks, agents, commands, prompts, rules and plugins as core.
  • The first refresh reads every transcript (about 26 GB, 9 minutes on this install, 2026-10-12); later runs reread only files whose size or mtime changed. The ledger is bucketed by UTC hour and rolled into local days at read time, so travel never forces a rebuild.
  • Anthropic's live incident feed holds only the last 50 incidents. Older history comes from history.json?page=N, three months a page, parsed by parseHistoryStamp.
  • X recent counts reach back seven days only, hourly in UTC. Daily refreshes build the series forward; an hour with no sample reads "no data", and X cannot fire until a week of hours exists.
  • Share hosts block scripts, so the page is pure inline SVG with no script, and the link given is the /raw one after it answers 200 text/html.

Examples

Example 1: the last day

User: "/witm 1d"
→ Report intent, --window 1d --share
→ "5 frustration episodes in the last day: 0 the model, 0 our harness, 0 both, 5 unclear", one sentence per episode with its evidence, the evidence table, and the /raw share link

Example 2: one bad evening

User: "Was Opus broken last Tuesday night?"
→ Day intent for that date
→ Each episode that evening with its sentence: which incident was open at the time, how many provider errors hit inside the window, what we changed in the 12 hours before

Workflows & Routing · 3

Each workflow is one job the skill runs. The trigger phrases route your request to the right one — this is the skill's routing table.

  1. 01
    Report Intents/Report.md

    /witm 1d, was it the model, was it us or the model lately, model or harness this week

  2. 02
    Day Intents/Day.md

    was it the model on Tuesday, what happened on October 7

  3. 03
    Refresh Intents/Refresh.md

    refresh WasItTheModel, backfill, sample X counts

Tools · 5

Deterministic executables the workflows call — the code that does the real work, not prompt scaffolding.

  • Episodes.ts
  • Ledger.ts
  • Page.ts
  • Sources.ts
  • WasItTheModel.ts

How to Invoke

Say any of these to your DA and LifeOS activates the WasItTheModel skill automatically:

  • "was it the model"
  • "witm"
  • "WasItTheModel"
  • "was the model broken"
  • "was it us or the model"
  • "is the model degraded"
  • "model nerfed"
  • "model having a bad day"
  • "why was yesterday so bad"
  • "model or harness"
  • "blame the model"
  • "check the status page against my sessions"

Or invoke explicitly:

Skill("WasItTheModel")

Want LifeOS to do this for you?

Install LifeOS on your machine — your DA gets the WasItTheModel skill plus 57 others, all hooked into one Life OS.