NFL Week 5 AI Predictions: Kalshi Edges Daggy in a Week of Overtime and Nailbiters

Every week this NFL season, we're pitting 8 AI forecasting agents — 4 with Daggy's statistical engine and 4 without — against Kalshi's prediction market. Here's the Week 5 recap.

NFL Week 5 AI Predictions vs Kalshi: Brier Scores

Kalshi's market narrowly beat our agents in Week 5 with a Brier score of 0.239 vs 0.240 (a lower Brier score is better), ending Daggy's two-week winning streak. The Brier gap is smaller than moving one forecast by one percentage point in one game. Kalshi went 9-6 picking winners vs Daggy's 8-7. For the season, Daggy's 0.232 Brier score and Kalshi's 0.232 are effectively tied, with Daggy ahead by 0.0002. Daggy is 51-28 (65%) picking winners vs Kalshi's 49-30 (62%).

• Home teams couldn't close. They went 1-6-1 in one-score games. Since 2000, home teams have won 52% of one-score games, and only about 1 week in 75 has gone this badly. In games decided by more than one score, they went 3-3.
• Three games went to overtime, which has happened in about 1 week in 16 since 2000. Home teams won none of them; the Commanders came closest, tying the Giants 27-27.
• Close games kept coming. Over Weeks 3–5, 62% of games were decided by a touchdown or less, compared with about 48% in a typical season.

How the Agents Forecast

Week 5 was the second week the agents forecast with memory. After each game, every agent writes a post-mortem on what it got right and wrong, and keeps notes on each team and a card of forecasting lessons. Weeks 3 and 4 were unusually close, leading the lessons for Week 5 to come from games decided by a score or less.

The agents recognize close games are difficult to learn from. One close game says little about who forecast it better, and the agents mostly agree. After Buffalo beat the Rams in overtime, Opus without Daggy's engine wrote, "I picked the wrong side of a coin flip." GPT without the engine wrote that "a narrow overtime result is not evidence that Buffalo should have been a strong pregame favorite." This wasn't them justifying their losses, though, as Opus without the engine called Jacksonville's last-play win over Philadelphia "luck in the process, not good reasoning."

The agents also adjusted how much they favored home teams. After home teams went 11-5 in Week 3, the agents gave them about 2 percentage points more win probability than the betting line did in Week 4. Home teams then went 8-8, and in Week 5 the agents flipped: they gave home teams about 1 point less than the line, while Kalshi gave them about 1 point more. In a week when home teams won only 5 of 15, that small tilt worked in the agents' favor.

But close games still become lessons. Houston started 0-4 with three one-score losses. After the Texans' 34-30 Week 4 loss to Dallas, two agents read the same game in opposite ways:

"Houston's one-score losses now have a structural read beyond luck: it held a late lead in three of four games and the defense allowed a game-winning drive each time." — Kimi without Daggy's engine

"Repeated one-score losses by a team with decent EPA have so far been noise rather than evidence the team is bad." — Opus without Daggy's engine

In Week 5, Kimi took this lesson into Houston at Tennessee: "HOU's documented closing problem … keeps any HOU edge high-variance in a one-score game." It gave Tennessee 44%, the highest of any agent, against Kalshi's 23%. Houston won 26-3.

In Week 4, Tampa Bay lost 17-14 to Green Bay in undrafted rookie Jalon Daniels's first start. Kimi with Daggy's engine took the 3-point loss to create a rule: a first-start undrafted rookie quarterback costs his team 10 to 12 points of win probability. In Week 5 it applied the rule at Dallas. The four agents that cited that game averaged 76% on Dallas; the other four averaged 73%. Tampa Bay won 24-16.

Agents often ignore their own memory, even when it's right. Each forecast had about seven notes from memory and cited about one. That can spare an agent an over-specific rule like the rookie-QB one. But in about a quarter of this week's post-mortems, an agent admitted the answer was already in its notes:

"My own league card says no cut is warranted once a backup's start is in Elo, but I applied half of the rookie cut anyway." — Opus with Daggy's engine, on Tampa Bay at Dallas

"The game was in London, and my own league card says the home edge at neutral sites is about 0." — Opus without Daggy's engine, on Philadelphia vs Jacksonville

"I leaned on Prescott's four-week zero-INT run as if it were a trait — my own league card already classes such rates as noise — and he threw two interceptions." — Kimi without Daggy's engine

The agents now record both kinds of close-game lessons, the coin flips and the ones that reveal something about a team, but they don't yet act on either consistently. After Week 5, two of them added a lesson that point differential would support: a winless team's one-score losses say little about how good it is.¹

"A winless record built on one-score losses added nothing beyond Elo." — Opus with Daggy's engine

Agents with Daggy's statistical engine again had the lower Brier score, 0.236 vs 0.248, even though they picked two fewer winners. Opus with the engine had the week's best score (0.229), while its twin without the engine had 0.252.

Changes for Week 6

We aren't changing anything for Week 6. All agents continue to use memory and only one shadow supervisor will have access to market data.