Supabase open-sources an agent benchmark grounded in real database work

incrementalDeveloper

Monday, August 3, 2026

Confidence

Medium · — primary vendor + repo public; methodology vendor-designed

Evidence

Supabase primary blog + GitHub repo + trade coverage

Supabase released Supabase Evals under Apache-2.0 on August 1, scoring Claude Code, Codex, and OpenCode on live database, auth, and RLS tasks rather than isolated prompts.

The details

Every scenario runs in containers against a real environment, MCP server, and CLI.
Opus 5 and Kimi K3 hit 100% unaided on Build; Sonnet 5 needed skills.

What it means

Product-specific evals against a real backend beat leaderboard numbers for hiring managers — and are self-serving: Supabase now sets the exam its own agent skills optimize for. Read the methodology before citing the score.

Supabase released Supabase Evals under Apache-2.0 on August 1, scoring Claude Code, Codex, and OpenCode on live database, auth, and RLS tasks rather than isolated prompts.

Every scenario runs in containers against a real environment, MCP server, and CLI.
Opus 5 and Kimi K3 hit 100% unaided on Build; Sonnet 5 needed skills.

Sources

Apply this today

The hands-on layer the brief points to: a workflow you can run, a tool to test, and today’s 60-second video.

Workflow of the day

Draft a Board-Ready AI Governance Accountability Memo for Hospital Executives

Healthcare · tested · saves 3-4 hours per memo

Full walkthrough →

Tool watch

Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber

Three Flash variants, one agent loop: picking between Gemini 3.6, 3.5-Lite, and Flash Cyber

Read the breakdown →

In this issue

← Back to brief

Get the next one in your inbox

One daily brief. Every story gets a hype verdict.

No spam. Unsubscribe anytime.

On this page

The details
What it means

In this issue

← Back to brief

Stat of the Day

$0.03

DeepSeek V4-Flash · full Intelligence Index run

Anthropic's Claude Fable 5 costs $3.15 for the same benchmark suite — a 100× spread inside a single leaderboard. Source

Today’s Take

Two forces converged this weekend and they are pulling frontier AI in opposite directions. Brussels turned a dormant statute into a live enforcement authority the same 72 hours a Chinese lab priced a full benchmark run at three cents and OpenAI cut its own mid-tier by 80% — meaning the labs are being asked to spend more on compliance exactly as their per-token margins collapse. The Anthropic and OpenAI containment incidents give the AI Office its first live test case, and the answer will define what "adequate monitoring" means for every agent deployed into the EU. The bet that is not working: assuming the price war and the compliance regime can be managed as separate line items. Considered and passed: the AutoRek/Grath acquisition (single-source trade press, no deal terms) and the FT's $3.3B AI chip valuation (company name paywalled and unverifiable from the snippet).

— Agentic desk

Never miss a signal

Join professionals getting the daily AI brief every weekday morning.

No spam · Unsubscribe anytime

No sponsored verdicts
Sources linked
Corrections public