8 companies admitted · updated 2026-08-25
senku. aifirst / evidence
AI-first registry · evidence

What each grade rests on

Every dimension of every admitted row, the reason it scored, and each citation a stranger can open. The one-line view is on the registry.

The bar

EVERY ADMITTED ROW, OUT OF 12 Block 12 OpenAI 11 Shopify 11 Stripe 11 Cloudflare 10 Ramp 10 Spotify 10 GitHub 8 the bar: 7 of 12
Drawn from the rows: each admitted company's total and the bar.

A company is admitted with human-graded evidence on at least 4 of the 6 dimensions and a total of at least 7 of 12. A dimension with no qualifying public evidence carries a dated line and no digit.

Block · 12 of 12

graded by Oleg Malkov · 2026-08-25 · every cited artifact fetched and read on the grading date.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 2 · repeated and concrete
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 2 · repeated and concrete
Human and societal consequences
Level 2 · repeated and concrete
Why each dimension scored, with 18 citations across 8 articles

graded by Oleg Malkov · 2026-08-25 · every cited artifact fetched and read on the grading date. Claim dates: the article's own published date, recorded when the artifact was fetched. 18 citations, all first-party, all current.

Named operating systems

Level 2 · repeated and concrete

Names Buzz, Block App Kit, Block Metrics Store, Block Data MCP, Query Expert MCP, CodeCrucible, and MeshLLM, and states each system's operating job; the naming recurs across the archive.

Names Buzz as the channel-driven workspace where people, agents, repos, and decisions work together in one signed room, with its protocol and storage design.

engineering.block.xyz/blog/buzz · observed 2026-08-23

Names Block App Kit as the sanctioned platform for internal apps, with its managed frontend, compute, and persistence layer and its agent-skill distribution route.

engineering.block.xyz/blog/from-localhost-to-launched-safely-shipping-apps-that-anyone-can-build · observed 2026-08-23

Names Block Metrics Store, Block Data MCP, and Query Expert MCP and states their distinct jobs: governed metric definitions, deterministic pre-approved SQL, and exploratory RAG retrieval.

engineering.block.xyz/blog/building-the-data-foundation-for-automated-analytics · observed 2026-08-23

Measured production results

Level 2 · repeated and concrete

Reports Block App Kit at over a thousand internal apps with weekly active users grown from the low hundreds into the thousands across more than fifty orgs, and the data platform at 1,100+ governed metrics with about a third of the company (2.1K monthly users) on Query Expert by March 2026; benchmark figures are published separately from these operating numbers.

Reports weekly active users climbing from the low hundreds into the thousands, a catalog of well over a thousand distinct apps across more than fifty orgs, and roughly four in five users outside engineering.

engineering.block.xyz/blog/from-localhost-to-launched-safely-shipping-apps-that-anyone-can-build · observed 2026-08-23

Reports 1,100+ governed metrics, 1,812 Block Data MCP and 1,899 Query Expert users in Q1 2026, 400M queries monthly, and a third of the company (2.1K monthly users) on Query Expert by March 2026.

engineering.block.xyz/blog/building-the-data-foundation-for-automated-analytics · observed 2026-08-23

Reports a 2,000+ app migration run with agent swarms, with Terminal-Bench benchmark figures labelled as benchmarks and kept separate from the migration account.

engineering.block.xyz/blog/effective-teams-buzz · observed 2026-08-23

Adverse evidence

Level 2 · repeated and concrete

Publishes a pilot dashboard that shipped incorrect data from a sampling-logic flaw, a first single-MCP design users rejected ('I can't trust this') that was discarded and rebuilt as two servers, and CodeCrucible's initial missed detection with stated recall and precision bounds.

Admits a pilot dashboard displayed excellent visuals over incorrect data from a subtle sampling-logic flaw, and states the fix: authoritative sources plus test scaffolding.

engineering.block.xyz/blog/from-localhost-to-launched-safely-shipping-apps-that-anyone-can-build · observed 2026-08-23

Admits the initial single-MCP design failed both audiences and records finance leadership's 'I can't trust this'; the design was discarded and rebuilt as two specialized servers.

engineering.block.xyz/blog/building-the-data-foundation-for-automated-analytics · observed 2026-08-23

States the Copy Fail vulnerability was initially missed without steering, that recall is bounded by the first pass and precision by the batch-local audit, and that repeated at-scale rescanning is unsolved.

engineering.block.xyz/blog/codecrucible-a-blueprint-for-llm-driven-sast · observed 2026-08-23

Reproducible mechanism

Level 2 · repeated and concrete

Publishes the identity-delegation token flow end to end, CodeCrucible's chunking, budget-gate, and deduplication blueprint, and Buzz's object-store guarantees specified in TLA+ with a backend conformance suite.

Publishes the four-component delegation design: edge identity issuer, token exchange, consent control plane, and authorization data plane, with effective capabilities computed as the intersection of user grants and delegated consent.

engineering.block.xyz/blog/whos-asking-identity-delegation-for-ai-agents-and-service-meshes · observed 2026-08-23

Publishes the SAST blueprint another engineer could challenge: semantic late chunking with intact file boundaries, budget gates, per-phase prompts, and CWE-keyed deduplication.

engineering.block.xyz/blog/codecrucible-a-blueprint-for-llm-driven-sast · observed 2026-08-23

Publishes Git-on-object-storage as content-addressed packfiles plus one mutable manifest pointer, with three object-store guarantees specified in TLA+ and a required backend conformance suite.

engineering.block.xyz/blog/buzz · observed 2026-08-23

Engineer authored cadence

Level 2 · repeated and concrete

Eight first-party practitioner articles between April and August 2026 cover distinct internal systems: agent collaboration, an internal app platform, data governance, security scanning, identity delegation, and distributed inference.

A practitioner account of configuring five production agents: instruction layers, memory tiers, harness and provider choices, and effort tuning.

engineering.block.xyz/blog/configuring-agents-in-buzz · observed 2026-08-23

A principal engineer's account of distributed inference over pooled idle compute, including its latency and best-effort limits.

engineering.block.xyz/blog/buzz-sharing-compute-powered-by-meshllm · observed 2026-08-23

A multi-role engineering account (tech lead, data engineers, AI engineer, platform engineering) of the data foundation build.

engineering.block.xyz/blog/building-the-data-foundation-for-automated-analytics · observed 2026-08-23

Human and societal consequences

Level 2 · repeated and concrete

A whole article keeps the human's identity and bounded consent attached to every agent action, with revocation and non-delegable capabilities; Buzz states that authorization does not erase authorship and argues against infrastructure lock-in of identities, history, and work; the MeshLLM post states plainly that prompts sent to another machine are visible to its owner.

Designs agent authority as 'this engineer, through this software actor acting on their behalf, with this bounded authority, for this request', with user revocation and capabilities that cannot be delegated at all.

engineering.block.xyz/blog/whos-asking-identity-delegation-for-ai-agents-and-service-meshes · observed 2026-08-23

States that authorization does not erase authorship, that owner revocation disconnects agents, and that running the infrastructure shouldn't mean owning the identities, history, or work.

engineering.block.xyz/blog/buzz · observed 2026-08-23

States plainly that prompts and any data sent to another machine are visible to its owner, and that compute should be shared only with people you trust.

engineering.block.xyz/blog/buzz-sharing-compute-powered-by-meshllm · observed 2026-08-23

OpenAI · 11 of 12

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 2 · repeated and concrete
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 2 · repeated and concrete
Human and societal consequences
Level 1 · claimed or partial
Why each dimension scored, with 12 citations across 3 articles

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass. Claim dates: the research run's date, standing in for the article's own publication date. 12 citations, all first-party, all current. openai.com refuses automated clients, so each of those citations also links the archive snapshot the graded passage was read from.

Named operating systems

Level 2 · repeated and concrete

Names the recurring documentation gardener and Symphony and states their operating roles.

openai.com/index/harness-engineering/ · archive snapshot · observed 2026-08-21

openai.com/index/open-source-codex-orchestration-symphony/ · archive snapshot · observed 2026-08-21

Measured production results

Level 2 · repeated and concrete

Reports roughly 1,500 merged pull requests, a million-line codebase, internal use, and up to 500% more landed pull requests on some teams.

openai.com/index/harness-engineering/ · archive snapshot · observed 2026-08-21

openai.com/index/open-source-codex-orchestration-symphony/ · archive snapshot · observed 2026-08-21

Adverse evidence

Level 2 · repeated and concrete

States early progress was slower than expected, names underspecification, review capacity, context switching, forgotten sessions, and stalled tasks.

openai.com/index/harness-engineering/ · archive snapshot · observed 2026-08-21

openai.com/index/open-source-codex-orchestration-symphony/ · archive snapshot · observed 2026-08-21

Reproducible mechanism

Level 2 · repeated and concrete

Publishes repository structure, mechanical invariants, worktree observability, task control-plane design, and an open orchestration specification.

openai.com/index/harness-engineering/ · archive snapshot · observed 2026-08-21

openai.com/index/open-source-codex-orchestration-symphony/ · archive snapshot · observed 2026-08-21

Engineer authored cadence

Level 2 · repeated and concrete

Multiple first-party engineering articles extend the same operating record across systems.

openai.com/index/harness-engineering/ · archive snapshot · observed 2026-08-21

openai.com/index/open-source-codex-orchestration-symphony/ · archive snapshot · observed 2026-08-21

openai.com/index/building-self-improving-tax-agents-with-codex/ · archive snapshot · observed 2026-08-21

Human and societal consequences

Level 1 · claimed or partial

Treats human attention, taste, review, and judgment as explicit system boundaries, without wider societal analysis.

openai.com/index/harness-engineering/ · archive snapshot · observed 2026-08-21

Shopify · 11 of 12

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 2 · repeated and concrete
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 2 · repeated and concrete
Human and societal consequences
Level 1 · claimed or partial
Why each dimension scored, with 15 citations across 4 articles

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass. Claim dates: the research run's date, standing in for the article's own publication date. 15 citations, all first-party, all current.

Named operating systems

Level 2 · repeated and concrete

Names River, Aquifer, Dispatch, Roast, and Boba and states their jobs.

shopify.engineering/under-the-river · observed 2026-08-21

shopify.engineering/building-an-agentic-harness-that-outlasts-the-model · observed 2026-08-21

shopify.engineering/introducing-roast · observed 2026-08-21

Measured production results

Level 2 · repeated and concrete

Reports one in eight merged pull requests, median session and tool-call counts, and more than 50,000 internal Quick sites.

shopify.engineering/under-the-river · observed 2026-08-21

shopify.engineering/quick · observed 2026-08-21

Adverse evidence

Level 2 · repeated and concrete

Names painful migration breaks, scaling failures, architectures that had to be discarded, and models producing more unconfirmed findings.

shopify.engineering/under-the-river · observed 2026-08-21

shopify.engineering/building-an-agentic-harness-that-outlasts-the-model · observed 2026-08-21

Reproducible mechanism

Level 2 · repeated and concrete

Explains Aquifer profiles and modes and publishes the structured Roast workflow mechanism.

shopify.engineering/under-the-river · observed 2026-08-21

shopify.engineering/introducing-roast · observed 2026-08-21

Engineer authored cadence

Level 2 · repeated and concrete

Multiple first-party practitioner articles cover distinct internal systems.

shopify.engineering/under-the-river · observed 2026-08-21

shopify.engineering/quick · observed 2026-08-21

shopify.engineering/building-an-agentic-harness-that-outlasts-the-model · observed 2026-08-21

shopify.engineering/introducing-roast · observed 2026-08-21

Human and societal consequences

Level 1 · claimed or partial

Addresses shared learning, human intervention, and merchant-data security, but not the broader social-political layer.

shopify.engineering/under-the-river · observed 2026-08-21

shopify.engineering/building-an-agentic-harness-that-outlasts-the-model · observed 2026-08-21

Stripe · 11 of 12

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 2 · repeated and concrete
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 2 · repeated and concrete
Human and societal consequences
Level 1 · claimed or partial
Why each dimension scored, with 10 citations across 4 articles

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass. Claim dates: the research run's date, standing in for the article's own publication date. 10 citations, all first-party, all current.

Named operating systems

Level 2 · repeated and concrete

Names Minions and the Knowledge AI Platform and explains their jobs.

stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents-part-2 · observed 2026-08-21

stripe.dev/blog/meet-stripes-knowledge-ai-platform · observed 2026-08-21

Measured production results

Level 2 · repeated and concrete

Reports over 1,300 human-reviewed Minions pull requests weekly and measured outcomes from roughly a dozen steering experiments.

stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents-part-2 · observed 2026-08-21

stripe.dev/blog/ai-steering-experiments · observed 2026-08-21

Adverse evidence

Level 2 · repeated and concrete

Publishes approaches that failed and the continuing integration problems they exposed.

stripe.dev/blog/ai-steering-experiments · observed 2026-08-21

Reproducible mechanism

Level 2 · repeated and concrete

Explains blueprints, deterministic nodes, agent loops, isolation, and mandatory human review.

stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents-part-2 · observed 2026-08-21

Engineer authored cadence

Level 2 · repeated and concrete

A multi-part practitioner series plus a separate internal platform article.

stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents · observed 2026-08-21

stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents-part-2 · observed 2026-08-21

stripe.dev/blog/meet-stripes-knowledge-ai-platform · observed 2026-08-21

Human and societal consequences

Level 1 · claimed or partial

Addresses agent authority and the limits of guidance, but not broader social consequences.

stripe.dev/blog/ai-steering-experiments · observed 2026-08-21

Cloudflare · 10 of 12

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 1 · claimed or partial
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 2 · repeated and concrete
Human and societal consequences
Level 1 · claimed or partial
Why each dimension scored, with 10 citations across 3 articles

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass. Claim dates: the research run's date, standing in for the article's own publication date. 10 citations, all first-party, all current.

Named operating systems

Level 2 · repeated and concrete

Names iMARS, Town Lake, and Skipper and states their operating jobs.

blog.cloudflare.com/internal-ai-engineering-stack/ · observed 2026-08-21

blog.cloudflare.com/our-unified-data-platform/ · observed 2026-08-21

Measured production results

Level 2 · repeated and concrete

Reports a bounded 30-day adoption, request, and token record.

blog.cloudflare.com/internal-ai-engineering-stack/ · observed 2026-08-21

Adverse evidence

Level 1 · claimed or partial

Names fragmentation, sampling, stale data, and external dependency as limitations, without a comparable company-caused incident.

blog.cloudflare.com/our-unified-data-platform/ · observed 2026-08-21

Reproducible mechanism

Level 2 · repeated and concrete

Publishes system architecture and agent-tracing mechanics.

blog.cloudflare.com/our-unified-data-platform/ · observed 2026-08-21

blog.cloudflare.com/agents-on-cloudflare/ · observed 2026-08-21

Engineer authored cadence

Level 2 · repeated and concrete

Multiple first-party engineering articles cover distinct internal systems.

blog.cloudflare.com/internal-ai-engineering-stack/ · observed 2026-08-21

blog.cloudflare.com/our-unified-data-platform/ · observed 2026-08-21

blog.cloudflare.com/agents-on-cloudflare/ · observed 2026-08-21

Human and societal consequences

Level 1 · claimed or partial

Addresses sensitive payload recording and privacy controls, but not the wider consequences seen in Deriv's record.

blog.cloudflare.com/agents-on-cloudflare/ · observed 2026-08-21

Ramp · 10 of 12

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 1 · claimed or partial
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 2 · repeated and concrete
Human and societal consequences
Level 1 · claimed or partial
Why each dimension scored, with 12 citations across 3 articles

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass. Claim dates: the research run's date, standing in for the article's own publication date. 12 citations, all first-party, all current.

Named operating systems

Level 2 · repeated and concrete

Names Inspect and Ramp Research and states their operating jobs.

engineering.ramp.com/post/why-we-built-our-background-agent · observed 2026-08-21

engineering.ramp.com/post/meet-ramp-research · observed 2026-08-21

Measured production results

Level 2 · repeated and concrete

Reports Inspect at about 30% of merged frontend/backend pull requests and Ramp Research usage across questions, conversations, and users.

engineering.ramp.com/post/why-we-built-our-background-agent · observed 2026-08-21

engineering.ramp.com/post/meet-ramp-research · observed 2026-08-21

Adverse evidence

Level 1 · claimed or partial

Names a human-in-loop approach that did not scale and evaluation blind spots, without a comparable incident record.

engineering.ramp.com/post/meet-ramp-research · observed 2026-08-21

Reproducible mechanism

Level 2 · repeated and concrete

Publishes sandbox, tool, interface, context-layer, and intermediate-step evaluation designs.

engineering.ramp.com/post/why-we-built-our-background-agent · observed 2026-08-21

engineering.ramp.com/post/meet-ramp-research · observed 2026-08-21

Engineer authored cadence

Level 2 · repeated and concrete

Multiple first-party practitioner articles cover coding, analytics, and security agents.

engineering.ramp.com/post/why-we-built-our-background-agent · observed 2026-08-21

engineering.ramp.com/post/meet-ramp-research · observed 2026-08-21

engineering.ramp.com/post/100-vulnerabilities-patched-with-0-humans · observed 2026-08-21

Human and societal consequences

Level 1 · claimed or partial

States PII exclusion and human review boundaries, without broader societal analysis.

engineering.ramp.com/post/meet-ramp-research · observed 2026-08-21

engineering.ramp.com/post/100-vulnerabilities-patched-with-0-humans · observed 2026-08-21

Spotify · 10 of 12

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 2 · repeated and concrete
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 2 · repeated and concrete
Human and societal consequences
No qualifying public evidence found in the reviewed sources · reviewed 2026-08-21
Why each dimension scored, with 10 citations across 3 articles

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass. Claim dates: the research run's date, standing in for the article's own publication date. 10 citations, all first-party, all current.

Named operating systems

Level 2 · repeated and concrete

Names Honk, Fleet Management, Fleetshift, and Chirp and explains their roles.

engineering.atspotify.com/2025/11/spotifys-background-coding-agent-part-1 · observed 2026-08-21

engineering.atspotify.com/2026/6/code-with-claude-coding-is-no-longer-the-constraint · observed 2026-08-21

Measured production results

Level 2 · repeated and concrete

Reports more than 1,500 agent pull requests, 60–90% time savings, weekly adoption, and pull-request frequency.

engineering.atspotify.com/2025/11/spotifys-background-coding-agent-part-1 · observed 2026-08-21

engineering.atspotify.com/2026/6/code-with-claude-coding-is-no-longer-the-constraint · observed 2026-08-21

Adverse evidence

Level 2 · repeated and concrete

Names three failure modes and reports the judge vetoing about a quarter of thousands of sessions, with only half recovering.

engineering.atspotify.com/2025/12/feedback-loops-background-coding-agents-part-3 · observed 2026-08-21

Reproducible mechanism

Level 2 · repeated and concrete

Shows CLI, verifier, judge, tracing, access boundary, and surrounding infrastructure.

engineering.atspotify.com/2025/11/spotifys-background-coding-agent-part-1 · observed 2026-08-21

engineering.atspotify.com/2025/12/feedback-loops-background-coding-agents-part-3 · observed 2026-08-21

Engineer authored cadence

Level 2 · repeated and concrete

A multi-part first-party engineering series continues into a later operational account.

engineering.atspotify.com/2025/11/spotifys-background-coding-agent-part-1 · observed 2026-08-21

engineering.atspotify.com/2025/12/feedback-loops-background-coding-agents-part-3 · observed 2026-08-21

engineering.atspotify.com/2026/6/code-with-claude-coding-is-no-longer-the-constraint · observed 2026-08-21

Human and societal consequences

No qualifying public evidence found in the reviewed sources · reviewed 2026-08-21

GitHub · 8 of 12

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass.

Named operating systems
Level 2 · repeated and concrete
Measured production results
Level 2 · repeated and concrete
Adverse evidence
Level 1 · claimed or partial
Reproducible mechanism
Level 2 · repeated and concrete
Engineer authored cadence
Level 1 · claimed or partial
Human and societal consequences
No qualifying public evidence found in the reviewed sources · reviewed 2026-08-21
Why each dimension scored, with 6 citations across 3 articles

graded by Oleg Malkov · 2026-08-25 · transcribed from the research record; no fetch on this pass. Claim dates: the research run's date, standing in for the article's own publication date. 6 citations, all first-party, all current.

Named operating systems

Level 2 · repeated and concrete

Names Qubot and shows its analytics job and evaluation system.

github.blog/ai-and-ml/github-copilot/how-we-built-an-internal-data-analytics-agent/ · observed 2026-08-21

Measured production results

Level 2 · repeated and concrete

Reports production A/B deltas and workflow-level token reductions with sample rules.

github.blog/ai-and-ml/how-we-made-github-copilot-cli-more-selective-about-delegation/ · observed 2026-08-21

github.blog/ai-and-ml/github-copilot/improving-token-efficiency-in-github-agentic-workflows/ · observed 2026-08-21

Adverse evidence

Level 1 · claimed or partial

Shows avoidable delegation, stale paths, and tool failures, but no comparable incident.

github.blog/ai-and-ml/how-we-made-github-copilot-cli-more-selective-about-delegation/ · observed 2026-08-21

Reproducible mechanism

Level 2 · repeated and concrete

Describes ground-truth cases, parallel trials, saved results, and aggregation metrics.

github.blog/ai-and-ml/github-copilot/how-we-built-an-internal-data-analytics-agent/ · observed 2026-08-21

Engineer authored cadence

Level 1 · claimed or partial

First-party practitioner writing is present, but the reviewed sample is less sustained across internal systems.

github.blog/ai-and-ml/github-copilot/improving-token-efficiency-in-github-agentic-workflows/ · observed 2026-08-21

Human and societal consequences

No qualifying public evidence found in the reviewed sources · reviewed 2026-08-21

Corrections

The house builds every listing; no company joined, consented or endorsed one. A company can challenge any claim or ask for removal. Corrections stay on the page with what was wrong. Write to oleg@mlkv.org.