← Running in production

Running in production

July 2026

• Jul 31, 2026 AI cost tracking & telemetry In the briefing →

  AWS — Bedrock takes explicit cache breakpoints on the GPT-5.6 family now, so you mark what gets reused instead of hoping a prefix matches. Cached input reads at a 90% discount and writes cost 1.25x, which puts the break-even near 20% of tokens served from cache. Minimum 1,024 tokens per section, four breakpoints per request, TTL of at least 30 minutes. (official)

• Jul 31, 2026 AI cost tracking & telemetry In the briefing →

  Grafana — Agent Observability reached general availability in Grafana Cloud, built first for Grafana's own Assistant. It records latency, cost, token usage and error rates next to full conversation transcripts, scores runs with both deterministic checks and LLM judges, and tracks experiments across agent versions and model swaps. Custom evaluators cover cases like prompt-injection detection. (official)

• Jul 30, 2026 AI cost tracking & telemetry In the briefing →

  Grafana — gcx and the Grafana MCP server both reached general availability, aimed at a workflow Grafana calls telemetry-driven development: before an agent writes the change, it queries production for the real request rate and latency, stands up a local LGTM stack, generates a k6 script shaped by actual traffic, and puts dashboard links in the PR. gcx is the exploratory CLI; the MCP server is the opinionated path, and it installs into Claude Code as a plugin. (official)

• Jul 30, 2026 AI cost tracking & telemetry In the briefing →

  OpenTelemetry — Demo 3.0 adds an agentic tier worth borrowing from: a LangGraph ReAct service, an MCP server exposing the demo's own capabilities as tools, and a gen-ai normalizer processor that rewrites assorted AI telemetry into the official gen_ai.* conventions. Two things break on upgrade — custom attributes moved from app.* to demo.*, and docker-compose.yaml is replaced by modular compose.yaml files. (official)

• Jul 29, 2026 AI cost tracking & telemetry In the briefing →

  Databricks — Every coding agent inside Databricks routes through one gateway, and Unity AI Gateway Budgets enforces two limits on it. A small daily cap catches runaway loops: at 90% an engineer gets a Slack message and can self-raise in increments with one click, uncapped, resetting nightly. A monthly cap governs real spend and needs manager approval, moving in coarse 2× and 5× steps and expiring with the project. Spend across Claude, GPT, Gemini and open-source models lands in Unity Catalog. (official)

• Jul 28, 2026 AI cost tracking & telemetry In the briefing →

  Grafana — Agent Observability went GA tracking token usage and cost alongside the usual latency and error counts, with evaluators scoring conversations continuously and policy guards enforced inline. Assistant Investigations and scheduled Automations also reached GA; Watchers, which monitors from a written description of your system rather than alert rules, is in public preview. (official)

• Jul 27, 2026 AI cost tracking & telemetry In the briefing →

  Simon Willison — A resale market exists for your API key, and it prices unprotected endpoints. Willison flags an investigation into the token relay market, largely Chinese, where resellers front pooled credentials through open-source proxies like one-api and new-api and sell access below list. The pools are filled from free trials, stolen cards, and any LLM endpoint left exposed. His ask of vendors is narrow and overdue: hard spend caps per API key. (source)

• Jul 27, 2026 AI cost tracking & telemetry In the briefing →

  Token accounting — Rendering source code as an image is billed very differently depending on whose API you send it to. A cross-provider measurement of 675 matched text/image request pairs across five languages, nine file lengths, and 15 model aliases found aggregate input-token reductions of 86.5%, 80.6%, and 75.8% for the three distinct accounting signatures. The break-evens diverge sharply: Anthropic and OpenAI count images lower at every size tested, while Gemini charges 6.95× as many tokens for a 20-line file and only crosses below text around 200 lines. Accounting only — no claim about accuracy. (paper)

• Jul 26, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  Pricing moved one way all week: Opus 5 at $5/$25, Gemini 3.6 Flash near $1.50/$7.50 on fewer output tokens, Cursor claiming 60% savings from routing.

• Jul 19, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  Pricing was the recurring thread: Fable 5's included window ran through July 19 before flipping to $10/$50 credits on July 20, while Kimi K3 landed at Sonnet-tier $3/$15 for a model Moonshot frames as Opus-4.8-class.

• Jul 12, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  The dominant thread all week: Fable 5's $10/$50 metered cutover, OpenAI workspace agents leaving free preview into credit metering (July 6), and CNBC's finding that Chinese open-weight models now hold 30–46% of US tokens on OpenRouter as frontier prices climb.

• Jul 9, 2026 AI cost tracking & telemetry In the briefing →

  Anthropic — Fable 5's included-access window was extended to July 12 — correcting this feed's earlier read that it ended July 7. Hours before the July 7 cutover, Anthropic announced from its official account that Fable 5 stays included at up to 50% of weekly limits on Pro, Max, Team, and eligible seat-based Enterprise plans through 11:59pm PT July 12 — five extra days at no additional cost — after which access moves to metered usage credits at the unchanged $10/$50 per million input/output tokens. Anthropic also reiterated it wants to fold Fable 5 into standard subscriptions later, once it has the compute to support that. It matters because the July 7 and July 8 briefings reported the promo as already closed and the model as already metered; the extension pushes the real steady-state cost cutover to after July 12, so teams that rewired production onto Fable 5 have a few more days of included access before every call meters against a per-token budget. (official, source)

• Jul 8, 2026 AI cost tracking & telemetry In the briefing →

  US companies are shifting AI spend toward cheaper Chinese open-weight models as frontier prices climb. Per CNBC, the share of tokens US companies run on Chinese models via OpenRouter has stayed above 30% every week since February 8 and spiked as high as 46%, against an ~11% average over the prior 12 months. The driver is cost: open-source Chinese models run 60–90% cheaper than the leading OpenAI and Anthropic models, and Z.ai's GLM 5.2 landed within a percentage point of Claude Opus 4.8 on one closely watched agentic benchmark at roughly a fifth of the cost — posting the fastest adoption of any model Vercel tracked in 2026 (daily token volume ~27×, customers ~80× in its first full week). Named migrations: AI startup Lindy moved 100% of its traffic from Claude to DeepSeek in June, while Shopify and Airbnb have touted Alibaba's Qwen 3. It matters because it puts a market number on the cost thread this week has tracked — Fable 5's $10/$50 rate, workspace-agent metering — showing frontier-tier price hikes translating directly into US buyers substituting toward open-weight Chinese models rather than absorbing the increase. (CNBC)

• Jul 7, 2026 AI cost tracking & telemetry In the briefing →

  Anthropic — Fable 5's promotional access ended today; the model is now billed through metered usage credits. Since its July 1 global return, Fable 5 had been included at up to 50% of weekly usage limits on Pro, Max, Team, and select Enterprise plans through July 7; as of today that allowance expires and continued Fable 5 use moves to usage credits — the metered top-up mechanism paid Claude plans use once their included limits are exhausted. The underlying API rate is $10 / $50 per million input/output tokens, roughly double Claude Opus 4.8 and the most expensive rate Anthropic has published for a generally available model. In practice, teams that rewired production onto Fable 5 during the promo now meter every call against a per-token budget rather than a weekly cap — and without usage credits enabled in the Console, Fable 5 access simply stops when the included allowance runs out rather than degrading gracefully. It matters because it closes the last open variable from the export-control saga — the actual steady-state cost of the model that drove a month of briefings — and sets an unusually high floor for frontier-tier spend at the same moment OpenAI (workspace agents, July 6) and GitHub (Copilot credit pools) are converting always-on AI from flat plans to per-run metering. (official, rate, usage credits)

• Jul 6, 2026 AI cost tracking & telemetry In the briefing →

  OpenAI — workspace agents left free preview today and are now metered in credits. The extended free window that OpenAI set when it made workspace agents generally available across Business, Enterprise, and Edu (on May 22) ended July 6, and as of today agent runs invoked inside ChatGPT are billed in credits. Workspace agents are Codex-powered cloud agents — an evolution of GPTs that keep running after the user logs off, run on schedules, retain memory across projects, and execute multi-step workflows that write code and call connected apps. OpenAI's rate card puts a typical end-to-end run on GPT-5.5 at 5–25 credits (a worked example of 20K input + 80K cached-input + 5K output tokens lands at ~7.25 credits); runs invoked outside ChatGPT — e.g. an agent replying in a Slack channel — stay in free preview for now. It matters because it converts always-on autonomous agents from a free experiment into a metered line item teams have to budget per run — the same shift to usage-based agent billing that GitHub Copilot made in June, and exactly the per-run spend that Anthropic's Enterprise cost controls last week were built to govern. (official, rate card)

• Jul 5, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  Claude Enterprise gained admin usage/cost analytics, model entitlements, spend-threshold alerts, and an Analytics API with named Datadog and CloudZero integrations.

• Jul 4, 2026 AI cost tracking & telemetry In the briefing →

  Anthropic — new admin analytics and cost controls landed for Claude Enterprise. The admin console now breaks down usage and cost by group and by user, showing the output a team produced — artifacts created, files edited, skills and connectors used — directly next to what it cost. Three levers ride alongside the visibility: model defaults and entitlements let admins set which Claude model new conversations start with across chat, Cowork, and Claude Code (so routine work doesn't default to the most expensive tier) and restrict which models a group can reach; spend-threshold alerts notify admins at 75% and 90% of an org-level spend limit while users get in-app warnings at 75% and 95% and can request an increase without leaving Claude; and an Analytics API exposes the same usage-and-cost data programmatically, with named integrations into Datadog Cloud Cost Management and CloudZero so finance and IT can fold Claude spend into the FinOps tooling they already run. It matters because it moves Claude cost governance from after-the-fact invoice reconciliation to a live, per-user, per-model control plane — the same instrumentation-and-budget-guardrail pattern the observability vendors have been building toward for LLM spend, now shipped first-party by the model provider rather than bolted on by a third party. (official)