← Running in production

Running in production

September 2026

• Sep 30, 2026 AI cost tracking & telemetry In the briefing →

  From October 30, OpenAI's $200 Pro plan drops from 20x to 10x the Plus allowance in ChatGPT Work and Codex, and GPT-6 Pro messages fall from 200 to 100 a week. A new $500 Pro plan carries 25x plus Ultrafast access. Existing subscribers get 62,500 usage credits expiring December 31; the five-hour limit stays gone. (source)

• Sep 27, 2026 AI cost tracking & telemetry In the briefing →

  Anthropic resumed billing for refusals that arrive before any output when stop_details.category is bio, frontier_llm or reasoning_extraction, at the normal rates of the model that ran. It chose those three categories for their low false-positive volume; cyber and general-harms refusals stay free, and mid-stream refusals were already billed. A benign case surfaced two days later: a New Stack reviewer asked Opus 5.5 to count job orderings, and after 19 minutes and 112,733 output tokens the API returned a refusal stop reason with no text. (official, source)

• Sep 27, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  Anthropic now bills pre-output refusals in three stop_details categories, and one benign Opus 5.5 job ran 112,733 tokens before refusing. OpenAI rebuilt prompt caching with explicit breakpoints and a 30-minute window, Claude Code moved auto mode's classifier tokens off the bill, and the Copilot app exports OpenTelemetry traces. (refusals, caching, classifier, OTel)

• Sep 25, 2026 AI cost tracking & telemetry In the briefing →

  Grafana borrowed the error budget for agent behaviour. Score conversations on groundedness, fulfilment, leakage and injection resistance; set an SLO such as 95% fulfilled over 30 days; treat the remaining 5% as budget to spend on prompt and model experiments. Burn-rate alerts catch the slow quality drift that a single bad-day spike hides. (source)

• Sep 24, 2026 AI cost tracking & telemetry In the briefing →

  The OpenTelemetry project surveyed 81 users running OTel against Prometheus-compatible backends, and the friction has measurably dropped: the share calling the two hard to use together fell from 29% to 10%, with ease-of-use averaging 3.6 out of five against 3.1 a year ago. The number worth noting is 49% — nearly half run Prometheus exporters and OTel receivers side by side instead of migrating off either. (official)

• Sep 23, 2026 AI cost tracking & telemetry In the briefing →

  OpenAI rebuilt prompt caching for the GPT-6 line: higher default hit rates, a 30-minute reuse window for eligible shared prefixes, explicit breakpoints so a developer chooses what gets cached, and diagnostics for why a hit missed. Cached input runs at a tenth of the uncached rate — $0.20 a million on Sol, $0.01 on Luna. (official)

• Sep 23, 2026 AI cost tracking & telemetry In the briefing →

  The GitHub Copilot app now exports OpenTelemetry traces of agent sessions to an endpoint an enterprise names in managed-settings.json. Prompt and response content is excluded by default, which is the setting to read before switching this on. Agent activity lands where the rest of your traces already live. (official)

• Sep 22, 2026 AI cost tracking & telemetry In the briefing →

  Claude Code 2.1.278 moves auto mode's safety classifier to the server for Claude API and Enterprise users and on Bedrock, Vertex, Foundry and gateways, so the classifier's own tokens stop reaching your bill. A new Auto mode server row in /status says which side is running it, and a session that falls back to the billed path now warns. CLAUDE_CODE_AUTO_MODE_SERVER=0 opts out. (official, official)

• Sep 20, 2026 AI cost tracking & telemetry In the briefing →

  Open-weight models carried 56% of tokens through Vercel's AI Gateway in August, up from under 10% in December, and took 14% of the spend. Anthropic held 64% — at least 61 cents of every dollar every month since December. Price per token fell 23.2% over the month and the median team paid 7.6% less. Which models run your volume and which consume your budget are now separate questions. (source)

• Sep 20, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  Open-weight models carried 56% of tokens through Vercel's AI Gateway in August and 14% of the spend, while Anthropic held 64% of the dollars. Mozilla fit the open-to-closed capability gap at 4.4 months, GitHub's usage metrics API now counts agentic CLI use per MCP server, and Copilot's auto model selection took an Efficiency/Balance/Intelligence preference — a preference, not a budget. (Vercel, Mozilla, metrics, auto)

• Sep 18, 2026 AI cost tracking & telemetry In the briefing →

  GitHub's Copilot usage metrics API now breaks agentic CLI use down by skill, custom agent, MCP server, slash command and plugin, each with an interaction count and a distinct-use count, at org and enterprise level over 1- and 28-day windows. Which MCP servers anyone actually connects to has been guesswork; this is the first place to look it up. (official)

• Sep 17, 2026 AI cost tracking & telemetry In the briefing →

  Mozilla's State of Open Source AI, on data current to September 1, fits the open-to-closed capability gap at 4.4 months against METR's task-horizon curve, close to Epoch AI's four-month estimate. The best open model trails the closed leader by three points on Artificial Analysis' intelligence index at 60% of the price, and sits two points behind Fable 5 at 30%. The report's reading: open by default, pay for closed on expert work, heavy retrieval and long context. (source, source)

• Sep 15, 2026 AI cost tracking & telemetry In the briefing →

  Copilot's auto model selection now takes direction instead of deciding alone: an Efficiency / Balance / Intelligence setting says whether it should favour cost, latency or quality, and it still drops to a small model for a docstring whatever you picked. Rolling out to VS Code, the Copilot CLI and the GitHub Copilot app. Billing stays usage-based on whichever model answers, and the 10% discount for usage billed through auto is unchanged. (official)

• Sep 13, 2026 AI cost tracking & telemetry In the briefing →

  AWS makes the case that per-token price is the wrong unit and proposes cost per successful outcome — everything spent on right and wrong attempts, divided by the correct answers. On AIME, GPT-5.6-Luna worked out to $0.0021 a correct answer against $0.0139 for GPT-5.4-Mini; on DeepSearchQA agent runs the gap widened to $0.05 against $0.40. The agent figure is the one worth copying, since re-sent context makes a trajectory's cost grow quadratically. (official)

• Sep 13, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  AWS proposed cost per successful outcome in place of price per token, and OpenTelemetry's GenAI conventions turn out to be emitted already by VS Code Copilot, Codex and Claude Code. Infostealers are lifting Claude sessions and burning subscriber quota, with Anthropic refunding but declining to hand over itemised logs. Ramp card data put AI spend per employee down nearly 10% in August, and OpenAI stopped selling $200 Pro seats. (outcome, OTel, sessions, Ramp, Pro)

• Sep 12, 2026 AI cost tracking & telemetry In the briefing →

  Copilot usage metrics now count the dedicated VS Code Agents window, generally available, with daily_active_vscode_agent_users and totals_by_vscode_agent in the aggregate reports. The figures stay separate from editor Agent Mode, so an existing adoption dashboard keeps reporting the old number until someone adds the new fields. (official)

• Sep 12, 2026 AI cost tracking & telemetry In the briefing →

  Bedrock AgentCore Evaluations scores live agent traffic with 13 LLM-as-judge evaluators — goal success, tool selection accuracy, instruction following — plus three deterministic matchers that check the order of tool calls. Sampling runs from 0.01% to 100% asynchronously, adding no latency. That rate is the budget dial: every scored trace is a second model call. (official)

• Sep 11, 2026 AI cost tracking & telemetry In the briefing →

  OpenAI stopped selling $200 Pro subscriptions to new customers, a week after Astra shipped, saying that tier strains its systems hardest. Plus, Go and API access continue, and no restart date was given. A capacity signal like this is worth reading before you plan a rollout on the newest model. (source)

• Sep 11, 2026 AI cost tracking & telemetry In the briefing →

  StackGen's account of running agents in production puts the diagnostic unit at the session, not the request: nested spans for every LLM call, tool execution and subagent handoff, each carrying latency and token cost, exported asynchronously so telemetry trouble never blocks the agent. Cost control is pre-execution — hard iteration caps, per-tool limits, a block on identical consecutive calls. (source)

• Sep 10, 2026 AI cost tracking & telemetry In the briefing →

  AI spend per employee at the top 1% of firms fell nearly 10% in August, to $7,205, on Ramp card data covering 70,000 companies. The share of companies paying for anything AI barely moved — up 0.4%, to 56%. Average token cost is down to $0.68 per million from a March peak of $1.15, which explains part of the drop; August holidays plausibly explain the rest. (source)

• Sep 9, 2026 AI cost tracking & telemetry In the briefing →

  The Claude session theft has a vendor-response half now. Anthropic has been emailing subscribers that infostealer malware is lifting Claude login sessions off their machines and burning the quota attached; one user watched 10% of a month's tokens go on August 4–5 with no work of his own. Sessions were invalidated and a partial refund issued — £44.49 against a $200 plan — but the company declined to hand over itemised usage logs, which is the only thing that would let a customer reconstruct what ran. (source)

• Sep 7, 2026 AI cost tracking & telemetry In the briefing →

  OpenAI published what its own research organisation spends on coding agents, and the curve is steep: the median researcher went from near zero in February to roughly $150 a day in June and about $600 a day by late August, priced at API rates. The 90th percentile is past $7,000 a day. Across the org that works out to 3.1 agent-workdays for every human workday — a ratio that sat below 1 as recently as May. (official, source)

• Sep 7, 2026 AI cost tracking & telemetry In the briefing →

  The OpenTelemetry project walked through its GenAI semantic conventions end to end: gen_ai.usage.input_tokens and gen_ai.usage.output_tokens on spans, gen_ai.client.token.usage and gen_ai.client.operation.duration as histograms, message content captured as events. The part to act on is that VS Code Copilot, OpenAI Codex and Claude Code already emit this — point them at an OTLP endpoint and per-request cost stops being whatever a vendor dashboard chooses to show. (official)

• Sep 6, 2026 AI cost tracking & telemetry In the briefing →

  Google benchmarked Gemma 3 12B and 27B on TPU v6e and found the workload shape, not the parameter count, decides what to deploy. Generation traffic walls the 27B at 64 concurrent users, plateauing at 4.12x throughput while the 12B climbs to 8.19x; prefill-heavy classification puts both near 6x. Autoscale on end-to-end latency rather than CPU. (official)

• Sep 6, 2026 AI cost tracking & telemetry Weekly recap Read the recap →

  GitHub published the arithmetic behind its own agent bill and the counter-intuitive part stuck: trimming individual tool outputs *raised* total cost, because the agent burned turns recovering what had been cut. Jamf built Bedrock spend enforcement from S3 logs, an Athena view and a Lambda for under $10 a month, and Shopify compressed a 6,000-token system prompt into 1,500 learned gist tokens. (GitHub, Jamf, Shopify)

• Sep 4, 2026 AI cost tracking & telemetry In the briefing →

  Shopify compressed the Sidekick agent's system prompt from about 6,000 tokens to 1,500 learned gist tokens — not a summary, a trained representation that leaves the model behaving as though it read the original. At 350 requests a minute, time to first token fell from 438ms to 354ms and end-to-end latency from 6.8s to 4.2s, freeing GPU allocation. (source)

• Sep 4, 2026 AI cost tracking & telemetry In the briefing →

  Nvidia released PAIR in beta, a free router that finds compatible machines on a local network and sends independent inference requests to whichever has GPU headroom. It fronts Ollama and LM Studio, adapts as devices come and go, and covers RTX 20-series and newer, RTX PRO, DGX Spark and Apple M4 silicon on Windows, macOS and Linux. (official)

• Sep 3, 2026 AI cost tracking & telemetry In the briefing →

  GitHub published the arithmetic behind cutting Copilot's agent bill, and every item is small: compressing repetitive build and test log noise while leaving git diff output untouched (5.5%), dropping unused line-number prefixes from the file viewer (3–5%), halving the task-tool prompt for 1,300 fewer tokens a turn (2.9%), batching background-task results (2.3%). Their counter-intuitive result: trimming individual tool outputs *raised* total cost, because the agent spent turns recovering what had been cut. (official)

• Sep 2, 2026 AI cost tracking & telemetry In the briefing →

  Jamf wired Bedrock spend enforcement out of parts most teams already run. Bedrock logs invocations to S3, an Athena view prices the tokens per user per day, and a Lambda on a 15-minute EventBridge schedule rewrites an IAM customer-managed policy keyed to the SAML subject. Engineers lose Opus at 80% of budget and Sonnet at 100%, Haiku always stays. No re-authentication, resets daily, and the whole thing runs under $10 a month for hundreds of engineers. (official)

• Sep 1, 2026 AI cost tracking & telemetry In the briefing →

  OpenAI has reportedly started billing some enterprise accounts on outcomes rather than tokens, under custom contracts tied to revenue won or support cost removed. It has published no pricing, named no success criterion, and announced nothing. The retries move onto the provider's side of the ledger: an agent that takes twenty calls to finish is now OpenAI's cost, not the customer's. (source) (unconfirmed)