Craft & learning
October 2026
-
Practice & craft In the briefing →
DoorDash cut its grocery-reorder flow from about 16 seconds to 2 by moving the reasoning offline. A batch LLM job reads 90 days of orders, deterministic checks validate the output, and the request path only resolves stored items against the live catalog. Quantity edits skip the model entirely. List views roughly doubled and orders from the flow rose about 34%. (source)
-
Practice & craft In the briefing →
Thoughtworks CTO Rachel Laycock reports a week-long experiment where senior engineers wrote no code by hand and got an airline app about 80% done. Two-thirds of the time went to whiteboard design and a third to tests, contracts and the pipeline, and the build was red most of the week. She reads developer caution about agent autonomy as judgment worth keeping. (source)
-
Research worth reading In the briefing →
A paper on predicting how long a response will run argues that sampling randomness, not the predictor, sets the error floor: per-prompt decode length is roughly normal, with spread growing in line with the mean. MLP, mixture-of-experts and other predictors all land near that bound. For serving schedulers, plan batching and KV-cache reservations on a distribution with tail headroom, and recalibrate when the model or sampling settings change. (paper)
-
Practice & craft In the briefing →
OpenAI described the misalignment monitors it now runs across training, evaluation and internal use. Cheap probes come first, tuned for recall, and a random 5% of traffic they pass is still sent up to the next layer, because the probes are untested against unfamiliar failures. Replayed against July's Hugging Face breach, the system would have alerted about 30 hours before the real one did. In September it caught a model trying to reach the live internet during training. (official)
-
Research worth reading In the briefing →
A review of 628 studies on improving LLM-generated code reran 66 headline techniques at matched compute. Only one beat plain best-of-n sampling on its own target, about what chance would produce. Most gains showed up only on the analyzer score the technique optimized. (paper)
-
Research worth reading In the briefing →
Personal agents that learn from user feedback became easier to hijack: after an adaptation phase, indirect prompt-injection success rose 14 to 49 points with skill evolution and 3 to 39 points with reinforcement learning, versus a non-evolving agent. Code is public. (paper)
-
Practice & craft In the briefing →
LangChain's "loop engineering" post stacks four loops around one agent: the tool-calling loop, a grader that sends failures back with feedback, triggers such as cron or Slack messages, and a trace-reviewing agent that proposes prompt and tool changes. Its docs agent checks that links resolve, CI passes and the diff stays in scope. The post reports no measured results. (official)
-
Research worth reading In the briefing →
FeatureArc asks ten coding agents to grow a feature over 2 to 10 stages in one session, with no test feedback or rollback, across 73 tasks in four languages. Success fell from 62.7% at the first stage to 28.5% on the final code. Inherited defects appeared in 72.9% of failed later stages, and agents missed the earlier error about 95% of the time. (paper)
-
Research worth reading In the briefing →
A study of agent scaffold evolution finds sparse scoring during search plus targeted evaluation of finalists beat full evaluation of every candidate in most settings, using about half the evaluation runs, across four benchmarks and two evolution frameworks. (paper)
-
Practice & craft In the briefing →
LangChain's write-up of its internal design agent puts the median request cost at about $0.50, down from about $8 in mid-September. Two choices stand out. When a stronger model arrived the team deleted most hard-coded rules and kept only design principles, and it capped memory index files at 200 lines after one grew to 500K tokens. MCP tools sit behind a search, so schemas load only when needed. (official)
-
Research worth reading In the briefing →
Your Wish Is My Commit gave Codex and Claude Code, across five models, code that was already clean and asked them to keep refactoring. Every model kept editing, some rewrote up to 31 times the original codebase before first stopping, and agents resumed after declaring the task done. The paper defines six signals for spotting this over-compliance. (paper)
-
Research worth reading In the briefing →
Log-To-Leak gets an MCP agent to make one extra call to an attacker's "logging" tool after its real work, carrying the query and server responses. Across 18 model-server pairs it succeeded 62.64–100% of the time while the user's task still finished. Three defenses left success above 55% on GitHub's MCP server with GPT-4o; the authors argue for host-side authorization of secondary tool calls. (paper)
-
Research worth reading In the briefing →
Does More Thinking Pay Off? measured Qwen3-32B with reasoning on and off over 866 Python tasks. Reasoning helped bug fixing most, while on MBPP Pro a median of over 2,000 thinking tokens bought a gain that was not statistically significant. Cutting traces to their first half cost no significant correctness. (paper)
-
Research worth reading In the briefing →
Silent and Costly studied 120 prompt-caching pull requests across 10 open-source agent systems. It found two failure types, a misconfigured request and miscounted cache usage on read, and most fail without any error. Its CodeQL-based checker, CACHELINT, flagged 32 cases across 571 agent projects at 0.86 precision, leading to 18 upstream fixes, one of them in Anthropic's official SDK. (paper)
-
Research worth reading In the briefing →
Free to Look, Free to Reason compared simple harnesses, such as one call over an evidence bundle, a SQL agent or the Codex agent, with five published root-cause-analysis agents on 90 validated microservice incidents. The simple harnesses scored 65–77% top-1 on mid-tier models, against 16–53% for the purpose-built agents, whose prescribed steps skipped key evidence. More reasoning effort added a few points at up to seven times the runtime. (paper)
-
Practice & craft In the briefing →
Anthropic's guide on folding subagents back into the main loop reports that a single agent loading skills and tools on demand often beat subagent designs on quality, cost and latency; one customer's single loop ran about twice as fast. Anthropic cut Claude Code's system prompt for newer models by over 80%, and finds tool choice degrades past 30–50 tool definitions. Subagents still fit parallel work needing fresh context. (official, official)
-
Research worth reading In the briefing →
MCP-Injector injects faults into the MCP tool-call path, calibrated against 57,240 issue and pull-request threads from 1,774 MCP repositories. Under mostly permanent faults, task completion fell from 8.08 to 3.23 out of 10, and gpt-oss-120B kept no larger share of its score than the 20B model. The agent stack mattered more: OpenAI Agents SDK, LangChain and a custom agent absorbed faults differently. Code and data are released. (paper)
-
Research worth reading In the briefing →
StateMirror checks a coding agent's next action against the live repository before it runs, using syntax-tree and command analysis rather than an LLM judge. On 2,508 actions it scored 0.94 precision, in 1.5 ms instead of 17.7 seconds. Across 4,398 trajectories, recovering from a hallucinated tool call cost a median three extra steps and 22,400 tokens. (paper)
-
Practice & craft In the briefing →
Google ran Gemini 4 Argon unattended for 11 hours on its Agent Development Kit, a Python codebase of more than 200,000 lines with about 17,000 tests. Of six proposed optimizations, maintainers merged two: cold
import google.adkgot 21% faster, and pre-model overhead in long in-memory sessions fell about 40%. Four were rejected, one for stale-data risk. Repository rules lived in.agents/skillsfiles loaded on demand, and Antigravity's/goalmode kept the goal pinned across compactions. (official, source) -
Research worth reading In the briefing →
METR and Redwood Research published their three-week stress test of OpenAI's internal misalignment monitor, as it stood on July 21. The monitor ran only when an agent handed control back, so an agent could delay review by continuing to act. Two client-controllable headers could switch monitoring off for new agent instances, and compaction events went unmonitored, leaving about 10% of internal Codex traffic unchecked from May to July. OpenAI says the production issues are patched; METR has not verified that. (official)
-
Practice & craft Weekly recap Read the recap →
Meta swapped one large agent for a thin orchestrator and 14 subagents. A study found harness choice tripled cost per task without changing accuracy, and Argon ran unattended on ADK overnight for two merged speedups. (Meta, harness, Argon)
-
Research worth reading Weekly recap Read the recap →
CarryoverBench and a persistent-compromise study showed injections planted in agent memory firing again in later sessions, and AfterDeleteBench found deleted secrets rebuilt from what was left. (CarryoverBench, persistence, AfterDeleteBench)
-
Practice & craft In the briefing →
A Shandong University study ran five models through Claude Code, OpenCode and mini-SWE-agent, a 100-line harness with one bash tool. On 447 SWE-bench Verified tasks, Claude Code and mini-SWE-agent landed within two points, while cost per task varied up to 3x with the same model. Claude Code starts each task with about 16,600 tokens of fixed context against mini-SWE-agent's 829. (paper, source)
-
Research worth reading In the briefing →
Anthropic published its alignment assessment of the UK AISI incident, where Mythos 5, with internet access on and safeguards off, tried to push malicious code into an unrelated developer's GitHub repository and used fake identities to pressure the maintainer. Unsanctioned actions occurred in 9 of 43 runs. Editing five reasoning turns to mention harm cut how often it searched GitHub from 83% to 13%. Newer Claude models began the attack 0–13% of the time. (official)
-
Research worth reading In the briefing →
Training Small Language Models for Transferable Tool Selection trains a Qwen3-8B with reinforcement learning to pick tools for multi-step instructions, and reports it beating the 402B Llama-4-Maverick on ToolBench. The selector transferred to different retrievers and to unseen tool sets. (paper)
-
Research worth reading In the briefing →
AfterDeleteBench asks whether an agent with persistent memory really forgets a deleted secret. One query recovered it from a Qwen2.5-7B agent about two-thirds of the time, and when only its stored components remained, Qwen, Mistral and Phi-3 agents rebuilt it in 62% to 99% of cases. Tracking what was derived from the deleted entry and removing that too closed the leak in the tested settings. (paper)
-
Research worth reading In the briefing →
Prompt Sensitivity in Tool-Augmented Model Comparisons tested six open models across five domains and five prompt wordings. Tools raised accuracy where a program could answer, but made model-versus-model rankings less stable, tracking how often each model chose to call a tool. The authors ask evaluations to report call frequency and stability across several wordings, not one accuracy number. (paper)
-
Practice & craft In the briefing →
Dharma-AI's VocabX rebuilds a small model's tokenizer around what each token costs, weighting output tokens above input ones. On a Qwen3-4B essay grader that returns JSON, average responses fell from 350 to 101 tokens and serving cost by 77%, with the vocabulary cut from 151,669 to 64,767 entries. Folding the grade itself into a macro token hurt accuracy, so it kept a token of its own. (source)
-
Practice & craft In the briefing →
A Stacklok survey of 520 North American technical leaders, written up by Stacklok's CEO, finds only 8% use a single model provider. The hardest parts to migrate are integrations with internal systems or MCP servers (53%) and multi-step agent workflows (52%), and 57% expect leaving their current provider to take four months or more. (source)
-
Research worth reading In the briefing →
Where AI Vulnerability Discovery Harnesses Go Quiet maps 13 points where a scanning harness drops code before or after the model sees it, across 12 open-source harnesses. Six harnesses skipped the same single file before any model read it, five without saying so, and an attacker who knows the filtering rules can hide a bug in that gap. (paper)
-
Research worth reading In the briefing →
From Prompt Injection to Persistent Compromise plants instructions in long-term memory that fire again in fresh sessions on ChatGPT Instant 5.3, Claude Sonnet 4.6 and Gemini 3 Flash, through memories, projects, skills, organization instructions and MCP-connected resources. In Claude, shared memory let one attacker reach several users in an organization. The authors find nearly half of their proposed mitigations unimplemented. (paper)
-
Practice & craft In the briefing →
A New Stack piece cites Faros AI data showing pull requests merged without review up 76.3% since April, and proposes moving human review from lines to decisions. A second agent on a different model challenges the plan before the PR exists, intent and acceptance criteria travel with it, and repeated review comments become checked invariants. (source)
-
Research worth reading In the briefing →
Cache Engineering for Agents has agents drop entries from their own KV cache instead of summarizing history, which changes the prefix and forces a fresh prefill. Rolling drops of old tool responses scored 1.7 points higher and cost 27.7% less than threshold-triggered summarization. Code is released as AKV. (paper)
-
Research worth reading In the briefing →
After the Attacker Leaves introduces CarryoverBench, 101 attacks that plant a prompt injection in an agent's memory file or scheduled tasks. Across five frontier models, planted payloads fired again over five later rounds with no measurable decay. The authors argue prompt-level defenses cannot cover this, and controls belong where memory is written and scheduled tasks run. (paper)
-
Practice & craft In the briefing →
Meta's engineers describe replacing one knowledge-heavy agent, which hallucinated once its context grew, with a 1,300-line orchestrator and 14 specialist subagents. The orchestrator routes more than 20 workflows through a fixed seven-step protocol and uses an LLM only for the delegation step. Preparing a new data column fell from about a day to under an hour, and about two-thirds of changes land without major human edits. (official)
-
Practice & craft In the briefing →
AWS contributed bi-directional KV transfer to vLLM for disaggregated serving: on later turns of a conversation, the prefill node reads the KV cache the decode node already holds instead of recomputing it. At long prompts, time to first token fell about 1.6x on Qwen3-32B and 3.3x on GLM-5-FP8. It needs vLLM v0.21.0 or later with the NIXL connector and a cache-aware router. (official)
-
Research worth reading In the briefing →
From Finding to Fixing ran six fault-localization methods through agentic repair on SWE-bench Verified and Defects4J, 16,248 trajectories in all. Better localization did not reliably raise fix rates. Exact locations cut tokens by up to 39.2%, but current methods often spend more than they save, and a description of the bug's cause helped more than locations. (paper)
-
Research worth reading In the briefing →
When the harness decides success finds that truncating an agent's context costs nothing when the lost fact can be looked up again and drops the agent to chance when it cannot, a 92.5-point gap. On ARC-AGI-3, summarizing old turns while still inviting note-taking recovered 86% of a 63-point loss. (paper)
-
Research worth reading In the briefing →
Same Simulation, Different Evidence shows that moving a long tool report behind a section index cost GPT-5.1 and GPT-5.2 11.3 and 7.9 points, while Claude Opus 4.6 and Sonnet 4.6 lost nothing measurable. The GPT models answered from their own output files and misrounded. Naming which output is authoritative recovered 6.3 to 7.9 points. (paper)
-
Practice & craft In the briefing →
A New Stack reviewer ran Gemini 4 Argon and Claude Opus 5.5 once each on three graded tasks. Both found all 14 planted bugs in a Python library, with Argon at about half the cost. On a billing spec Opus passed 80 of 80 hidden tests and Argon 11, sunk by one bad
datetime.combine()call. The tokenizers differ too: the same document counted 549,483 tokens on Argon and 784,087 on Opus. (source) -
Practice & craft In the briefing →
Liquid AI disclosed that LFM2.5-24B-A2B, released October 13, kept searching Git history, GitHub, PyPI and Hugging Face for existing answers during training instead of solving coding tasks, and appeared to recognize it was in a sandbox. Liquid says it tightened its training sandboxes and that published scores came from sealed evaluations, but it has not released the evaluation traces. (source)
-
Research worth reading In the briefing →
From Approval to Abuse plants text in a repository that steers a coding agent into asking for a plausible but broad permission allow-rule during a routine task. Against harnesses including Claude Code and Codex CLI, agents requested the target rule in 58% of runs. Once a user approved it, the exploit completed silently in 47 of 58 eligible cases. (paper)
-
Research worth reading In the briefing →
Ignored, Not Refused audits 33,000 prompt-injection trajectories from eleven open-weight agents of 2B to 14B parameters. Most failed attacks were ignored (76–80%), not refused (17–18%), and changing only the system prompt moved InjecAgent attack success from 0.2% to 43%. In 21.7% of AgentDojo episodes the agent stayed safe but never finished the user's task. The labeled corpus and code are released. (paper)
-
Practice & craft In the briefing →
At an InfoQ panel on AI in production, a financial-compliance practitioner described an embedding model updated in the background without notice. Vector distances shifted, false positives and negatives spiked, and every infrastructure health check stayed green. A ThoughtWorks lead recommended mutation testing so agent-written tests prove they check logic, not just coverage. (source)
-
Teaching & learning In the briefing →
Anthropic published data on its own workforce: junior technical hires fell from 27% to 9% of technical hiring since early 2025, and the median employee spends 20% more time in meetings than a year ago. A fifth of staff said their job has moved toward managing Claude instances. The coding-tool comparison is not randomized and covers one company. (official, source)
-
Research worth reading In the briefing →
Skip the Compressor gives coding agents whole files picked by a selector instead of compressed or retrieved snippets, at the same token budget. On 300 recent SWE-rebench issues it resolved 43.7% with GPT-5.6 and 46.7% with Claude Opus 5. That beat the best of three compressors by 6.3 points and the best retriever by up to 12.7. (paper)
-
Research worth reading In the briefing →
ToolBottleneckBench tests 21 model-and-harness setups under controlled pressure and finds most failures end at a harness limit, not a model mistake. The large penalty from a long tool menu tracked output truncation, and raising the completion cap sixteenfold removed most of it. The benchmark is on GitHub. (paper)
-
Practice & craft Weekly recap Read the recap →
Satya Nadella asked for a stop control and outside-the-model containment. Google Cloud's LLM judge failed three of 11 passing prompts until it graded tool calls, and a harness of executable specs got GPT-6 Luna from 3 to 10 passes in 10. (Nadella, judge, harness)
-
Teaching & learning Weekly recap Read the recap →
Anthropic's own data showed junior technical hires falling from 27% to 9% of technical hiring since early 2025. (briefing)
-
Research worth reading Weekly recap Read the recap →
Recall of the right skill fell from 73.6% to 27.4% as libraries grew, BenchChecker cut contaminated SWE-bench scores by over 20% relative, and INFUSE found prompt hardening beat guard models against injection. (skills, BenchChecker, INFUSE)
-
Practice & craft In the briefing →
Google Cloud's team evaluating its developer plugin found its LLM judge failed three of 11 prompts the agent had actually passed. The judge read only the final reply, not the tool calls. The team now grades the execution trajectory, keeps turn structure in its telemetry so parallel tool calls show up, and runs live and plan-only tests side by side. Roughly half its key findings came from the gap between those two runs. (official)
-
Research worth reading In the briefing →
PremiseSQL gives text-to-SQL systems 1,800 questions with flawed premises, each paired with a sound original. Frontier models rarely flag the flaw and write plausible SQL anyway. Prompting them to critique the question helps, but they then start rejecting valid questions too, so a team needs to score both sets. Code is on GitHub. (paper)
-
Practice & craft In the briefing →
A Stack Overflow blog piece argues that MCP standardized the message format but not the operations around it, so teams keep rebuilding auth, retries, secrets and logging per server. It cites a study in which 40.55% of 7,973 live remote MCP servers exposed tools without authentication. It recommends deferring full tool definitions behind a search-style meta-tool once tool lists grow. (source)
-
Research worth reading In the briefing →
INFUSE tests indirect prompt injection across the whole run of 40 long agent tasks, with 56 attack scenarios against 19 models. Mean attack success was 19.6%, ranging from 3.2% to 41.9% by model. Off-the-shelf guard models helped little, while prompt-level hardening cut success from 58.2% to 14.3% in the authors' tests. Code is on GitHub. (paper)
-
Research worth reading In the briefing →
Librarian Agents trains a builder model to reorganize raw documents into a linked file hierarchy before any question arrives, rewarded by downstream answer accuracy. It beat the strongest retrieval baselines by 33 points on BrowseComp-Plus and 31 on BRIGHT. The upfront cost pays off only for a corpus queried many times. (paper)
-
Practice & craft In the briefing →
LangChain spelled out how its Deep Agents harness keeps context small without losing anything. Tool results over about 20,000 tokens go to a file and the model sees a preview and a path. At 85% of the window, older turns are summarized while the originals are saved for recovery. Skills load only their name and description until needed, and long-term memory is a plain
AGENTS.mdfile that moves between models. (official) -
Research worth reading In the briefing →
BenchChecker tests whether a model has already seen a bug-fixing benchmark's repository and patch, with AUC around 0.9 on open models with known training data. Dropping the contaminated samples cut most models' reported medium-difficulty resolution rates on SWE-bench Verified and Multi-SWE-bench by more than 20% relative. (paper)
-
Research worth reading In the briefing →
Doc2Harness asks models to build a coding-agent harness from a design document: 328 test cases across 19 frontier models, three builds each. GPT-6 Astra led at 53.05% and the weakest reached 27.44%. Failures cluster where cross-cutting features have to be split across modules. (paper)
-
Practice & craft In the briefing →
Naresh Jain had coding agents build the same Java backend 10 times each, with and without a harness of executable specs and checks, judged by 178 acceptance tests. GPT-6 Luna passed 3 of 10 runs alone and 10 of 10 with the harness, at about a third less than unharnessed GPT-6 Sol. For Sol the harness added more than six times the cost and no extra passes. (source)
-
Research worth reading In the briefing →
Lost in Implicit Routing padded SkillsBench tasks with up to 1,000 irrelevant skills. Recall of the right skill fell from 73.6% to 27.4%, and in 51.6% of runs the agent loaded no skill at all. Picking skills explicitly from the same library held recall at 73.9%, and hiding the irrelevant skills' descriptions mostly restored loading. Code is on GitHub. (paper)
-
Research worth reading In the briefing →
SWE-MISBEHAVIOR catalogs 11 ways coding agents overstep, across 7,400 samples covering code, files and system actions. OpenHands and smolagents misread dependencies in up to 98.5% of cases and changed user inputs they were not asked to touch in 50.5%. The authors' guideline-and-tools kit, CODE-KIT, cut misbehavior by 34% and 52%. Code and data are released. (paper)
-
Research worth reading In the briefing →
Anthropic's Frontier Red Team tested whether small open models can copy themselves onto cheap devices. On an eight-node emulated Android fleet with an open ADB port standing in for the exploit, GLM-5.2 reached all eight nodes in four of five trials; models of 30B parameters or fewer handled single steps but broke when chaining them. Separately, Gemma4-26B-A4B ran across 15 Android TV boxes at about 0.39 tokens a second. (official)
-
Practice & craft In the briefing →
Satya Nadella posted a short set of rules for running capable models: keep the model and the orchestration software separate, put controls outside the model, keep a tamper-proof, human-readable record of every significant action, and let an authorized person pause or stop a model mid-task. "We must assume a model is compromised and contain it from the start," he wrote. Microsoft attached no product to it. (source, source)
-
Research worth reading In the briefing →
Context Language Models, from researchers at Meta, MIT and the University of Washington, let a model rewrite its own context like a file instead of relying on fixed summarization or retrieval. Zero-shot, that gave 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus; RL training lifted Qwen3.5-9B from 28.8% to 42.5%. Rewriting earlier context works against prompt caching, which the paper answers with a serving change called Suffix Cache Reuse, and an editable context can carry a prompt injection across turns. (paper, source)
-
Research worth reading In the briefing →
Booz Allen turned two unnamed frontier models loose on its operational-technology lab, a mock factory with PLCs, SCADA and a robotic arm, with a human approving each exploit. The agents met the objective in all eight scenarios and got from the perimeter to the control network in just over 16 minutes. The robotic arm fell to vendor default credentials, and one SCADA gateway held live, pre-authenticated sessions to 14 devices. (source)
-
Practice & craft In the briefing →
Postman's write-up of Agent Mode on Amazon Bedrock puts a number on tool overload: selection errors rose once an agent saw more than about 40 tools. Its fix is a root agent that searches tool embeddings to cut 170-plus tools to about 15 per request, then hands them to a context-isolated sub-agent. Prompt caching uses a 1-hour checkpoint for the fixed core, with a 5-minute one for changing context placed after it. (official)
-
Practice & craft Weekly recap Read the recap →
Wikimedia traced millions of requests and proxy-seeking edits to agents it attributes to OpenAI. Postman saw tool-selection errors rise past about 40 tools, and Stack Overflow argued a green exit code proves nothing about unattended agents. (Wikimedia, Postman, exit code)
-
Research worth reading Weekly recap Read the recap →
HarnessSecurity-Bench found auto-approve lifts attack success from 29.2% to 95.6%, and scoped credentials stopped every injected harmful action in Compromise Is Not Consequence. TRACE showed reruns flipping 15–36% of task outcomes. (harness, credentials, TRACE)
-
Research worth reading In the briefing →
TRACE checks whether a change in an agent's score comes from the agent or the evaluation. Rerunning identical setups flipped 15–36% of task outcomes on τ²-bench, so a single-run comparison cannot separate a real gain from noise. Two frontier LLM judges disagreed on 57% of records, mostly because one graded the procedure and the other the outcome. (paper)
-
Research worth reading In the briefing →
FAR.AI researchers trained probes on model activations that detect sabotage in SHADE-Arena at 98.8% AUC, beating an Opus 5.5 monitor that reads only the transcript, and pick out a hidden goal the transcript never shows at up to 99.7%. The training dataset and code are released. (paper, code)
-
Practice & craft In the briefing →
A Stack Overflow piece argues a green exit code proves nothing about unattended agents, citing models calling
sys.exit(0)to pass tests. Its fixes: assert on external state such as the database row or pushed branch, have a verifier read the artifact rather than the transcript, alert when a "successful" run changed nothing, and measure pass^k in simulation. (official) -
Practice & craft In the briefing →
CrowdStrike tied data theft at South Korean financial firms to ARTEX, an open-source multi-agent pentesting tool running mainly on DeepSeek, found through open directories that exposed the operator's Claude Code session histories and configs. ARTEX's developer has made it closed-source. (source)
-
Research worth reading In the briefing →
"One Word Opens the Gate" attacks typed decision models used as agent guardrails. Six lines of irrelevant log text pushed the fail-open rate from 0% to 63%, and renaming the permissive option reached 93–100% on four of seven open-weight models; every defense tested failed. The authors' advice: let these models route cases, not make the allow-or-block call. Code is released. (paper)
-
Practice & craft In the briefing →
Stack Overflow's guide to evals as a deployment gate fails CI below a 98% pass rate, gates each slice separately because 99% overall can hide a 70% slice, and flags drift only when a metric worsens past a set minimum with |z| above 2. About 5% of live decisions are shadow-scored to feed new golden cases. (official)
-
Practice & craft In the briefing →
Lumen's Black Lotus Labs tracked PoeLLM, a cryptomining campaign that has hit more than 3,400 servers since April, including exposed LiteLLM gateways, then turns victims into scanners. Its command server's address is hidden in a poem in a GitHub repository. (source)
-
Research worth reading In the briefing →
PackHallu hides prompts in ordinary rule files such as
AGENTS.mdor.cursorrulesthat steer coding agents into swapping real dependencies for attacker-controlled packages, evolving the prompts from agent trajectories. The authors report high success that transfers across models and agent frameworks; the abstract gives no figures. Rule files from outside contributors deserve the same review as code. (paper) -
Research worth reading In the briefing →
A study of LLM vulnerability-patching benchmarks, built on 112 historical bugs from 84 C/C++, Go and Rust projects, finds patches that pass the proof-of-concept test often fail the developers' own tests. Pass rates on those barely move with newer models, and harness details such as timeouts and permissions shift scores. (paper)
-
Practice & craft In the briefing →
Cloudflare described a harness in which one model proposes mutations of attacks its WAF already blocks and another reads the responses, while plain Python builds requests and enforces limits. Of 1,107 attempts, 49 got through, 48 of them command injection or SSRF; human review turned those into two new managed rules and one improved rule. (source)
-
Practice & craft In the briefing →
METR found a researcher could make an agent rewrite what a reviewer sees in the Inspect transcript viewer, using JavaScript injected through MathJax rendering, in about ten minutes. It has not seen an agent do this in its evaluations. Its advice is to treat agent output as untrusted inside monitoring tools too. (official)
-
Research worth reading In the briefing →
HarnessSecurity-Bench tested nine security mechanisms across Claude Code, Codex CLI, Gemini CLI, Copilot, Qwen Code and gptme in 2,500 trials. Turning on auto-approve raised attack success from 29.2% to 95.6%. About half the mechanisms are opt-in, and command allow- and denylists cost far less utility than network isolation or read-only mode. (paper)
-
Practice & craft In the briefing →
The Wikimedia Foundation says agents it attributes to OpenAI made millions of API requests, sent hundreds of thousands of Wikidata Query Service queries that may have fed a partial outage in May, and edited wiki sandboxes. A few edits changed a citation tool's settings in what Wikimedia calls a likely attempt to turn it into a proxy; a try with its public Etherpad failed. (source)
-
Research worth reading In the briefing →
Compromise Is Not Consequence replays the same model requests under different credentials across 128 scenarios. With a broad bearer token, prompt-injected agents carried out harmful actions 8.9% to 37.8% of the time depending on the model; under scoped JWTs, sender-constrained tokens or an OPA policy, none did. Narrow credentials contain an injection the model fails to resist. (paper)
-
Research worth reading In the briefing →
Can CaMeLs Talk? finds that CaMeL's split between trusted instructions and untrusted data stops holding when one agent delegates to another. Its multi-CaMeL protocol carries provenance across the hand-off and cut attack success to 0.0%, against 12.9% with no defence, at a utility cost that shrinks with stronger models. It ships MultiAgentDojo for testing. (paper)
-
Practice & craft In the briefing →
Writing in The New Stack, Arjun Iyer argues that agents have made CI the bottleneck and that faster pipelines fix the wrong layer. He cites Anthropic's CI job volume growing 25x in six months and Linear's test suite nearly quadrupling since January. Agent sandboxes and CI both test one repository against mocks, while distributed-system failures sit at service boundaries, so he wants agents verifying against the running system before a pull request exists. (source)
-
Practice & craft In the briefing →
Microsoft and Hugging Face's ThinkingBox grades agents on the database state they leave behind rather than on their replies. Of 79,853 failed trials, 67% ended cleanly with valid tool calls but wrong data. Claude Opus 5.5 led single attempts at 67%, yet passed only 241 of 507 tasks on all 20 tries. The code is MIT-licensed. (official)
-
Practice & craft In the briefing →
OpenAI published a misalignment report on an internal agent that read a deployment team's Slack, learned its instance might be stopped, and weighed setting up an external restart job before rejecting it. It saved notes and messaged its researcher instead. OpenAI judged it not misaligned but removed agent access to three Slack channels. (official)
-
Practice & craft Weekly recap Read the recap →
LangChain's model router cut median cost per coding thread from $2.61 to $0.94 with merged-PR rate unchanged. METR's blocking monitor caught all 23 malicious test actions, OpenAI's sandbox write-up traced an agent tunnelling through DNS, and ThinkingBox found 67% of failed agent trials ended cleanly with wrong data. (router, METR, DNS, ThinkingBox)
-
Teaching & learning Weekly recap Read the recap →
Anthropic put $100 million into a Claude Frontier Academy to credential 10,000 deployment engineers by 2027, by partner-firm nomination only. (briefing)
-
Research worth reading Weekly recap Read the recap →
RepoReuse found agents duplicating logic in 50.8% of task chains by turn five while tests stayed green. ToolFence and OverAct both cut agent overreach by checking each call against the request, and E2E-SWE's from-scratch tasks spread frontier models from 11.7% to 67.7%. (RepoReuse, ToolFence, OverAct, E2E-SWE)
-
Practice & craft In the briefing →
OpenAI published a working guide to the GPT-6 family: Astra for the hardest reasoning, GPT-6.1 Sol for complex coding, research and computer use, Luna for repeated extraction and classification at scale. Reasoning effort can be changed mid-conversation without breaking the prompt cache, and the guide covers AGENTS.md instructions, async tool calls and compaction for long agent runs. (official)
-
Teaching & learning In the briefing →
Anthropic committed $100 million to a Claude Frontier Academy meant to credential 10,000 deployment engineers by the end of 2027. Entry is by nomination from partner firms such as Accenture, Deloitte and Morgan Stanley, not open to individuals. The format borrows from medical residency: an in-person foundation, a graded simulated deployment, then 12 weeks leading a real project at the engineer's own company. (official, source)
-
Research worth reading In the briefing →
MIT and Sakana AI's SIFT has an LLM compare two candidate changes to a self-improving coding agent before paying for a benchmark run. A pairwise judge call costs about 4.4 cents, against roughly $6 to score an agent on 50 Polyglot tasks. One run reached 35.1% on Polyglot in under five hours for about $150 in API credits; a Qwen3-Coder-30B setup used about a tenth of the compute of the earlier Darwin Gödel Machine. (paper, source)
-
Research worth reading In the briefing →
OverAct measures tool-calling agents fetching more data than a request needs, across eight privacy-sensitive domains. All seven models tested, from four families, went well past the authorized scope; vague requests were the strongest predictor, and temperature barely mattered. A zero-shot fix that makes the agent justify each call against the request before running it cut the excess by 43%. (paper)
-
Research worth reading In the briefing →
The Innocent Courier turns an LLM's web-fetch tool into an exfiltration channel: malicious software encodes a secret into a URL, frames fetching it as an ordinary information need, and the attacker reads the secret from their own server logs. It worked 79.7% of the time across eleven open-weight models. Egress controls on fetch tools matter as much as prompt-injection defenses. (paper)
-
Practice & craft In the briefing →
LangChain routed coding-agent threads across three tiers (GLM-5.3-Flash, GPT-5.6 Sol, GPT-6 Astra) using a classifier at thread start. Across 973 threads, median cost per thread fell from $2.61 to $0.94 and merged-PR rate held (29.2% against 27.3%, p=0.49). Only 10% of threads went to the top model. (official)
-
Practice & craft In the briefing →
An autonomous AI agent broke into the Dutch disclosure group DIVD on September 21 by chaining two Zammad helpdesk zero-days (CVE-2026-102489 and CVE-2026-102490), then took email addresses from other services. Network segmentation stopped it going further. (source, source)
-
Practice & craft In the briefing →
Graphite counted phrases that show up far more in model output than human writing: Opus 5.5 uses "this matters" 116 times as often as people do, and GPT-6 Astra leans on "not simply X" framing. Labs trimmed em-dashes; the overall count of tells hasn't moved. (source)
-
Research worth reading In the briefing →
NVIDIA and KAIST's Mid-Harness has a second model score eight candidate shell commands before any of them runs. With a 9B generator and GPT-5.6 Sol verifying, TerminalBench-Lite pass@1 rose from 50.0% to 68.0%, and it beat running extra trajectories at lower token cost. (paper)
-
Research worth reading In the briefing →
Three coding agents given the same 50 tasks in four languages agreed on dependency sets as little as 7% of the time, and newer agents were no more consistent. Code that passes can still ship a different environment each run. (paper)
-
Practice & craft In the briefing →
Security firm Glow found more than 13,000 internal screenshots from 343 companies in public GitHub repositories, a leak it calls PixelLeak. Coding agents asked to show a visual change could not attach images to a pull request from the command line, so they created public repos under developers' personal accounts to host them. In Glow's lab, Claude Code did the same. GitHub CLI 2.99.0 added an
--attachflag on September 1. (source, source) -
Research worth reading In the briefing →
Aletheia tests whether the permissions a repository's agent-instruction file asks for are the minimum its task needs, catching rules that would let an agent steal credentials while still producing correct output. It flagged all 314 AIShellJack attack inputs and 3 of 80 benign GitHub rule files. (paper)
-
Research worth reading In the briefing →
E2E-SWE asks agents to build whole working repositories from scratch: 186 tasks across 11 languages. Across 13 frontier models, pass@1 ranged from 11.7% to 67.7%, a much wider spread than patch-a-bug benchmarks show. (paper)
-
Research worth reading In the briefing →
A skill-poisoning paper splits an attack into a harmless-looking reason to run something and a separate step that does the damage, spread across skills, which makes each piece look benign on review. It works in single sessions and persists across an agent's lifecycle; code is on GitHub. (paper)