Mesh Refinement

It’s Monday, and so it’s time for the next issue of Chronicles. Last week was very significant because it represented a tectonic shift from using expensive, efficient models to cheap and still very efficient models. In other words, AI is getting democratized.

Let’s start with the hardware. This week we’ve seen interesting news from OpenAI, which published the first benchmarks for their Jalapeño chip, a custom inference chip designed with Broadcom. It’s pretty impressive. They published over 700 tokens per second per user on DeepSeek R1 at concurrency of one, and about 1400 on Kimi K2.5 and GPT-OSS. The variety here matters because the chip has been benchmarked not only on OpenAI’s own models but on open-weight models from other vendors as well. The chip is critical in the ongoing price war with the Chinese providers. Remember that last week we had data on cutting prices on their Sol model by 20% on input and 33% on output, which is a promotional window that expires on 21 November, and a week before that they cut the price of Luna by a whopping 80%.

Last week was all about the harness. Gone are the times when the models were all things-in-themselves. Now, as with early Homo erectus, the tools are the factor of survival. If we look at the types of harness-related posts and announcements, we will see three trends.

Harness products

Several companies have released either agents or infra for agents at once.

OpenAI has released the execution framework that their Codex (the CLI one) is based on. Unsurprisingly called Harness, it is their answer to the Claude Agent SDK and the GitHub Copilot SDK. The Harness provides the execution loop, memory, tools, and other necessities of agentic life. You can use it in three different ways. First, you can just use codex exec to run non-interactive jobs. Second, you can use the Codex SDK for building workflows. And last, you can use app-server to build apps that require conversation handling. One point worthy of attention is that they claim that the Harness improved the performance of the GPT-5.6 Sol model from 13.3% to 38.3% on ARC-AGI-3. It’s unclear what harness (not capitalized) was used as a baseline, though.

Over the last week, we’ve seen some announcements from both the highest-end and lowest-end sides of open-weight models. At the same time, labs are starting to think that maaaybe, just maybe, we move too fast and we need to stop and think. So, in their free time, they are starting price wars.

Models, hi

Last week we saw a bunch of releases of very large open-weight models. Z.ai shipped GLM-5.3, which is relatively small, just 743B, DeepSeek published their 1.6T V4-Pro weights under the MIT license, and Alibaba surprised with a whopping 2.4-trillion-parameter Qwen3.8. Although they are open, doing anything meaningful with models of this size requires hardware not a lot of individuals have, which defeats their openness a little. On the other hand, they open possibilities for 3rd-party hosting and incentivize the price wars I will talk about below.

This article opens a series (I expect it to become one) of posts in which I review the AI-related events of the past week and try to figure out how they fit into the bigger picture.

The main highlights of the past week are:

1. We continue to see real-world security breaches caused by AI system evaluations, and it looks like another race.

2. Major regulations came into force.

3. Data centres face increasing opposition from local communities. Companies seek workarounds.

Lately, there have been numerous alerts about security vulnerabilities connected to indirect prompt injection attacks. The main conclusion is: LLMs are gullible, nobody knows how to make them completely robust. Therefore, no AI system is safe.

It is, of course, completely true, and the attention those attacks attract is most definitely welcome, but the main question is left unanswered: what is to be done about it?

Purists would push for banning all external inputs that could lead to prompt injection, but, frankly, I wouldn’t be so rigid. A lot of genuinely useful applications do rely on external input, and so, I am afraid, we have to retreat to the last resort: engineering discipline.

There are various schemes that use a clever interaction of different models to minimize their exposure to attacks (see, for example, the CaMeL paper). I believe, though, that we don’t pay enough attention to simple input sanitization.

A lot of such attacks rely on text hidden from the human, but visible to the machine. Detecting and removing such text is relatively trivial (in a programmatic sense), although it will require working not on the text but on the container level. This technique (called Content Disarm & Reconstruction, by the way) has been around for quite some time. Honestly, I am surprised that it is not implemented everywhere. It wouldn’t prevent all attacks, but it would make an unsophisticated attacker’s life harder.

And to those who insist that 99% in security is a failing score, I would like to remind them of a fundamental security principle: no system is 100% secure. The role of security is to make an attack more costly than the potential benefit gained from it (see the Gordon–Loeb model).

With this paradigm in mind, we can build reliable and secure AI systems, even with insecure individual components.

Some of the things you need to know about the latest GPT-5 release that evangelists don’t talk about:

1. GPT-5 is not one model. What they call GPT-5 outside of API context is a router that sends your request to a model that it thinks would work most efficiently on it. You need to look at the OpenAI’s promise to provide access to everyone in this light. They provide access to the router, and you don’t know the specific configuration it applies to you and whether you will actually be able to test the most powerful model. That would undoubtedly cause completely different experiences for different users.

2. GPT-5 is not a PhD. It is a pretty capable model (at least, one of them) that excels at some tasks. You can expect improvements in:

   • coding capabilities (they are impressive according to some vibe tests, but according to the SWE-bench Verified benchmark, it has just a minor lead over Claude Opus 4.1);
   • tool calling capabilities, which are the most important for agentic workloads;
   • other tasks where OpenAI had access to immediate feedback.

   While those are important improvements for a lot of areas, they don’t make the model PhD-level. The hallucination about the airfoil during the demo perfectly demonstrates that it still internalizes the most common belief, not the most current. It is a very hard problem to solve and actually a major roadblock on the way to AGI.

3. Diminished hallucinations are a double-edged sword. On the one hand, the less a model hallucinates, the better, as you can trust it more. On the other hand, the more you trust the model, the more likely you are to miss actual hallucinations. In the real world, the model that never hallucinates is the best. The model that hallucinates in 0.01% of cases can be more dangerous than one that hallucinates in 10% of cases.

My personal impression so far is that it still has the same issue as the previous models from OpenAI, namely that it is really superficial without careful prompting. It provides you with the most shallow analysis it can get away with and hides this fact by using the very well-structured responses.

Google Opal

Google has started a public preview for its new tool for graphical creation of multi-step AI workflows. While not a tool for production use, it is a great helper for building personal tools (and everybody should build personal tools, really, it is the biggest differentiator now).

It works on the Gemini platform with Gemini models (well, of course), and since it is a preview tool, it is free for now. In the future it will most likely use the Gemini Plan if the user has it. I am not sure about custom API keys, as this looks like a general public tool, but we’ll see.

Right now it is available only in the US.

X is buzzing with a new Horizon Alpha model that is beating all previous models on various vibe tests (read: unicorns on bicycles and so on) singlehandedly. This model has 256k context window, which is a solid, albeit not the most impressive, number.

Most probably it is a new OpenAI model (GPT-5?), as they already did the same trick before. You can try it on openrouter.ai completely for free, but remember not to provide it with any private data, as it is collected and used for model improvement.

Claude Code Sub Agent

Anthropic has added the ability to create and use specialized sub-agents in Claude Code. These sub-agents use a separate context window, which allows you to run separate tasks without polluting the main context, limiting the context rot effect. You can run the created sub-agents manually or let Claude Code decide when to use them.

What can be a good sub-agent? Anything that needs to be an expert in its area, doesn’t need to share the context with the main agent, can have dedicated tools, and can be designed to run self-sufficient tasks. To give you a taste of what can be made into a sub-agent, here are a couple of examples:

1. Git sub-agent to which you can offload various git-related operations. As it can use just the local git state, and doesn’t need to have access to the global context, it is a good choice for a tool with its own context window.
2. A book-writing sub-agent (I know, but I create book-like documents in a specific style for self-education). You provide it with a high-level plan, a set of materials, and one or two sections as examples, and let it work on a section separately from other sub-agents.

The second example would really benefit from the ability to run several sub-agents in parallel, but it looks like I’m asking too much.

Quoting Arvind Narayanan:

If we compared AI capabilities against humans with no access to tools, such as the internet, we would probably find that AI already outperformed humans at many or most cognitive tasks we perform at work. But of course this is not a helpful comparison and doesn’t tell us much about AI’s economic impacts. We are nothing without our tools.