Alibaba benchmark: coding agents still require human corrections
- AI agentica

A group of researchers from Alibaba and several Asian universities has introduced Co-Design Bench, a new benchmark for evaluating coding agents on editing requests drawn from real sessions. Instead of using synthetic or already saturated exercises, the test asks agents to intervene on existing web applications, including from multimodal inputs such as images, documents, audio, and video. The goal is to measure not only whether code is produced, but whether the final application truly works after changes that reflect problems users encounter in real-world contexts.
The result tempers the optimistic reading of traditional benchmarks. According to the article, the best-performing model among the ten evaluated satisfies on average about 70 percent of a task’s requirements, but fully completes only 26.4 percent of cases. In practice, agents often manage to cover much of the work, but leave incomplete details or errors that require manual intervention. The gap is even more evident when compared with older tests, where agentic models score above 99 percent and the differences between systems are almost invisible.
For enterprise audiences, this data matters for at least two reasons. The first is operational: a good partial outcome does not equal a closed task, especially when a change affects logic, interface, content, and interactions within the same assignment. The study in fact notes that performance drops for all models when requests combine multiple types of change, which is precisely the typical condition in real projects. The second reason is governance: the benchmark evaluates only the functions that truly operate within the application, with a dual check designed to detect simulated results or rigidly coded outputs. For those who must decide on adoption in regulated environments, the distance between “almost correct” and “correct” therefore remains decisive.
Another relevant element concerns the execution environment, not just the model. The researchers observe that the same system can achieve very different results depending on the harness, that is, the program through which the agent reads, modifies, and executes the code. This suggests that rankings based only on the model name describe only part of the problem. In an enterprise evaluation, model, execution tools, verification criteria, and human review should be considered as a single work system.
The news does not show that coding agents are of little use; rather, it shows that their value today lies mainly in accelerating parts of the development cycle, not in fully replacing technical oversight and human responsibility. For organizations that want to introduce agents into software processes, the point is not to chase full automation, but to design workflows with verification, correction, and traceability of changes.
Source: The Decoder, November 8, 2026