Advances in coding and agent capabilities have repeatedly reshaped artificial intelligence model rankings. They have also emerged as some of the fastest routes to commercial value.
At the same time, AI agents are taking on longer and more complex tasks, prompting a growing number of general-purpose foundation models to add native multimodal capabilities.
An enterprise usage report OpenAI released in January found that image uploads ranked third among the ChatGPT tools most commonly used in R&D roles, behind search and data analysis.
As AI agents begin generating websites, operating software, and checking their own work, vision is taking on a broader role. Visual input is no longer limited to images users provide. It is becoming a feedback mechanism that models can use to inspect their output, identify errors, and adjust their actions.
In July, Moonshot AI released Kimi K3, a mixture-of-experts model with 2.8 trillion total parameters and a context window of one million tokens. Alongside its coding and long-horizon agent capabilities, native multimodality has become one of Kimi K3’s defining features.
Native multimodality refers to training images, text, and other forms of data together so they jointly shape the model during pretraining or continual pretraining. These capabilities are then refined during post-training and incorporated into an agent’s perception and decision-making.
After its release, Kimi K3 topped Arena’s Frontend leaderboard with a score of 1,679. The leaderboard ranks models through blind user comparisons of the interactive websites they generate. It evaluates not only whether the code works, but also how the finished page looks.
Browser development platform Puter tested Kimi K3 by deliberately introducing five visual discrepancies into a website. By comparing screenshots of the target page with the rendered version, Kimi K3 identified all five discrepancies without producing a false positive.
Its multimodal capabilities allowed it to recognize visual problems that emerged only after the code ran, providing a basis for another round of revisions. Moonshot AI calls this iterative process between code and screenshots “vision in the loop.” After writing code, the model views the resulting page and continues making adjustments based on what it sees.
That process depends on the model’s ability to interpret visual information.
Moonshot AI is not alone. Alibaba Group and ByteDance are also pursuing native multimodality, treating vision as a foundational capability of general-purpose models. Alibaba’s Qwen3.8-Max and ByteDance’s Doubao-Seed-2.1 both present visual understanding and multimodal input as major features.
By contrast, the latest general-purpose foundation models from DeepSeek, Z.ai, and Tencent‘s Hunyuan remain primarily text-based.
DeepSeek founder Liang Wenfeng has said that training AI effectively does not require a world model or even multimodality. Yet he has also said multimodality is something that ultimately must be developed.
The two positions are not necessarily contradictory. Model developers broadly agree on the long-term value of multimodality. Their disagreement concerns timing and cost.
As coding capabilities improve rapidly, decisions about whether to train vision alongside language, and how much model capacity, data, and computing power to allocate to it, are shaping the technical strategies of China’s leading model developers.
Why AI agents need eyes
Is native multimodality necessary? As agents take on increasingly long sequences of tasks, the question is becoming more concrete.
“A lot of the time, I simply prefer to upload a screenshot,” said a researcher from a leading foundation model team. “Text might be able to achieve the same result, but you would have to spend a lot of context describing the visual relationships.”
Visual input can make many tasks more convenient, although users do not always perceive its value in the same way.
“Most people may not notice the difference, but multimodality is very convenient for developers,” the researcher said.
The demand is not limited to software development. The researcher said people around him who do not work in AI most commonly use models to create PowerPoint presentations.
“Professionals may need more capabilities and more modalities, but ordinary users will also use them in specific scenarios,” he said.
A text-only model cannot directly interpret visual input. In practice, a system can use optical character recognition, or OCR, or a separate vision-language model to convert an image into text, labels, coordinates, or structured fields. It can then pass that information to a text model for reasoning.
These tools can be connected through an agent framework or Model Context Protocol, or MCP, allowing visual interpretation to function as an external component.
From this perspective, giving an agent access to visual input is not only an R&D challenge. It is also a product design decision.
A well-tuned system can use a large language model, or LLM, as its central controller and delegate visual interpretation to a smaller downstream model. This architecture is sufficient for many routine tasks.
But the multimodal researcher said the modular approach has limits.
“If you call a tool, you are still dealing with two models,” he said. “One LLM is ‘blind,’ while a smaller sidekick downstream can see clearly.”
In a natively multimodal model, he said, the communication channel between visual input and the language backbone has greater bandwidth. The system does not need to compress a scene into text before passing it to another model for judgment.
The difference also extends to post-training.
“If the model can do it itself, that is certainly better,” he said.
During vision-based reinforcement learning, a natively multimodal model can examine a page or image it has generated and incorporate the visual result into the same training trajectory. When an external tool is used, the perceptual result must still be passed between two models.
The distinction becomes more apparent during long-horizon tasks that require a model to inspect a screen repeatedly. An external vision tool must first convert the screen into text and then pass it to the main model for judgment. A natively multimodal model can interpret changes on the page directly and decide what to do next.
In an interview with 36Kr, one researcher recalled the first time he used GPT-5.6 Sol to operate a PC-based agent. The experience, he said, was “stunning.”
GPT-5.6 Sol could open a browser automatically, handle authentication and permissions, push local code to a platform, log in to a training platform, and launch a task.
“Without native multimodality, the model has no eyes, and its feedback is limited to text,” the multimodal researcher said. “But a great deal of user-facing output is visual, including websites, images, and videos. The model needs to understand those results and then use them to provide feedback to itself.”
There is still no consensus on whether vision is simply a tool that helps models complete tasks or an indispensable part of understanding the world.
OpenAI co-founder and former chief scientist Ilya Sutskever once said during a fireside chat with Nvidia CEO Jensen Huang that text is “a projection of the world.” Humans have compressed a vast number of real-world rules into language, suggesting that a model that continues learning from text may still develop an understanding of the world.
Turing Award winner and Meta chief AI scientist Yann LeCun has reached nearly the opposite conclusion.
“The vast majority of human knowledge is not expressed in text,” he said.
Language contains information that humans have already filtered and summarized. For models to understand objects, space, and action, they may ultimately need to learn from images, videos, and interactions with the physical world.
Text is highly condensed, while images and videos are closer to raw signals. Those signals contain aspects of the world that may never have been recorded in text. They also require more data, computing power, and training before they can be distilled into knowledge that generalizes.
“Multimodality is certainly not decisive, but it is important,” an AI model researcher said. “Basic image- and video-based question answering is already fairly saturated. The next step should be to bring multimodality into real production scenarios and complex tasks.”
Expectations for flagship models are also changing as agent tools reach more mainstream users.
In late June, Z.ai chief scientist Jie Tang asked users on X which features the next version of GLM should include. One answer appeared repeatedly in the comments: vision.
Several sources previously told 36Kr that the next generation of general-purpose foundation models from Z.ai and Tencent’s Hunyuan could move further toward native multimodality.
The direction is becoming clearer. Yet foundation model developers still face a resource allocation problem when deciding whether to commit fully to native multimodality.
The cost of giving AI models eyes
“For an LLM, multimodality is more like an add-on,” said a researcher at a leading foundation model company. “The core of model development is not multimodality. It is the ability to complete tasks.”
From the standpoint of task completion, seeing more does not necessarily mean a model can do more. A more immediate concern is that adding multimodal capabilities could weaken an LLM’s existing language, reasoning, and coding performance.
“The more tasks there are, the harder they are to balance,” a multimodal researcher said. “If you are doing only one task, you can keep optimizing in one direction. If you are doing two tasks at the same time, vision and text, you either need a larger model or more data.”
Adding vision is not as simple as adding another input interface. Images and text differ in structure, information density, and learning speed. When both are used to train the same backbone parameters, visual data competes with text, code, mathematics, and reasoning for model capacity. Their training objectives can also interfere with one another.
The Kimi K2.5 technical report documented this effect in an ablation study examining when modalities should be combined. When visual data was introduced during the middle or later stages of training, the model’s text capabilities initially declined before gradually recovering.
“If you want to build a unified, natively multimodal model, the resources required will definitely amount to more than one plus one,” the researcher said. “You cannot simply add together the training volume and parameters of two separate models and expect to obtain the same result.”
Beyond the difficulty of training, native multimodality expands the range of possible applications while increasing computing requirements.
Apple’s technical report on its MM1 multimodal large model found that the vision encoder, image resolution, and number of visual tokens all affect performance. Higher resolutions and greater numbers of visual tokens generally increase both training and inference costs.
For tasks that require reading only a few fields, having a general-purpose model inspect an entire image repeatedly may be less cost-effective than using OCR or a specialized model.
Kimi K3 is the clearest current example of the resource-intensive approach. Designed to provide vision, coding, and agent capabilities in one model, it has 2.8 trillion total parameters and activates about 104 billion parameters for each token.
Kimi K3 follows an early-fusion training approach similar to the one used for Kimi K2.5. Experiments with Kimi K2.5 showed that when the total number of image and text tokens remained fixed, introducing vision earlier and at a lower proportion generally produced better results than adding it during later stages of training.
The final approach continuously mixed text and visual data at a fixed ratio during joint pretraining on about 15 trillion image and text tokens.
Kimi K3 also trained its roughly 400 million-parameter MoonViT-V2 vision encoder from scratch instead of initializing it with weights from SigLIP, a contrastive image-text pretraining model. It fed visual representations directly into the next-token prediction objective.
According to the technical report, the approach improved training stability while preserving visual performance.
“Every time you add more multimodal data, text comprehension, coding, or mathematics may deteriorate, so you then have to keep adjusting the ratios,” a researcher said. “It tests a foundation model team’s ability to integrate everything into a single model.”
A few weeks after Kimi K3’s release, DeepSeek presented a different strategy.
The official version of DeepSeek V4-Flash has 284 billion total parameters and activates 13 billion parameters for each token, about one-tenth and one-eighth of Kimi K3’s respective figures. DeepSeek left the architecture and parameter scale unchanged, focusing the update on post-training.
Across the coding and agent benchmarks published by DeepSeek, the official version significantly outperformed the preview release. It also surpassed the previous preview version of V4-Pro.
On Arena’s overall WebDev leaderboard, which evaluates website development and multiturn tool use, the model’s public score rose to 1,577 at one point, close to GLM-5.2. Only a handful of models ranked higher, including Kimi K3, Claude Fable 5, and GPT-5.6 Sol.
DeepSeek V4-Flash showed that post-training can still produce substantial improvements in core capabilities such as coding without adding vision or significantly increasing the model’s parameter count.
As long as this approach continues to deliver rapid gains, foundation model companies are unlikely to reach a common answer on how many resources they should allocate to vision in advance.
Alibaba’s newly released Qwen3.8-Max has taken a broader, all-in-one direction similar to Kimi K3. It has 2.4 trillion total parameters and activates 95 billion parameters for each token while supporting native vision, coding, and Cowork capabilities.
The strategies chosen by foundation model developers are not determined by technical findings alone.
“In the end, the decision over how large a model should be and which selling points it should emphasize may come down to just a few people, or simply the boss,” an industry source said.
The research backgrounds of core team members, intended product use cases, and training costs can all shape a company’s priorities.
The talent market is also responding. After Kimi K3’s release, 36Kr learned that more headhunters had begun appearing in cafes and restaurants near Moonshot AI’s offices. “Multimodality” had reportedly become the focus of their visits.
That does not mean every company will immediately combine extremely large parameter counts with native multimodality. Nor does it mean multimodality will replace coding as the main field of competition.
Several sources said coding remains the price of admission. A company whose models cannot reach the leading tier in coding may have little opportunity to compete in the next stage of the market.
For China’s foundation model developers, the divide is not rooted in conflicting views about the long-term path toward artificial general intelligence. It reflects different assessments of timing and cost.
For now, competitive rankings, product development, and commercialization remain more heavily concentrated on coding and agent capabilities.
The competition is governed by two clocks.
Coding and agent rankings can change within months or even weeks. The transformation of models from language systems into systems capable of understanding the world will take much longer.
The first clock determines which companies can keep pace today. The second may ultimately determine how far they can go.
KrASIA features translated and adapted content that was originally published by 36Kr. This article was written by Li Zhaofeng for 36Kr.