FB Pixel no scriptMultimodal AI breakthrough could come within two years, SenseTime scientist says
MENU
KrASIA
Features

Multimodal AI breakthrough could come within two years, SenseTime scientist says

Written by Cheng Zi Published on   16 mins read

Share
Photo source: Dreamstime (Sidney De Almeida, ID: 276160626).
SenseTime sees visual understanding and controllable generation as steps toward AI that can reason and act across physical environments.

“There were Einsteins and Newtons before computers existed. We don’t think they were less intelligent just because they couldn’t code.”

SenseTime co-founder and chief scientist Lin Dahua used the analogy in an interview with 36Kr to explain why coding ability does not define the limits of intelligence. It also helps explain the company’s long-term commitment to native multimodal artificial intelligence, which processes different forms of information within a shared model.

Coding agents are among the first AI applications to demonstrate a viable commercial model. Public information indicates that Anthropic’s annualized revenue exceeded USD 65 billion by the end of July, about seven times its level at the end of 2025. When might multimodal AI deliver similarly clear gains in capability and commercial results? That question recurred throughout the conversation. Over the longer term, Lin believes “the potential of AI and AGI extends far beyond coding.”

DeepSeek founder Liang Wenfeng has said the company will eventually work on multimodal AI but views it as a “component,” rather than intelligence itself, in the pursuit of AGI, or artificial general intelligence.

Around the same time, Nvidia founder and CEO Jensen Huang unveiled Cosmos 3, a world foundation model for robotics and autonomous driving. It combines visual reasoning, environment simulation, and action prediction in one architecture. Huang said Cosmos 3 can “understand and reason about the physical world, generate it, simulate it in the loop, and even be the policy itself.”

Lin does not see this as a question of who is right or wrong. Different companies face different tasks, resources, and long-term visions, he said, so their choices will naturally differ.

SenseTime’s roots in computer vision have shaped its move into native multimodal AI and spatial intelligence. Multimodality runs through both its model development and its product strategy.

Its SenseNova 6.8 series comprises multimodal agent models that plan and execute extended tasks and coordinate multiple agents. The U series unifies visual understanding, generation, and editing, with capabilities extending into video and 3D. Its applications include Seko, an AI agent that links storytelling, storyboarding, and video generation to create short films, and Office Raccoon, an agent for office work and data analysis.

In April, SenseTime released and open-sourced SenseNova U1, a native model that unifies understanding and generation. It removes the separate vision encoder and variational autoencoder (VAE), processing language, visual meaning, and pixel generation together in a single autoregressive architecture.

Nearly four months later, SenseTime released U1.5 Lite. It extends the architecture to continuous editing while improving high-resolution generation and the ability to follow complex instructions.

SenseTime reported first-half revenue of RMB 2.91 billion (USD 432.6 million) in its 2026 interim results, up 23.4% year-on-year. Generative AI revenue reached RMB 2.33 billion (USD 346.4 million), nearly 80% of the total.

Driven by growth in its AI business, SenseTime achieved profitability on an IFRS basis for the first time, giving it more room to sustain long-term investment in multimodal AI.

Photo of Lin Dahua, chief scientist at SenseTime.
Lin Dahua, chief scientist at SenseTime. Photo source: SenseTime.

The following transcript has been edited and consolidated for brevity and clarity.

36Kr: Liang Wenfeng sees multimodal AI as a “component,” similar to search, rather than the main path toward raising the ceiling of intelligence. You, however, have argued that once multimodal capabilities are fully integrated, a standalone language model will no longer be necessary. Where does the disagreement come from?

Lin Dahua (LD): I actually have a lot of respect for DeepSeek. Every company needs a primary focus. DeepSeek wants its language and coding capabilities to rank among the best in the industry while making its model architecture as efficient as possible. From its position, that choice is entirely reasonable. When it needs to process images, it can connect a multimodal component, so at least the product’s functionality is complete.

SenseTime faces somewhat different tasks. Office work, healthcare, retail, robotics, and world models all require visual signals. We concluded very early on that jointly modeling vision and language is fundamental to bringing AI into real industries and the physical world.

SenseTime also works on coding. We regard programming ability as an important foundation for models, but we do not treat complex software engineering as our sole primary focus.

At a deeper level, the key question is how we understand the upper limits of intelligence. Strong coding capability is a result of greater intelligence, but coding as a form of work is not its essence. Intelligence is the ability to bring together complex information and carry out reasoning, simulation, execution, and correction over extended tasks.

36Kr: Is native multimodal AI necessary for intelligence that can handle extended tasks to enter the real world?

LD: If a model only writes code, multimodal capability may not be necessary. But once tasks extend beyond coding into broader industries and the real world, many situations require it.

Coding is currently the most mature medium for demonstrating reasoning over extended tasks because the data conditions are so favorable. GitHub contains enormous amounts of data about the development process, code can be executed, test cases can provide deterministic feedback, and a mature software engineering market supports it. That is why coding was the first to establish a complete path from training to commercialization.

But AI will not stop at coding. Even in front-end development, after a model writes the code, it still needs to render the page and see whether it looks good or contains obvious errors. That also requires visual capability.

36Kr: Researchers working on multimodal large language models and native multimodal models have said that adding visual data significantly increases the difficulty of pretraining and post-training. If the data mix is poorly handled, language, math, and reasoning capabilities can even deteriorate. How do you view the training difficulty and cost of native multimodal models?

LD: If you are simply attaching a vision encoder to a language model, the additional computation at inference time is actually negligible compared with a language backbone containing hundreds of billions or even one trillion parameters. The real difficulty is training.

The first issue is the data mix. If you add visual capabilities to a model, you need to add the corresponding data while preventing its existing language, math, and reasoning abilities from deteriorating. That is what I call the “alignment tax.” It is not simply a matter of adjusting the ratio from 1:9 to 2:8. You also need a large number of controlled experiments, and every model upgrade may require the mix to be adjusted again.

The second issue is reinforcement learning. Introducing a new modality makes the training objectives and technical approaches more complicated. How to build feedback signals for vision-related tasks is itself still an open problem. Often, the most expensive part is the time spent exploring methods before the final large-scale training run. For a model company, the issue may go beyond whether it can afford the financial cost. One more experiment can mean releasing a model a month later than a competitor.

That is also why we first iterate with lightweight models containing eight billion parameters. The approach to unified understanding and generation is not yet settled. It is more efficient to stabilize the architecture, data mix, and training method before scaling up.

36Kr: U1 and U1.5 are still focused mainly on image generation. From a user’s perspective, what experiences does this unified architecture, which does not rely on a traditional vision encoder, make possible that specialized models cannot provide?

LD: For users, the most direct change is that the model first understands the content and then decides how to generate it. When given several pieces of complex information, it has to judge what matters, then determine the structure, layout, and visual presentation. Infographics naturally require understanding, reasoning, and generation to be connected, which is why we started with that use case.

The U series also has a longer-term goal: spatial intelligence and physical intelligence. We once considered developing spatial intelligence alongside image generation, but each additional capability significantly increases training complexity. So we temporarily set aside the more complicated spatial intelligence work and let U1 establish an approach to unified understanding and generation first. That allowed us to develop a technical understanding of how language and visual elements can be integrated within the same representation space.

You cannot spend three years working on long-term technology with your head down and deliver no interim results. SenseTime is a listed company. It needs to keep producing results that users can perceive and that create value for the industry. Image generation is the first capability this approach has unlocked, but it is not the endpoint.

36Kr: U1.5 focuses on improving text generation, aesthetics, and continuous editing. How does it change the old image generation process of repeatedly “drawing cards” and hoping for a usable result?

LD: After U1 launched, actual use showed that text generation and local details still needed improvement. From April through June, we kept refining the infographic version and gradually corrected the text-generation issues.

Later, we found that even when all the text was correct, an image could still look like a PowerPoint made by a student, with no sense of design. So we brought in professional designers to teach the model that lesson in aesthetics. Aesthetic judgment and logical ability are two different things. Coding and a sense of beauty are also two different things. No matter how capable a model becomes, aesthetic judgment does not emerge automatically.

Accurate text and attractive visuals still address only the quality of a single generation. Real design work does not end after one generation. A complete set of composition and revision requirements can run to 1,000 or 2,000 Chinese characters. The model has to understand all those constraints while changing only the specified area and leaving everything else as untouched as possible.

The old one-shot generation process was more like drawing cards from a deck. You might generate five images in a row, each with an error in a different place, and end up with none that are usable. Continuous editing allows users to preserve what is already correct and change only what has a problem.

Continuous editing is a foundational capability. It connects the understanding of requirements, content generation, and repeated revisions into a complete process. That is what we call “agentic generation.” The model keeps iterating on the same work until it is ready to deliver.

36Kr: Tencent’s Yao Shunyu has said users are willing to pay for the strongest models and that enterprises assessing value for money need to consider performance first. A more expensive model can sometimes save money by completing a task correctly in one attempt. U1.5 Lite emphasizes value for money, while U1 Pro aims to extend the limits of capability. How does SenseTime view these two types of demand?

LD: A model first has to be useful before price is worth discussing. U1.5 Lite is an open-source model with eight billion parameters that places greater emphasis on efficiency and value for money, making it suitable for many everyday tasks. U1 Pro aims to extend the limits of capability. It costs much more, but the goal is to deliver professional-grade results.

It is a little like the difference between a compact camera and a professional DSLR. Some people are willing to pay more for top-tier results, while others care more about efficiency. The Lite version may have a higher probability of small defects, but those defects are easy to edit, so users do not have to start over and draw another card. The two products serve different needs.

36Kr: Enterprises can use one general-purpose large model for an entire task or have an agent coordinate multiple specialized models. Which approach is more economical in real workflows?

LD: A model first needs a complete set of basic capabilities; different models can then specialize. There is an “impossible triangle” here: breadth of capability, professional depth, and cost. You can certainly build a supermodel that is both broad and deep, but it will inevitably be very large and very expensive.

Companies do not hire a “superhuman” for every position either. In reality, the more practical approach is still to have different specialized models form a team, although each model still needs sound general intelligence of its own.

For SenseTime, our multimodal agent model SenseNova 6.8 is more like a manager. It handles extended tasks and coordinates multiple specialized agents. It also needs “eyes” to understand images and decide what action to take next. The U series is more like a visual specialist, developing very high levels of detailed visual perception, controllable generation, and spatial intelligence. The two have different roles and work together to complete tasks.

36Kr: SenseTime’s AI content creation platform Seko can produce videos and can also connect to specialized models such as Seedance. Compared with a standalone video generation model, what problem is Seko primarily trying to solve?

LD: Seko is meant to deliver a complete video creation workflow, but not every step has to be completed by SenseTime’s own models. Final video rendering can call specialized models such as Seedance. What we focus on is how the story unfolds, how the storyboard is generated, and how the work is adjusted after reviewing the result. That closed loop of creation and iteration, the creative process itself, is what truly reflects the creator’s level of intelligence, and that is what we want to master.

Video quality depends largely on high-quality data and investment in resources. SenseTime focuses more on enabling the model to understand creative intent, determine what needs adjustment, and connect understanding, generation, and revision. That also distinguishes Seko’s role from that of a specialized video generation model.

36Kr: Image generation, video generation, 3D, and spatial intelligence all require sustained investment. When resources are limited, what criteria does SenseTime use to decide which capabilities to prioritize?

LD: We are really focused on two relationships: whether language intelligence can influence visual imagination and generation, and whether visual generation and imagination can, in turn, influence reasoning.

So when allocating resources, we focus on work that can deepen our understanding of those two questions.

Image generation, video, and 3D may look like different directions, but all address those two underlying relationships. That is why we do not try to push every metric to its limit. In video, for example, cinematic image quality will not be our main competitive focus. We will continue working on portrait generation, primarily to ensure the quality of generated images, but it will not become our principal direction.

We chose infographics as a major generation use case mainly because they require real intelligence to organize and communicate information effectively within limited visual space. In office applications, we pay particular attention to data analysis and forecasting because they place very high demands on intelligence and can help extend the limits of capability.

When everyone else is competing intensely on coding, we can work at our own pace. If everyone shifted into spatial intelligence, the pressure on us might be a little greater.

36Kr: How much longer will it take for world models to achieve true generalization?

LD: There are already many things that work in specific scenarios. They just cannot generalize yet. Given a particular environment, today’s vision-language-action models or world action models can become highly specialized. But move them into another environment and they may no longer have even basic judgment.

A true GPT moment requires both breadth and depth. Breadth means understanding many different scenarios. Depth means sustaining reasoning and action over extended tasks. Spatial capabilities also need to be combined with complex logic. Otherwise, if a model suddenly loses the ability to think halfway through a task, commercialization will remain very difficult.

I think we still need another one to two years before we see systems with a certain degree of generalization. Model capabilities will improve little by little. There will not be a single day when everything suddenly switches over.

36Kr: World models and spatial intelligence are widely seen as constrained by data. Is the real shortage in the data itself or in the methods used to work with it?

LD: First, we should distinguish between two concepts. A world model needs to learn about the world and its physical laws. Spatial intelligence is about how people understand space, think within space, and act within it. At least for spatial intelligence, I do not think the world lacks video. There is an enormous amount of public video and first-person video, and companies in our ecosystem are also collecting first-person data. The real problem is that this data has not been fully utilized.

Data also needs to be used in stages. As in language model training, pretraining should absorb material broadly. Mid-training should introduce tasks and reasoning processes from specific domains, and post-training should adapt the model to specific tasks.

At present, people mainly use video to train generative models, teaching them what scenes look like and how the next frame changes. They have not made full use of video to improve intelligence.

For example, in a movie, someone might suddenly look terrified after seeing something. The model should not only see the expression. It should also understand why the person is terrified and how that reaction relates to what is happening. That is understanding.

So there are two things to do. We need to keep collecting video suited to spatial intelligence. We also need to ask the right questions about video and establish relationships between events. The latter has not yet been done effectively at scale.

36Kr: U1.5 can already perform continuous editing on images. How will that approach extend into video and 3D?

LD: From the perspective of spatial intelligence, 3D may actually have a higher priority than video. The essence of spatial intelligence is understanding the 3D world and being able to act and interact within three-dimensional space. The basic principles behind continuous editing and feedback loops are similar, but once you move into video and 3D, the data becomes more complex and new problems emerge.

The main technical approach is already relatively clear. The next step is to solve the specific problems one by one. U2 will also incorporate more spatial intelligence, with stronger modeling and understanding of three-dimensional space. It will not simply turn static images into longer videos.

36Kr: Coding has already produced clear feedback in terms of revenue and valuations, while the payback period for multimodal AI appears longer. How does SenseTime stay committed while still producing results along the way?

LD: SenseTime does work on coding. We simply have not made complex software engineering our main focus. We have always remained firmly committed to our long-term vision and allocated resources to the key priorities needed to pursue it.

Ultimately, the market will judge whether this path can produce customers and revenue. SenseTime needs to prove the commercial viability of multimodal AI through user growth, repeat purchases, and paid usage of products such as Office Raccoon, our AI productivity agent, rather than merely releasing a larger model.

We have discussed internally whether to pursue a model with one trillion parameters. We concluded that we should advance in line with the underlying principles of the technology. At the right time, we will build larger models based on innovative architectures and technologies, rather than simply increase scale to compete with similar products.

We do not want to delay exploration of the real problems just to expand model scale more quickly. Competition among large models is exceptionally intense right now. Whether a model company’s understanding is ahead of the curve and whether it can iterate quickly enough are extremely important, even decisive to success or failure. For us, parameter count is only a specific technical choice. What matters more is moving fast enough in the direction we believe in so that our understanding stays at the forefront.

More specifically, the appropriate parameter count depends on the maturity of the technical approach. During frontier exploration, using too many parameters before the approach has been sufficiently validated can slow iteration. Once it is more mature, you need to scale up to explore the upper limits of capability. Our multimodal agent models have therefore entered the scale-up stage, while unified understanding and generation still presents architectural and training questions that need to be explored.

36Kr: You mentioned that once coding capabilities crossed a certain threshold, people could suddenly feel that the technology was genuinely useful. What are the corresponding thresholds for design and spatial intelligence?

LD: Coding did not suddenly become several orders of magnitude more capable than before. Its capabilities simply accumulated to a point where people started to feel that it was genuinely useful.

The inflection point for design is already close. Model controllability and visual quality are steadily improving. The next step is to enter real workflows, such as directly generating content that can continue to be used in tools such as Figma. Moving from looking good to being useful involves many problems that can be solved with better data and product features. I think the timeline can be measured in months.

The inflection point for spatial and visual reasoning will come later, but its impact will be deeper. Without a “brain” equipped with spatial intelligence, robots can only perform fixed actions in fixed environments. It is very difficult for them to independently complete extended tasks in complex physical spaces.

36Kr: The cost of image and video generation requests continues to fall, but professional customers care more about the quality of the final deliverable. Do you think multimodal AI will enter a price war?

LD: If everyone is at the same capability level, users will naturally choose the cheaper model. But if competition revolves only around tokens, margins will be pushed very low and models really will become utilities.

I agree that AI will become a utility, but utilities are not the most profitable industries of this era. The truly valuable part is what problem you ultimately solve for the user.

Lawyers are not paid based on the number of words in a judgment, and doctors are not paid based on the number of words in a medical record. Users are willing to pay when the lawyer wins the case or the doctor cures the illness. Intelligence is not sold by the pound. Tokens are only a unit of measurement. What ultimately matters is whether the problem you solve is valuable.

In the future, companies could charge by task or by final deliverable. AI and customers could also work together and share the value created by the result. Users do not care whether a model consumes one million or 100 million tokens. They only care whether the job gets done.

36Kr: Would you encourage more companies to invest in native multimodal AI?

LD: I would rather say that people should remain confident in the long-term prospects of multimodal AI. Many industries have a fundamental need for visual capabilities. Demand for multimodal AI is real, but we need to ask the right questions. We cannot remain stuck at simple visual question answering.

As for whether each company or practitioner chooses coding or multimodal AI, neither path is inherently superior. What matters is choosing a direction you genuinely believe in based on your own experience and judgment, rather than simply following the crowd.

36Kr: Over the next one or two years, will multimodal AI reach a critical point similar to coding agents?

LD: At this stage, the significance of coding goes beyond coding itself. It has shown people that AI can genuinely do things and create commercial value, rather than merely write poetry or make pictures. That has given the entire industry a great deal of confidence.

I think multimodal AI will cross a similar threshold within the next one to two years. Its eventual form may differ from today’s products. It may not simply be an image or video generation tool, but its foundation will still be the combination of vision, language, reasoning, and action.

As data and applications continue to mature, multimodal AI will reach a moment when new capabilities begin to emerge. Of course, we hope SenseTime will be among the first to get there.

KrASIA features translated and adapted content that was originally published by 36Kr. This article was written by Xiao Xi for 36Kr.

Note: RMB figures are converted to USD at rates of RMB 6.73 = USD 1 based on estimates as of September 16, 2026, unless otherwise stated. USD conversions are approximate and, where appropriate, rounded for ease of reference. They may not fully match prevailing exchange rates.

Share

Loading...

Loading...