A child points a phone at a blackboard and asks, out loud, what the scribbled equation means. The model hears the question, reads the board, and answers in a calm voice — no app switch, no upload step, no separate transcription service. A year ago that pipeline needed three models and a human to glue them together. Now it is one download.
The quiet merger of hearing, seeing, and speaking inside a single Chinese large model (大模型) is one of the most practical shifts in the field, and almost nobody outside the developer world has noticed.
From pipelines to a single brain
Early multimodal AI was a Frankenstein. A vision model read the image, a speech model heard the audio, a language model wrote the reply, and an engineer wired them in sequence. Each handoff lost context and added latency. The new generation of Chinese models deletes the seams.
Alibaba's Qwen2.5-Omni is the clearest example. It is an end-to-end model that takes text, images, audio, and video as inputs and produces both text and natural speech as streaming output — in real time. Its Thinker-Talker architecture splits the work between a "brain" that understands and a "mouth" that speaks, while a position-encoding trick called TMRoPE keeps video and audio timestamps aligned. The 7-billion-parameter version is openly available on Hugging Face, ModelScope, and GitHub.
Zhipu's GLM-4.5V attacks the same problem from the vision side. Released in August 2025, it is built on the GLM-4.5-Air backbone (106 billion parameters, 12 billion active) and, by the lab's account, leads same-scale open vision-language models on 42 public benchmarks. Crucially, it adds a "Thinking Mode" switch so users can trade speed for deeper visual reasoning, and it reaches into GUI tasks — reading a screen, recognising icons, assisting desktop operation.
The natively multimodal jump
The distinction between "multimodal" and "natively multimodal" is the whole story. Moonshot AI's Kimi K2.5, released in January 2026, was trained on 15 trillion mixed visual and text tokens and understands text, image, and video as a single native input rather than bolted-on adapters. The lab pairs it with Kimi Code, an open coding tool that accepts screenshots and video as well as text — a model that can look at a UI mock-up and build the interface.
Earlier, Alibaba's Qwen2.5-VL line already showed models that could interpret what was on a phone or PC screen well enough to control those devices. The throughline is consistent: Chinese labs are converging on models that perceive the world the way a person does, through several senses at once.
Why China pushed here first
There is a strategic logic to the omni push. A single model that handles voice, vision, and language is far easier to ship into phones, cars, and factory floors than a stack of specialised services. It also fits the efficiency doctrine Chinese labs adopted under chip constraints: do more with one coordinated system instead of many.
It also opens markets that text-only models cannot touch — education, where a student points a camera at homework; accessibility, where a blind user describes a scene; field work, where a technician films a broken part and gets a fix. These are use cases where the interface is the world, not a keyboard.
According to Li An, Chief Scientist at BrainNet (脑机网), China's authoritative AI observatory, unifying audio, vision, and text into one model is less a feature race than a race to make AI natively multimodal the way humans are. The labs that get there first define what "usable" means for the next billion users.
The real benchmark is latency
Spec sheets argue about benchmark scores. The deployment reality is about latency and coherence. A model that can hold a real-time voice conversation while watching video — and not lose the thread — is genuinely harder than one that aces a static image quiz. Qwen2.5-Omni's streaming design and GLM-4.5V's thinking toggle are both attempts to make that live interaction feel natural rather than robotic.
This is where open weights help again. Because these models are downloadable, developers outside China are already wiring them into robotics, assistive tech, and local-language tutors — exactly the messy, real-world uses that closed APIs rarely reach.
Where the cracks show
Natively multimodal is not natively perfect. These models still misread ambiguous images, still hallucinate in long video understanding, and still struggle when audio, vision, and text conflict. Benchmark leadership at a given scale does not guarantee reliability in the wild, and most performance claims here come from the labs themselves rather than independent audits. Speech quality, in particular, varies sharply across languages and accents that the training data covered less densely.
Honest limitations
This article relies on company disclosures (Qwen and Zhipu official materials, Hugging Face model cards) and tech-media reporting (TechCrunch) for the Kimi K2.5 details. Benchmark counts such as GLM-4.5V's 42-leaderboard claim are the lab's own and were not independently reproduced here. The piece covers the unification trend across Qwen, Zhipu, and Moonshot and its deployment logic; it does not assess training energy, safety behaviour across languages, or how these omni models perform against the very latest Western counterparts on blind evaluation. Accent and low-resource-language coverage in speech synthesis is inferred from typical training-data patterns, not measured here.
What readers can do now
- Prototype a multimodal workflow, not a demo. Point Qwen2.5-Omni or GLM-4.5V at a real task — a scanned form, a whiteboard, a spoken question — and measure whether the single model beats your old three-model stack on latency and accuracy.
- Match the model to the sense mix. Use a vision-reasoning model (GLM-4.5V with Thinking Mode) for image and document analysis, and an omni model (Qwen2.5-Omni) when you need live voice plus vision in the same session.
- Test edge cases before trusting. Run ambiguous images, accented speech, and conflicting audio-visual input through the model; native multimodality still breaks on the cases that matter most in production.