Alibaba Releases Qwen3-Omni With Open Access to Text, Image, Audio and Video AI

Four kinds of input, with answers in text or speech

Alibaba Cloud has unveiled Qwen3-Omni, an artificial intelligence model that accepts text, images, audio and video in one system and answers with text or natural speech. As of September 24, 2025, the release gives developers access to a multimodal assistant under Apache 2.0, a permissive license allowing free download, modification and commercial deployment.

Alibaba reports support for 119 languages in text, 19 for speech input and 10 for speech output. It also claims leading results across a broad set of evaluations: the best scores on 22 of 36 benchmarks and leadership among open source models on 32. Those figures put multilingual communication and audio and video understanding at the center of the launch.

Qwen3-Omni is designed to handle several kinds of information together. An assistant could listen to a spoken question, examine an image or video and then speak an answer. Its outputs remain text and speech, rather than generated images or video, keeping the response process focused on conversation.

The release includes three public variants: Instruct for mixed inputs and spoken responses, Thinking for more demanding reasoning, and Captioner for detailed descriptions of audio with a reduced tendency to invent information. Developers can download the models, use examples in the official repository or access a hosted API, including a faster Flash serving option.

Alibaba's late night launch also included a new Qwen3 TTS system for converting text into speech and an image editing model. Together, the releases show an effort to supply more of the components needed for applications that read, see, listen and talk. For businesses, the immediate choice is between hosting Qwen3-Omni themselves and paying for access through a service.

Where the release fits in the AI competition

OpenAI helped popularize assistants combining text, image and audio through GPT-4o. Google extended video analysis through Gemini 2.5 in some product tiers. Qwen3-Omni enters that competition with a system built from the outset to process all four input types together.

Alibaba describes its approach as a single stack trained from input through to output. The model can examine still images and documents, recognize text through optical character recognition, identify relevant image regions and answer questions about visual content. Video capabilities include identifying scenes and tracking events, while audio processing supports listening and speech recognition.

Access is a central difference. Qwen3-Omni's downloadable weights, the learned numerical settings that determine its behavior, let businesses run the model on their own infrastructure and adapt it for particular sectors. Proprietary alternatives restrict hosting on customer infrastructure or charge for interactions through their services.

The launch comparison also places Qwen3-Omni alongside Google's open Gemma family, described as limited to text rather than offering the same multimodal scope. Alibaba is targeting organizations that want one open model for voice, vision and text instead of separate systems for each task.

Businesses moving beyond demonstrations need an assistant that can respond while an interaction is still happening. Qwen3-Omni combines that goal with control over deployment location, customization and spending. Its competitive appeal rests on both its reported performance and the freedom to decide where the system runs.

How quickly can it speak back?

Alibaba publishes theoretical first packet latency figures of about 234 milliseconds for audio and 547 milliseconds for video. First packet latency measures how long the system takes to begin returning a response, rather than the time needed to finish the whole answer.

Those short delays are intended to support the rhythm of ordinary conversation. Streaming allows an assistant to send parts of an answer as they become ready, reducing long silences while the rest is being produced. Efficient streaming also matters when users and servers are far apart.

Alibaba's figures come from internal systems. Actual response times will depend on hardware, the number of simultaneous users and network conditions. The reported ability to stay below a real time factor of one under load means processing can keep pace with incoming media, an important requirement for live conversations.

For a customer speaking to an assistant, both measures matter: how soon the answer starts and whether the system continues keeping up. A quick beginning followed by delays would still make conversation awkward. Deployment teams therefore need to assess the full interaction under the conditions their users will encounter.

The Thinker and Talker divide the work

Qwen3-Omni uses two main components called Thinker and Talker. Thinker handles understanding and reasoning across text, images and audio. It interprets incoming information, plans a response and composes content. Talker concentrates on turning the response into natural speech that can be delivered progressively.

Separating these jobs allows the speech system to work at its own pace while the language component plans ahead. Both use Mixture of Experts, a design in which a model contains many specialist networks but activates only a few for each token, or small unit of information it processes.

This approach increases throughput when many requests arrive together without activating every model parameter at every step. It is intended to combine a large pool of learned capabilities with a smaller amount of active computation for each piece of a response.

The speech pipeline represents audio through several sets of discrete codes, generated in sequence. A lightweight convolutional network called Code2Wav converts those codes into sound waves. The design aims to reduce delay while preserving voice quality, including timbre, the distinctive sound of a voice, and prosody, its rhythm and intonation.

Talker is also separated from Thinker's text representations and draws directly on audio and visual features. That gives spoken responses access to information about what is being heard and seen, helping the voice stay aligned with events on screen during an interaction.

Training across sound, images and text

Alibaba says an Audio Transformer encoder, the component that turns incoming sound into a form the model can process, was trained on roughly 20 million hours of supervised audio. Supervised training uses examples paired with information that guides what the system should learn.

The broader training process took place in stages. The team first aligned the encoders that handle different inputs, then trained on around two trillion tokens spanning text, audio, images and video. A later stage extended the amount of information the system could handle during long sessions.

Further training included supervised fine tuning, distillation and optimization for both Thinker and Talker. Fine tuning refines behavior with selected examples, while distillation transfers useful behavior from another model or training process. These steps followed initial training across large text collections and extensive audio and visual material.

Alibaba presents Qwen3-Omni as natively multilingual and multimodal: a system trained to work across languages and types of information together. Its stated aim is to preserve natural speech while the language model continues planning what comes next.

Three variants and different limits

The public models carry the Qwen3-Omni-30B-A3B name. Instruct handles mixed media, including audio, video and text, and can return text and speech. Thinking concentrates on difficult reasoning, mathematical steps and planning, with text output. Captioner specializes in detailed audio descriptions with fewer hallucinations, the term for plausible information an AI invents.

The hosted Flash option is intended for faster interactions through Alibaba Cloud. It gives teams another route into the system when they want to begin with a service rather than install and maintain the model themselves.

Language support varies by task. The model supports 119 languages for text, 19 for spoken input and 10 for spoken output, with Cantonese among the languages and dialects highlighted. That range can help organizations serve users with different scripts and accents, although the number of supported languages is smaller for speech than for text.

Thinking Mode offers a context window of up to 65,536 tokens, while Non Thinking Mode reaches 49,152. A context window is the amount of information a model can keep available while working on a response. The Thinking limit is 16,384 tokens larger, or about one third more than the Non Thinking limit.

The listed maximum input and output are 16,384 tokens, and the longest reasoning chain can extend to 32,768 tokens. These limits describe different parts of a session: the information held in context, what can enter or leave, and the internal reasoning sequence used for a difficult task.

What Alibaba's benchmark results show

Alibaba reports evaluations across 36 benchmarks covering text and reasoning, speech and audio, images and vision, and video tasks involving both sound and pictures. It says Qwen3-Omni achieves state of the art scores on 22 and leads open source models on 32.

The company also says the model outperforms GPT-4o and Gemini 2.5 Pro on many audio and audiovisual evaluations. The tasks include speech recognition, audio understanding, answering questions about video and sound together, describing video and generating voices. In tasks dominated by text, Alibaba says performance matches members of its Qwen text family.

Each benchmark measures a particular task under particular conditions. Instruction wording, scoring rules and the material used in a test can affect results. Vendors also choose which measurements they publish, so buyers need to examine the categories most relevant to their intended application.

A business using telephone recordings faces different conditions from a clean demonstration: background noise, varied microphones, regional accents and interruptions. Video applications must also handle rapid scene changes. Testing with the organization's own samples can reveal how the model performs under those conditions.

Cost and throughput belong in the same assessment. A team needs to know how many requests its deployment can handle, how quickly it responds and how often it produces an incorrect answer. Alibaba's broad evaluation program reflects the range of tasks the model is designed to perform, while practical testing determines whether it fits a particular workload.

What developers can build

Qwen3-Omni's repository and demonstrations include cookbooks for common tasks, allowing developers to begin with working examples before making extensive changes. The applications span communication, document work, media analysis and assistants that respond to their surroundings.

• Multilingual speech recognition for calls, meetings and media.
• Speech translation across widely used language pairs, including spoken replies.
• Descriptions of environmental sounds and music tagging.
• Optical character recognition for documents and screenshots, with answers tied to relevant regions.
• Questions and answers about images, complex diagrams and dashboards.
• Video descriptions, highlights and event detection to help navigate long clips.
• Questions combining audio and video for training, support and instructional content.
• Live technical support that listens, examines a problem and talks through a repair.
• Agent workflows that use external tools and APIs to complete tasks.

System prompts can set the assistant's persona, tone and response format. Streaming allows it to deliver partial answers while reasoning continues, supporting quick clarifications rather than leaving users waiting through long pauses.

Sensitive applications also need protection around the model. Measures described for deployment include stronger prompt controls, filters for text and audio, and a policy engine that records context and decisions. These components help organizations govern how an assistant responds and how its actions can be reviewed.

What Apache 2.0 changes for businesses

Apache 2.0 permits commercial use, distribution, modification and sublicensing. It also includes a patent grant, giving businesses a defined patent license when incorporating the model into products. Enterprises can adapt Qwen3-Omni without negotiating a separate model license for each sector or use.

Organizations can run the system on private computing clusters or trusted clouds, tune it with their own data and distribute software without paying model usage fees to Alibaba. Running the system still requires computing infrastructure and operations, while hosted API access has its own charges.

Control over deployment reduces dependence on a single provider. For regulated industries, keeping a multimodal system on infrastructure they manage can help with requirements governing where data is stored and how processing is audited.

Alibaba describes a growing Qwen ecosystem, with hundreds of models released since its first checkpoints, downloads in the hundreds of millions and tens of thousands of derivatives published by developers on public model hubs. That pool of models and community work can shorten the move from an initial prototype to a working application.

Infrastructure choices remain central to cost. GPU memory, the bandwidth needed to move audio and video from storage, and fast networking for streaming all affect responsiveness. A downloadable model gives an organization control over those decisions, together with responsibility for operating the service.

Hosted pricing and the route to deployment

Alibaba's hosted API uses token billing per 1,000 tokens, with separate rates for text, audio, image or video inputs and for text or audio outputs. New accounts receive a free quota of one million tokens across modalities for 90 days after activation. Exact monetary rates are not specified in the launch details.

The introductory quota is intended to support feasibility testing, prompt development and evaluation on internal data. Teams can start through the API while infrastructure specialists estimate the resources needed for a deployment they operate themselves.

Some organizations may use an open model for routine work and a proprietary API for unusual cases or specialized features. That approach gives them a way to balance spending, flexibility and access to capabilities outside their main system.

Developers can obtain the weights and examples through Qwen3-Omni on GitHub. The quickstart covers tools including Transformers and vLLM. Docker images and demonstrations with web interfaces are also available for initial trials.

Model hubs provide checkpoints and community guides for fine tuning and evaluation. Hosted access offers another starting point for teams that want to experiment with streaming speech before committing to their own infrastructure.

Privacy, accuracy and safeguards

Assistants that watch and listen can receive sensitive information in voices, images, documents and video. Deployment plans therefore need clear consent procedures, secure storage and deletion policies. Legal teams should review how information moves from a user's device to servers and logs, with retention periods matched to applicable rules.

Voice synthesis also carries risks of misuse because it can imitate a style even when the model does not produce a particular target voice by default. Organizations need to consider that risk alongside the benefits of natural spoken responses.

Accuracy testing must cover the conditions in which the system will work. Audio captioning can misinterpret sounds in busy environments. Video descriptions can lose context when scenes change. Long sessions can allow instructions to drift as more information enters the conversation.

Recommended evaluations include adversarial prompts designed to challenge safeguards, coverage of accents and dialects, and red teaming to test safety boundaries. Teams should also examine hallucinations, bias and unsafe content before giving the model consequential responsibilities.

Users need a way to flag mistakes after deployment. Those reports can help organizations identify recurring problems and improve the service while retaining a record of how it behaves. For businesses adopting Qwen3-Omni, the release combines greater control over a capable multimodal system with the work of making that system reliable for its intended users.

Key Points

• Alibaba Cloud released Qwen3-Omni under Apache 2.0, allowing download, modification and commercial deployment with a patent grant.
• The model accepts text, images, audio and video and returns streaming text or speech.
• Instruct, Thinking and Captioner serve conversation, demanding reasoning and detailed audio description.
• Alibaba reports theoretical first packet latency of about 234 milliseconds for audio and 547 milliseconds for video.
• Language support covers 119 languages for text, 19 for speech input and 10 for speech output.
• Alibaba claims leading scores on 22 of 36 benchmarks and leadership among open source models on 32.
• Context windows reach 65,536 tokens in Thinking Mode and 49,152 in Non Thinking Mode.
• Hosted access includes a quota of one million tokens for new accounts during the first 90 days after activation.
• Organizations can operate the model themselves, with infrastructure, accuracy, privacy and safety controls forming part of deployment.