Vision-Language-Action (VLA) models are an architecture that folds camera footage and natural-language instructions into a single system, outputting a robot's actions (motor commands) directly. Against a tradition in robotics that designed perception (recognition) and control (planning and execution) as separate pipelines, the core idea here is a shift in mindset: integrating both into a single transformer model.

An illustration of Vision, Language, and ActionVision + Language → Action

No license-free official logo could be found for the major VLA companies (Physical Intelligence, etc.), so a simple icon representing the integration of vision, language, and action was created in its place.

Lineage: From RT-1 to RT-2 to OpenVLA

Google's RT-1 was an early flagship example, and its successor RT-2 developed the idea further, redirecting knowledge from a large model pretrained on web-scale vision and language data toward generating robot actions. On the open-source side, models like Open-VLA, Octo, and CLIP-RT have followed with transformer-based architectures built around cross-modal attention (a mutual attention mechanism spanning vision and language). What all of these share is that "seeing," "understanding," and "acting" aren't split into separate modules — a single network is trained end to end, from input (footage plus instruction) all the way through to output (action).

Internal Structure: Three Subsystems

A VLA model's computational architecture can be understood as roughly three subsystems. First, a vision encoder converts raw pixel data from the camera into a structured feature representation — often a combination of pretrained image encoders like SigLIP or DINOv2. Next comes a projection module (usually a multi-layer perceptron) that maps the extracted visual features into the embedding space the language model works in. Finally, a decoder-only transformer processes the concatenated sequence of visual and language (instruction) tokens and generates the final action.

Generating Action: Two Design Approaches

Once vision and language are integrated, there are broadly two design approaches for how the actual motor command (the action) gets generated.

One uses discrete action tokens: continuous action values (joint angles, velocities, and the like) are discretized into a fixed number of bins (256, say), and token IDs from the language model's vocabulary that see little other use are repurposed to represent actions. The advantage is that vision, language, and action can all be handled uniformly as the same kind of "sequence of tokens," generating the action sequence a_{1:T} autoregressively — conditioned on the observation o, generating each next token based on the ones generated before it.

p(a_{1:T} \mid o) = \prod_{t=1}^{T} p(a_t \mid a_{<t}, o)

This formulation has exactly the same shape as next-word prediction in natural-language generation, and it captures the core idea of VLA — handling vision, language, and action within the same probabilistic model.

The other approach uses a diffusion-based action head: the transformer's job is reduced to producing a "readout token" that summarizes the visual and language information, and generating the actual continuous action values is handed off to a diffusion model. Discrete action tokens, being generated in stages, can sometimes fall short on real-time responsiveness and smoothness, whereas a diffusion-based action head tends to produce smoother, continuous action output. Combinations of the two are increasingly common too — Octo, for instance, feeds image and language tokens through a transformer as a single sequence, then passes its output (the readout token) as a condition into a diffusion-based action head.

What This Design Means in Practice

A classical control system can only handle a situation it wasn't designed for by following its script and nothing else. The advantage of a VLA model is the possibility of generalizing to conditions it was never explicitly trained on. Its range of application is expanding fast, from warehouse automation to surgical assistance robots, and it's increasingly being positioned as the core technology behind Embodied AI. At the same time, heavy reliance on large-scale human demonstration data remains a major challenge, both in data collection and training cost.

Physical Intelligence's π0 Lineage

One of the names drawing the most attention in VLA recently is π0 (pi-zero), from Physical Intelligence, a startup that has raised over $400 million. It was announced as a general-purpose robot policy, combining large-scale multi-task, multi-robot data collection with a new network architecture, capable of handling a diverse range of tasks — folding laundry, clearing a table, scooping coffee beans. Physical Intelligence open-sourced π0's code and weights as the openpi repository, and has continued releasing improved versions since — π0.5, π0.6, π0.7. π0 base was pretrained on the Open X-Embodiment (OXE) dataset together with the company's own seven robot platforms, and is designed as a general-purpose model meant for fine-tuning: usable zero-shot for tasks already covered by the pretraining data, but built with fine-tuning as the expected path to a specific use case.

Physical Intelligence's Latest: π0.7 and Beyond

Tracking Physical Intelligence's own official blog, the π0 line has kept adding capability steadily through 2026. March 2026 brought two back-to-back releases: a method that uses an "RL Token" extracted from the VLA model to rapidly refine real-robot manipulation tasks via online reinforcement learning, and "Multi-Scale Embodied Memory," which integrates long- and short-term memory to handle tasks running longer than ten minutes. π0.7, announced in April, is positioned as a steerable model — one that can be guided externally — exhibiting emergent capabilities that amount to a "step-change" in generalization performance. The center of gravity in development appears to be shifting from single-shot demonstration imitation toward capabilities much closer to genuinely autonomous robot operation: memory, online learning, steerability.

LIBERO Saturating, and the Race for Speed

LIBERO, the simulation benchmark that's become standard in VLA research, has seen multiple methods reach task-success rates in the high 90s by the second half of 2026 — accuracy is effectively approaching saturation. Temporal Forcing (August 2026), which strengthens temporal-context understanding by aligning with a 4D scene representation, posts 98.8% on LIBERO while lifting success on a harder "hidden placement" task from 20.0% to 43.3% — a sign that as the standard benchmark saturates, the field's evaluation focus is shifting toward tougher generalization tasks.

With accuracy converging across methods, research focus has also shifted toward optimizing inference speed. DriftingVLA (August 2026), which replaces the conventional multi-step diffusion process with a single forward pass, maintains 98.32% success on LIBERO while reporting a 3.36-fold speedup in action-chunk generation compared to iterative methods. AdaVLA (August 2026, IROS 2026), which accelerates already-trained models without retraining them, introduces a metric that estimates generation confidence from the curvature of the flow-matching trajectory, achieving 1.87x and 2.24x speedups on π0.5 and X-VLA respectively with no additional training. SMILE (August 2026), aimed at smooth motion generation over long-horizon tasks, likewise maintains 98.0% success on LIBERO while reporting a 1.1x speedup through a design that predicts B-spline coefficients — "faster without sacrificing accuracy" has become one of the main axes of VLA research in the latter half of 2026.

The Spread of 2026

Research in the VLA space has continued to appear at a fast pace well into 2026. ABot-M0, which incorporates action-manifold learning; ACE-Brain-0.5, a unified foundation model for embodied agentic AI; Kairos, which integrates a world model with action — research is broadening beyond simple "see and move" toward modeling, remembering, and acting in the physical world in an integrated way. VLA as a concept is itself being repositioned — from a robotics-specific technology into a core element of the broader "Physical AI" framework.

The Connection to newbot

newbot doesn't currently use a VLA model — it achieves autonomous driving through classical path planning via Nav2 and a sensor-based obstacle guard (see "How Mapping and Autonomous Navigation Work" for details). That said, the rapid progress in the VLA space hints at room for future development — navigation directed not by "specify coordinates on the map" but by "give an instruction in natural language."

References

#VLA #Robotics #Embodied AI