Research theme · part of Robotics and Semantic Systems
Foundation Models and Visuomotor Policies
Connecting foundation models and learned robot skills with explicit world models and explainable task control.
We have a long history of research on robot skills and task-level programming, using explicit, ontology-based world models to represent objects, relationships and the knowledge needed to perform tasks. This work forms the basis of SkiROS, our framework for representing, composing and executing robot skills.
Building on this foundation, we investigate how foundation models and visuomotor policies, which turn visual observations into robot actions, can support more adaptable robot behaviour. Our aim is a neuro-symbolic middle ground: combining the performance and flexibility of state-of-the-art learning methods with explicit knowledge representation and explainability. We study how learned skills can be connected to world models and structured task control, so that task-level decisions can be inspected and explained.
Our current work includes learning from human demonstrations, generative visuomotor policies such as normalizing flows, and vision-language-action models. We connect these methods to task-level control, including behaviour trees, and evaluate them on physical robots in the Robotlab. Our Quest2ROS2 framework supports the collection of demonstrations through virtual-reality teleoperation.
A concrete target is our 2015 demonstration of Christmas gift wrapping with an ABB YuMi robot. Programming that demonstration required two person-months of work. We aim to recreate it using modern physical AI-based robot skills, providing a practical setting in which to examine programming effort, adaptability and explainability within a complete robot task.
Members
Publications
A theme-tagged selection. The full and authoritative list is Publications.
-
Quest2ROS2: A ROS 2 Framework for Bi-manual VR Teleoperation Open access
-
Normalizing flows are capable models for bi-manual visuomotor policy
-
Addressing Failures in Robotics Using Vision-Based Language Models (VLMs) and Behavior Trees (BT)
-
Towards self-reliant robots: skill learning, failure recovery, and real-time adaptation, integrating behavior trees, reinforcement learning, and vision-language models for robust robotic autonomy
Projects
Projects tagged with this theme. All of the division's projects are on Projects.
-
A Robust and Reliable Vision-Language-Action Interface
-
Automate Sweden
-
Bi-manual Robot Learning
-
Efficient Transfer Learning for Robot Skills
-
NextG2Com: Next Generation Communication and Computational Infrastructures and Applications
-
Radical Innovation of Adaptive Cobot Technology (RIACT)
-
ScalABLE 4.0: Scalable automation for flexible production systems
-
SkiROS: Skills for Robotic Systems for Enabling Agile Production
Last updated: 2026-11-10