Mixed-Reality Robot Training Platform

• A step manager, with user actions as command objects (start, next, back, skip, reset) and timing per step.
• Interfaces for work objects, human tools and robots, with reusable behaviours for magnetic placement, welding, screwing and brushing.
• A TCP client that sends a task to the ABB controller, waits for 'done' and reads joint values to move the virtual robot's axes.
• A Modbus TCP link to a PLC. Inputs are written as coils and outputs are read as discrete inputs, mapped to PLC signal components.
• Three demo scenes, with step text loaded from JSON files.

Trainees follow instructions step by step with their hands. They work on virtual or real objects, next to a virtual or real collaborative robot. We built it at the ASSAR Industrial Innovation Arena in Skövde in 2022, as a platform for programming ABB GoFa and Universal Robots robots.

Work objects, human tools and robots can each switch between real and virtual during a session, and the process logic adapts. A training is a list of steps made in the scene from reusable behaviours, so a new process needs no new code.

There are three demo scenes: human-robot collaborative welding, battery assembly with a robot handing over parts, and a PLC-driven assembly. The interface is in English and Swedish. The public code does not open as a ready build, because paid and third-party assets are removed.