Alejandro RubioWorkcell

Projects

AGV Path Planning in Mixed Reality

AGV simulation in mixed reality on a Varjo XR-3. You build the scene with your hands, plan routes with A* and export the waypoints.

Type

Mixed reality

Context

Bachelor's project, ASSAR Industrial Innovation Arena

Period

2022

Stack

Unity 2020.3 (C#) · Varjo XR-3 · Ultraleap hand tracking · A* path planning · Chart.js

Area

[Mixed reality] [Robot programming]

Co-developed.

Media

Avoiding the safety zone
Mixed-reality view of the floor with a captioned workflow step, cyan floor grid and virtual obstacles.
Planning steps
Mixed-reality capture with cyan walls and a magenta path in a lab space.
Second mixed-reality capture of a magenta planned path between cyan walls.
Mixed-reality capture with tracked hands shown as wireframe models.
Mixed-reality capture with tracked virtual hands and interface elements.
Mixed-reality capture with a user-detected indicator and shoes visible on the floor.
User detection
Top-down Unity view of a grid with obstacles and planned paths.
Top-down view of the planner
Search variants and speed
Depth occlusion and user detection
Hand tracking
Virtual cubes over real objects

Video

AGV path planning in mixed reality, Varjo XR-3Open on YouTube ↗
AGV path planning in mixed reality, demo 2Open on YouTube ↗

What was built

  • Four states: floor placing, object placing, target placing and simulating.
  • A grid built from the floor's scale, with blocked nodes from physics checks and a safety zone around obstacles.
  • A* with Manhattan and diagonal variants (cost 10 for a straight step, 14.14 for a diagonal one).
  • Path smoothing, and a robot that follows look-points and slows down near the goal.
  • Export and import of workspaces and waypoints as JSON, and top-down PNG snapshots.

The user places a virtual floor, obstacles and start and end points with hand tracking, using a QR marker in the middle of a physical workspace. A virtual AGV then plans and drives the route. It is a simulation only. There is no physical AGV.

Two A* variants run side by side, one with Manhattan and one with diagonal movement. There is a configurable safety zone around obstacles, and the path is smoothed. The interface shows calculation time and distance, and warns when there is no path or a collision.

Workspaces, waypoints and top-down images are exported to files, and waypoints are plotted in a browser. We did the work at ASSAR from January to June 2022. The bachelor's thesis followed in 2023.

Facts

ItemValue
Unity version2020.3.17f1
Example exports2 workspaces, 8 waypoint files, 13 snapshots
Final-year project mark (Alicante)10 / 10
Bachelor Degree Project (Skövde)Grade A
[CHANNEL] ↗[THESIS (RUA)] ↗[CODE] ↗