HoloLens Operator Assistant

Back in 2018 when I was working at Diginext we had an R&D Augmented Reality project with a real industrial use-case. The HoloLens app’s goal was to assist Operators in their task : assemble a lot of long electric cables with specific inputs and outputs, following a “map” on a big table. Difficulties: pick up correct cable and not mix them up ; follow the exact plan and wire them correctly. Maps are complex and errors are easily made.

HoloLens app helps the operator to:

• confirm cables’ reference (thanks to a barcode reader)
• highlight his expected position in AR
• control the work at the end to avoid errors

My developer responsibilities on this project were:

• integrate HoloLens SDK into Diginext’s software, the authoring tool “Inscape” (similar to Unity)
• correctly wire the software code with the HoloLens api, and create some specific tools to let the Inscape’s user build an AR app and build it to HoloLens
• design and implement our industrial use-case into Inscape (using “Blueprints-like” visual programming), to create the HoloLens app
• run tests with the industrial client in real situation
• improve UX based on operators’ feedback

Technologies : mainly C++ (Inscape code + HoloLens SDK) and visual programming (similar to UE’s Blueprints) to create the AR experience.

This video shows (in our office, not in real situation) the first step of the use case, before operator starts his real task: positioning the map’s hologram and make it match real map’s position, located on the table. I created UX and controls, building on top of HoloLens’ basic gesture recognition.

This video demonstrates the operator’s use-case: positioning electric cables on the “map” and fixing them / wiring them. In the video I go quickly as it was just a demo to show our current app’s development state.