I owned the visual design and technical animation for key areas of the HoloLens 2 OS. HoloLens shell design is a team effort requiring collaboration with artists, UX designers, and engineers, but I drove the design and integration of hand-tracked manipulation and input. My focus was on Hologram manipulation UI, distant hand interactions, and button FX.
HoloLens 2 was announced at MWC Barcelona, and the second-gen device’s new hand tracking capabilities were on full display in a live keynote demo. Working closely with developer (and demo presenter) Julia Schwarz, I created the volumetric UI visuals and animations to show off these new interactions.
The HoloLens 2 announce demo features an early version of the Hologram interaction UX I created for the operating system. I also designed and integrated the volumetric sliders and button visuals, responsible for models, materials, and animation states. I helped artists optimize pieces of Hologram content, built animation states for various interaction effects, and integrated these with our sliders. The Wind Farm ultimately was the only Hologram we included in the demo. I optimized the asset, made the ocean shader, and set up its animation states.
Most of my focus for HoloLens 2 was on Direct Manipulation, enabling the user to use their hands to intuitively interact with Holograms. I designed and implemented shader-based visual effects and affordances for resizing Holograms and App Slates, interacting with distant content, and pressing buttons. The rendered videos above are from the HoloLens docs, and demonstrate some of these core system interactions.
I was awarded a Microsoft patent for my visual design contributions to distant Hologram interactions. When the user interacts with far-away objects, graphical dashed-line beams project from their wrists to aid in aiming. These beams reflect Hologram manipulation states, and the dashed line is visually consistent to aid in gauging distance.
In-Device screenshots of Hologram adjust UX. Holograms respond to the proximity of user’s hands, and the same Adjust affordances are applied to both 3D holograms and app slates, with nested prefabs making designs scalable across both experiences.
I developed the look and feel of the UX through various shader explorations, and created flexible spatial layout logic and animation state machines to create a versatile asset and clean user experience.