https://github.com/microsoft/MoveBox-for-Microsoft-Rocketbox

The following paper was published at IEEE AIVR 2020 to coincide with the release of this toolbox,and gives more details of the features included:

Mar Gonzalez-Franco, Zelia Egan, Matthew Peachey, Angus Antley, Tanmay Randhavane, Payod Panda, Yaying Zhang, Cheng Yao Wang, Derek F. Reilly, Tabitha C Peck, Andrea Stevenson Won, Anthony Steed and Eyal Ofek (2020) “MoveBox: Democratizing MoCap for the Microsoft Rocketbox Avatar Library”. IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)

See also the publication page at Microsoft.

We will be using both tools in our upcoming work on social virtual reality and character animation.

Rocketbox was part of Havok which was bought by Microsoft in 2015. When I was at Microsoft Research on sabbatical in the second half of 2018/early 2019, the idea of making the avatars available came up, and in March 2020, Microsoft made the Rocketbox avatars available for research free of charge. You can just check out the avatars from github. I had a small hand in setting this up, by contributing a script to automatically fix up the avatars when loaded as .fbx files into recent versions of Unity.

More tools will be coming, and hopefully we will now be able to share open source versions of some of our experiments to facilitate reproduction and extension of our work.

Anthony

One of our contributions has been helping to lead an effort to share best practice around running experiments remotely as well as fostering a community of potential collaborators to run each others’ experiments to get some initial feedback on work. There is a Slack community that is sharing experience, and some of us wrote up some of the discussion in an article for Interactions Magazine

We also contributed to a panel discussion in the Frontiers in VR seminar series “XR for bringing people together in times of lockdown and post-COVID”

Finally Anthony was a contributor to the ACM’s Virtual Conferences, A Guide to Best Practices A community resource from the ACM Presidential Task Force on What Conferences Can Do to Replace Face-to-Face Meetings

• Anthony talked on a panel at the VR in VR workshop. Vinoba Vinayamoorthy captured a nice image of the panel on Zoom
• Drew presented our paper t IEEE VR, Directing Versus Attracting Attention: Exploring the Effectiveness of Central and Peripheral Cues in Panoramic Videos. PDF here
• Anthony was a co-author on two papers resulting from his sabbatical at Microsoft Research. The first Using Facial Animation to Increase the Enfacement Illusion and Avatar Self-Identification
• The second Measuring System Visual Latency through Cognitive Latency on Video See-Through AR devices, also has github source for the timer that was used.
• We had a poster on Docking Haptics. There is a longer ArXiv paper that describes this prototype.

Sebastian won the 2018 VGTC Virtual Reality Best Dissertation Award for his dissertation on Low Latency Rendering with Dataflow Architectures. He gave a short talk about his work.

Drew presented his paper Perception of Volumetric Characters’ Eye-Gaze Direction in Head-Mounted Displays.

Abstract: Volumetric capture allows the creation of near-video-quality content that can be explored with six degrees of freedom. Due to limitations in these experiences, such as the content being fixed at the point of filming, an understanding of eye-gaze awareness is critical. A repeated measures experiment was conducted that explored users’ ability to evaluate where a volumetrically captured avatar (VCA) was looking. Wearing one of two head-mounted displays (HMDs), 36 participants rotated a VCA to look at a target. The HMD resolution, target position, and VCA’s eye-gaze direction were varied. Results did not show a difference in accuracy between HMD resolutions, while the task became significantly harder for target locations further away from the user. In contrast to real-world studies, participants consistently misjudged eye-gaze direction based on target location, but not based on the avatar’s head turn direction. Implications are discussed, as results for VCAs viewed in HMDs appear to differ from face-to-face scenarios

Anthony gave a keynote speech at PERCAR: The Fifth IEEE VR Workshop on Perceptual and Cognitive Issues in AR. He talked about Embodied Cognition in AR/VR.

Here is one of the official photographers photos of Drew talking.

Showing a video from one of the VR group’s early experiments, circa 1994.

Anthony gave an invited talk on The Impact of Avatars on Close Quarters Interaction at the Virtual Humans and Crowds in Immersive Environments (VHCIE) 

Abstract: There is a compelling theory emerging of how embodiment inside immersive virtual environments enables participants to use their bodies in natural and fluid ways. In this talk, I will discuss recent work on how avatar representation and embodiment affect collaboration in social virtual reality. Our lab-based works shows how users utilize information about avatars in quite complex and surprising ways, and our studies of consumers in their homes shows some of the barriers that users experience in using avatars for extended periods. I will then discuss how these set some near-term challenges to the field, and review some immediate ways forward that could have significant impact on the utility of social virtual reality.

We had a poster about some work I helped with during my stay at Microsoft, with Mar Gonzales and Parastoo Abtahi: Individual Differences in Embodied Distance Estimation in Virtual Reality

Finally, Anthony was on a panel about Virtual Reality Curriculum.

Profiling Distributed Virtual Environments by Tracing Causality

Sebastian J Friston, Elias J Griffith, David Swapp, Alan Marshall, Anthony Steed

Abstract: In this paper we explore a new technique to profile distributed virtual environments. Profiling is a key part of the optimisation process, but techniques based on static analysis or passing meta-data have difficulty following causality in concurrent and distributed systems. Our technique is based on taking hashes of the system state in order to abstract away platform-specific details, facilitating causality tracing across process, machine and even semantic boundaries. Across three case studies, we demonstrate the efficacy of this approach, and how it supports a variety of metrics for comprehensively bench-marking distributed virtual environments.

This was a result of our CASMS project.

A Comparison of Virtual and Physical Training Transfer of Bimanual Assembly Tasks

Mara Murcia-López, Anthony Steed

Abstract: As we explore the use of consumer virtual reality technology for training applications, there is a need to evaluate its validity compared to more traditional training formats. In this paper, we present a study that compares the effectiveness of virtual training and physical training for teaching a bimanual assembly task. In a between-subjects experiment, 60 participants were trained to solve three 3D burr puzzles in one of six conditions comprised of virtual and physical training elements. In the four physical conditions, training was delivered via paper- and video-based instructions, with or without the physical puzzles to practice with. In the two virtual conditions, participants learnt to assemble the puzzles in an interactive virtual environment, with or without 3D animations showing the assembly process. After training, we conducted immediate tests in which participants were asked to solve a physical version of the puzzles. We measured performance through success rates and assembly completion testing times. We also measured training times as well as subjective ratings on several aspects of the experience. Our results show that the performance of virtually trained participants was promising. A statistically significant difference was not found between virtual training with animated instructions and the best performing physical condition (in which physical blocks were available during training) for the last and most complex puzzle in terms of success rates and testing times. Performance in retention tests two weeks after training was generally not as good as expected for all experimental conditions. We discuss the implications of the results and highlight the validity of virtual reality systems in training.

The Effect of Transition Type in Multi-View 360° Media

Andrew MacQuarrie, Anthony Steed

Abstract: 360° images and video have become extremely popular formats for immersive displays, due in large part to the technical ease of content production. While many experiences use a single camera viewpoint, an increasing number of experiences use multiple camera locations. In such multi-view 360° media (MV360M) systems, a visual effect is required when the user transitions from one camera location to another. This effect can take several forms, such as a cut or an image-based warp, and the choice of effect may impact many aspects of the experience, including issues related to enjoyment and scene understanding. To investigate the effect of transition types on immersive MV360M experiences, a repeated-measures experiment was conducted with 31 participants. Wearing a head-mounted display, participants explored four static scenes, for which multiple 360° images and a reconstructed 3D model were available. Three transition types were examined: teleport, a linear move through a 3D model of the scene, and an image-based transition using a Möbius transformation. The metrics investigated included spatial awareness, users movement profiles, transition preference and the subjective feeling of moving through the space. Results indicate that there was no significant difference between transition types in terms of spatial awareness, while significant differences were found for users movement profiles, with participants taking 1.6 seconds longer to select their next location following a teleport transition. The model and Möbius transitions were significantly better in terms of creating the feeling of moving through the space. Preference was also significantly different, with model and teleport transitions being preferred over Möbius transitions. Our results indicate that trade-offs between transitions will require content creators to think carefully about what aspects they consider to be most important when producing MV360M experiences.

Anthony organised a panel on How Should Social Virtual Reality Work? He was a last minute replacements for a panelist on The future impact of neuroscience and cognitive psychology on virtual environments

• Drew is presenting: Cinematic Virtual Reality: Evaluating the Effect of Display Type on the Viewing Experience for Panoramic Video, Andrew MacQuarrie and Anthony Steed
• The following papers have UCL co-authors
  • Efficient Hybrid Image Warping for High Frame-Rate Stereoscopic Rendering, Andre Schollmeyer, Simon Schneegans, Stephan Beck, Anthony Steed, and Bernd Froehlich
  • A Psychophysical Experiment Regarding Components of the Plausibility Illusion, Richard Skarbez, Solène Neyret, Frederick P. Brooks Jr., Mel Slater, and Mary C. Whitton
  • The Plausibility of a String Quartet Performance in Virtual Reality, Ilias Bergstrom, Antonio Sergio Azevedo, Panos Papiotis, Nuno Saldanha, and Mel Slater
• We have two posters:
  • The AR-Rift 2 Prototype, Anthony Steed, Yonathan Widya Adipradana, and Sebastian Friston
  • Object Location Memory Error in Virtual and Real Environments, Mengxin Xu, María Murcia-López, and Anthony Steed
• We are helping with three panels:
  • Virtual Social Interaction (Anthony Steed & Antonia Hamilton)
  • Instantaneous Beaming to Distance Places – A Possible and Desirable Future? (Anthony Steed & Mel Slater)
  • Big metal VR in the HMD and Unity era (a last minute panel at the SEARIS Workshop) (Anthony Steed)
  • Virtual Reality and Neuroscience (Mel Slater)

David Swapp gave a talk at the Digital Catapult event Immersed in the UK.

Fascinating speech from David Swapp from @UCL 15 years old Immersive #AR #VR Lab w/ 4‑wall caves & more! #DigiColLab https://t.co/HH2C8xalm4 pic.twitter.com/3uDD3abU3t

— Thomas Gere (@Tom_Gere) September 20, 2016