Holodeck: Case Studies in Exploring AI Use in Automatic VR World Generation

Abraham G. Campbell (), Ghofran Abourayana, Molly Maloney, Simran, Sam Jacob, Quinn Berryman, Mikey Dolan, Gueric Thomas, Valentin Boutouria, Michal Lanecki and Xuanhui Xu
Additional contact information
Abraham G. Campbell: University College Dublin, School of Computer Science
Ghofran Abourayana: University College Dublin, School of Computer Science
Molly Maloney: University of Michigan
Simran: University College Dublin, School of Computer Science
Sam Jacob: University College Dublin, School of Computer Science
Quinn Berryman: University College Dublin, School of Computer Science
Mikey Dolan: University College Dublin, School of Computer Science
Gueric Thomas: de Modélisation et de leurs Applications, Institut Supérieur d’Informatique
Valentin Boutouria: de Modélisation et de leurs Applications, Institut Supérieur d’Informatique
Michal Lanecki: University College Dublin, School of Computer Science
Xuanhui Xu: University College Dublin, School of Computer Science

A chapter in XR and Metaverse, 2026, pp 459-470 from Springer

Abstract: Abstract Recent advancements in AI, particularly through transformer and diffusion architectures, have unlocked possibilities once considered science fiction, such as generating 3D objects from simple prompts and creating interactive avatars capable of dynamic responses and role-playing. This paper presents three case studies examining different workflows for leveraging the context of objects or avatars to assign specific properties, with the vision of enabling the development of immersive applications within LLM-powered virtual environments. The case studies explore how object properties, such as scale and relational position, can be defined and how agent-driven avatars can reflect their capabilities and select animations, while outlining challenges for future developers, including refining prompt design for conciseness and inference speed, diversifying output types, and establishing objective evaluation methods. Using the open-source Holodeck project as a basis, these concepts are demonstrated through three case studies: one on object scaling, animations, and reflective behaviors of a teaching assistant avatar, and finally a workflow and LLM benchmark for exploring objects’ relational properties to each other. This work aims to highlight the potential for integrating contextual properties into generated virtual worlds, allowing non-expert practitioners to help build VR applications in the future.

Keywords: Virtual reality; Human–computer interaction; Procedural generation; LLMs (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:prbchp:978-3-032-11983-4_34

Ordering information: This item can be ordered from
http://www.springer.com/9783032119834

DOI: 10.1007/978-3-032-11983-4_34

Access Statistics for this chapter

More chapters in Springer Proceedings in Business and Economics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().