With recent advances in generative AI, there has been much discussion about how it can be used in games, virtual worlds, and the metaverse. So far, the focus has been on how we can speed up the production pipeline, given that many parts of game development are capital and labor intensive.

But the potential of generative AI goes far beyond production processes. In fact, generative AI is already built into the game loop in products such as AI Dungeon, Hidden Door, and Replica. Similarly, AI opponents are successful. games like diplomacy Players must understand the language and craft complex strategies. These examples are just the beginning.

Is there a way to establish a level to aim for in game development?

A few years ago, the Society of Automotive Engineers (SAE) A five-level schema to assemble the AI ​​in the vehicle It looks like. Level 1 is just basic driver assistance systems such as automatic braking. Up to Level 5, the vehicle operates at human (or superhuman) performance levels in any situation without human supervision.

Game developers often like “up-and-coming” hierarchies, which got me thinking about how to apply this idea to game development (and natural extensions like simulations, metaverses, and virtual worlds of all kinds).

Self-driving cars work, but for Virtual worlds allow you to express yourself creatively and influence the product itself.

Our initial focus on generative AI for games was for its potential to accelerate the production pipeline at game studios, but a more holistic view of games requires consideration of all creators touching them. there is.

Where creativity happens has a lot to do with where creativity happens: outside or in-game, inside established game systems, or built from scratch. The main participants in the creative continuum are:

• game studio (This includes everything from one-man indies to large AAA teams): These developers create experiences using engines and tools that exist outside of a specific game framework. increase. Examples include games built with Unity and Unreal or bespoke engines. Elden Ring, Grand Theft Auto, Call of Duty, World of Warcraft and most games ever made.
• of game You can create content while playing. This dates back to roguelike games (including Nethack and its graphical successors such as Diablo). Recent examples involving real AI include AI dungeons. replicas; or the product Hidden Door is talking about.
• of player It gives you opportunities for creative expression within the game loop. To some extent, this is all the games that exist, because for the most part there is creativity in the form of tactics and strategy. Games that exist to provide a form of “sandbox” play, such as Minecraft, Sim City, and Star Wars Galaxies, make creative expression one of the primary reasons to play. There’s also the social creativity that exists through guilds and various cooperative and competitive mechanisms that appeared in MMOs, and creativity around meta-experiences such as live streaming.
• remodeler Extend your game experience with tools outside the game loop. This includes Minecraft mods, Terraris mods, and World of Warcraft UI mods. Older examples date back to Neverwinter Nights and Quake. Before that, there was a hacker community creating their own versions of early games using sector editors and disassemblers.

Most commercial games today are made by game studios (although there are many notable exceptions within platforms like Roblox). Level here has mostly to do with how quickly you get an idea out of your head and onto the screen.

• Level 0: no automation. No significant automation (although it can kitbash a bit). Creators build their own engine and backend.
• Level 1: Support Creators. Use a 3D engine and/or backend platform as a configuration framework. Leverage off-the-shelf tools and code.
• Level 2: Partial Automation. A non-technical tool for creative input. Tech eliminates the previous step. Serverless and containers used in online games.
• Level 3: conditional generation. Part of our pipeline uses generative AI. Humans are heavily involved in loops, concept/story iteration, and code generation.
• Level 4: Highly Generative. Full automation of at least one creative pipeline. For example, a text prompt to create a fully usable 3D model (you need to remove all the complexity from UV unwrapping, bake light, etc.).
• Level 5: Directly from imagination. “Holodeck” level creativity. Multiple generative pipelines are comprehensively integrated into one composition.

A modder is someone who creates an experience that fits an established game system, rather than building their own game from scratch.

• Level 0: Hacker. How it all started: people with old fashioned sector editors, disassemblers, etc.
• Level 1: Tools provided by the game. The game comes with tools like a map editor and an accessible interface to apply ready-made creative tools.
• Level 2: Script Interface. The game either provides an API for creating mods or provides an embedded scripting interface for creating mods.
• Level 3: conditional generation. Some parts of the pipeline (concept art, stories, etc.) use generative AI, and humans are heavily involved in the loop.
• Level 4: Highly Generative. Integrated generative tools. For example, tools such as map editors call on generative AI to accelerate creation.
• Level 5: Directly from imagination. A complete mod was born from an idea. Iterative “sculpting” of mods with prompts used for refinement.

Players are creative too. Today, we’re blending the distinction between players and modders — and one day we’ll start seeing games that allow players to do many of the things that require modding today.

• Level 0: Limited creativity. It’s just the tactics and strategies you use to master the game.
• Level 1: Player as Content. Players modify the game by influencing the social system and economy, but have no meaningful impact on the rest of the built-in content.
• Level 2: Worldbuilding. Players influence the shape of the world itself, including how maps and structures live in a persistent world.
• Level 3: Emergent narrative. Player actions affect persistent content in the world, including living stories and content.This is something you see in most MMORPGs, especially eve online.
• Level 4: Integrated generative. What used to be the realm of modding is now part of the game loop with in-loop generated AI.
• Level 5: Directly from imagination. If players can dream it, they’ll make it happen. The game system adapts it to the rules in an appropriate way.

Finally, the game itself is the source of generated content. As generative AI technology improves, we will see more and more sophisticated content, animated characters, and game systems that are hard to imagine today.

• Level 0: static game. The game doesn’t change much between gameplay experiences. Set the level progression, etc.
• Level 1: Procedural content. Rules-based procedural content, including unique maps in play. A basic AI opponent may also be included.
• Level 2: Live Generative Content. Experiences include “live” generated content (text, images, music, or NLP-recognized characters) during play.
• Level 3: conditional generation. Generative content-in-the-loop leverages rules-based systems (such as RLHF) to maintain consistency and structure.
• Level 4: Highly Generative. The game “Game as a Dungeon Master” understands what the player likes most and personalizes/adapts the experience.
• Level 5: Creativity with full computation. Explore the search space of human fun, like our game designer friends coming up with new systems.

What level of generative technology are you aiming for in your own game or virtual world? What features do you envision? It’s all about looking beyond productivity tools (which are surprisingly important) and exploring what it means to use generative AI as part of the core game loop of an entirely new experience. I would recommend it to anyone.

• The Direct-from-Imagination Era Has Begun introduces convergence technology that allows you to go from idea to experience in an instant.
• In Composability is the Most Powerful Creative Force in the Universe, some insight into how we are always building on the shoulders of giants and how technologies such as frameworks and AI play an important role. introduces the concept of
• Computational Creativity is my article on how an individual can finally create something at the level of sophistication that exists in Elden Ring. Initially he thought it would take ten years, but it’s starting to feel like too long.
• The Generative AI for Virtual Worlds Market Map extends the ideas in this article to a map of companies building or leveraging generative AI to drive Level 3 and beyond.

If you enjoyed this, you may also like having the core concept in deck form, convenient to share with colleagues and friends: