Author: Ada Hoffmann

Game design is a very complex field, incorporating a huge number of both artistic and technical specialties. Could a computer design its own games? The answer, surprisingly, is yes – or a qualified yes.

Technically, computers already create many parts of the video games that you enjoy. It’s called procedural content generation. Computers can fill in certain parts of a game on their own, based on certain rules, to avoid making the programmer do everything by hand.

Most procedural content generation is pretty formulaic. For example, a computer might procedurally generate textures. Or a Roguelike game, given some rules about the size of rooms and what monsters might spawn in them, might randomly generate rooms and randomly place monsters. (Minecraft and many other games randomly generate terrain, too.) Computers can use fractals and other computer graphics rules to make settings full of trees, mountains, and other natural forms which all look detailed and different from each other, instead of making a human artist manually paint absolutely everything.

Procedural generation can be more complex than this go. For example, there’s the Director AI in Left 4 Dead which not only creates levels but adjusts their difficulty, based on the players’ current state, to keep the pace of the game exciting. And difficulty/pace is only one of many variables that a computer could use to guide its decisions, making them, arguably, artistic decisions, rather than random/meaningless ones.

So much for generating parts of games, but could a computer design a whole game? Arguably, yes. One big project in this vein is Michael Cook’s ANGELINA, which handles everything from level design to a game’s aesthetics and graphics to generating new game mechanics to make playable games. Whether or not people want to play these games is another question – but it should be noted that ANGELINA has taken part in game design competitions with humans, and was not the worst in the bunch.

Video games are very much not my thing, but if you want to know more about computers that design their own games, you can start with Michael Cook’s blog; a lot of other relevant researchers in that field are linked to from there.

How do you teach a computer to write a story? At first, the task seems impossible – to understand everything about the world that could be represented in a story, the computer would have to know as much as you or me. However, several research groups are up to the challenge.

Early groups worked by creating story templates that could be selected between, modified slightly, or filled in with words or concepts from a list. (You or I can also do this sort of thing, if we have a free afternoon, with some simple string manipulation functions. It’s not hard, and is a lot like a very carefully crafted Mad Lib.) However, the more interesting task in computational creativity is to make a system that can understand the characters involved in a story, figure out what they would do, and fit that into a storylike structure without a predefined instruction about how the story should go.

Systems that write stories – TALESPIN, MINSTREL, and MEXICA among them – typically take a single topic, like talking animals or Aztec warriors, and are programmed with a large knowledge base about objects, social reasoning, the actions that can be taken by an agent in this setting, etc.

Unfortunately (based on my not very thorough current searching) it’s much easier to find computer scientists talking about the details of these systems and exactly what knowledge they encode (details which are probably beyond the scope of these posts) than to find examples of the stories they make. The programs that exist today are quite interesting, able to calculate relatively advanced concepts like why people take specific actions, what basic actions are available or unavailable to a specific person, and what might be the consequence of that action. But the examples I can find are all a bit disappointing in a specific way:

The jaguar knight went hunting with the tlatoani. The fisherman hated the tlatoani. The fisherman attacked the tlatoani. The tlatoani attacked the fisherman. The jaguar knight made prisoner the fisherman. (from MEXICA by Iván Román and Rafael Pérez y Pérez)

It seems that, while current computers can represent a logical sequence of events for a story, describing the story in evocative, interesting, emotionally deep, or immersive ways is still a long way off.

This is no problem for certain applications, though. The storytelling advances made by these programs are useful for certain commercial applications in sports, finance, or elsewhere. Programs analyzing data in these fields can automatically generate stories based on their data, which make the most pertinent points easily comprehensible to humans:

Store 9, your sales of Item 6 are far below the other stores in your region. If you are able to up your sales of this product by only 5 units a day, you will be able to increase your profits next month by $1,123. The sales of this product for other stores in your region seem to indicate that this is completely achievable. (example from Narrative Science‘s website)

Computer programs can also automatically generate usable news stories about sports, finance, and other formulaic topics. Not exciting, perhaps; but also not terrible for a practice that grew out of Mad Libs.

I don’t often mention my graduate research on my writing blog. I use different names for each pursuit, and I like to keep them a little bit separate! But lately I’ve been questioning that approach, and when I told the folks at Can*Con about my research, they asked if I’d want to do a presentation or panel about it. I said yes. But I don’t know what in my field is most interesting to the average SFF fan, and nobody wants a presentation by an obsessed grad student who drones on and on about little technical details! So I’m going to do a blog series as a dry-run, giving very introductory information to see what people’s reactions are.

And what is my graduate research, you ask? My graduate research is how to make computers creative. I’m interested in making AI that can write, draw, or sing the way people do.

A lot of people don’t know much about this field, even at my own university. I’m going to do a series in 8 parts to tell you what kind of writing, drawing, singing etc our computers can already do right now.

Since we tend to think of visual art first when we hear the word “art”, I’ll start with visual art. Are you ready?
Continue reading “Creative AI, part 1: Visual Arts”