Creative AI, part 1: Visual Arts

I don’t often mention my graduate research on my writing blog. I use different names for each pursuit, and I like to keep them a little bit separate! But lately I’ve been questioning that approach, and when I told the folks at Can*Con about my research, they asked if I’d want to do a presentation or panel about it. I said yes. But I don’t know what in my field is most interesting to the average SFF fan, and nobody wants a presentation by an obsessed grad student who drones on and on about little technical details! So I’m going to do a blog series as a dry-run, giving very introductory information to see what people’s reactions are.

And what is my graduate research, you ask? My graduate research is how to make computers creative. I’m interested in making AI that can write, draw, or sing the way people do.

A lot of people don’t know much about this field, even at my own university. I’m going to do a series in 8 parts to tell you what kind of writing, drawing, singing etc our computers can already do right now.

Since we tend to think of visual art first when we hear the word “art”, I’ll start with visual art. Are you ready?

We don’t often think of computers as visual artists, but the seed of computer art was planted almost as soon as computers were available. In the 60s, modern artists – Manfred Mohr, Frieder Nake, Roman Verostko, and many others – were quick to adopt computer programs to adopt new techniques and effects.

A computer-generated artwork consisting of a lot of small lines and shapes drawn in black and scattered around the page.
“a formal language”, an early computational artwork by Manfred Mohr.
A computer-generated artwork depicting a sphere covered in tiny sigils. The sigils consist of different subsets of the outer lines and diagonals of a square.
“sphereless”, also by Manfred Mohr, from about the same time period


Since then, using algorithms to make certain kinds of art has become standard practice. Game designers and computer animators use programs to help fill in backgrounds and landscapes. And everyone knows you can now use Photoshop to alter an image, or even generate one from scratch! But these programs are more like tools that a human artist uses than artists in their own right.

As computers became more sophisticated, so did the kind of art one could create with them, and some of these kinds of art give the computer more autonomy. For instance, many artists today produce works using artificial life and evolutionary algorithms. Line- and colour-producing subroutines are set loose, and are subject to mutations and selection pressures, so that the visible artwork continually evolves.

A computer-generated image in black and white. It looks like a tangle of liquid, black and white lines.
“Dual Bodies”, an evolutionary artwork by Karl Sims. Created through the interactive evolution of procedural textures.
A computer-generated image that looks like a purple, shell-like, biological spiral. Small tendrils cover the outer edge and there is also a tail-like appendage.
An unnamed evolutionary artwork by William Latham.


The output of these programs can be quite surprising and interesting, but some researchers turned to a more ambitious question. Could a computer learn to create realistic artworks in the same manner as a human?

Harold Cohen thought so. A visual artist and programmer, Cohen created a computer program called AARON. At first, AARON drew abstract images, but over the course of many years, Cohen programmed it with more and more knowledge about objects, their physical properties, and how to portray them. AARON grew into a program that could produce infinitely many artworks, print, and physically colour them itself.

A computer-generated picture of a woman with a hand to her head standing behind a lush, leafy green plant. The background is mottled and blue.
A picture by AARON
A computer-generated still life of plants in pots of different colours, on a yellow table, with a pink background.
An image physically painted by AARON.


AARON is impressive, but there are limits to its abilities. It can’t learn new styles, or discover new objects to paint, unless Cohen specifically programs them in.

Are pre-programmed styles like this “creative”? Many researchers don’t think so – which is why current projects try for tricky, process-based effects that AARON couldn’t do.

Steve DiPaola’s recent project “Darwin’s Gaze”, for instance, combines evolutionary techniques with a program that can change its goals depending on what looks promising at the time. Sometimes it doubles down towards the goal of copying an existing painting, and sometimes it chooses to free-associate and explore. But this freedom of choice, so far, is limited to a single project based on applying filters to try to copy a single human painting.

A photograph of a number of images attached to a gallery wall. The images are curves in different colours which almost but not quite resemble faces. At the bottom is a small portrait of Charles Darwin, the image that the other images are being evolved to resemble.
A photograph from the exhibition of “Darwin’s Gaze”.

And then there’s Simon Colton’s project, The Painting Fool, which switches styles based on the emotional effect it deems most appropriate and writes out an explanation – “framing information” – for its actions.

A portrait of a man. The portrait is painted in broad, impressionistic strokes in grayish greens, and its aspect ratio is squeezed. A portrait of a man, done in squiggly green lines on a canvas background. The man's eyes and one eyebrow are highlighted in dramatic blood red.

A portrait of a man, done in thick squiggly lines. The lines are mostly purple and blue, but there are also red, orange, yellow, and green lines. The background is white.
Three different portraits of the same man, done by The Painting Fool.


What do you think? Are you impressed by any of these pictures? Would you call an art-making computer creative? Why or why not? Leave a comment. 😀