Microsoft’s new drawing bot is a robo-Salvador Dali – even if it doesn’t mean to be

Alex Lee January 19, 2018

It’s slightly embarrassing to say, but AI is clearly more artistically inclined than I am. While I gracelessly botched my Fine Art GCSE before I even got to the final hurdle, Microsoft’s new AI is already able to sketch drawings from written descriptions.

Microsoft’s new drawing bot is a robo-Salvador Dali - even if it doesn’t mean to be

The drawing bot, as it’s simply called, takes the information from text and turns it into pictures, pixel by pixel. It uses something called a Generative Adversarial Network (GAN) running on two machine learning models. One of these models generates images from text, while the other is a discriminator, a model that uses descriptions of text to test how authentic the drawing is, so that it can’t be fooled. The AI has been trained on a combination of images and captions and the GAN understands what a bird looks like and can draw a bird thanks to the caption associated with the bird.

“If you go to Bing and search for a bird, you get a bird picture. But here the pictures are created by the computer, pixel by pixel, from scratch,” XIaodong He, principal researcher and research manager in the Deep Learning Technology Centre at Microsoft’s research lab in Redmond, said. “These birds may not exist in the real world, they are just an aspect of our computer’s imagination of birds.”

The team even found that the drawing bot was, in a way, beginning to think for itself. Instead of merely following the instructions of the description of a bird, it would often draw a bird on a branch or add other details, mirroring the typical kind of drawings the machine has been fed and trained on.

While the AI is a better artist than I, there are a few fatal flaws in the system, and let’s just say, the drawing bot is a little… avant-garde. Take a look at the drawings and you’ll see it’s managed to draw birds with blue beaks and fruit stands which hold mutated bananas.

In addition, the bot is terrible with ultra-complex commands. Tell it to draw a bird with a green crown, yellow wings and a red belly, for example, and the GAN, will serve up a bird that swirls with red, green, and yellow in the middle to compensate. There’s no artistic flare, all the drawings are in that same style – even the absurd drawing of a red, floating bus over a lake has that same AI-style. So while for now, it’s intriguing, I don’t think any artists need to worry.

Some of the technology draws on AI projects that Microsoft has built in the past. Last year, the company released Seeing AI, an app aimed at the visually impaired which is able to complete multiple tasks. From describing anything that the user points their phone at, to recognising people who have been saved in their contacts, and reading out documents and scanning barcodes, Microsoft has been clearly working hard on AI. Similarly, Facebook has been using AI to caption photos for visually impaired users of its social network.

Major tech companies have been using AI in art projects as of late. Earlier this week, Google released a feature to their Arts and Culture app which matches a picture of yourself to one in art history, with hit and miss results. It joins Google’s Appsperiments, a collection of aps that helps you take better photos.