Making imaginary cities with doodles and neural networks

Thomas McMullan Read more November 28, 2016

Everybody loves an Italo Calvino quote, so here’s one care of artist and programmer Gene Kogan with Opendot Lab: “With cities, it is as with dreams: everything imaginable can be dreamed.”

Making imaginary cities with doodles and neural networks

The Invisible Cities project, with its heavy nod in the direction of the famous 20th-century Italian writer, makes much of the echoes between the logic of cities and dreams. While Google’s DeepDream uses a neural network to transform images into surreal, algorithmically generated hallucinations, Kogan and Co have instead trained a neural network to turn pictures into fictitious maps of cities.

“We were fascinated by the possibility of generating new and non-existent but realistic images using conditional adversarial neural networks that remember a certain set of features from what it has seen in the past; the same process that we humans undergo when we dream,” the team explains, presumably with a long intake of breath at the end.

Using open-sourced geographic data from OpenStreetMap, and applying a specific colour scheme to roads, buildings, green spaces and areas of water, the team were able to teach a neural network to learn the correspondence between different aerial images. They then trained their system to layer specific tile types pulled from a number of real-world cities: Venice, Milan and Los Angeles.

This basically lets the system make a generative map from the simplified, colour-coded image. The team could generate a version of, say, Milan, but in the style of, say, Los Angeles. The layout of the city would look the same, but instead of red-roofed villas, the new city would be made of concrete complexes.

Interestingly, the team left the background plain white, which allows the model to fill in the gaps with what it thinks should logically appear. “This translated to unexpected results as the [neural network] could interpret the same white patch of land as an airport, a maize field, a dumpster, or many other unexpected objects,” the team write. “Curiously, the model of Venice learned to hallucinate fake boats in the canals, where none had been present in the original images!”

As well as rejigging existing cities, the technique allows the team to feed in drawings of totally imaginary places, or abstract shapes, and the neural network will translate them into satellite maps in the style of Venice, Milan or Los Angeles.

In a way, this marks a movement in the opposite direction from DeepDream. While that neural network takes coherent images and makes them strange – almost incomprehensible in some cases – Invisible Cities turns abstract doodles into comprehensible, mappable places.