Abstract

We use a technique called Deep Dream to generate hallucinating images. In order to do so, first, an artificial neural network was trained on historical watermarks. The watermarks have been digitized by the Universität Leipzig and are mostly some forms of oxhead (German Ochsenkopf). We have managed to train a network that can classifiy watermarks with an accurary of over 95%. With the network, Deep Dream lets us feed arbitrary images into it to create dreamy images. The algorithm enhances strutures or features in an input images that resembles those of the data that is was trained on, most dominantly the oxheads. On this website, you can see the results of applying Deep Ochsenkopf to works of five visual artists from the collection "Kreuzberger Boheme" of the FHXB Friedrichshain-Kreuzberg Museum.

Detailed Description

In recent years, Machine Learning and especially so-called 'Deep Learning' with Artificial Neural Networks have established themselves as megatrends. In this project, we use this latest technological development and combine it with some medieval cultural goods to create some very modern art. But first things first, what inspired us to do so?

Deep Dream image showing parts of dogs
Deep Dream image which started with white noise. Source

In 2015, Google presented Deep Dream which is a technique to better understand how artificial neural networks see the world. The algorithm enhanced features or structures it recognizes in an input image. As a side effect, it creates hallucinating images which some people compare to the effect of psychedelic drugs such as LSD. But an artificial neural network can only output something it has learned. During the time Deep Dream went viral, the networks were mostly trained on dogs because there was a machine learning competition on dog breeds. As a consequence, most pictures were dreaming of puppies. But there is more to the world than just puppies. So we want to dream about something completely different: historical watermarks.

Historical Watermarks
Categories of historical watermarks. More information about watermarks.

Since the beginning of the paper production in the 13th century, watermarks were used to indicate origin and quality of the paper. They were made out of wire and produced a negative impression on the paper. Nowadays, the marks are used to put historical writings into context and estimate age. A collection of over 5000 items were digitized and categorized by the Universitätsbibliothek Leipzig. We want a system to learn this data. But training a network from scratch is beyond our resources that is why we fine-tuned a pre-trained one. This means that we took a network that was trained for weeks on multiple computers on a different data set and adapt it to our need. The result was a network that could assign watermarks to different categories with an accuracy of over 95%.

After that, we can feed arbitrary images into the network to find structures that resemble those watermarks. Because the watermarks are mainly different variations of oxheads (Ochsenköpfe), it also produces oxhead structures. Most dominantly, it picked up the round upper face with a horn going up in the middle. We dreamed on images of the collection "Kreuzberger Boheme" of the FHXB Friedrichshain-Kreuzberg Museum and you can the results on this website.

The whole process turned out to be rather difficult because of a lack of appropriate tooling, tutorials, and documentation. Training the network in particular turned out to be rather difficult because there are so many different knobs to fiddle and parameters to tune. It took us over five weeks to produce a network that picks up high-level structures of the marks. The result of this project will also be – besides working code – a long follow-up blog post to give step for step instructions on how to produce your own deep dreams.