Recognition

Forget artificial intelligence – in the brave new world of big data, it’s artificial idiocy we should be looking out for. —Tom Chatfield

The average person struggles to understand art. Artificial Intelligence could help solve that problem

If you went to a liberal arts university, or a reasonably well-funded high school, you likely took a course mysteriously referred to as “Art Appreciation.” Your instructor, who might have been an art history major if you were lucky, spent the duration of the semester trying to convince everyone that art was cool while simultaneously imposing a list of dates and names one was required to memorize in order to pass the sanctioned tests. The courses are required as part of the accreditation for most university programs in hopes that the institutions produce reasonably well-rounded graduates.

No, it doesn’t work.

A minority of the US population actually appreciates art on any level. Not because they don’t want to, mind you. Ask most people if they like art and they’ll tell you they do.  However, actually appreciating the work requires a deeper understanding than just “liking” art. Giving a work the “thumbs up” on Facebook is a long way from being able to discuss the work’s merits or the motivations of the artist.

The general concept of art, for many people, is that it fall into two categories: the pieces where one can tell what the picture is, or those where they can’t. We “like” those pieces we find easy to understand, especially those brightly colored paintings of the Romantic period. More challenging to the common aesthetic are those pieces that demand abstract thought, almost any major work from the late 19th century forward. Hang a Jackson Pollock painting and wait. Someone will inevitably ask if it was painted by an elephant or a four-year-old. Every time.

 Maybe We Need A Little Help

One of the reasons we have so much difficulty with comprehending visual art is that we are challenged to connect it to what is going on in our own lives. After all, much of the artwork that we find in traditional museums pre-dates most of us by at least 100 years or more. What was relevant to the artist is not necessarily still relevant in a contemporary setting. We are often asked to have an understanding not only of art styles, but history, fashion, and politics in ways that are completely lost to us. People wander through a gallery without hardly a clue as to why a work is important or what makes one more valuable than the other.

The curators of museums feel your pain. They want visitors to appreciate the collections they’ve worked so hard to assemble. Understanding how different works by various artists connect not only to each other but also to our own lives is a struggle every curator feels at one point or another. Obviously, the curators see the connections, but they do so drawing heavily not only on their depth of accumulated knowledge but on their own life experiences with art. Transferring that experience from themselves to their guests is almost impossible.

Enter the Tate Britain, one of the world’s premiere art museums. Each year, the Tate offers its IK Prize for promoting the use of technology in the exploration of art. Named after philanthropist Irene Kreitman, the prize this year was awarded to a team in Treviso, Italy for their entry, Recognition. Created by Fabrica with the help of JoliBrain, an artificial intelligence firm, and Microsoft, which helped fund the project, Recognition uses artificial intelligence to match photographs from current photojournalists with works of art at the Tate. The presumption is that by relating works of art with modern photographs, we might better understand the art.

How It Works

There are, obviously, a lot of questions to ask about an artificial intelligence program’s ability to understand and comprehend art. The folks at Fabrica are quick to explain that this is an experimental program and that a more accurate technology would require years more research and input. Recognition is not flawless by any stretch of the imagination. Yet, by forcing consideration of a strict set of criteria, even the mistakes help us to more deeply examine both the structure of art and of the images around us. Consider the criteria Recognition uses for comparison. These definitions are taken directly from the program itself:

1. Object recognition is a process for identifying specific objects. Its algorithms rely on matching, learning, or pattern recognition using appearance-based or feature-based analysis.
2. Facial recognition is a process for identifying human faces. In addition to locating the human faces in an image, it determines the age, gender, and emotional state of each subject it finds.
3. Composition recognition is a process for identifying prominent shapes and structures, visual layout, and colours.
4. Context recognition is a process which analyses the titles, dates, tags, and descriptions associated with each image. By reading this text, it’s also how recognition learns how to write a caption for each match.

Images with close similarity in these four categories are selected as a match, and displayed in Recognition‘s gallery. You can watch the process online through November 27. I’ve been watching it most of the morning, absolutely mesmerized. Recognition makes three or four matches an hour. Some of them immediately make sense. Others require more thought. And, inevitably, there are those where one has to conceded that maybe the computer got it wrong. But hey, at least it tried.

 How Much Do We Want To Learn?

Visitors to the exhibit at the Tate have the ability to help Recognition learn by making better suggestions to the matches that it makes. Given that the exhibition is only open through the end of November, however, means that there is a distinct limit to how much the program could learn. If one were to leave it running for five years or more, I would suspect that its ability to match photographs to works of art would become extremely accurate, but such an exhibition would require a tremendous amount of funding and even the Tate’s extensive pockets don’t run that deep.

Yes, there are more than a few detractors. One needs to have a better-than-average knowledge of the Tate’s catalog in order to participate and interact with any level of accuracy. The interface isn’t exactly friendly and the errors are sometimes so mindboggling as to leave one disappointed in the entire technology.

Yet, it would seem that artificial intelligence has a lot it can teach us. The technology is still very much in its infancy. If anything, it teaches us that understanding art is a learning process itself. Applying strict analysis to what has traditionally been seen as a subjective opinion forces different modes and conditions of examination. Art viewers are forced to take on different considerations that may not be especially comfortable. Making those adjustments, however, are what deepens our own understanding of art.

I wish I had time to fly to London and watch other people interacting with Recognition. I think it would be interesting to study how even a basic interaction with the program alters one’s perception of the art pieces displayed in the gallery. This is an exciting exhibition, one that could genuinely improve public appreciation and understanding of art, which could eventually translate into better public funding for the arts and for artists.

At least, we can hope for that outcome, can’t we?