Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Breaking the Machine: Finding Art-Historical Value in the Failure of Big Data and Computing

Matthew Lincoln
February 18, 2017

Breaking the Machine: Finding Art-Historical Value in the Failure of Big Data and Computing

Presented at The College Art Association 2017 annual conference.

In his 1962 book "The Shape of Time", George Kubler wrote, "it is disturbing to those who value the individuality of a thing to have that individuality diminished by classifications and generalizations." (36) This suspicion is as relevant as ever, as specialists in machine learning make ever-more boisterous claims about their ability to mine insights from artistic databases, such as a recent project that claimed to identify "the most creative paintings of all time." (Elgammal and Saleh 2015, http://arxiv.org/abs/1506.00711) Art historians have rightly challenged the implicit impartiality of these researchers' data sources and methods, and contested the intellectual coherence of their stated goals.

However, what might a digital art history look like that deploys machine learning precisely in order to see where the machine breaks? This paper will explore the conditions in which computational models fail to predict or classify, demonstrating how these instances can provoke new questions or analytical approaches. I will discuss a case study in which supervised machine classification was used to analyze a database of seventeenth-century Haarlem still life paintings. The computer's comparative ability, or inability, to discern the shape of different artists' oeuvres based on limited information about composition or subject matter suggested how these painters, as well as those who imitated them, defined their artistic personae. Where the machine breaks, and where it succeeds, also help to surface how decisions as fundamental as one's chosen framework for visual description can fundamentally alter the synthetic historical narratives that one ultimately composes.

Matthew Lincoln

February 18, 2017

More Decks by Matthew Lincoln

Other Decks in Research


  1. @matthewdlincoln Breaking the Machine Finding Art-Historical Value in the Failure

    of Big Data and Computing Matthew Lincoln, Ph.D Data Research Specialist College Art Association, New York February 18, 2017
  2. Sam Segal, A Flowery Past: A Survey of Dutch and

    Flemish Flower Painting from 1600 Until the Present (1982). @matthewdlincoln
  3. Henry Duval Gregory, “Tabletop Still Lifes in Haarlem, c. 1610-1660:

    A Study of the Relationships between Form and Meaning” (Ph.D. diss., University of Maryland, 2003). @matthewdlincoln
  4. 1.  Lemon (peeled) 2.  Candle (extinguished) 3.  Oyster 4.  Beer

    5.  Tazza (overturned) 6.  … Willem Claes Heda, Banquet Piece with Mince Pie, 1638. National Gallery of Art, Washington @matthewdlincoln
  5. 1.  Meat pie (Turkey) 2.  Mince Pie 3.  Oyster 4. 

    Wine (white) Pieter Claesz, Still life with Turkey Pie, 1627. Rijksmuseum, Amsterdam. @matthewdlincoln
  6. Compositional •  Orientation (por./land.) •  Disposition (wedge? pyramid?) •  Viewpoint

    (high/low) •  Cropping (tight/expansive) •  Height •  Width Symbolic •  Significant Motifs •  Illusionistic Signature? @matthewdlincoln
  7. @matthewdlincoln Some painters had motifs more predictable than compositions • 

    2 out of 5 misattributed by motif •  1 out of 6 misattributed by composition N = 12 (great performance for such a small sample)
  8. @matthewdlincoln Others had more predictable motifs •  1 out of

    4 ptgs misattributed by composition •  1 out of 6 misattributed by motif
  9. @matthewdlincoln Systematic misattributions point to strong affinities •  100% of

    Mahu’s works mistakenly given to PC judging by motifs alone
  10. Data are subjective This is a good thing Where models

    fail can be even more informative than where they succeed At its best, computing accentuates uncertainty and contingency @matthewdlincoln