Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Breaking the Machine: Finding Art-Historical Value in the Failure of Big Data and Computing

A597f983f2a3599765ef8a68ed9e5c4b?s=47 Matthew Lincoln
February 18, 2017

Breaking the Machine: Finding Art-Historical Value in the Failure of Big Data and Computing

Presented at The College Art Association 2017 annual conference.

In his 1962 book "The Shape of Time", George Kubler wrote, "it is disturbing to those who value the individuality of a thing to have that individuality diminished by classifications and generalizations." (36) This suspicion is as relevant as ever, as specialists in machine learning make ever-more boisterous claims about their ability to mine insights from artistic databases, such as a recent project that claimed to identify "the most creative paintings of all time." (Elgammal and Saleh 2015, http://arxiv.org/abs/1506.00711) Art historians have rightly challenged the implicit impartiality of these researchers' data sources and methods, and contested the intellectual coherence of their stated goals.

However, what might a digital art history look like that deploys machine learning precisely in order to see where the machine breaks? This paper will explore the conditions in which computational models fail to predict or classify, demonstrating how these instances can provoke new questions or analytical approaches. I will discuss a case study in which supervised machine classification was used to analyze a database of seventeenth-century Haarlem still life paintings. The computer's comparative ability, or inability, to discern the shape of different artists' oeuvres based on limited information about composition or subject matter suggested how these painters, as well as those who imitated them, defined their artistic personae. Where the machine breaks, and where it succeeds, also help to surface how decisions as fundamental as one's chosen framework for visual description can fundamentally alter the synthetic historical narratives that one ultimately composes.

A597f983f2a3599765ef8a68ed9e5c4b?s=128

Matthew Lincoln

February 18, 2017
Tweet

Transcript

  1. @matthewdlincoln Breaking the Machine Finding Art-Historical Value in the Failure

    of Big Data and Computing Matthew Lincoln, Ph.D Data Research Specialist College Art Association, New York February 18, 2017
  2. The Daisy Linda Ward Collection, Ashmolean Museum, Oxford Image source:

    BBC Four @matthewdlincoln
  3. River Clegg, “An Honest Museum Audio Tour,” The New Yorker,

    December 5, 2016 @matthewdlincoln
  4. Jan van Huysum, Amsterdams Historisch Museum @matthewdlincoln

  5. Jan van Huysum, Amsterdams Historisch Museum @matthewdlincoln

  6. Sam Segal, A Flowery Past: A Survey of Dutch and

    Flemish Flower Painting from 1600 Until the Present (1982). @matthewdlincoln
  7. The Daisy Linda Ward Collection, Ashmolean Museum, Oxford Image source:

    BBC Four @matthewdlincoln
  8. Henry Duval Gregory, “Tabletop Still Lifes in Haarlem, c. 1610-1660:

    A Study of the Relationships between Form and Meaning” (Ph.D. diss., University of Maryland, 2003). @matthewdlincoln
  9. Low viewpoint @matthewdlincoln Willem Claes Heda, Banquet Piece with Mince

    Pie, 1638. National Gallery of Art, Washington
  10. High viewpoint @matthewdlincoln Pieter Claesz, Still life with Turkey Pie,

    1627. Rijksmuseum, Amsterdam.
  11. 1.  Lemon (peeled) 2.  Candle (extinguished) 3.  Oyster 4.  Beer

    5.  Tazza (overturned) 6.  … Willem Claes Heda, Banquet Piece with Mince Pie, 1638. National Gallery of Art, Washington @matthewdlincoln
  12. 1.  Meat pie (Turkey) 2.  Mince Pie 3.  Oyster 4. 

    Wine (white) Pieter Claesz, Still life with Turkey Pie, 1627. Rijksmuseum, Amsterdam. @matthewdlincoln
  13. Compositional •  Orientation (por./land.) •  Disposition (wedge? pyramid?) •  Viewpoint

    (high/low) •  Cropping (tight/expansive) •  Height •  Width Symbolic •  Significant Motifs •  Illusionistic Signature? @matthewdlincoln
  14. @matthewdlincoln (image credit: Shih 2013) Decision Trees and “Random Forest”

    learning
  15. @matthewdlincoln Willem Claes Heda, Banquet Piece with Mince Pie, 1638.

    National Gallery of Art, Washington
  16. @matthewdlincoln above-median proximity

  17. @matthewdlincoln below-median proximity

  18. @matthewdlincoln Willem Claes Heda, Banquet Piece with Mince Pie, 1638.

    National Gallery of Art, Washington
  19. @matthewdlincoln More Predictable More Unpredictable OOB error estimates

  20. @matthewdlincoln Some painters had motifs more predictable than compositions • 

    2 out of 5 misattributed by motif •  1 out of 6 misattributed by composition N = 12 (great performance for such a small sample)
  21. @matthewdlincoln Others had more predictable motifs •  1 out of

    4 ptgs misattributed by composition •  1 out of 6 misattributed by motif
  22. @matthewdlincoln Misattributes almost 100% of Gerrit Heda’s compositions to his

    father
  23. @matthewdlincoln Also misattributes 100% of Hedas by motif, but splits

    these mistakes between PC and WCH
  24. @matthewdlincoln Systematic misattributions point to strong affinities •  100% of

    Mahu’s works mistakenly given to PC judging by motifs alone
  25. Data are subjective This is a good thing Where models

    fail can be even more informative than where they succeed At its best, computing accentuates uncertainty and contingency @matthewdlincoln
  26. Matthew Lincoln matthewlincoln.net @matthewdlincoln