What the Listener Keeps: An Autoethnography Comparing a Documenter's Memory, AI-Generated Imaginings, and Photographs of Others' Cherished Objects

The 19th International Symposium on Visual Information Communication and Interaction (VINCI ’26)

[Cultural Computing Special Track Full Paper]What the Listener Keeps: An Autoethnography Comparing a Documenter's Memory, AI-Generated Imaginings, and Photographs of Others' Cherished Objects
[Authors] Chinatsu Ozawa, Yoichi Ochiai, Masaki Takeda

Transcript

1. Cultural Computing Special Track “Cultural Reconstruction and Creation” What the Listener Keeps: An Autoethnography Comparing a Documenter’s Memory, AI-Generated Imaginings, and Photographs of Others’ Cherished Objects ◎Chinatsu Ozawa, Yoichi Ochiai, Masaki Takeda
2. What the Listener Keeps Background Cultural records (craft catalogs, oral histories, etc.) = things that people have cherished ＝ Digital Nature Group / Chinatsu Ozawa Secondary witness Narrator Documenter Recorded data the owner or maker of the object curators, researchers, family members audio, photographs, notes Tell Keep Archives have treated this “in-between” position as if it were transparent. 2
3. What the Listener Keeps Background Common practice Digital Nature Group / Chinatsu Ozawa Audio, photographs etc. Kept in the archive The documenter’s own memory Discarded as noise Secondary witness Documenter Our focus • The weight, smell, and texture of the object (senses) • Where in the room it sat, and what sat next to it (situation) • The owner’s face when holding it, and the mood of the room (scene) 3
4. What the Listener Keeps Background Generative AI for restoring lost information People expect to use it when old recordings or transcripts survive but no photograph of the object exists. Digital Nature Group / Chinatsu Ozawa Audio → Generative AI → “Restored” image → False memory What is missing from the narration? What can generative AI not fill in? 4
5. What the Listener Keeps Research Question Records are made by a “documenter,” who stands between the owner and the record. The documenter’s own memory is discarded, and no one has checked what is lost. Generative AI now tries to fill that gap, but it can create false memories. And the person who uses the AI is the documenter. Digital Nature Group / Chinatsu Ozawa RQ What does the documenter remember, and what do they forget, about another person’s cherished object? Methods The first author becomes the documenter and compares their own memory with the audio, the photographs, and the AI images. 5
6. What the Listener Keeps Methods: Autoethnography The first author becomes the documenter and examines their own memory. Digital Nature Group / Chinatsu Ozawa 1. Listen to the recording 2. Write down what you remember 3. Have the AI generate an image 4. Look at the AI image 5. Compare with the photograph record select Documenter Narrator’s cherished object Documenter’s memory written down before seeing any image Image made by AI from the narrator’s words alone correct Compare 3 types Photograph an empty envelope if it cannot be published 6
7. What the Listener Keeps Methods: Interview • Six private collectors Digital Nature Group / Chinatsu Ozawa • • • • (antiques, old books, local materials) About 1.5 hours each, 9 hours in total Topic: the objects they cherish, and how they keep them We photographed all objects; three narrators asked us not to publish theirs This talk: two objects we can show — daruma molds (antiques shop, Sendai) and books (used bookshop, Tono) Iwate Pref. Miyagi Pref. Tohoku region, Japan 7
8. What the Listener Keeps Methods: 2 objects we selected Digital Nature Group / Chinatsu Ozawa Daruma mold Face-out books on Tono folklore Yomuyo, an antiques shop in Sendai Kappa Books, a used bookshop in Tono • Wooden molds for making papier-mâché daruma dolls • Paper is laid over the mold, cut along the back, then the mold is removed • “The daruma everyone buys is a copy. The mold is the original, but it • Some books are shelved with the cover facing out • The bookseller picks the ones with strong covers, so the book “speaks for never meets the public eye.” itself” • “The cover is everything now. Being picked up is a feat.” 8
9. What the Listener Keeps Results Overview The documenter remembered how the object was made and used, but lost its decades of age and its smell and texture. Digital Nature Group / Chinatsu Ozawa 1. The narration carries the procedure, but not the decades of age the object has acquired. 2. The documenter’s own memory also has large holes, such as smell and texture, and the documenter filled unseen parts with their own experience. 3. Seeing the AI’s wrong images let the documenter put into words features of the object they had never been able to say. 9
10. What the Listener Keeps Results1 Digital Nature Group / Chinatsu Ozawa The narration carries the procedure, but not the decades of age the object has acquired. The AI drew the procedures in the narration (laying paper over the mold and cutting the back; shelving books with the cover facing out), though not accurately. But the narration did not say that the mold had been worn by decades of use, that the books were old, or that hands had rubbed the surfaces. So the AI drew everything as if it were new. The AI does not leave unsaid information blank. It fills it in as “new.” 10
11. What the Listener Keeps Results2 Digital Nature Group / Chinatsu Ozawa The documenter’s own memory also has large holes, such as smell and texture, and the documenter filled unseen parts with their own experience. The author could not recall the smell of the bookshop or the texture of the pages, though both must have been there. The author had also forgotten that there were two molds side by side, and the conversation held in front of them. The photograph brought these back. When imagining an object they had not seen, the author filled it in with memories of things from their childhood home. The documenter did see the object, but their memory is less reliable than they think. 11
12. What the Listener Keeps Results3 Digital Nature Group / Chinatsu Ozawa Seeing the AI’s wrong images let the documenter put into words features of the object they had never been able to say. Only when the AI drew the mold as new could the author say, “the wood has matured over years of use” and “hands have polished the surface smooth.” When the AI drew the cut as a thin seam, the author noticed that the real cut was thick and deep, the trace of a human hand. Re-examining the photograph next to the AI image also revealed details the author had missed on site. The AI image is wrong as a record, but it works as a tool that asks the documenter, “What was it really like?” 12
13. What the Listener Keeps Findings AI is not a tool for restoring, but a tool for eliciting. Digital Nature Group / Chinatsu Ozawa • When we ask AI to recreate a lost object, it only fills unsaid parts with newness or leans toward regional stereotypes. It does not get closer to the real thing. • The useful way to use AI is not to make a correct image, but to show a wrong image to someone who saw the object and let them speak. 13
14. What the Listener Keeps Limitation Digital Nature Group / Chinatsu Ozawa This study does not prove anything about generative AI in general. It records one case in which the AI’s confident output diverged from memory and photographs. • • • One author studied two objects, in one region, over one summer. The visitor data are only eleven short texts. The narrators themselves did not see the AI images or the comparisons. 14
15. What the Listener Keeps Conclusion Question: What does the documenter remember, and what do they lose, about another person’s cherished object? Digital Nature Group / Chinatsu Ozawa Answer: They remembered how it was made and used, but lost its decades of age and its smell and texture. People who have not seen the object, humans and AI alike, imagine it as new and fill it in with their own experience. AI images are useless as records, but useful as a tool that asks the person who saw the object, “What was it really like?” 15