How to write a good CHI paper (that might just get accepted)

Presentation from a workshop given at the Dyson School of Design Engineering at Imperial College London for PhD students on how to write CHI papers, targeting #chi2024.

With big thanks to those who came before, especially Lennart Nacke and Lisa Anthony, Brett Mensh and Konrad Kording.

Transcript

1. How to write a good CHI paper (that may just get accepted) Sebastian Deterding Imperial College London
2. Why me? EiC ACM Games journal – Dozens of CHI/HCI reviews – 2x CHI associate chair – 6 CHI full papers, 3 of which honourable mentions –
3. Aims Understand quirks of writing for CHI 1. Identify your fitting contribution and subcommittee 2. Structure your abstract and introduction (as short forms of the full paper) 3.
4. Writing a good CHI paper is 70% writing a good paper and 30% knowing the quirks of CHI
5. Agenda 14:00-14:15 I: Getting to know CHI 14:15-14:45 II: Making a contribution 14:45-15:05 Activity: Framing contributions 15:05-15:20 Q&A Break 15:30-15:45 III: Structuring your paper 15:45-16:30 Activity: Structuring introductions
6. I Say hello to CHI
7. What is CHI? "ACM SIGCHI Conference on Human Factors in Computing Systems" – Biggest, most important conference in human-computer interaction: 4,670 attendees in 2023, running 40+ years – Highly selective: 879/3,182 submissions = 27.6% acceptance rate in 2023. Honorable mentions = top 5% accepted papers, best papers = top 1% – CHI 'full' paper is equivalent to top journal paper in other fields, recognised in many field (CS, informatics, communication, design, ...) – Like HCI, a broad tent for very diverse research communities –
8. How writing for CHI is different Broad tent with diverse communities and contributions – Short, one-off revision cycle = feasible revisions are limited to 4 weeks rewriting & reanalysis – Annual flagship HCI conference = rewards novel and timely contributions to HCI (often = previous CHI papers) – "Implications for design" –
9. The submission process for 2024 Two-step: Abstract by Sep 7, full paper Sep 14 – Submit PDF and source files – Use Overleaf with the conference template - word template is finicky! – New target length: 8,000 words excluding references, figures, tables (Longer needs good reason) – Share anonymised underlying data, analysis code, etc. if you can as supplementary materials or via repositories (OSF allows anonymous share-only link) – ACM submission is finicky: study and check in advance –
10. Figures speak Insert a figure on p. 1 if you can. – Check if someone can follow your paper just from figures & tables – Include a video figure if you can (first anonymised) –
11. The review process
12. The review process CHI uses a peculiar 'one-off' revise & resubmit cycle Every paper gets 2 AC + 2 external reviews – Decision recommendations: A, ARR, RR, RRX, X – If any AC recommends A/ARR/RR, paper is invited to revise & resubmit on Nov 7 (ca. 40%) – Resubmit Dec 12: Clean & marked changes document, response to ACs – Subcommittees decide based on AC recommendations, published Jan 19 – Publication-ready files due Feb 22 – Remote videos due March 28 –
13. Other submissions to consider at CHI Workshop papers: 2-4pp position pieces. Typically Sat/Sun before main conference, great entry into community for newcomers – Interactivity: Showcasing prototypes – Student competitions (game, design, research) – Late-breaking work: Poster + video – alt.chi: Weird, provocative papers –
14. II Making a Contribution
15. A paper gets accepted to CHI if ... it makes a contribution to HCI that is valid, original, transparent, and clear Handily, these are the review criteria: https://chi2024.acm.org/submission- guides/guide-to-a-successful- submission/ – Contribution and validity/rigour really matter – 1 Julie R. Williamson, Understanding CHI Reviews, 2023. https://chi2023.acm.org/2022/09/22/understanding-chi-reviews-analysis-of- chi2022-revise-and-resubmit/ 1.
16. Accepted papers do three things They clearly identify how they move HCI forward (contribution and originality) 1. They make reviewers confident that the contribution is well-founded and not overstated (validity and transparency) 2. They are easy to read (clarity) 3.
17. ... One more thing Don't use up the reviewer's reservoir of goodwill with ... poor formatting, typos, references – unsupported or overstating detail claims – long, waffling text – insensitive, uncritical language –
18. What is "a contribution"? NOT "filling a gap". There are infinite gaps to fill – Identify a problem/opportunity that matters in real life and to the academic audience you write for – Find/make something that improves how we think about and act on that problem – Express how we should think/act differently as a result – NOT a contribution "We do the first qualitative/teenager/... study on X" -> Why should results differ? Why does that matter? – "Sense of ownership drives recycling" -> How does this matter to HCI? – "We made X using mobile/LLMs/..." -> Why is that needed, beneficial, previously hard to do? – "Our system works/is liked" -> What does that tell us beyond your system for designing systems? – "60% of surveyed users dislike X" -> What does that mean for research and practice? –
19. What is a contribution to the CHI community? Advances a particular subcommittee ... – ... with general relevance to HCI: Check and reference review or agenda papers at CHI, interactions, other HCI journals – Explicitly states contributions in abstract, introduction, and conclusion – Honest about limitations and scope: Reviewers judge whether your paper supports your claims. Precise, humble claims = no attack surface. – Fits a particular genre of contribution – CHI 2024 gives a handy guide: https://chi2024.acm.org/submission-guides/contributions-to-chi/ –
20. Genres of contributions1 Jacob O. Wobbrock and Julie A. Kientz. 2016. Research contributions in human-computer interaction. interactions 23, 3 (May + June 2016), 38–44. https://doi.org/10.1145/2907069 1.
21. System contributions aka 'We made a thing' Make a new thing that expands and improves what interfaces, users, designers can do – Finding and realising a new working solution (space) to an important use case that generalises beyond the system itself – Usually documents a built artefact that enables new possibilities or insights, overcomes an important limitation, or (more rarely) improves on a benchmark – Validates that the system works with a user study, reflection, critical reflection, benchmarking – Interface Artifacts or Techniques = 'frontend' new forms of interaction – Systems, Tools, Architectures, and Infrastructure = 'backend' new software/hardware enabling interactions – Innovation, Creativity, and Vision = speculative novel ways of doing things –
22. Study contributions aka 'We collected and analysed evidence' Make new valid empirical observations that advance our understanding of how people design or interact with interfaces and/or improve how we design interfaces – New grounded concepts, new effects, challenging existing beliefs and models, settling a debate – Understanding Users = Making new observations about people interacting with computers – Validation and Replication = Rigorously testing an important prior empirical result whose reliability or generalisability is in doubt –
23. Method contributions aka 'Here's how to do things differently' New validated methods for designing interactions or researching HCI that allow us to do new things or do things better – Usually validated with critical reflection and documentation how using method improved one's own practice; or testing whether it improves practice of other practitioners – Can be Design Methods for designers or Research Methods for researchers –
24. Meta contributions aka 'Our field needs to change' Intervene in and advance how HCI researchers think about and do research with compelling new concepts and arguments – Usually validated with logical argument grounded in accurate, informed referencing of prior work inside and outside HCI, sometimes thought experiments or critical practice reflections how this new perspective could or did change things – Theory = New concept or model for understanding, studying, designing something that demonstrably generates novel inferences compared to existing theory – Argument = New important provocation to HCI researchers to change how to think or act – Systematic review = New synthesis of existing work identifying limitations, needed future research, or consensus and clarity –
25. How to find and write for your subcommittee Subcommittees = communities with their own literature, methods and standards, writing style, and typical contribution (study, system, or meta) – Different standards! E.g., Accessibility cares about engaging stakeholders, inclusive language, ... – These partially mirror smaller SIGCHI conferences: Blending Interaction = UIST, Computational Interaction = IUI, Games & Play = CHI PLAY, ... (alternative submission places) – Check https://chi2024.acm.org/subcommittees/selecting-a- subcommittee/ – Read text & example papers to find your fit and standards –
26. Topical subcommittees Accessibility and Aging – Health – Games and Play – Learning, Education, and Families – Privacy and Security – Specific Application Areas: Designing for ICT4D, creativity, rural, mobility, urban, civic, more- than-human, ... – Visualization - making new visualisation techniques or new insights into visualisation usage & experience –
27. Research genre subcommittees Blending Interaction: Engineering Interactive Systems & Tools - making systems for new kinds of interaction – Developing Novel Devices: Hardware, Materials, and Fabrication - making new physical tools – Computational Interaction - heavily quant/formal studies of (data-driven) algorithms for understanding, modelling, optimising interactions – Critical Computing, Sustainability, and Social Justice – critical, often more humanist or designerly work on social issues – Design – Research through, on, and for design – Interacting with Devices: Interaction Techniques & Modalities - making new kinds of interaction techniques – Interaction Beyond the Individual - CSCW/CSCL, often more ethnographic studies of collaboration – Understanding People: Quant, Qual, Mixed/alternative – empirical studies of people interacting with computers. – User Experience and Usability - Work informing UX/interaction/interface design, usability practice –
28. Activity: Phrase your contribution Solitary work 5 min solitary work + 15 min discussion – Go to blanked for sharing – On one-post-it, write: – Your contribution 1. The type of contribution you make (system, study, method, meta) and subcommittee in which you make it (e.g., Design, Computational Interaction, ...) 2. Your name 3. If you haven't submitted an abstract, take a current paper you are working on or past paper – Discussion You call out issues and questions you encountered – I will go through some post-its to highlight noteworthy things –
29. Q&A
30. Break! Be back in 5 minutes
31. III Structure your paper
32. What goes in a paper Your paper needs to answer 7 questions in order: Context What is the Problem in the World? 1. What is the Problem in the Field? 2. What is the Actual Problem you needed to solve here? 3. Content How did you solve the problem? 1. How do we know your solution is valid? 2. What did you find? 3. Contribution How does this move us forward? 1.
33. Your paper is a fractal1 The full paper answers 1-7 in detail – The introduction condenses 1-7 into 1 page – The abstract (and conclusion) condense 1-7 into 150 words – The title condenses 7 into 5- 10 words – Check Mensh & Kording, Ten simple rules for structuring papers – Mensh B, Kording K (2017) Ten simple rules for structuring papers. PLoS Comput Biol 13(9): e1005619. https://doi.org/10.1371/journal.pcbi.1005619 1.
34. Example Abstract1 All taken from Lisa Anthony: Writing a successful CHI paper, 2017. 1.
35. Example Abstract
36. Example Abstract
37. Example Abstract
38. How to write Start with introduction/abstract, then paper, then title – Bullet point, then write out – Be boring: use a standard structure – Guide yourself with targets: 40 paragraphs of 150-200 words each – Some suggest to merge Introduction and Background –
39. Standard Study Paper Introduction (4-5para) Background (5-6para) Method (4-5para) What is the Problem in the World? Foreshadow 2-7, paper structure 1. What is the Problem in the Field? [What concepts do we need to know to understand your work. Related Work: What have others done, what is missing] 1. What is the Actual Problem you needed to solve here? [Your research question/objective] 2. How did you solve the problem? [Your study approach] 1. How we know your solution is valid? [Detail explanation and justification of your approach] 2. Results (10-14para) Discussion (8-10para) Conclusion (1para) What did you find? [Detail results] 1. How does this move us forward? [How do findings move the Problem forward from Related Work?][How do they move the Field forward?] [Implications for design: How do findings change what practitioners should do][Limitations: Of method, of generalisability] 1.
40. Standard System Paper Introduction Background What is the Problem in the World? Foreshadow 2-7, paper structure 1. What is the Problem in the Field? 1. What is the Actual Problem you needed to solve here? 2. System - 9-11 paras Evaluation - 7-9 paras Discussion Conclusion How did you solve the problem? [Design process/principles informing your system] 1. What did you find [Your system/approach/algorithm] 2. How do we know your solution is valid? [User study or similar validation] 1. How does this move your field forward? 1. How does this move the world forward? 2.
41. Activity: Structure your introduction Group work Discussion 30 min group work + 20 min discussion – Pair up in groups of 3 – Check blanked for sharing – Individually, open your google doc and take 7 minutes to bullet point your introduction out, based on your abstract and reworked contribution – In your group, read and critique each others' outlines (7 minutes each) – You call out issues and questions you encountered – I will go through some documents to highlight noteworthy things –
42. Thank you!