How to write a good CHI paper (that might just get accepted)

Presentation from a workshop given at the Dyson School of Design Engineering at Imperial College London for PhD students on how to write CHI papers, targeting #chi2024.

With big thanks to those who came before, especially Lennart Nacke and Lisa Anthony, Brett Mensh and Konrad Kording.

Transcript

1. How to write a
   good CHI paper
   (that may just get
   accepted)
   Sebastian Deterding
   Imperial College London
   

   View Slide

2. Why me?
   EiC ACM Games journal
   –
   Dozens of CHI/HCI reviews
   –
   2x CHI associate chair
   –
   6 CHI full papers, 3 of which
   honourable mentions
   –
   

   View Slide

3. Aims
   Understand quirks of writing for
   CHI
   1.
   Identify your fitting
   contribution and subcommittee
   2.
   Structure your abstract and
   introduction (as short forms of
   the full paper)
   3.
   

   View Slide

4. Writing a good CHI
   paper is 70% writing
   a good paper and
   30% knowing the
   quirks of CHI
   

   View Slide

5. Agenda
   14:00-14:15 I: Getting to know CHI
   14:15-14:45 II: Making a contribution
   14:45-15:05 Activity: Framing
   contributions
   15:05-15:20 Q&A
   Break
   15:30-15:45 III: Structuring your paper
   15:45-16:30 Activity: Structuring
   introductions
   

   View Slide

6. I
   Say hello to CHI
   

   View Slide

7. What is CHI?
   "ACM SIGCHI Conference on Human Factors in Computing
   Systems"
   –
   Biggest, most important conference in human-computer
   interaction: 4,670 attendees in 2023, running 40+ years
   –
   Highly selective: 879/3,182 submissions = 27.6%
   acceptance rate in 2023. Honorable mentions = top 5%
   accepted papers, best papers = top 1%
   –
   CHI 'full' paper is equivalent to top journal paper in
   other fields, recognised in many field (CS, informatics,
   communication, design, ...)
   –
   Like HCI,
   a broad tent for very diverse research
   communities
   –
   

   View Slide

8. How writing for CHI is different
   Broad tent with diverse communities and
   contributions
   –
   Short, one-off revision cycle = feasible
   revisions are limited to 4 weeks rewriting &
   reanalysis
   –
   Annual flagship HCI conference = rewards novel
   and timely contributions to HCI (often =
   previous CHI papers)
   –
   "Implications for design"
   –
   

   View Slide

9. The submission process for 2024
   Two-step: Abstract by Sep 7, full paper Sep 14
   –
   Submit PDF and source files
   –
   Use Overleaf with the conference template - word
   template is finicky!
   –
   New target length: 8,000 words excluding references,
   figures, tables (Longer needs good reason)
   –
   Share anonymised underlying data, analysis code,
   etc. if you can as supplementary materials or via
   repositories (OSF allows anonymous share-only link)
   –
   ACM submission is finicky: study and check in
   advance
   –
   

   View Slide

10. Figures speak
    Insert a figure on
    p. 1 if you can.
    –
    Check if someone can
    follow your paper
    just from figures &
    tables
    –
    Include a video
    figure if you can
    (first anonymised)
    –
    

    View Slide

11. The review
    process
    

    View Slide

12. The review process
    CHI uses a peculiar 'one-off' revise & resubmit cycle
    Every paper gets 2 AC + 2 external reviews
    –
    Decision recommendations: A, ARR, RR, RRX, X
    –
    If any AC recommends A/ARR/RR, paper is invited to
    revise & resubmit on Nov 7 (ca. 40%)
    –
    Resubmit Dec 12: Clean & marked changes document,
    response to ACs
    –
    Subcommittees decide based on AC recommendations,
    published Jan 19
    –
    Publication-ready files due Feb 22
    –
    Remote videos due March 28
    –
    

    View Slide

13. Other submissions to
    consider at CHI
    Workshop papers: 2-4pp position pieces.
    Typically Sat/Sun before main conference,
    great entry into community for newcomers
    –
    Interactivity: Showcasing prototypes
    –
    Student competitions (game, design, research)
    –
    Late-breaking work: Poster + video
    –
    alt.chi: Weird, provocative papers
    –
    

    View Slide

14. II
    Making a
    Contribution
    

    View Slide

15. A paper gets accepted
    to CHI if ...
    it makes a contribution to HCI that
    is valid, original, transparent, and
    clear
    Handily, these are the review
    criteria:
    https://chi2024.acm.org/submission-
    guides/guide-to-a-successful-
    submission/
    –
    Contribution and validity/rigour
    really matter
    –
    1
    Julie R. Williamson, Understanding CHI Reviews, 2023.
    https://chi2023.acm.org/2022/09/22/understanding-chi-reviews-analysis-of-
    chi2022-revise-and-resubmit/
    1.
    

    View Slide

16. Accepted papers do three
    things
    They clearly identify how they move
    HCI forward (contribution and
    originality)
    1.
    They make reviewers confident that the
    contribution is well-founded and not
    overstated (validity and transparency)
    2.
    They are easy to read (clarity)
    3.
    

    View Slide

17. ... One more thing
    Don't use up the reviewer's
    reservoir of goodwill with
    ...
    poor formatting, typos,
    references
    –
    unsupported or overstating
    detail claims
    –
    long, waffling text
    –
    insensitive, uncritical
    language
    –
    

    View Slide

18. What is "a
    contribution"?
    NOT "filling a gap". There are
    infinite gaps to fill
    –
    Identify a problem/opportunity
    that matters in real life and
    to the academic audience you
    write for
    –
    Find/make something that
    improves how we think about and
    act on that problem
    –
    Express how we should think/act
    differently as a result
    –
    NOT a contribution
    "We do the first
    qualitative/teenager/... study
    on X" -> Why should results
    differ? Why does that matter?
    –
    "Sense of ownership drives
    recycling" -> How does this
    matter to HCI?
    –
    "We made X using
    mobile/LLMs/..." -> Why is that
    needed, beneficial, previously
    hard to do?
    –
    "Our system works/is liked" ->
    What does that tell us beyond
    your system for designing
    systems?
    –
    "60% of surveyed users dislike
    X" -> What does that mean for
    research and practice?
    –
    

    View Slide

19. What is a contribution to the CHI
    community?
    Advances a particular subcommittee ...
    –
    ... with
    general
    relevance to HCI: Check and reference review or
    agenda papers at CHI, interactions, other HCI journals
    –
    Explicitly states contributions in abstract, introduction, and
    conclusion
    –
    Honest about limitations and scope: Reviewers judge whether your
    paper supports your claims. Precise, humble claims = no attack
    surface.
    –
    Fits a particular genre of contribution
    –
    CHI 2024 gives a handy guide:
    https://chi2024.acm.org/submission-guides/contributions-to-chi/
    –
    

    View Slide

20. Genres of contributions1
    Jacob O. Wobbrock and Julie A. Kientz. 2016. Research contributions in
    human-computer interaction. interactions 23, 3 (May + June 2016), 38–44.
    https://doi.org/10.1145/2907069
    1.
    

    View Slide

21. System contributions aka 'We made a thing'
    Make a new thing that expands and improves what interfaces, users,
    designers can do
    –
    Finding and realising a new working solution (space) to an important
    use case that generalises beyond the system itself
    –
    Usually documents a built artefact that enables new possibilities or
    insights, overcomes an important limitation, or (more rarely) improves
    on a benchmark
    –
    Validates that the system works with a user study, reflection,
    critical reflection, benchmarking
    –
    Interface Artifacts or Techniques = 'frontend' new forms of
    interaction
    –
    Systems, Tools, Architectures, and Infrastructure = 'backend' new
    software/hardware enabling interactions
    –
    Innovation, Creativity, and Vision = speculative novel ways of doing
    things
    –
    

    View Slide

22. Study contributions aka 'We
    collected and analysed evidence'
    Make new valid empirical observations that advance our
    understanding of how people design or interact with
    interfaces and/or improve how we design interfaces
    –
    New grounded concepts, new effects, challenging existing
    beliefs and models, settling a debate
    –
    Understanding Users = Making new observations about
    people interacting with computers
    –
    Validation and Replication = Rigorously testing an
    important prior empirical result whose reliability or
    generalisability is in doubt
    –
    

    View Slide

23. Method contributions aka 'Here's
    how to do things differently'
    New validated methods for designing interactions
    or
    researching HCI that allow us to do new things or do
    things better
    –
    Usually validated with critical reflection and
    documentation how using method improved one's own
    practice; or testing whether it improves practice of
    other practitioners
    –
    Can be Design Methods for designers or Research
    Methods for researchers
    –
    

    View Slide

24. Meta contributions aka 'Our field needs
    to change'
    Intervene in and advance how HCI researchers think about and do
    research with compelling new concepts and arguments
    –
    Usually validated with logical argument grounded in accurate,
    informed referencing of prior work inside and outside HCI,
    sometimes thought experiments or critical practice reflections
    how this new perspective could or did change things
    –
    Theory = New concept or model for understanding, studying,
    designing something that demonstrably generates novel inferences
    compared to existing theory
    –
    Argument = New important provocation to HCI researchers to
    change how to think or act
    –
    Systematic review = New synthesis of existing work identifying
    limitations, needed future research, or consensus and clarity
    –
    

    View Slide

25. How to find and write for your
    subcommittee
    Subcommittees = communities with their own literature,
    methods and standards, writing style, and typical
    contribution (study, system, or meta)
    –
    Different standards! E.g., Accessibility cares about
    engaging stakeholders, inclusive language, ...
    –
    These partially mirror smaller SIGCHI conferences:
    Blending Interaction = UIST, Computational Interaction =
    IUI, Games & Play = CHI PLAY, ... (alternative submission
    places)
    –
    Check https://chi2024.acm.org/subcommittees/selecting-a-
    subcommittee/
    –
    Read text & example papers to find your fit and standards
    –
    

    View Slide

26. Topical subcommittees
    Accessibility and Aging
    –
    Health
    –
    Games and Play
    –
    Learning, Education, and Families
    –
    Privacy and Security
    –
    Specific Application Areas: Designing for ICT4D,
    creativity, rural, mobility, urban, civic, more-
    than-human, ...
    –
    Visualization - making new visualisation techniques
    or new insights into visualisation usage &
    experience
    –
    

    View Slide

27. Research genre subcommittees
    Blending Interaction: Engineering Interactive Systems & Tools - making systems
    for new kinds of interaction
    –
    Developing Novel Devices: Hardware, Materials, and Fabrication - making new
    physical tools
    –
    Computational Interaction - heavily quant/formal studies of (data-driven)
    algorithms for understanding, modelling, optimising interactions
    –
    Critical Computing, Sustainability, and Social Justice – critical, often more
    humanist or designerly work on social issues
    –
    Design – Research through, on, and for design
    –
    Interacting with Devices: Interaction Techniques & Modalities - making new
    kinds of interaction techniques
    –
    Interaction Beyond the Individual - CSCW/CSCL, often more ethnographic studies
    of collaboration
    –
    Understanding People: Quant, Qual, Mixed/alternative – empirical studies of
    people interacting with computers.
    –
    User Experience and Usability - Work informing UX/interaction/interface
    design, usability practice
    –
    

    View Slide

28. Activity: Phrase your
    contribution
    Solitary work
    5 min solitary work + 15 min
    discussion
    –
    Go to blanked for sharing
    –
    On one-post-it, write:
    –
    Your contribution
    1.
    The type of contribution you
    make (system, study, method,
    meta) and subcommittee in which
    you make it (e.g., Design,
    Computational Interaction, ...)
    2.
    Your name
    3.
    If you haven't submitted an
    abstract, take a current paper you
    are working on or past paper
    –
    Discussion
    You call out issues and questions
    you encountered
    –
    I will go through some post-its to
    highlight noteworthy things
    –
    

    View Slide

29. Q&A
    

    View Slide

30. Break!
    Be back in 5
    minutes
    

    View Slide

31. III
    Structure your
    paper
    

    View Slide

32. What goes in a
    paper
    Your paper needs to answer 7
    questions in order:
    Context
    What is the Problem in the
    World?
    1.
    What is the Problem in the
    Field?
    2.
    What is the Actual Problem
    you needed to solve here?
    3.
    Content
    How did you solve the
    problem?
    1.
    How do we know your solution
    is valid?
    2.
    What did you find?
    3.
    Contribution
    How does this move us
    forward?
    1.
    

    View Slide

33. Your paper is a
    fractal1
    The full paper answers 1-7 in
    detail
    –
    The introduction condenses 1-7
    into 1 page
    –
    The abstract (and conclusion)
    condense 1-7 into 150 words
    –
    The title condenses 7 into 5-
    10 words
    –
    Check Mensh & Kording, Ten
    simple rules for structuring
    papers
    –
    Mensh B, Kording K (2017) Ten simple rules for structuring papers. PLoS
    Comput Biol 13(9): e1005619. https://doi.org/10.1371/journal.pcbi.1005619
    1.
    

    View Slide

34. Example Abstract1
    All taken from Lisa Anthony: Writing a successful CHI paper, 2017.
    1.
    

    View Slide

35. Example Abstract
    

    View Slide

36. Example Abstract
    

    View Slide

37. Example Abstract
    

    View Slide

38. How to write
    Start with introduction/abstract, then
    paper, then title
    –
    Bullet point, then write out
    –
    Be boring: use a standard structure
    –
    Guide yourself with targets: 40
    paragraphs of 150-200 words each
    –
    Some suggest to merge Introduction and
    Background
    –
    

    View Slide

39. Standard Study Paper
    Introduction (4-5para)
    Background (5-6para)
    Method (4-5para)
    What is the Problem in the World?
    Foreshadow 2-7, paper structure
    1.
    What is the Problem in the Field? [What
    concepts do we need to know to
    understand your work. Related Work: What
    have others done, what is missing]
    1.
    What is the Actual Problem you needed to
    solve here? [Your research
    question/objective]
    2.
    How did you solve the problem? [Your
    study approach]
    1.
    How we know your solution is valid?
    [Detail explanation and justification of
    your approach]
    2.
    Results (10-14para)
    Discussion (8-10para)
    Conclusion (1para)
    What did you find? [Detail results]
    1.
    How does this move us forward?
    [How do findings move the Problem
    forward from Related Work?][How do they
    move the Field forward?]
    [Implications for design: How do
    findings change what practitioners
    should do][Limitations: Of method, of
    generalisability]
    1.
    

    View Slide

40. Standard System
    Paper
    Introduction
    Background
    What is the Problem in the World?
    Foreshadow 2-7, paper structure
    1.
    What is the Problem in the Field?
    1.
    What is the Actual Problem you
    needed to solve here?
    2.
    System - 9-11 paras
    Evaluation - 7-9 paras
    Discussion
    Conclusion
    How did you solve the problem?
    [Design process/principles
    informing your system]
    1.
    What did you find [Your
    system/approach/algorithm]
    2.
    How do we know your solution is
    valid? [User study or similar
    validation]
    1.
    How does this move your field
    forward?
    1.
    How does this move the world
    forward?
    2.
    

    View Slide

41. Activity: Structure your
    introduction
    Group work
    Discussion
    30 min group work + 20 min discussion
    –
    Pair up in groups of 3
    –
    Check blanked for sharing
    –
    Individually, open your google doc
    and take 7 minutes to bullet point
    your introduction out, based on your
    abstract and reworked contribution
    –
    In your group, read and critique each
    others' outlines (7 minutes each)
    –
    You call out issues and questions you
    encountered
    –
    I will go through some documents to
    highlight noteworthy things
    –
    

    View Slide

42. Thank you!
    

    View Slide