$30 off During Our Annual Pro Sale. View Details »

How to write a good CHI paper (that might just get accepted)

How to write a good CHI paper (that might just get accepted)

Presentation from a workshop given at the Dyson School of Design Engineering at Imperial College London for PhD students on how to write CHI papers, targeting #chi2024.

With big thanks to those who came before, especially Lennart Nacke and Lisa Anthony, Brett Mensh and Konrad Kording.

Sebastian Deterding

July 18, 2023
Tweet

More Decks by Sebastian Deterding

Other Decks in Research

Transcript

  1. How to write a
    good CHI paper
    (that may just get
    accepted)
    Sebastian Deterding
    Imperial College London

    View Slide

  2. Why me?
    EiC ACM Games journal

    Dozens of CHI/HCI reviews

    2x CHI associate chair

    6 CHI full papers, 3 of which
    honourable mentions

    View Slide

  3. Aims
    Understand quirks of writing for
    CHI
    1.
    Identify your fitting
    contribution and subcommittee
    2.
    Structure your abstract and
    introduction (as short forms of
    the full paper)
    3.

    View Slide

  4. Writing a good CHI
    paper is 70% writing
    a good paper and
    30% knowing the
    quirks of CHI

    View Slide

  5. Agenda
    14:00-14:15 I: Getting to know CHI
    14:15-14:45 II: Making a contribution
    14:45-15:05 Activity: Framing
    contributions
    15:05-15:20 Q&A
    Break
    15:30-15:45 III: Structuring your paper
    15:45-16:30 Activity: Structuring
    introductions

    View Slide

  6. I
    Say hello to CHI

    View Slide

  7. What is CHI?
    "ACM SIGCHI Conference on Human Factors in Computing
    Systems"

    Biggest, most important conference in human-computer
    interaction: 4,670 attendees in 2023, running 40+ years

    Highly selective: 879/3,182 submissions = 27.6%
    acceptance rate in 2023. Honorable mentions = top 5%
    accepted papers, best papers = top 1%

    CHI 'full' paper is equivalent to top journal paper in
    other fields, recognised in many field (CS, informatics,
    communication, design, ...)

    Like HCI,
    a broad tent for very diverse research
    communities

    View Slide

  8. How writing for CHI is different
    Broad tent with diverse communities and
    contributions

    Short, one-off revision cycle = feasible
    revisions are limited to 4 weeks rewriting &
    reanalysis

    Annual flagship HCI conference = rewards novel
    and timely contributions to HCI (often =
    previous CHI papers)

    "Implications for design"

    View Slide

  9. The submission process for 2024
    Two-step: Abstract by Sep 7, full paper Sep 14

    Submit PDF and source files

    Use Overleaf with the conference template - word
    template is finicky!

    New target length: 8,000 words excluding references,
    figures, tables (Longer needs good reason)

    Share anonymised underlying data, analysis code,
    etc. if you can as supplementary materials or via
    repositories (OSF allows anonymous share-only link)

    ACM submission is finicky: study and check in
    advance

    View Slide

  10. Figures speak
    Insert a figure on
    p. 1 if you can.

    Check if someone can
    follow your paper
    just from figures &
    tables

    Include a video
    figure if you can
    (first anonymised)

    View Slide

  11. The review
    process

    View Slide

  12. The review process
    CHI uses a peculiar 'one-off' revise & resubmit cycle
    Every paper gets 2 AC + 2 external reviews

    Decision recommendations: A, ARR, RR, RRX, X

    If any AC recommends A/ARR/RR, paper is invited to
    revise & resubmit on Nov 7 (ca. 40%)

    Resubmit Dec 12: Clean & marked changes document,
    response to ACs

    Subcommittees decide based on AC recommendations,
    published Jan 19

    Publication-ready files due Feb 22

    Remote videos due March 28

    View Slide

  13. Other submissions to
    consider at CHI
    Workshop papers: 2-4pp position pieces.
    Typically Sat/Sun before main conference,
    great entry into community for newcomers

    Interactivity: Showcasing prototypes

    Student competitions (game, design, research)

    Late-breaking work: Poster + video

    alt.chi: Weird, provocative papers

    View Slide

  14. II
    Making a
    Contribution

    View Slide

  15. A paper gets accepted
    to CHI if ...
    it makes a contribution to HCI that
    is valid, original, transparent, and
    clear
    Handily, these are the review
    criteria:
    https://chi2024.acm.org/submission-
    guides/guide-to-a-successful-
    submission/

    Contribution and validity/rigour
    really matter

    1
    Julie R. Williamson, Understanding CHI Reviews, 2023.
    https://chi2023.acm.org/2022/09/22/understanding-chi-reviews-analysis-of-
    chi2022-revise-and-resubmit/
    1.

    View Slide

  16. Accepted papers do three
    things
    They clearly identify how they move
    HCI forward (contribution and
    originality)
    1.
    They make reviewers confident that the
    contribution is well-founded and not
    overstated (validity and transparency)
    2.
    They are easy to read (clarity)
    3.

    View Slide

  17. ... One more thing
    Don't use up the reviewer's
    reservoir of goodwill with
    ...
    poor formatting, typos,
    references

    unsupported or overstating
    detail claims

    long, waffling text

    insensitive, uncritical
    language

    View Slide

  18. What is "a
    contribution"?
    NOT "filling a gap". There are
    infinite gaps to fill

    Identify a problem/opportunity
    that matters in real life and
    to the academic audience you
    write for

    Find/make something that
    improves how we think about and
    act on that problem

    Express how we should think/act
    differently as a result

    NOT a contribution
    "We do the first
    qualitative/teenager/... study
    on X" -> Why should results
    differ? Why does that matter?

    "Sense of ownership drives
    recycling" -> How does this
    matter to HCI?

    "We made X using
    mobile/LLMs/..." -> Why is that
    needed, beneficial, previously
    hard to do?

    "Our system works/is liked" ->
    What does that tell us beyond
    your system for designing
    systems?

    "60% of surveyed users dislike
    X" -> What does that mean for
    research and practice?

    View Slide

  19. What is a contribution to the CHI
    community?
    Advances a particular subcommittee ...

    ... with
    general
    relevance to HCI: Check and reference review or
    agenda papers at CHI, interactions, other HCI journals

    Explicitly states contributions in abstract, introduction, and
    conclusion

    Honest about limitations and scope: Reviewers judge whether your
    paper supports your claims. Precise, humble claims = no attack
    surface.

    Fits a particular genre of contribution

    CHI 2024 gives a handy guide:
    https://chi2024.acm.org/submission-guides/contributions-to-chi/

    View Slide

  20. Genres of contributions1
    Jacob O. Wobbrock and Julie A. Kientz. 2016. Research contributions in
    human-computer interaction. interactions 23, 3 (May + June 2016), 38–44.
    https://doi.org/10.1145/2907069
    1.

    View Slide

  21. System contributions aka 'We made a thing'
    Make a new thing that expands and improves what interfaces, users,
    designers can do

    Finding and realising a new working solution (space) to an important
    use case that generalises beyond the system itself

    Usually documents a built artefact that enables new possibilities or
    insights, overcomes an important limitation, or (more rarely) improves
    on a benchmark

    Validates that the system works with a user study, reflection,
    critical reflection, benchmarking

    Interface Artifacts or Techniques = 'frontend' new forms of
    interaction

    Systems, Tools, Architectures, and Infrastructure = 'backend' new
    software/hardware enabling interactions

    Innovation, Creativity, and Vision = speculative novel ways of doing
    things

    View Slide

  22. Study contributions aka 'We
    collected and analysed evidence'
    Make new valid empirical observations that advance our
    understanding of how people design or interact with
    interfaces and/or improve how we design interfaces

    New grounded concepts, new effects, challenging existing
    beliefs and models, settling a debate

    Understanding Users = Making new observations about
    people interacting with computers

    Validation and Replication = Rigorously testing an
    important prior empirical result whose reliability or
    generalisability is in doubt

    View Slide

  23. Method contributions aka 'Here's
    how to do things differently'
    New validated methods for designing interactions
    or
    researching HCI that allow us to do new things or do
    things better

    Usually validated with critical reflection and
    documentation how using method improved one's own
    practice; or testing whether it improves practice of
    other practitioners

    Can be Design Methods for designers or Research
    Methods for researchers

    View Slide

  24. Meta contributions aka 'Our field needs
    to change'
    Intervene in and advance how HCI researchers think about and do
    research with compelling new concepts and arguments

    Usually validated with logical argument grounded in accurate,
    informed referencing of prior work inside and outside HCI,
    sometimes thought experiments or critical practice reflections
    how this new perspective could or did change things

    Theory = New concept or model for understanding, studying,
    designing something that demonstrably generates novel inferences
    compared to existing theory

    Argument = New important provocation to HCI researchers to
    change how to think or act

    Systematic review = New synthesis of existing work identifying
    limitations, needed future research, or consensus and clarity

    View Slide

  25. How to find and write for your
    subcommittee
    Subcommittees = communities with their own literature,
    methods and standards, writing style, and typical
    contribution (study, system, or meta)

    Different standards! E.g., Accessibility cares about
    engaging stakeholders, inclusive language, ...

    These partially mirror smaller SIGCHI conferences:
    Blending Interaction = UIST, Computational Interaction =
    IUI, Games & Play = CHI PLAY, ... (alternative submission
    places)

    Check https://chi2024.acm.org/subcommittees/selecting-a-
    subcommittee/

    Read text & example papers to find your fit and standards

    View Slide

  26. Topical subcommittees
    Accessibility and Aging

    Health

    Games and Play

    Learning, Education, and Families

    Privacy and Security

    Specific Application Areas: Designing for ICT4D,
    creativity, rural, mobility, urban, civic, more-
    than-human, ...

    Visualization - making new visualisation techniques
    or new insights into visualisation usage &
    experience

    View Slide

  27. Research genre subcommittees
    Blending Interaction: Engineering Interactive Systems & Tools - making systems
    for new kinds of interaction

    Developing Novel Devices: Hardware, Materials, and Fabrication - making new
    physical tools

    Computational Interaction - heavily quant/formal studies of (data-driven)
    algorithms for understanding, modelling, optimising interactions

    Critical Computing, Sustainability, and Social Justice – critical, often more
    humanist or designerly work on social issues

    Design – Research through, on, and for design

    Interacting with Devices: Interaction Techniques & Modalities - making new
    kinds of interaction techniques

    Interaction Beyond the Individual - CSCW/CSCL, often more ethnographic studies
    of collaboration

    Understanding People: Quant, Qual, Mixed/alternative – empirical studies of
    people interacting with computers.

    User Experience and Usability - Work informing UX/interaction/interface
    design, usability practice

    View Slide

  28. Activity: Phrase your
    contribution
    Solitary work
    5 min solitary work + 15 min
    discussion

    Go to blanked for sharing

    On one-post-it, write:

    Your contribution
    1.
    The type of contribution you
    make (system, study, method,
    meta) and subcommittee in which
    you make it (e.g., Design,
    Computational Interaction, ...)
    2.
    Your name
    3.
    If you haven't submitted an
    abstract, take a current paper you
    are working on or past paper

    Discussion
    You call out issues and questions
    you encountered

    I will go through some post-its to
    highlight noteworthy things

    View Slide

  29. Q&A

    View Slide

  30. Break!
    Be back in 5
    minutes

    View Slide

  31. III
    Structure your
    paper

    View Slide

  32. What goes in a
    paper
    Your paper needs to answer 7
    questions in order:
    Context
    What is the Problem in the
    World?
    1.
    What is the Problem in the
    Field?
    2.
    What is the Actual Problem
    you needed to solve here?
    3.
    Content
    How did you solve the
    problem?
    1.
    How do we know your solution
    is valid?
    2.
    What did you find?
    3.
    Contribution
    How does this move us
    forward?
    1.

    View Slide

  33. Your paper is a
    fractal1
    The full paper answers 1-7 in
    detail

    The introduction condenses 1-7
    into 1 page

    The abstract (and conclusion)
    condense 1-7 into 150 words

    The title condenses 7 into 5-
    10 words

    Check Mensh & Kording, Ten
    simple rules for structuring
    papers

    Mensh B, Kording K (2017) Ten simple rules for structuring papers. PLoS
    Comput Biol 13(9): e1005619. https://doi.org/10.1371/journal.pcbi.1005619
    1.

    View Slide

  34. Example Abstract1
    All taken from Lisa Anthony: Writing a successful CHI paper, 2017.
    1.

    View Slide

  35. Example Abstract

    View Slide

  36. Example Abstract

    View Slide

  37. Example Abstract

    View Slide

  38. How to write
    Start with introduction/abstract, then
    paper, then title

    Bullet point, then write out

    Be boring: use a standard structure

    Guide yourself with targets: 40
    paragraphs of 150-200 words each

    Some suggest to merge Introduction and
    Background

    View Slide

  39. Standard Study Paper
    Introduction (4-5para)
    Background (5-6para)
    Method (4-5para)
    What is the Problem in the World?
    Foreshadow 2-7, paper structure
    1.
    What is the Problem in the Field? [What
    concepts do we need to know to
    understand your work. Related Work: What
    have others done, what is missing]
    1.
    What is the Actual Problem you needed to
    solve here? [Your research
    question/objective]
    2.
    How did you solve the problem? [Your
    study approach]
    1.
    How we know your solution is valid?
    [Detail explanation and justification of
    your approach]
    2.
    Results (10-14para)
    Discussion (8-10para)
    Conclusion (1para)
    What did you find? [Detail results]
    1.
    How does this move us forward?
    [How do findings move the Problem
    forward from Related Work?][How do they
    move the Field forward?]
    [Implications for design: How do
    findings change what practitioners
    should do][Limitations: Of method, of
    generalisability]
    1.

    View Slide

  40. Standard System
    Paper
    Introduction
    Background
    What is the Problem in the World?
    Foreshadow 2-7, paper structure
    1.
    What is the Problem in the Field?
    1.
    What is the Actual Problem you
    needed to solve here?
    2.
    System - 9-11 paras
    Evaluation - 7-9 paras
    Discussion
    Conclusion
    How did you solve the problem?
    [Design process/principles
    informing your system]
    1.
    What did you find [Your
    system/approach/algorithm]
    2.
    How do we know your solution is
    valid? [User study or similar
    validation]
    1.
    How does this move your field
    forward?
    1.
    How does this move the world
    forward?
    2.

    View Slide

  41. Activity: Structure your
    introduction
    Group work
    Discussion
    30 min group work + 20 min discussion

    Pair up in groups of 3

    Check blanked for sharing

    Individually, open your google doc
    and take 7 minutes to bullet point
    your introduction out, based on your
    abstract and reworked contribution

    In your group, read and critique each
    others' outlines (7 minutes each)

    You call out issues and questions you
    encountered

    I will go through some documents to
    highlight noteworthy things

    View Slide

  42. Thank you!

    View Slide