$30 off During Our Annual Pro Sale. View Details »

Construction of a Digital Exam Grading System using TeX

doraTeX
August 11, 2019

Construction of a Digital Exam Grading System using TeX

Presentation slides at TUG 2019 (the 40th annual meeting of the TeX Users Group), Palo Alto, California.

doraTeX

August 11, 2019
Tweet

More Decks by doraTeX

Other Decks in Technology

Transcript

  1. $POTUSVDUJPOPGB%JHJUBM&YBN
    (SBEJOH4ZTUFNVTJOH5F9
    :VTVLF5&3"%"
    5FUTVSZPLVLBJ 5PLZP&EVDBUJPOBM*OTUJUVUF
    +BQBO
    August 11, 2019
    TUG 2019 @ Palo Alto

    View Slide

  2. "CPVU.F
    Yusuke TERADA from Tokyo, Japan
    A private preparatory school teacher for
    high-school students
    Japanese TeX Development Community
    A contributor to TeXShop development
    Major work: TeX2img https://tex2img.tech/

    View Slide

  3. 56(!+BQBO

    View Slide

  4. View Slide

  5. View Slide

  6. 6UJMJ[F5F9GPSQSPNPUJOH
    PQFSBUJPOBMF⒏DJFODZJOTDIPPM

    View Slide

  7. TUBHF4FMFDUJPO
    ✦ Examinees: over 570,000 students
    ✦ Multiple-choice test
    ✦ Automatically scored by computers
    ✦ Used as cut-offs for the 2nd stage
    First Stage
    National Center Test for

    University Admissions

    View Slide

  8. TUBHF4FMFDUJPO
    ✦ Examinees: students who passed the 1st stage

    (Required score depends on each university)
    ✦ Style: descriptive paper test
    ✦ Graders: professors at each university
    Second Stage
    Exams conducted by

    individual universities

    View Slide

  9. 1SFQBSBUPSZ4DIPPMT
    Privately-run schools which provide education
    supplementary to public education
    Aim to help students prepare entrance exams
    Conduct practice exams on a regular basis

    View Slide

  10. 4QFDJpDBUJPOTPG1SBDUJDF&YBNT
    Paper-and-pencil exam in a descriptive manner
    Examinees: over 800 students
    Graded by humans
    Assigned time for grading: less than one week
    Fairness is strictly required
    Graders correct answers and add comments
    Return feedback to examinees

    View Slide

  11. Answer sheet

    View Slide

  12. Pile of answer sheets

    View Slide

  13. #FGPSF5F9
    1. Sort answer sheets by hand
    2. Divide a pile of answer sheets into some
    piles
    3. Graders score their assigned problems in
    parallel
    4. Exchange the piles to each other

    View Slide

  14. Divided piles

    View Slide

  15. Phase 1
    Grader 1
    Grader 2
    Grader 3

    View Slide

  16. Phase 2
    Grader 1
    Grader 2
    Grader 3

    View Slide

  17. Phase 3
    Grader 1
    Grader 2
    Grader 3

    View Slide

  18. #FGPSF5F9
    5. Transcribe hand-written scores
    into Excel
    6. Cross-check the transcribed
    scores visually

    View Slide

  19. #FGPSF5F9
    5. Transcribe hand-written scores
    into Excel
    6. Cross-check the transcribed
    scores visually
    Insanely Inefficient

    View Slide

  20. #FGPSF5F9
    5. Transcribe hand-written scores
    into Excel
    6. Cross-check the transcribed
    scores visually
    Insanely Inefficient
    Improving efficiency
    using IT is imperative!

    View Slide

  21. 3FRVJSFNFOUTGPSUIF

    /FX%JHJUBM(SBEJOH4ZTUFN
    Paper-and-pencil style must be maintained.
    Multiple graders should be able to score in parallel
    on the digital system.
    Privacy and fairness should be secured.
    The system must be run on macOS/iOS.
    Feedback to examinees should be returned in paper.

    View Slide

  22. 4PMVUJPO
    Construct a Digital Exam Grading System
    Utilize TeX as a PDF manipulation tool

    View Slide

  23. 5F9TZTUFNBTB1%'
    NBOJQVMBUJPOUPPM
    pdfTeX has a lot of powerful primitives
    that manipulate PDF since early times.
    Recently (x)dvipdfmx has been enhanced
    in terms of PDF manipulation.

    View Slide

  24. 3FDFOUFOIBODFNFOUTPG
    Y
    EWJQEGNY
    extractbb is automatically invoked when needed. (TL2015)
    page option of \includegraphics is supported. (TL2015)
    pagebox option of \includegraphics is supported. (TL2016)
    \includegraphics supports direct input of Adobe
    Illustrator files. (TL2014)
    pdfpages package supports (x)dvipdfmx. (TL2015)
    (x)dvipdfmx supports rotated PDF pages. (TL2018)

    View Slide

  25. \includegraphicsPQUJPOT
    \includegraphics[page=2, pagebox=artbox,
    viewport=0 20 40 80, scale=2.5, clip]
    {foobar.pdf}
    pageɿPDF page number to include
    pageboxɿmediabox/cropbox/trimbox/bleedbox/artbox
    viewportɿbounding box relative to pagebox
    scaleɿmagnification scale factor
    clipɿhide outside of bounding box

    View Slide

  26. 5IFXPSLqPXPGUIFOFXMZCVJMU
    EJHJUBMHSBEJOHTZTUFNVTJOH5F9

    View Slide

  27. 4UFQ1SFQBSF"OTXFS4IFFUT
    Make individualized answer sheets using TeX

    View Slide

  28. View Slide

  29. Collection of \rule's

    View Slide

  30. Criterial marks used
    for image correction
    Drawn
    with TikZ

    View Slide

  31. Individualized sheet
    QR code generated by qrcode package

    View Slide

  32. RSDPEFQBDLBHF
    QR code generator written in pure TeX
    Developed by Anders Hendrickson
    Included in CTAN and TeX Live

    View Slide

  33. RSDPEFQBDLBHF
    QR code generator written in pure TeX
    Developed by Anders Hendrickson
    Included in CTAN and TeX Live
    Each block is drawn with \rule

    View Slide

  34. After conducting an exam, scan the collected answer sheets
    and obtain scanned-PDF.
    4UFQ4DBOBOTXFSTIFFUT

    View Slide

  35. Scanned answer sheets

    View Slide

  36. 4UFQ3FBE TPSUBOEDPSSFDUJNBHFT
    Identify the examinee of each sheet
    based on the embedded QR code
    Sort images by the numbers
    encoded in the QR codes
    Correct the scanned-images based
    on the criterial marks
    Paper orientation
    Translation
    Rotation
    Distortion

    View Slide

  37. %Z/".J,4
    DyNamic Asynchronous Mark-interpretation Kernel System
    Developed by Mitsuhiro DAIMON (Physics teacher at our school)
    Written in Swift and Objective-C
    Run natively on macOS
    Use macOS Cocoa API for PDF handling
    Adopt OpenCV as its image-processing engine
    Being developed on GitHub

    (but currently not open-sourced, sorry.)

    View Slide

  38. View Slide

  39. Recognize images with
    high accuracy

    View Slide

  40. By using


    Split each answer sheet
    Make grading forms
    4UFQ.BLFHSBEJOHGPSNT
    \includegraphics[page=...,viewport=...]

    View Slide

  41. View Slide

  42. Grading Form

    View Slide

  43. (SBEJOH'PSN
    Import only specific problems
    assigned to each
    grader

    View Slide

  44. (SBEJOH'PSN
    Remove name and import QR code
    For privacy and
    fairness

    View Slide

  45. (SBEJOH'PSN
    Distinguish background
    by color
    White area: feedback
    Gray area: note
    \pagecolor{black!20!white}

    View Slide

  46. Evaluation criteria
    (SBEJOH'PSN

    View Slide

  47. (SBEJOH'PSN
    Mark-columns for scores

    View Slide

  48. iPad Pro + Apple Pencil + GoodNotes
    Hardware Software
    4UFQ1FSGPSNHSBEJOHXJUIJ1BE

    View Slide

  49. After grading

    View Slide

  50. Write feedback within white area
    Fill scores

    View Slide

  51. 4UFQ&YQPSUHSBEJOHGPSNTBT1%'

    View Slide

  52. 4UFQ3FBETDPSFTXJUI%Z/".J,4
    Collect (examinee number, score) data
    examinee number
    score

    View Slide

  53. Restore answer sheets to
    return from grading forms
    4UFQ3FTUPSFBOTXFSTIFFUT

    View Slide

  54. Grading forms of a single examinee

    View Slide

  55. Grading forms of a single examinee
    Merge feedback areas into one sheet

    View Slide

  56. Restored answer sheet

    View Slide

  57. Each area is pasted from grading forms with
    \includegraphics

    View Slide

  58. Examinee's name is restored from the
    original PDF with \includegraphics

    View Slide

  59. TeX overprints score digits read by
    DyNAMiKS on the sheet image

    View Slide

  60. 4UFQ1SJOUTIFFUTBOESFUVSOGFFECBDL
    Print the restored answer sheets in color
    Return them to students in classrooms

    View Slide

  61. Analysis sheet

    View Slide

  62. Analysis sheet
    Automatically generated
    by TikZ + tcolorbox

    View Slide

  63. 8PSLqPX4VNNBSZ
    1. Make individualized answer sheets by TeX and conduct an exam.
    2. Scan the answer sheets into PDF.
    3. Sort and correct the scanned-PDF with DyNAMiKS.
    4. Split them and generate grading forms by TeX.
    5. Perform grading with iPad Pro + Apple Pencil + GoodNotes.
    6. Read scores by DyNAMiKS.
    7. Restore answer sheets and overprint scores by TeX.
    8. Generate analysis sheets by TikZ + tcolorbox.
    9. Print sheets and return them to students in paper.

    View Slide

  64. "MMJO0OF"QQ
    macOS GUI App

    View Slide

  65. 3FTVMU
    Time for grading: shortened
    Fairness: enhanced
    Risk of losing answer sheets: disappeared
    Transcription by hand: no longer necessary

    View Slide

  66. 3FTVMU
    Time for grading: shortened
    Fairness: enhanced
    Risk of losing answer sheets: disappeared
    Transcription by hand: no longer necessary
    Efficiency: improved

    View Slide

  67. %J⒏DVMUZ
    "Too many images in a document"
    × 800 students

    View Slide

  68. texk/dvipdfm-x/spc_pdfm.c
    texk/dvipdfm-x/pdfximage.h

    View Slide

  69. texk/dvipdfm-x/spc_pdfm.c
    texk/dvipdfm-x/pdfximage.h
    Not enough!

    View Slide

  70. texk/dvipdfm-x/spc_pdfm.c
    texk/dvipdfm-x/pdfximage.h
    Not enough!
    This limitation was
    abolished at TeX Live 2018

    View Slide

  71. 'VUVSF8PSL
    Handwritten Digit Recognition

    View Slide

  72. 'VUVSF8PSL
    MNIST

    View Slide

  73. 'VUVSF8PSL
    Our digits MNIST

    View Slide

  74. $PODMVTJPO
    TeX is useful also as a PDF manipulation tool.
    TeX has enormous potential to automate daily tasks
    and enhance efficiency.
    Explore further utilization of TeX 

    as an automation tool!

    View Slide