Construction of a Digital Exam Grading System using TeX

Presentation slides at TUG 2019 (the 40th annual meeting of the TeX Users Group), Palo Alto, California.

Transcript

1. $POTUSVDUJPO
   PG
   B
   %JHJUBM
   &YBN
   (SBEJOH
   4ZTUFN
   VTJOH
   5F9 :VTVLF
   5&3"%"
   5FUTVSZPLVLBJ
   5PLZP
   &EVDBUJPOBM
   *OTUJUVUF
   +BQBO August 11, 2019 TUG 2019 @ Palo Alto
2. "CPVU
   .F Yusuke TERADA from Tokyo, Japan A private preparatory school teacher for high-school students Japanese TeX Development Community A contributor to TeXShop development Major work: TeX2img https://tex2img.tech/
3. 56(
   
   !
   +BQBO
6. 6UJMJ[F
   5F9
   GPS
   QSPNPUJOH
   PQFSBUJPOBM
   F⒏DJFODZ
   JO
   TDIPPM
7. TUBHF
   4FMFDUJPO ✦ Examinees: over 570,000 students ✦ Multiple-choice test ✦ Automatically scored by computers ✦ Used as cut-offs for the 2nd stage First Stage National Center Test for
 University Admissions
8. TUBHF
   4FMFDUJPO ✦ Examinees: students who passed the 1st stage
 (Required score depends on each university) ✦ Style: descriptive paper test ✦ Graders: professors at each university Second Stage Exams conducted by
 individual universities
9. 1SFQBSBUPSZ
   4DIPPMT Privately-run schools which provide education supplementary to public education Aim to help students prepare entrance exams Conduct practice exams on a regular basis
10. 4QFDJpDBUJPOT
    PG
    1SBDUJDF
    &YBNT Paper-and-pencil exam in a descriptive manner Examinees: over 800 students Graded by humans Assigned time for grading: less than one week Fairness is strictly required Graders correct answers and add comments Return feedback to examinees
11. Answer sheet
12. Pile of answer sheets
13. #FGPSF
    5F9 1. Sort answer sheets by hand 2. Divide a pile of answer sheets into some piles 3. Graders score their assigned problems in parallel 4. Exchange the piles to each other
14. Divided piles
15. Phase 1 Grader 1 Grader 2 Grader 3
16. Phase 2 Grader 1 Grader 2 Grader 3
17. Phase 3 Grader 1 Grader 2 Grader 3
18. #FGPSF
    5F9 5. Transcribe hand-written scores into Excel 6. Cross-check the transcribed scores visually
19. #FGPSF
    5F9 5. Transcribe hand-written scores into Excel 6. Cross-check the transcribed scores visually Insanely Inefficient
20. #FGPSF
    5F9 5. Transcribe hand-written scores into Excel 6. Cross-check the transcribed scores visually Insanely Inefficient Improving efficiency using IT is imperative!
21. 3FRVJSFNFOUT
    GPS
    UIF
 /FX
    %JHJUBM
    (SBEJOH
    4ZTUFN Paper-and-pencil style must be maintained. Multiple graders should be able to score in parallel on the digital system. Privacy and fairness should be secured. The system must be run on macOS/iOS. Feedback to examinees should be returned in paper.
22. 4PMVUJPO Construct a Digital Exam Grading System Utilize TeX as a PDF manipulation tool
23. 5F9
    TZTUFN
    BT
    B
    1%'
    NBOJQVMBUJPO
    UPPM pdfTeX has a lot of powerful primitives that manipulate PDF since early times. Recently (x)dvipdfmx has been enhanced in terms of PDF manipulation.
24. 3FDFOU
    FOIBODFNFOUT
    PG
    Y EWJQEGNY extractbb is automatically invoked when needed. (TL2015) page option of \includegraphics is supported. (TL2015) pagebox option of \includegraphics is supported. (TL2016) \includegraphics supports direct input of Adobe Illustrator files. (TL2014) pdfpages package supports (x)dvipdfmx. (TL2015) (x)dvipdfmx supports rotated PDF pages. (TL2018)
25. \includegraphics
    PQUJPOT \includegraphics[page=2, pagebox=artbox, viewport=0 20 40 80, scale=2.5, clip] {foobar.pdf} pageɿPDF page number to include pageboxɿmediabox/cropbox/trimbox/bleedbox/artbox viewportɿbounding box relative to pagebox scaleɿmagnification scale factor clipɿhide outside of bounding box
26. 5IF
    XPSLqPX
    PG
    UIF
    OFXMZ
    CVJMU
    EJHJUBM
    HSBEJOH
    TZTUFN
    VTJOH
    5F9
27. 4UFQ
    
    1SFQBSF
    "OTXFS
    4IFFUT Make individualized answer sheets using TeX
29. Collection of \rule's
30. Criterial marks used for image correction Drawn with TikZ
31. Individualized sheet QR code generated by qrcode package
32. RSDPEF
    QBDLBHF QR code generator written in pure TeX Developed by Anders Hendrickson Included in CTAN and TeX Live
33. RSDPEF
    QBDLBHF QR code generator written in pure TeX Developed by Anders Hendrickson Included in CTAN and TeX Live Each block is drawn with \rule
34. After conducting an exam, scan the collected answer sheets and obtain scanned-PDF. 4UFQ
    
    4DBO
    BOTXFS
    TIFFUT
35. Scanned answer sheets
36. 4UFQ
    
    3FBE
    TPSU
    BOE
    DPSSFDU
    JNBHFT Identify the examinee of each sheet based on the embedded QR code Sort images by the numbers encoded in the QR codes Correct the scanned-images based on the criterial marks Paper orientation Translation Rotation Distortion
37. %Z/".J,4 DyNamic Asynchronous Mark-interpretation Kernel System Developed by Mitsuhiro DAIMON (Physics teacher at our school) Written in Swift and Objective-C Run natively on macOS Use macOS Cocoa API for PDF handling Adopt OpenCV as its image-processing engine Being developed on GitHub
 (but currently not open-sourced, sorry.)
39. Recognize images with high accuracy
40. By using
 
 Split each answer sheet Make grading forms 4UFQ
    
    .BLF
    HSBEJOH
    GPSNT \includegraphics[page=...,viewport=...]
42. Grading Form
43. (SBEJOH
    'PSN Import only specific problems assigned to each grader
44. (SBEJOH
    'PSN Remove name and import QR code For privacy and fairness
45. (SBEJOH
    'PSN Distinguish background by color White area: feedback Gray area: note \pagecolor{black!20!white}
46. Evaluation criteria (SBEJOH
    'PSN
47. (SBEJOH
    'PSN Mark-columns for scores
48. iPad Pro + Apple Pencil + GoodNotes Hardware Software 4UFQ
    
    1FSGPSN
    HSBEJOH
    XJUI
    J1BE
49. After grading
50. Write feedback within white area Fill scores
51. 4UFQ
    
    &YQPSU
    HSBEJOH
    GPSNT
    BT
    1%'
52. 4UFQ
    
    3FBE
    TDPSFT
    XJUI
    %Z/".J,4 Collect (examinee number, score) data examinee number score
53. Restore answer sheets to return from grading forms 4UFQ
    
    3FTUPSF
    BOTXFS
    TIFFUT
54. Grading forms of a single examinee
55. Grading forms of a single examinee Merge feedback areas into one sheet
56. Restored answer sheet
57. Each area is pasted from grading forms with \includegraphics
58. Examinee's name is restored from the original PDF with \includegraphics
59. TeX overprints score digits read by DyNAMiKS on the sheet image
60. 4UFQ
    
    1SJOU
    TIFFUT
    BOE
    SFUVSO
    GFFECBDL Print the restored answer sheets in color Return them to students in classrooms
61. Analysis sheet
62. Analysis sheet Automatically generated by TikZ + tcolorbox
63. 8PSLqPX
    4VNNBSZ 1. Make individualized answer sheets by TeX and conduct an exam. 2. Scan the answer sheets into PDF. 3. Sort and correct the scanned-PDF with DyNAMiKS. 4. Split them and generate grading forms by TeX. 5. Perform grading with iPad Pro + Apple Pencil + GoodNotes. 6. Read scores by DyNAMiKS. 7. Restore answer sheets and overprint scores by TeX. 8. Generate analysis sheets by TikZ + tcolorbox. 9. Print sheets and return them to students in paper.
64. "MMJO0OF
    "QQ macOS GUI App
65. 3FTVMU Time for grading: shortened Fairness: enhanced Risk of losing answer sheets: disappeared Transcription by hand: no longer necessary
66. 3FTVMU Time for grading: shortened Fairness: enhanced Risk of losing answer sheets: disappeared Transcription by hand: no longer necessary Efficiency: improved
67. %J⒏DVMUZ
    "Too many images in a document" × 800 students
68. texk/dvipdfm-x/spc_pdfm.c texk/dvipdfm-x/pdfximage.h
69. texk/dvipdfm-x/spc_pdfm.c texk/dvipdfm-x/pdfximage.h Not enough!
70. texk/dvipdfm-x/spc_pdfm.c texk/dvipdfm-x/pdfximage.h Not enough! This limitation was abolished at TeX Live 2018
71. 'VUVSF
    8PSL
    Handwritten Digit Recognition
72. 'VUVSF
    8PSL
    MNIST
73. 'VUVSF
    8PSL
    Our digits MNIST
74. $PODMVTJPO TeX is useful also as a PDF manipulation tool. TeX has enormous potential to automate daily tasks and enhance efficiency. Explore further utilization of TeX 
 as an automation tool!