Construction of a Digital Exam Grading System using TeX

33d84247eeffa16885b98b922d5843f7?s=47 doraTeX
August 11, 2019

Construction of a Digital Exam Grading System using TeX

Presentation slides at TUG 2019 (the 40th annual meeting of the TeX Users Group), Palo Alto, California.

33d84247eeffa16885b98b922d5843f7?s=128

doraTeX

August 11, 2019
Tweet

Transcript

  1. $POTUSVDUJPOPGB%JHJUBM&YBN (SBEJOH4ZTUFNVTJOH5F9 :VTVLF5&3"%" 5FUTVSZPLVLBJ 5PLZP&EVDBUJPOBM*OTUJUVUF +BQBO August 11, 2019 TUG

    2019 @ Palo Alto
  2. "CPVU.F Yusuke TERADA from Tokyo, Japan A private preparatory school

    teacher for high-school students Japanese TeX Development Community A contributor to TeXShop development Major work: TeX2img https://tex2img.tech/
  3. 56(!+BQBO

  4. None
  5. None
  6. 6UJMJ[F5F9GPSQSPNPUJOH PQFSBUJPOBMF⒏DJFODZJOTDIPPM

  7. TUBHF4FMFDUJPO ✦ Examinees: over 570,000 students ✦ Multiple-choice test ✦

    Automatically scored by computers ✦ Used as cut-offs for the 2nd stage First Stage National Center Test for
 University Admissions
  8. TUBHF4FMFDUJPO ✦ Examinees: students who passed the 1st stage
 (Required

    score depends on each university) ✦ Style: descriptive paper test ✦ Graders: professors at each university Second Stage Exams conducted by
 individual universities
  9. 1SFQBSBUPSZ4DIPPMT Privately-run schools which provide education supplementary to public education

    Aim to help students prepare entrance exams Conduct practice exams on a regular basis
  10. 4QFDJpDBUJPOTPG1SBDUJDF&YBNT Paper-and-pencil exam in a descriptive manner Examinees: over 800

    students Graded by humans Assigned time for grading: less than one week Fairness is strictly required Graders correct answers and add comments Return feedback to examinees
  11. Answer sheet

  12. Pile of answer sheets

  13. #FGPSF5F9 1. Sort answer sheets by hand 2. Divide a

    pile of answer sheets into some piles 3. Graders score their assigned problems in parallel 4. Exchange the piles to each other
  14. Divided piles

  15. Phase 1 Grader 1 Grader 2 Grader 3

  16. Phase 2 Grader 1 Grader 2 Grader 3

  17. Phase 3 Grader 1 Grader 2 Grader 3

  18. #FGPSF5F9 5. Transcribe hand-written scores into Excel 6. Cross-check the

    transcribed scores visually
  19. #FGPSF5F9 5. Transcribe hand-written scores into Excel 6. Cross-check the

    transcribed scores visually Insanely Inefficient
  20. #FGPSF5F9 5. Transcribe hand-written scores into Excel 6. Cross-check the

    transcribed scores visually Insanely Inefficient Improving efficiency using IT is imperative!
  21. 3FRVJSFNFOUTGPSUIF
 /FX%JHJUBM(SBEJOH4ZTUFN Paper-and-pencil style must be maintained. Multiple graders should

    be able to score in parallel on the digital system. Privacy and fairness should be secured. The system must be run on macOS/iOS. Feedback to examinees should be returned in paper.
  22. 4PMVUJPO Construct a Digital Exam Grading System Utilize TeX as

    a PDF manipulation tool
  23. 5F9TZTUFNBTB1%' NBOJQVMBUJPOUPPM pdfTeX has a lot of powerful primitives that

    manipulate PDF since early times. Recently (x)dvipdfmx has been enhanced in terms of PDF manipulation.
  24. 3FDFOUFOIBODFNFOUTPG Y EWJQEGNY extractbb is automatically invoked when needed. (TL2015)

    page option of \includegraphics is supported. (TL2015) pagebox option of \includegraphics is supported. (TL2016) \includegraphics supports direct input of Adobe Illustrator files. (TL2014) pdfpages package supports (x)dvipdfmx. (TL2015) (x)dvipdfmx supports rotated PDF pages. (TL2018)
  25. \includegraphicsPQUJPOT \includegraphics[page=2, pagebox=artbox, viewport=0 20 40 80, scale=2.5, clip] {foobar.pdf}

    pageɿPDF page number to include pageboxɿmediabox/cropbox/trimbox/bleedbox/artbox viewportɿbounding box relative to pagebox scaleɿmagnification scale factor clipɿhide outside of bounding box
  26. 5IFXPSLqPXPGUIFOFXMZCVJMU EJHJUBMHSBEJOHTZTUFNVTJOH5F9

  27. 4UFQ1SFQBSF"OTXFS4IFFUT Make individualized answer sheets using TeX

  28. None
  29. Collection of \rule's

  30. Criterial marks used for image correction Drawn with TikZ

  31. Individualized sheet QR code generated by qrcode package

  32. RSDPEFQBDLBHF QR code generator written in pure TeX Developed by

    Anders Hendrickson Included in CTAN and TeX Live
  33. RSDPEFQBDLBHF QR code generator written in pure TeX Developed by

    Anders Hendrickson Included in CTAN and TeX Live Each block is drawn with \rule
  34. After conducting an exam, scan the collected answer sheets and

    obtain scanned-PDF. 4UFQ4DBOBOTXFSTIFFUT
  35. Scanned answer sheets

  36. 4UFQ3FBE TPSUBOEDPSSFDUJNBHFT Identify the examinee of each sheet based on

    the embedded QR code Sort images by the numbers encoded in the QR codes Correct the scanned-images based on the criterial marks Paper orientation Translation Rotation Distortion
  37. %Z/".J,4 DyNamic Asynchronous Mark-interpretation Kernel System Developed by Mitsuhiro DAIMON

    (Physics teacher at our school) Written in Swift and Objective-C Run natively on macOS Use macOS Cocoa API for PDF handling Adopt OpenCV as its image-processing engine Being developed on GitHub
 (but currently not open-sourced, sorry.)
  38. None
  39. Recognize images with high accuracy

  40. By using
 
 Split each answer sheet Make grading forms

    4UFQ.BLFHSBEJOHGPSNT \includegraphics[page=...,viewport=...]
  41. None
  42. Grading Form

  43. (SBEJOH'PSN Import only specific problems assigned to each grader

  44. (SBEJOH'PSN Remove name and import QR code For privacy and

    fairness
  45. (SBEJOH'PSN Distinguish background by color White area: feedback Gray area:

    note \pagecolor{black!20!white}
  46. Evaluation criteria (SBEJOH'PSN

  47. (SBEJOH'PSN Mark-columns for scores

  48. iPad Pro + Apple Pencil + GoodNotes Hardware Software 4UFQ1FSGPSNHSBEJOHXJUIJ1BE

  49. After grading

  50. Write feedback within white area Fill scores

  51. 4UFQ&YQPSUHSBEJOHGPSNTBT1%'

  52. 4UFQ3FBETDPSFTXJUI%Z/".J,4 Collect (examinee number, score) data examinee number score

  53. Restore answer sheets to return from grading forms 4UFQ3FTUPSFBOTXFSTIFFUT

  54. Grading forms of a single examinee

  55. Grading forms of a single examinee Merge feedback areas into

    one sheet
  56. Restored answer sheet

  57. Each area is pasted from grading forms with \includegraphics

  58. Examinee's name is restored from the original PDF with \includegraphics

  59. TeX overprints score digits read by DyNAMiKS on the sheet

    image
  60. 4UFQ1SJOUTIFFUTBOESFUVSOGFFECBDL Print the restored answer sheets in color Return them

    to students in classrooms
  61. Analysis sheet

  62. Analysis sheet Automatically generated by TikZ + tcolorbox

  63. 8PSLqPX4VNNBSZ 1. Make individualized answer sheets by TeX and conduct

    an exam. 2. Scan the answer sheets into PDF. 3. Sort and correct the scanned-PDF with DyNAMiKS. 4. Split them and generate grading forms by TeX. 5. Perform grading with iPad Pro + Apple Pencil + GoodNotes. 6. Read scores by DyNAMiKS. 7. Restore answer sheets and overprint scores by TeX. 8. Generate analysis sheets by TikZ + tcolorbox. 9. Print sheets and return them to students in paper.
  64. "MMJO0OF"QQ macOS GUI App

  65. 3FTVMU Time for grading: shortened Fairness: enhanced Risk of losing

    answer sheets: disappeared Transcription by hand: no longer necessary
  66. 3FTVMU Time for grading: shortened Fairness: enhanced Risk of losing

    answer sheets: disappeared Transcription by hand: no longer necessary Efficiency: improved
  67. %J⒏DVMUZ "Too many images in a document" × 800 students

  68. texk/dvipdfm-x/spc_pdfm.c texk/dvipdfm-x/pdfximage.h

  69. texk/dvipdfm-x/spc_pdfm.c texk/dvipdfm-x/pdfximage.h Not enough!

  70. texk/dvipdfm-x/spc_pdfm.c texk/dvipdfm-x/pdfximage.h Not enough! This limitation was abolished at TeX

    Live 2018
  71. 'VUVSF8PSL Handwritten Digit Recognition

  72. 'VUVSF8PSL MNIST

  73. 'VUVSF8PSL Our digits MNIST

  74. $PODMVTJPO TeX is useful also as a PDF manipulation tool.

    TeX has enormous potential to automate daily tasks and enhance efficiency. Explore further utilization of TeX 
 as an automation tool!