Learning and Educational Technologies Research Unit Learning and Educational Technologies Research Unit Monday Lunch Meeting: Paper Introduction Bell, K., Hong, J., McKeown, N., & Voss, C. (2021). The Recon Approach: A New Direction for Machine Learning in Criminal Law. Berkeley Technology Law Journal, 37. Ogata Lab. D3 Hiroyuki Kuromiya

Learning and Educational Technologies Research Unit Abstract 2

Learning and Educational Technologies Research Unit Notice 3 This paper is not from the field of education, but criminal law. However, the concept itself seems to be applicable in education.

Learning and Educational Technologies Research Unit Contents 1. What is RECON approach? 2. Pilot study (example) 3. Limitations of RECON approach 4. Applicability in Learning Analytics 4

Learning and Educational Technologies Research Unit 1. What is RECON approach? 5

Learning and Educational Technologies Research Unit Current machine learning approach 6 Current machine learning approach à predictive approach This person will commit a crime in X%. Feedback to whom decisions are made.

Learning and Educational Technologies Research Unit RECON approach 7 RECON approach à flash lighting on the past Feedback to decision makers. You were likely to judge poor people as guilty than rich people.

Learning and Educational Technologies Research Unit Two functions 8 Reconnaissance Reconsideration Identifying which factors tend to influence human decision making. Identifying particular cases that appear to be inconsistent with most other decisions. ≒ Association mining ≒ Anomaly detection

Learning and Educational Technologies Research Unit Why we need RECON approach? 1. Humans are far from being perfect. • A machine learning from human judgement will likewise be imperfect. • Not to replace human judgement with machine learning. 2. Provide data-driven opportunities “to make things as little wrong as possible” • Develop tools that act like a flashlight on the past • Bring to light potential problems in decisions that humans have already made. 9

Learning and Educational Technologies Research Unit 2. Pilot study (example) 10

Learning and Educational Technologies Research Unit Context: parole decisions 11 In California, each year, the Board holds 6,000 parole hearings and decides whether a given individual to be eligible for release on parole. The questioning focuses on social history, the underlying crime, the record of conduct in prison, as well as plans for re-entry upon release.

Learning and Educational Technologies Research Unit Dataset and analysis method • Dataset • They acquired 35,105 parole hearing transcripts from 2007-2019 as well as demographic data like race/ethnicity. • Reconnaissance • They developed a tool to show what factors influence parole suitability decisions. Stakeholders are able to query the data for factors of their interest in response to the changing social and legislative landscape. • Reconsideration • They developed a tool that can identify the 10% of cases that are anomalous in the sense of having this same combination of factors but nevertheless resulting in a denial of parole. 12

Learning and Educational Technologies Research Unit Reconnaissance tool 13

Learning and Educational Technologies Research Unit Decision modeling • Regression analysis is often used to perform this type of task. However, it has at least two limitations: 1) it assumes the input and output variables are continuous numerical values, and 2) limited ability to capture the way of the decision making. • Nearest neighbors or decision trees are particularly well-suited to modeling decision making in a multi-step manner. 14

Learning and Educational Technologies Research Unit Reconsideration tool 15

Learning and Educational Technologies Research Unit 3. Limitations of RECON approach 16

Learning and Educational Technologies Research Unit Information extraction • In our developing of the RECON approach, we have focused a great deal on building NLP tools to identify and extract information from hearing transcripts. • To reliability extract information, NLP methods need to be developed to be capable of consuming long text all at once. 17

Learning and Educational Technologies Research Unit Access to data • Nearly all data about a decision-making process is held by the agency that makes those decisions. • The agency has some incentive to resist disclosing data to researchers seeking to implement a Recon Approach. • However, while the Recon Approach offers a way to improve discretionary decision-making in the longrun, it does so by exposing problems with the existing way in which decisions are made. 18

Learning and Educational Technologies Research Unit Regulatory capture • As explained above, existing members of the agency have an interest in minimizing the risk that the Recon Approach will uncover problematic issues that could disrupt the regular functioning of the existing agency. • This interest may express itself in the form of granting access to only selective data points. It may also express itself in granting access to data only on the condition that any resulting research must be reviewed and approved by the agency prior to publication. 19