Fin (Legal) Tech - Law’s Future from Finance’s Past - Professors Daniel Martin Katz + Michael J Bommarito

Fin (Legal) Tech - Law’s Future from Finance’s Past - Professors Daniel Martin Katz + Michael J Bommarito

Transcript

1. daniel martin katz blog | ComputationalLegalStudies.com corp | LexPredict.com law’s future from finance’s past page | DanielMartinKatz.com michael j bommarito blog | ComputationalLegalStudies.com corp | LexPredict.com page | bommaritollc.com Fin(Legal)Tech edu | chicago kent college of law edu | university of michigan cscs
2. Today we only want to talk about one thing …
3. Today we only want to talk about one thing … #Arbitrage
4. But if we are going to talk about #Arbitrage
5. Then we need to talk about why we *sometimes* 
 miss obvious opportunities
6. opportunities that have been right under our noses all along
8. we are all the hero of our own story
9. we are all in our own filter bubble
10. We must fight rigidity & impact of our own success
11. Must get exposed to new ideas (most innovation in law started elsewhere)
12. need to develop a
13. less law centric view of the world
14. see what the world is understand what it might become #arbitrage the precursor to invention
15. Three Types of Lawyers (as described by paul lippe)
16. play “whack-a-mole”, reacting to problems by creating fear and friction within organizations and the impression that there is a legal risk around every corner. Mediocre Lawyers
17. can help clients shape (perhaps distort) external perception of risk. Merely Clever Lawyers
18. design systems that balance risk and improve transparency, helping clients correctly price risk internally Great Lawyers
20. Lawyer VALUE PROPOSITION (From the Client’s Perspective) (internal or external client)
21. help price risk / help reduce information asymmetries transactional =
22. litigation = characterize (predict) risk/exposure shift the expected value of a lawsuit help price risk / help reduce information asymmetries transactional =
23. litigation = characterize (predict) risk/exposure shift the expected value of a lawsuit compliance = identify + prevent rogue behavior monitor behavior in (near) real time help price risk / help reduce information asymmetries transactional =
24. litigation = characterize (predict) risk/exposure shift the expected value of a lawsuit compliance = identify + prevent rogue behavior monitor behavior in (near) real time help price risk / help reduce information asymmetries transactional = regulatory = help identify (predict) the decisions of regulators / law makers and the risk associated with various outcomes
25. focusing every individual and every organization on the places where they actually provide a return on investment (ROI)
27. the analogy du jour
28. law = finance ( insurance as well )
29. law < > finance many elements in law look like finance did 25 - 50 years ago (on the long road from Black-Scholes to algorithmic trading)
30. this is an extension of this prior talk by mike bommarito
31. Dominant Model in Law expert centered pricing of risk
32. Dominant Model in Law lots of unintentional self insurance rarely (if ever) based upon explicit risk models
33. Cult of one 
 (or very small # of) person(s) thinking drives decisions with serious financial consequences
34. hard to move to more rigorous models given borderline pathological numerophobia among lawyers
35. Claim: fin(tech) offers lessons for many areas in law
36. thesis statement: the financialization of the law will be an important vector of the next decade(s) in law
37. The Two Major Branches in #FinTech
38. The Two Major Branches in #FinTech removing socially meaningless frictions (from financial processes)
39. The Two Major Branches in #FinTech removing socially meaningless frictions characterizing (pricing) increasingly 
 exotic 
 forms of risk (from financial processes)
40. #Fin(Legal)Tech application of those ideas and technologies to a wide range of law related spheres including litigation, transactional work and compliance.
42. #FinLegalTech
43. the path of fin(tech) has in part followed developments in artificial intelligence
44. There has been lots of recent interest in applying artificial intelligence to law
47. data driven AI rules based AI Competing Orientations in Artificial Intelligence
48. expert systems Computational Law Data Driven Rules Based prediction models and methods network analytic methods natural language processing self executing law visual law computable codes
49. we see a decent amount of rules based AI in legal industry
50. that is actually pretty consistent with path of A.I. in general
51. lots of issues with expert systems and/or rules based A.I. (without data or an evolutionary dynamic)
53. Ultimately we are trying to learn the rules / dynamics that underlie some class of activity
54. With that understood we want to be able to mimic / predict
56. rules based A.I. data driven A.I. 1980’s, 1990’s, Early 2000’s >
57. rules based A.I. data driven A.I. 1980’s, 1990’s, Early 2000’s rules based A.I. data driven A.I. 2005 - Present < > ~
59. A.I. State of the Art
60. A.I. State of the Art purely data centric
61. A.I. State of the Art purely data centric augment expert forecasts w/ data
62. iterative data < > rules A.I. State of the Art purely data centric augment expert forecasts w/ data
64. fin(tech) is commercial field where huge advances have been made in science of prediction
65. is an emerging field where the tools of predictive analytics are finally being employed fin(legal)tech
66. some PUBLIC examples (many more proprietary examples)
68. Here are just a few predictions that we are trying to accomplish in law
69. #Predict Relevant Documents #Predict Case Outcomes Data Driven Legal Underwriting Data Driven EDiscovery/Due Diligence (Predictive Coding) #Predict Rouge Behavior Data Driven Compliance #Predict Contract Terms/Outcomes Data Driven Transactional Work #Predict Regulatory Outcomes Data Driven Lobbying, etc.
71. There are 3 Known Ways to Predict Something fin(tech) Borrowing in part from
72. Experts, Crowds, Algorithms
73. example from our own work
74. predicting the decisions of the Supreme Court of the United States #SCOTUS
76. Experts
77. Columbia Law Review October, 2004 Theodore W. Ruger, Pauline T. Kim, Andrew D. Martin, Kevin M. Quinn Legal and Political Science Approaches to Predicting Supreme Court Decision Making The Supreme Court Forecasting Project:
78. experts
79. Case Level Prediction Justice Level Prediction 67.4% experts 58% experts From the 68 Included Cases for the 2002-2003 Supreme Court Term
80. these experts probably overfit
81. they fit to the noise and not the signal
83. if this were finance this would be trading worse than S&P500
84. #BuffetChallenge
85. #NoiseTrading
86. law is full of 
 noise predictors …
87. we need to evaluate experts and somehow benchmark their expertise
88. from a pure forecasting standpoint
89. the best known SCOTUS predictor is
91. the law version of superforecasting
93. Crowds
94. crowds
98. https://fantasyscotus.lexpredict.com/case/list/ We can generate Crowd Sourced Predictions
99. however, not all members of crowd are made equal
100. we maintain a ‘supercrowd’ which is the top n% of predictors up to time t-1
101. the ‘supercrowd’ outperforms the overall crowd (and also the best single player)
102. as of May 16, 2016
103. not enough crowd based decision making in institutions
105. “Software developers were asked on two separate days to estimate the completion time for a given task, the hours they projected differed by 71%, on average. When pathologists made two assessments of the severity of biopsy results, the correlation between their ratings was only .61 (out of a perfect 1.0), indicating that they made inconsistent diagnoses quite frequently. Judgments made by different people are even more likely to diverge.”
106. here is our commercial offering
107. design to unlock untapped expertise in organizations #Winning
108. Allowing for Frictionless Crowdsourcing #ManualUnderwriting
109. Allowing for Easy Data Aggregation #KeepingScore
110. https://lexsemble.com/
111. https://lexsemble.com/
113. Brief Aside About the Power of Crowd Sourced Prediction #LegalCrowdSourcing
114. “FantasyJustice predicted the long-shot appointment of Neil Gorsuch for the U.S. Supreme Court! “On November 20th, less than two weeks after the election, FantasyJustice predicted the Gorsuch appointment,” wrote our colleague, Michael Bommarito on the LexPredict blog. “And except for a few brief hours on November 23rd, Gorsuch never fell from that lead.
117. Algorithms
118. Columbia Law Review October, 2004 Theodore W. Ruger, Pauline T. Kim, Andrew D. Martin, Kevin M. Quinn Legal and Political Science Approaches to Predicting Supreme Court Decision Making The Supreme Court Forecasting Project:
119. Ruger, et al (2004) relied upon Brieman(1984)
120. Ruger, et al (2004) relied upon Brieman(1984) i.e. a single tree (as shown below)
122. Leo Brieman moved away from CART in Brieman (2001)
123. Breiman, L.(2001). Random forests. Machine learning, 45(1), 5-32. Published in Machine Learning (A Springer Science Journal)
124. One well-known problem with standard classification trees is their tendency toward overfitting
125. This is because standard decision trees are weak learners
126. Random forest is an approach to aggregate weak learners into collective strong learners (think of it as statistical crowd sourcing)
127. Random Forest: Group of Decision Trees Outperforms and is more Robust (less likely to overfit) than a Single Decision Tree
128. http://machinelearning202.pbworks.com/w/file/fetch/37597425/ performanceCompSupervisedLearning-caruana.pdf Random Forest (particularly with special config/ optimization) have proven to be unreasonably effective
129. Our algorithm is a special version of random forest https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2463244 available at
131. we have developed an algorithm that we call {Marshall}+ ~ random forest
132. Benchmarking since 1953 + Using only data available prior to the decision Mean Court Direction [FE] Mean Court Direction 10 [FE] Mean Court Direction Issue [FE] Mean Court Direction Issue 10 [FE] Mean Court Direction Petitioner [FE] Mean Court Direction Petitioner 10 [FE] Mean Court Direction Respondent [FE] Mean Court Direction Respondent 10 [FE] Mean Court Direction Circuit Origin [FE] Mean Court Direction Circuit Origin 10 [FE] Mean Court Direction Circuit Source [FE] Mean Court Direction Circuit Source 10 [FE] Difference Justice Court Direction [FE] Abs. Difference Justice Court Direction [FE] Difference Justice Court Direction Issue [FE] Abs. Difference Justice Court Direction Issue [FE] Z Score Difference Justice Court Direction Issue [FE] Difference Justice Court Direction Petitioner [FE] Abs. Difference Justice Court Direction Petitioner [FE] Difference Justice Court Direction Respondent [FE] Abs. Difference Justice Court Direction Respondent [FE] Z Score Justice Court Direction Difference [FE] Justice Lower Court Direction Difference [FE] Justice Lower Court Direction Abs. Difference [FE] Justice Lower Court Direction Z Score [FE] Z Score Justice Lower Court Direction Difference [FE] Agreement of Justice with Majority [FE] Agreement of Justice with Majority 10 [FE] Difference Court and Lower Ct Direction [FE] Abs. Difference Court and Lower Ct Direction [FE] Z-Score Difference Court and Lower Ct Direction [FE] Z-Score Abs. Difference Court and Lower Ct Direction [FE] Justice [S] Justice Gender [FE] Is Chief [FE] Party President [FE] Natural Court [S] Segal Cover Score [SC] Year of Birth [FE] Mean Lower Court Direction Circuit Source [FE] Mean Lower Court Direction Circuit Source 10 [FE] Mean Lower Court Direction Issue [FE] Mean Lower Court Direction Issue 10 [FE] Mean Lower Court Direction Petitioner [FE] Mean Lower Court Direction Petitioner 10 [FE] Mean Lower Court Direction Respondent [FE] Mean Lower Court Direction Respondent 10 [FE] Mean Justice Direction [FE] Mean Justice Direction 10 [FE] Mean Justice Direction Z Score [FE] Mean Justice Direction Petitioner [FE] Mean Justice Direction Petitioner 10 [FE] Mean Justice Direction Respondent [FE] Mean Justice Direction Respondent 10 [FE] Mean Justice Direction for Circuit Origin [FE] Mean Justice Direction for Circuit Origin 10 [FE] Mean Justice Direction for Circuit Source [FE] Mean Justice Direction for Circuit Source 10 [FE] Mean Justice Direction by Issue [FE] Mean Justice Direction by Issue 10 [FE] Mean Justice Direction by Issue Z Score [FE] Admin Action [S] Case Origin [S] Case Origin Circuit [S] Case Source [S] Case Source Circuit [S] Law Type [S] Lower Court Disposition Direction [S] Lower Court Disposition [S] Lower Court Disagreement [S] Issue [S] Issue Area [S] Jurisdiction Manner [S] Month Argument [FE] Month Decision [FE] Petitioner [S] Petitioner Binned [FE] Respondent [S] Respondent Binned [FE] Cert Reason [S] Mean Agreement Level of Current Court [FE] Std. Dev. of Agreement Level of Current Court [FE] Mean Current Court Direction Circuit Origin [FE] Std. Dev. Current Court Direction Circuit Origin [FE] Mean Current Court Direction Circuit Source [FE] Std. Dev. Current Court Direction Circuit Source [FE] Mean Current Court Direction Issue [FE] Z-Score Current Court Direction Issue [FE] Std. Dev. Current Court Direction Issue [FE] Mean Current Court Direction [FE] Std. Dev. Current Court Direction [FE] Mean Current Court Direction Petitioner [FE] Std. Dev. Current Court Direction Petitioner [FE] Mean Current Court Direction Respondent [FE] Std. Dev. Current Court Direction Respondent [FE] 0.00781 0.00205 0.00283 0.00604 0.00764 0.00971 0.00793 TOTAL 0.04403 Justice and Court Background Information Case Information 0.00978 0.00971 0.00845 0.00953 0.01015 0.01370 0.01190 0.01125 0.00706 0.01541 0.01469 0.00595 0.02014 0.01349 0.01406 0.01199 0.01490 0.01179 0.01408 TOTAL 0.22814 Overall Historic Supreme Court Trends 0.00988 0.01997 0.01546 0.00938 0.00863 0.00904 0.00875 0.00925 0.00791 0.00864 0.00951 0.01017 TOTAL 0.12663 Lower Court Trends 0.00962 0.01017 0.01334 0.00933 0.00949 0.00874 0.00973 0.00900 TOTAL 0.07946 0.00955 0.00936 0.00789 0.00850 0.00945 0.01021 0.01469 0.00832 0.01266 0.00918 0.00942 0.00863 0.00894 0.00882 0.00888 Current Supreme Court Trends TOTAL 0.14456 Individual Supreme Court Justice Trends 0.01248 0.01530 0.00826 0.00732 0.01027 0.00724 0.01030 0.00792 0.00945 0.00891 0.00970 0.01881 0.00950 0.00771 TOTAL 0.14323 0.01210 0.00929 0.01167 0.00968 0.01055 0.00705 0.00708 0.00690 0.00699 0.01280 0.01922 0.02494 0.01126 0.00992 0.00866 0.01483 0.01522 0.01199 0.01217 0.01150 TOTAL 0.23391 Differences in Trends
133. Total Cases Predicted Total Votes Predicted 7,700 68,964
134. Justice Prediction Case Prediction 70.9% accuracy 69.6% accuracy From 1953 - 2014
136. If You Want a Taste of How the Algorithm Works …
137. Quantitative Methods for Lawyers
138. http://www.quantitativemethodsclass.com/ Professor Daniel Martin Katz Intro Class
139. Legal Analytics Professor Daniel Martin Katz Professor Michael J Bommarito II
140. http://www.legalanalyticscourse.com/ Professor Daniel Martin Katz Professor Michael J. Bommarito II Advanced Class
142. Experts, Crowds, Algorithms
143. For most problems ... ensembles of these streams outperform any single stream
144. Humans + Machines
145. Humans + Machines >
146. Humans + Machines Humans or Machines >
147. question is how to assemble such streams for particular problems
148. so that we are not required to rely exclusively on experts
149. law is a field dominated by individual human experts
150. in most fields - significant quality improvements have been made by moving from experts to ensembles
151. Ensembles come in various forms
152. Here is a well known example
153. Poll Aggregation is one form of ensemble where the learning question is to determine how much weight (if any) to assign to each individual poll
154. poll weighting
155. it does not always get it right aggregation of similar biased signals does not work
156. Must Compare to Alternative Methods Dont compare to perfection
157. A Visual Depiction of How to build an ensemble method in our judicial prediction example
158. expert forecast crowd forecast learning problem is to discover how to blend streams of intelligence algorithm forecast ensemble method ensemble model
159. expert forecast crowd forecast learning problem is to discover how to blend streams of intelligence algorithm forecast ensemble method ensemble model via back testing we can learn the weights to apply for particular problems
161. Given our ability to offer forecasts of judicial outcomes, we wondered if this information could inform an event driven trading strategy ?
162. Paper Released August 24, 2015 http://arxiv.org/abs/1508.05751 available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2649726
167. lots of litigation decisions are just a version of this basic idea law = finance
168. lots of litigation decisions are actually implicit litigation finance (or self insurance) #fin(legal)tech
170. of course the most well known fin(legal)tech is current litigation finance industry
171. funding sources includes: institutional crowdfunding competitive funding platform
172. institutional www.burfordcapital.com/ http://www.gerchenkeller.com/ http://www.fulbrookmanagement.com/ http://www.longfordcapital.com/ http://www.benthamimf.com/
173. competitive funding platform think of it as https://mighty.com/
174. https://www.lexshares.com/ crowdfunding
176. are these funding decisions:
177. impressionistic investing ? are these funding decisions:
178. are these funding decisions: impressionistic investing ? or based on real underwriting ?
179. impressionistic investing ?
180. a lack of real transparency allows for massive returns lit(fin) [today]
181. greater transparency reduces margins + requires better underwriting lit(fin) [future] +
182. better underwriting requires leveraging experts + crowds + algorithms to allow for better predictions
184. fin(legal)tech pricing as
185. hourly rates alternative fees vs.
186. this is a really finance/insurance question
187. Who should bear the cost associated with an overrun?
188. As clients demand a shift toward AFA’s
189. Question is how to rigorously underwrite / predict costs of matters?
190. Weak Lawyer: Asserts dimensions along which their case is a special snowflake
191. Strong Lawyer - Starts Embracing
192. #LegalAnalytics #LegalData #PredictionScience #fin(legal)tech
194. #fin(legal)tech law firm pricing goal
195. it is *not* predicting cost of this particular matter where n=1
196. correctly characterize the distributional properties of a portfolio of matters
197. including identification of outliers both + and -
198. apply portfolio theory
199. to take n=1 and scale to n=many #fin(legal)tech
200. #fin(legal)tech #self insurance today this is how you would run a more rigorous version of
201. AIG to Launch Data- Driven Legal Ops Business in 2016 https://bol.bna.com/aig-to- launch-data-driven-legal- ops-business-in-2016/
202. #fin(legal)tech tomorrow? learn from legal ops service offering to build a commercial insurance product offering legal cost insurance ? other exotic insurance offerings?
203. #fin(legal)tech In such a world, Law Firm is *not* interfacing with client but rather insurance company regarding fees
205. fin(legal)tech Transactional Work as
206. we just discussed price of lawyers
207. now lets think about transactional value
208. Meet Bob
209. Meet Bob lawyer on a major corporate transaction
210. Meet Bob Bob is about to engage in yet another round of markup on deal terms lawyer on a major corporate transaction
211. Meet Bob Bob is about to engage in yet another round of markup on deal terms this round is likely to generate a delay on the expected close of the deal lawyer on a major corporate transaction
213. how much value is created by these modifications? how much delay will be introduced? vs.
214. Need a better understanding of the actual drivers of risk
215. Being able to compute the change in risk as a function of a change in deal terms
217. Outside of M+A Requires Mapping of Deal Terms to actual substantive outcomes #legaldata #legalanalytics
218. this is particularly important when non-lawyers are doing the negotiation (for example your global sales force)
220. fin(tech) fin(legal)tech vs. Additional Lessons
221. fin(tech) is commercial field where there have have been huge advances in working with unstructured data
222. 80%+ of the world’s data is unstructured data
223. in a variety of ways fin(tech) has already confronted this
224. Solution is to either let tech or human process that data
225. And humans are actually pretty good pattern detectors
226. But only for certain types of problems
227. Trading (HFT in particular) is about looking for anomalies 60 Seconds of HFT
229. Two Relevant Examples of Anomaly Detection in Law
230. example 1
231. the discovery + compliance convergence
236. a hard #bigdata problem in law (near real time) compliance FCPA, Product Defect, etc.
237. the goal is near real time monitoring
238. defect w/5 ‘airbag’ version 1.0 backdate w/5 ‘option’ etc.
239. near real time monitoring of version 2.0 a massive volume of communications
241. Behavior will change But Behavior Change will lag (i.e. rogue action will be done offline) (i.e. folks will craft incriminating communications at least for a while) Corp Security Beginning to mirror today’s NSA
242. thus, discovery (in part) becomes compliance and some (only some) litigation is avoided legal standards will still shift real time monitoring will generate lots of false positives
244. example 2
245. we all have a tell
246. lots of efforts to trade on sentiment lessons from fin(tech)
247. efforts to trade on the sentiment contained in these and other related documents
248. sentiment analysis
249. sentiment analysis
250. understanding your opponents or other key decision makers tells (and your own)
251. legal sentiment analysis a new source of competitive legal intelligence
253. combating complexity through information mgmt. lessons from fin(tech)
254. Information Management is a significant problem in legal
255. data that could inform operations is not collected / or not regularized
256. information necessary to undertake due diligence or other regulatory exercises is locked in an antiquated format (i.e. pdf, word, tif file)
257. Dodd-Frank RRP for SIFI’s (Systemically Important Financial Institution) EXAMPLE:
258. Resolution & Recovery Plans are Living Wills for Banks
259. “The living will is effectively a roadmap and simulation of the largest possible series of transactions in a bank’s lifetime, the type of analytical exercise that is common in electronic systems design or software testing, but unprecedented in law.”
260. Ideal RRP is a ‘War Game’ whereby a SIFI demonstrates it is robust to failure of various counterparties
261. but requires review and understanding of the set of agreements across all business lines (p&l’s)
262. problem is legal work product is not a pointable data object
265. horizontal integration of legal work product in the broader corporate technology ecosystem represents a source of immediate value creation
266. “Watson [and related machine learning technologies] will catalyze better organization of legal information and legal data, forcing organizations to better manage their current data and delivering substantial returns from this information management step alone....”
267. for example - contracts should be born (or processed) as computational objects to point straight into finance/acct and other relevant IT systems stored legal work product
268. sensor data + contracts talking to other contracts
269. #InternetofContracts which is a special case of the #InternetofLegalThings This is the which is a special case of the #InternetofThings #IOT
273. we are starting a decade(s) long process of overhauling the global financial infrastructure
274. it is a massive friction reduction exercise
275. Big 4 vs. Big Law who will get to drive this agenda? (i will bet on the big 4)
276. only time will tell …
277. #fin(tech)
278. #fin(tech)
279. #fin(tech)
280. #fin(tech)
281. #fin(tech)
282. #fin(tech)
283. but blockchain is important bitcoin is probably not that important
285. SOME CONCLUDING IMPLICATIONS
286. in order to support enterprise quality risk models remove unnecessary friction
287. < > Implication #1
288. every organization in law needs a data strategy
289. Capture, Clean, Regularize Data to support a range of tasks
290. Deploy Data for Specific Enterprise Applications Develop a data roadmap http://legaldatastrategy.com/
291. http://legaldatastrategy.com/
293. < > Implication #2
294. every organization in law needs a relevant human capital #LegalAnalytics
295. Either going to need homegrow your own talent
296. and/or work with organizations who can help (consultants, etc.)
298. Finally, we recently organized 
 this conversation
299. FinLegalTechConference.com November 4, 2016
300. Fin(Legal)Tech Conference finlegaltechconference.com @Illinois Tech - Chicago Kent College of Law
301. Fin(Legal)Tech Conference finlegaltechconference.com @Illinois Tech - Chicago Kent College of Law
302. Fin(Legal)Tech Conference finlegaltechconference.com @Illinois Tech - Chicago Kent College of Law Fin(Legal)Tech Conference finlegaltechconference.com @Illinois Tech - Chicago Kent College of Law Fin(Legal)Tech Conference finlegaltechconference.com @Illinois Tech - Chicago Kent College of Law
304. Associate Professor of Law Illinois Tech - Chicago Kent Affiliated Faculty Stanford CodeX Center for Legal Informatics College of Law Chief Strategy Officer LexPredict
305. Fellow Stanford CodeX Center for Legal Informatics Adjunct Professor University of Michigan Center for Study of Complex Systems Chief Executive Officer LexPredict
306. LexPredict.com
307. ComputationalLegalStudies.com BLOG
308. @ computational
309. TheLawLab.com
310. Michael J. Bommarito II @ mjbommar computationallegalstudies.com lexpredict.com bommaritollc.com university of michigan center for the study of complex systems @
311. Daniel Martin Katz @ computational computationallegalstudies.com lexpredict.com danielmartinkatz.com illinois tech - chicago kent college of law @