SocInfo22 Keynote: Modelling and Quantifying Uncertainty in Economic Indicators

Slide 1

Slide 1 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Keynote: Modelling and Quantifying Uncertainty in Economic Indicators Social Informatics 2022 (SocInfo 2022) October 19, 2022 Edgar Meij, Ph.D. Head of AI Search and Discovery @edgarmeij | [email protected]

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

© 2018 Bloomberg Finance L.P. All rights reserved. Bloomberg is just finance, right? ● A technology company founded in New York City in 1981 ● 325,000+ subscribers in 170 countries ● 20,000+ employees in 160+ locations, including 7,000+ software engineers – with 200+ engineers and data scientists working on AI and related problems ● Increased use of and contributions to open source software ● Increased presence in academic research

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

© 2022 Bloomberg Finance L.P. All rights reserved. Sanford Bernstein’s Toni Sacconaghi “And so, where specifically will you be in terms of capital requirements?” Real-time multi-modal data moves markets speech recognition entity recognition linking salience topic classification summarization Elon Musk “Excuse me. Next. Boring, bonehead questions are not cool. Next?”

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Latency matters entity recognition linking salience sentiment

Slide 22

Slide 22 text

© 2022 Bloomberg Finance L.P. All rights reserved. Scale of unstructured data at Bloomberg 80% of all pertinent data is unstructured company filings and press releases news and social media analyst reports and CRMs chats & email client feedback alternative data web sites AI >2M news articles and relevant social media posts ingested per day >1B indexed and searchable documents in multiple languages <500ms latency for monitoring and alerting Hundreds of millions of entities and relations in the Bloomberg KG

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

© 2018 Bloomberg Finance L.P. All rights reserved. The Fed ● The Federal Reserve conducts monetary policy "so as to promote the goals of maximum employment, stable prices, and moderate long-term interest rates” ● Mandate ○ Management and oversight of the production and distribution of the nation's currency ○ Sharing of information and statistics ○ Promoting economic and employment growth through changes to the interest rate

Slide 26

Slide 26 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. The Fed ● The Federal Open Market Committee (FOMC) ○ Eight scheduled meetings per year ■ reviewing economic and financial conditions ■ determining monetary policy ■ assessing the risks to its long-run goals of price stability and sustainable economic growth ○ Promotes economic and employment growth through changes to the interest rate

Slide 27

Slide 27 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Central bankers – Of hawks and doves ● A hawk is thought of as assigning a higher priority to fighting inflation ○ >> higher interest rates ● A dove supports output growth and lowering unemployment ○ >> lower interest rates ● Interest rates?

Slide 28

Slide 28 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Monetary transmission ● Central banks use the interest rate as its primary policy tool ○ Overnight rate at which banks lend to each other over the very short term ○ Interest rate at which banks can borrow from the central bank (directly or via bonds) ○ Shifts in this rate have a knock-on effect on the economy through, e.g., bank lending/mortgage/interest rates, and thus consumer spending and borrowing ● Price stability is critical to a healthy economy ○ If interest rates go up it increases the cost of borrowing, making credit and investment more expensive – used to slow an overheated economy ○ If rates go down it makes borrowing cheaper, encouraging credit and investment – used to stimulate a stagnant economy

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Monetary transmission ● If inflation rates are rising, then there is too much money in the economy relative to the amount of goods ○ Each dollar/euro/pound/… one has is worth less than it was a day earlier ● Increasing the interest rate is a way to remove money from the economy and thus to decrease inflation ○ Investors will pick less risky investments, people will save more and spend less, etc ○ Decreases employment opportunities, and may be detrimental long-term ● A decrease in rates allows more money to be spent, encouraging business growth – thus increasing the potential for employment

Slide 34

Slide 34 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Inflation and target rates ● Lag ○ Policymakers have to anticipate future inflation trends when deciding on rate levels in the present ○ Adherence to inflation targets can only be gauged with backward-looking statistics ○ These can range widely amid economic shocks "In short, if making monetary policy is like driving a car, then the car is one that has an unreliable speedometer, a foggy windshield, and a tendency to respond unpredictably and with a delay to the accelerator or the brake," -- Ben Bernanke, 2004

Slide 35

Slide 35 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Hawks vs. doves (and in-betweens?) ● “About 20,000 articles or reports from more than 30 newspapers and business reports of Fed watchers with reference to FOMC members are read.” ○ (Footnote: “Due to the particularity of the exercise, the process involved human reading rather than text mining/reading algorithms.”) Istrefi, Klodiana (2019). Fed Watchers’ Eyes: Hawks, Doves and Monetary Policy. Banque de France.

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Economic indicators ● Economic indicators provide measures of macroeconomic performance and stability ● Divided into three broad groups ○ Leading indicators, e.g., consumer confidence, jobless claims, and durable goods orders ○ Coincident indicators, e.g., short-term interest rate, retail sales, and CPI ○ Lagging indicators, e.g., GDP, corporate earnings, and PCE

Slide 39

Slide 39 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Example: Consumer Price Index (CPI) ● Measure of the average change over time in the prices paid by urban consumers for a market basket of consumer goods and services ● Based on surveys of consumer purchases ● Essentially compares the cost of living over time ○ Can be used to gauge inflation levels ● Market participants pay close attention to CPI for signs of inflation ○ Rising inflation can cause higher interest rates and reduce borrowing ○ Deflation can lower interest rates and encourage lending

Slide 40

Slide 40 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Example: Gross Domestic Product (GDP) ● The market value of all goods and services produced within a country during a given period ● Lagging indicator and one of the most popular gauges of economic health ● Can vary by political definition even if there is no difference in the economy ● Commonly released every three months

Slide 41

Slide 41 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Example: Personal Consumption Expenditures (PCE) ● Measure of consumer spending ● Include how much is spent every month on (non-)durable goods and services, so is comparable to CPI ● Takes GDP into account (measured/reported quarterly!) ○ Gaps are filled using retail sales every month ● PCE is used by the Federal reserve to measure inflation ○ Shows how people change their buying habits when prices change, providing a window into demand for products and services

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Contrasting, explaining ● Inflation up due to demand (economy, labor market up) or supply (labor market down, monetary policy)?

Slide 46

Slide 46 text

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Slide 49

Slide 49 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Risk analysis (shocks) ● What happens if … ○ VIX + 20% ? ○ Oil price + x %? ○ Pr(inflation > 4%)? ○ If inflation is high in the US, what is the probability of inflation being low in the EU?

Slide 50

Slide 50 text

Slide 51

Slide 51 text

© 2018 Bloomberg Finance L.P. All rights reserved. How are economic indicators created? ● Surveys ● Reports ● Estimates ● Nowcasting ○ Combine ■ Input indicators ■ “Alternative data” ● Weather, satellite imagery ● (Semi-)structured data ○ Transactions ○ Web crawls ● Textual data ○ Predict ■ Actual value ■ Above/Below expectation

Slide 52

Slide 52 text

© 2018 Bloomberg Finance L.P. All rights reserved. Data is the backbone of the financial markets ● Historically mostly “structured” market data (ticks/quotes/trades) ○ Well-understood ○ Enabling advanced forms of automation ● Increasingly non-traditional factors, based on “alternative” data, such as: ○ Satellite images / CO2 emissions over factories ○ Sentiment analytics on news ○ Shopping mall footfall traffic ○ Number of people riding the subway ○ Credit card transactions ○ “Pret index” ○ etc. Bl o o mb e r g Se c o nd M e a su r e L L C © 20 21 | Th i s d o c u me nt i s st r i c t l y p r i v a t e a nd c o nf i d e nti a l a nd m a y n o t b e c o p ie d , d is t r i b u t e d o r r e p r o d u c e d i n w h o l e o r i n p a r t w i t h o u t Bl o o mb e r g Se c o nd M e a su r e ' s p r i o r w r i t t e n c o nse nt . 3 W hat is Transaction Data? P u r c h a se d a t a i s t h e p r e f e r r e d so u r c e f o r c o nsu me r t r e nd s a nd b u si ne ss p e r f o r ma nc e i nsi g h t s. T r a nsa c t i o n d a t a p r o v i d e s b e ne f i t s t h a t o t h e r so u r c e s c a nno t : ● P o i nt - o f - sa l e i nsi g h t s ● T i me l i ne ss ● L o ng i t u d i na l a na l y se s Aw areness Interest Consideration Intent Evaluation Purchase Transaction Re c e i p t Alternative Data Sources: Se nt i me nt W e a t h e r Sa t e l l i t e G e o - l o c a t i o n W e b t r a f f i c A p p u sa g e Consumer Purchase Funnel

Slide 53

Slide 53 text

© 2018 Bloomberg Finance L.P. All rights reserved. Data is the backbone of the financial markets ● Increasingly non-traditional factors, based on “alternative” data, such as: ○ Satellite images / CO2 emissions over factories ○ Sentiment analytics on news ○ Shopping mall footfall traffic ○ Number of people riding the subway ○ “Pret index” ○ Credit card transactions ○ etc. ● But also “unstructured” data… Challenge: identify financially-relevant signal from noisy, complex tangentially-related datasets.

Slide 54

Slide 54 text

© 2018 Bloomberg Finance L.P. All rights reserved. But also “unstructured” data ● 80% of data exists in the form of “raw”, unstructured text, e.g., ○ News stories ○ Company filings, earnings call transcripts ○ Tweets/Reddit/Facebook/… posts ○ Research analyst reports, CRMs ○ Economic policy, govt communications ○ Press releases ○ Web sites ○ Chats & e-mail, client feedback ○ etc. ○ Lots of jargon and custom terminology (sometimes even firm-specific!) Our Challenge Identify financially-relevant signal from noisy, complex tangentially-related datasets

Slide 55

Slide 55 text

Slide 56

Slide 56 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. NLP – Ingredients ● What do you need? ○ Data (Text) ○ Annotations ○ Algorithms/Models ○ Compute ○ Human beings, to ■ operate compute ■ tune models ■ interpret results ■ annotate data ○ Universe of “entities” (and their relationships) ■ financial instruments, companies, people, products/brands, geopolitical, etc. ■ supply chain, issuers, corporate structure, domicile, c-suite, board members, etc. ○ Point-in-time

Slide 57

Slide 57 text

Slide 58

Slide 58 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. NLP in Practice ● Generation 1: Write a bunch of rules (“templates”, “grammars”) ○ High-precision ○ Slow, manual, difficult to maintain or update ● Generation 2: Count words, or train a statistical classifier ○ For sequence tagging: conditional random fields ○ For document classification: logistic regression, SVMs, decision trees/random forests ○ Need labeled data ● Generation 3: Deep learning and human in the loop ○ Need a lot more labeled data, or distant supervision ○ May be slower

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Slide 61

Slide 61 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. State-of-the-art NLP for monetary policy ● Circa 2000s – “counting words” ○ Gazetteers of positive/negative words ○ PMI over Google hits, e.g., ■ Lucca, D. O., & Trebbi, F. (2009). Measuring central bank communication: an automated approach with application to FOMC statements. National Bureau of Economic Research. ● Circa 2010s – Still counting words, LDA was on the rise ○ E.g., ■ Baker, S. R., Bloom, N., & Davis, S. J. (2016). Measuring economic policy uncertainty. The quarterly journal of economics, 131(4), 1593-1636. ■ Hendry, S (2012), Central Bank Communication or the Media’s Interpretation: What Moves Markets?, Bank of Canada Working Paper 2012-9. ■ Hansen, S, McMahon, M and Prat, A (2014), Transparency and Deliberation within the FOMC: a Computational Linguistics Approach, CEP Discussion Papers DP1276, Centre for Economic Performance, LSE.

Slide 62

Slide 62 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. State-of-the-art circa now ● Representation learning and reusing knowledge across tasks ○ Pretrain general language models to predict next words (BERT, GPT-k, …) ○ Use it to initialize a task-specific model ○ Achieve higher accuracy with fewer examples ○ Zero or few-shot learning ● Add relevant domain, task, and contextual knowledge to LMs ● Scaling out deep learning architectures ● Learning more from less annotations ○ Distant supervision ○ Transfer learning ○ Active learning

Slide 63

Slide 63 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. “Augmented” intelligence ● No model is static ○ New entities, new contexts, new relationships, new data, ... ● Human-experts-in-the-loop ○ Help prevent model drift and ameliorate lack of recall, precision ○ Provide important training data ● “Automation is augmentation, not replacement” ○ Need effective tools for humans to work with algorithms

Slide 64

Slide 64 text

Slide 65

Slide 65 text

Slide 66

Slide 66 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Method ● Boolean expression keyword retrieval, i.e., keyword match, and then count ○ Match ■ Economic terms ● “economic”, “economy” ○ AND ■ Uncertainty terms ● “uncertain” OR “uncertainty” ○ AND ■ Policy terms ● “regulation” OR “deficit” OR “deficits” OR “legislation” OR “legislative” OR “legislature” OR “congress” OR “congressional” OR “white house” OR “federal reserve” OR “the fed” OR “regulations” OR “regulatory” OR “legislative” OR “legislature”

Slide 67

Slide 67 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. (Dis)entangling multiple sources of uncertainty ● Real underlying uncertainty about economic outcomes ● Semantic uncertainty as expressed in language ● Annotator uncertainty about the labels ● Modeling uncertainty inherent to NLP/AI methods ● Limitations of keyword-based approaches ○ Lacking context ○ Missing lexical variations ○ Inability to capture common sense / inference K Keith, C Teichmann, B O’Connor, and E Meij (2020). Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty. In Proceedings of the Fourth Workshop on Natural Language Processing and Computational Social Science. Association for Computational Linguistics.

Slide 68

Slide 68 text

© 2018 Bloomberg Finance L.P. All rights reserved. Disentangling multiple sources of uncertainty Demand for new clothing is uncertain because several states may implement large hikes in their sales tax rates. Some economists claim that uncertainties due to government industrial policy in the 1930s prolonged and deepened the Great Depression. The outlook for the H1B visa program remains highly uncertain. As a result, some high-tech firms fear that shortages of qualified workers will cramp their expansion plans. Uncertainty about U.S. military actions in Iraq are placing upward pressure on oil prices. It remains unclear whether the government will implement new incentives for small business hiring. The looming political fight over whether to extend the Bush-era tax cuts makes it extremely difficult to forecast federal income tax collections in 2011. Uncertainty about prospects for war in Iraq has encouraged a build-up of petroleum inventories and pushed oil prices higher. 'The budget is uncertain,' Ms. Veneman said at a hearing of the Senate Agriculture Committee. 'I can't tell you where the budget is going to go with regard to anything.' Manufacturing businesses' confidence slipped in April for the second consecutive month, partly because of uncertainty about the Clinton Administration's plans, Cahners Economics Inc. said today. Adding to the pressures on the dollar was the growing view in the currency markets that the Federal Reserve was unlikely to act soon to push up interest rates further. Policy, Economy, Uncertainty, Causal Relation

Slide 69

Slide 69 text

© 2018 Bloomberg Finance L.P. All rights reserved. Analysis ● Baker et al. sample documents, obtain binary labels, and construct a “human- generated” index which has “a 0.86 correlation” with their index ○ Only 16% of documents have more than one annotator and of these, the agreement rates are moderate: 0.60 Krippendorff’s α (chance-adjusted agreement) ○ Baker et al. did not address whether this is a result of annotator bias, error in annotations, or true ambiguity in the text

Slide 70

Slide 70 text

© 2018 Bloomberg Finance L.P. All rights reserved. Small improvements: embeddings-based term expansion ○ Match ■ Economic terms: “economic” OR “economy” OR “economies” OR “financial” OR “economic” OR “growth” OR “recession” OR “slowdown” ○ AND ■ Uncertainty terms: “uncertain” OR “uncertainty” OR “unclear” OR “unsure” OR “uncertainties” OR “turmoil” OR “confusion” OR “worries” ○ AND ■ Policy terms: “regulation” OR “deficit” OR “deficits” OR “legislation” OR “legislative” OR “legislature” OR “congress” OR “congressional” OR “white house” OR “federal reserve” OR “the fed” OR “regulations” OR “regulatory” OR “legislative” OR “legislature”

Slide 71

Slide 71 text

© 2018 Bloomberg Finance L.P. All rights reserved. Method ● LogReg: multiclass logistic regression ○ Obtain vocabulary containing most frequent words from labeled documents ○ Maximize conditional log-likelihood in order to infer weights associated with these words, expressing how much they contribute towards the class label ○ Aggregate and use for document-level classification

Slide 72

Slide 72 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Results ● Text-as-data methods require task-specific validation ● Content validity: consider type of linguistic information one is measuring ○ For instance, mapping keywords to a document-level binary label collapses all types of semantic uncertainty, many of which cannot be identified via keywords alone ● Sensitivity: strengthen conclusions with multiple measurements and measurement approaches ○ In this paper, we demonstrate that no tight correlation exist between the keywords-based approach and aggregating outputs of a document classifier

Slide 73

Slide 73 text

Slide 74

Slide 74 text

Slide 75

Slide 75 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. FOMC meeting minutes ● 15+ pages, dense and technical prose ○ Rich structure ■ Numerous sections ■ Discussions about previous meetings, future outlook, etc. ○ Designed to influence markets ● Our experiment ○ 177 meeting documents ranging from January 2000 until August 2020 ■ Includes ad-hoc meetings ○ Split into ~100k sentences ○ Manually labelled 1.8k sentences ■ Hawkish vs dovish vs neutral Distribution of labels Negative 38.2% Neutral 27.4% Positive 34.4%

Slide 76

Slide 76 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Example sentences from FOMC meeting minutes Sentence Source Comments Regarding the labor market, many participants commented that the pace of employment gains, which was quite strong in May and June, had likely slowed. July 28-29, 2020 Sentiment is governed by the final clause. One of them judged that the low level of inflation compensation could reflect increased concern on the part of investors about adverse outcomes in which low inflation was accompanied by weak economic activity, and that it was important not to dismiss this possible interpretation. March 17-18, 2015 Negation does not invert sentiment. Recent data along with anecdotal reports indicated some loss of vigor in the nation’s housing markets, though overall activity was still at a high level. December 21, 1999 Mixed sentiment.

Slide 77

Slide 77 text

Slide 78

Slide 78 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Methods ● Dictionary-based: measure the ratio of positive to negative sentiment- bearing words ● Linear: multiclass logistic regression ○ Obtain vocabulary containing all words and word pairs from labeled sentences ○ Maximize conditional log-likelihood of all meetings in order to infer weights associated with these words and word pairs, expressing how much they contribute towards an overall positive, negative, or neutral sentiment ○ Final classifier: one model per sentiment class using one-versus-all scheme Neutral Positive Negative committees 1.02 increased 1.12 weakness 1.23 committee 0.93 gains 0.88 weak 1.14 had remained 0.87 strong 0.88 uncertainty 1.10 mixed 0.86 rise 0.86 below 0.82 information 0.78 up 0.82 short 0.78

Slide 79

Slide 79 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Methods ● Dictionary-based: measure the ratio of positive to negative sentiment- bearing words ● Linear: multiclass logistic regression ○ Obtain vocabulary containing all words and word pairs from labeled sentences ○ Maximize conditional log-likelihood of all meetings in order to infer weights associated with these words and word pairs, expressing how much they contribute towards an overall positive, negative, or neutral sentiment ○ Final classifier: one model per sentiment class using one-versus-all scheme ● Deep Learning: finetune a character-based Transformer, decomposing words into sub-word units and train using labeled sentences

Slide 80

Slide 80 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Results: F1 score Negative F1 score Neutral F1 score Positive F1 score Dictionary 0.62 0.28 0.43 Linear 0.6 0.41 0.66 Deep Learning 0.72 0.55 0.66 F1 score Chance 0.28 Dictionary 0.44 Linear 0.58 Deep Learning 0.66

Slide 81

Slide 81 text

Slide 82

Slide 82 text

Slide 83

Slide 83 text

Slide 84

Slide 84 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Results ● Information contained in FOMC’s meeting minutes can help forecast the target interest rate ● Strongly leads the FOMC’s decision to hike or cut rates by approximately eight meetings or one year ● Manual encoding coupled with deep learning performs better than baselines

Slide 85

Slide 85 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. Conclusions ● Economic indicators: interpret and predict macroeconomic activity ○ Used by governments, central banks, and finance professionals ○ Have real-life impact on everyone in society ● Open challenges and the role of AI/NLP ○ Find accurate and more real-time signals to determine effects of economic policy ○ How to incorporate shocks? ○ Deliver value by using NLP to add structure to unstructured data ■ At scale with high accuracy and, in some cases, with low latency ■ Link, connect, and identify patterns over entities, topics, and events ● Compute and implementations of algorithms are commodity ○ Finding, cleaning, labeling, and analyzing relevant data to train models is key ○ Collaborate with experts to train, understand, and operate models and drive (continuous) annotations

Slide 86

Slide 86 text

© 2018 Bloomberg Finance L.P. All rights reserved. © 2022 Bloomberg Finance L.P. All rights reserved. https://TechAtBloomberg.com/AI https://TechAtBloomberg.com/data-science-research-grant-program/ https://www.bloomberg.com/careers @edgarmeij | [email protected] Thank you