Deploying real time machine intelligence over large scale recruitment processes

Linguistics algorithms in a talent discovery context: applying machine intelligence over large numbers of resumes and job descriptions in real time to identify patterns within the structure and phrasing of language, surfacing relevant people and connecting talent with opportunities at city, country and continent level. The talk will focus on exhibiting embedding of artificial intelligence in existing flows: catering to the talent industry Opening deploys data science at scale, neural networks working in tandem to perform a multitude of business processes faster and with a high degree of accuracy.

Transcript

1. opening.io
 —
 
 Deploying real time machine intelligence over large scale recruitment processes.
2. Singularity?
 —
 
 Pick 5 random numbers from 0 to 10. No right or wrong answer, no trick, just pick 5 numbers crossing your mind right now.. Let’s play a game!
3. Most often these randomly picked numbers in our mind won’t repeat. We tend to perceive environment as more chaotic than it really is: 1,4,5,7,10 - 3,5,6,8,9 - 3,4,5,1,8 - … True randomness surrounding us is not grey noise but patterns - with Earth for instance we spot contiguous areas of blue, green, …
 
 ..or in the realm of numbers very often patterns like 1,2,2,2,9 - 8,7,8,1,7 - …
4. Small exercise
 —
 
 Trying to define how distance can be perceived: Intuition 
 
 Janice wants to buy Jenna an ice-cream. If she were to buy it and take it to Jenna, will it melt on the way?
 What’s the safest distance she can carry the ice-cream over without melting? What is 100%?
 
 In order to answer we probably need to know the distance between Janice and Jenna. However, other variables may hide as well within a myriad of details alongside position: the ice-cream may actually be a cake disguised as an ice-cream lasting way longer. 100% may be infinite!
 

5. Convention
 —
 
 Settling down to a distance measure. What if we looked from the center of the earth? 48˚ 163˚ 163˚ If we were to think in terms of angles
 0˚ could mean close, 180˚ far away We can even take away the earth and people become stars in the sky, with coordinates!
6. Extra context
 —
 
 Additional attributes to discover besides space coordinates Avenue: consider information as points within a higher dimensional latent space (a sky) and use known distance metrics (euclidian, angular, ..) to reason about their properties. Janice position + likes pistachio ice-cream + .. = [x1, y1, z1, pistachio, red_shoes, owns_two_bunnies, ..] 
 = [x1, y1, z1, p1, r1, o1, ..] (coordinates of a star in a higher dimension) Jenna position + likes frozen orange lollipops + … = [x2, y2, z2, orange, blue_shoes, owns_a_kitty, ..] = […]
7. Ideally
 —
 
 We would like to use the analogy for our particular use case. senior java developer marketing manager Where do we place these stars on the sky? How many coordinates will each star have?
8. Detour
 —
 
 A closer look at generic text, collecting consecutive windows of 10 words: Human language has meaning, words form ideas while operating together. Dublin is the capital of and largest city in Ireland is the capital of and largest city in Ireland. Dublin .. Dublin is located in the province of Leinster on the .. Later expanded as a Viking settlement, the Kingdom of Dublin It means it’s likely to find many more windows where Dublin and Ireland are found together than there are windows containing Dublin and some random word, tractor.
9. Co-occurrences
 —
 
 Let’s remember when words come together. We’ll mark down combinations of two words in the same window with 1 and random combinations of words from different windows with 0. (Dublin, is) = 1 (Dublin, the) = 1 (Dublin, capital) = 1 … (Dublin, Ireland = 1) .. (capital, largest = 1) .. .. (Viking, kingdom) = 1 (Viking, Dublin) = 1 Statistically, how many times more likely is for a word like “Sweden” to appear alongside “Viking” over the whole of Wikipedia than “Dublin”? (Dublin, tractor) = 0 (Dublin, chrysanthemum) = 0 (Ireland, snowboarding) = 0 … What’s happening here is important, we’re translating the objective from the realm of natural language to statistics and co-occurrences allowing to reformulate the question:
10. Objective
 —
 
 In essence, neural networks are simply groups of numbers and functions tweaked in such a way during training (the “learning” phase) so that once trained (during ‘inference’, live phase) they react in predictable ways to patterns in input similar to patterns discovered while analyzing historical data. In our case the goal is having the network “emulate” a value of 1 given an input like “Dublin” + “Ireland” and 0 given “Dublin” + “tractor”. Most likely also combinations like “Dublin”+”Irish” will evaluate to 1 while “ski” + “Dublin” will evaluate to 0*. *Levy/Goldberg consider this to be a factorization of the PMI matrix although there are oposing views) Words become numbers with each word having an unique vocabulary index
11. Machines to spot patterns
 —
 
 They form an important part of today’s AI backbone and are being deployed in the next wave of products and services across a wide range of markets and industries. Pictures become numbers as arrays of consecutive pixels Sentiment analysis Voice assistants Object detection (self-driving cars) Audio becomes numbers via frequency analysis (FFT) Language becomes numbers as statistics over text
12. A fresh look at the original example
 —
 
 Training with a custom setup. “activations” “objective” “observations” This last “layer” contains only numbers, (“activations”) before translating network computation back into business domain.
 What if they are the coordinates of our star?
13. Embedding
 —
 
 Representing “things” as numbers. Powerful and versatile concept allowing representation of instances in real-life (pictures, audio, language, sensor data, time observations,..) condensed as structures of numbers with known mathematical properties. Often cited example: In Opening:
14. Embedding
 —
 
 Latent space: Click on image preview to start video
15. Bonus
 —
 
 Generalization. In general it’s notoriously difficult to apply knowledge from one business domain to another. For instance language - Japanese has few (at surface level) characteristics in common with English. 
 
 However, numbers are invariant to such subtleties: as long as there are patterns in Japanese (and phonetic rules for Japanese definitely exist) a correlation could be established with their English counterparts: it’s how neural machine translation works in today’s systems (Google Translate for example). Example use case: parsing, entity (company names) recognition:
16. Data science meets recruitment
 —
 
 We apply linguistics algorithms over large numbers of resumes in real time to identify patterns within the structure and phrasing of jobs and CVs - telling recruiters who the most qualified candidates are for their job requirements and explain why. A cognitive engine powering talent databases
17. How is Opening different?
 —
 
 A solid foundation in sync with actual research Three central components in defining candidate/job similarity when analyzing abilities: 
 - skills distributions (topic modeling, flat or hierarchical clustering) - abilities within a global context: embedding information in a continuous latent space (word2vec & al.) - coherence of text in a local context (language modeling) Click on image preview to start video
18. Showcase
 —
 
 Fine-tuned intelligence Disambiguation
 
 The same set of abilities (example: C, Python, Databases, ..) can qualify a profile for both positions like “Senior Developer”
 as well as “Dev-Ops Engineer”. In a similar fashion, a “Sales manager” job description can refer to a Sales Manager in Pharma
 or a Sales Manager in IT, which very often tend to carry different requirements and connotations.
 
 Click on image preview to start video
19. Batch, asynchronous, bi-directional data synchronization via APIs Opening is a stateless, functional, reactive service bus mediating interactions between the data lake and various intelligence processes. It integrates with Salesforce via bi-directional data synchronization: collection/aggregation of data, business logic transformations followed by automatic updating of key metrics in Salesforce). Opening fully adopted functional reactive programming paradigms: 
 
 Responsiveness (real-time) Resilience (failure tolerance) Elasticity (scaling with load) Message-driven (decoupled functionality) 
 We build on top of actor model primitives (isolated, stateless) and regard final state as an aggregation of sequential transformations, revisioned and immutable. Opening is developed using Scala programming language thus having complete access to the wider Java ecosystem. Salesforce integration
 —
 

20. Which market segments can Opening address today from a technological perspective?
21. Recruitment agencies High back-log of candidates (50k - 1M+) Fluid job requirements Extra contextual data
 High volume of unsearchable / old data Short time-spans before data becomes invalid
22. Job boards Bad candidates experience (manual fill) Application ‘noise’ 
 Highest volume of unsearchable data (100k-10M) Short time-spans before data becomes invalid
23. Corporate Multiple sourcing channels (direct/email/..) High amount of noise and repetitive data Painful talent rediscovery (assessed previously) No ability to pool from the inside talent base
24. Applicant tracking systems Constant manual data curation needs, many profiles lost or untracked by the ATS (ex.: email/printed applications) Application ‘noise’ 
 High volume of unsearchable / old data
25. Data privacy Opening is a one-on-one transaction acting as an utility provider. We are not a data broker and by definition handle data ownership exclusively with its legitimate custodian.
26. Salesforce integration
 —
 
 Overview Opening already operates in Salesforce environments embedded within current processes.
 It can auto-populate unfilled jobs automatically and flag high value talent to recruiters in real time.
 

27. Tech stack allowing for potential new business avenues: “Active candidates” exchange By enabling an agency located in Prague to announce availability of surplus candidates (after filling their local needs) to a bid from New York (agency or corporate) Opening could potentially mediate a very important relation: - active candidates(vetted by Prague), available, matching open positions in a 3rd party market with a high demand, New York.
28. Thank you.