CREATION OF SIMULATED DATA AND SCORING SYSTEM BASED ON LINKEDIN DATA

I tried to attempt to generate data that will mimic linkedin data. Data from linkedin especially data on skills and endorsements is hard to get except if linkedin provides it directly. Scraped linked in data is also very limited. Therefore I decided to create a simulated data.
The scope of this data is limited to IT( information technology) industry only .
How I did it?
I reviewed the proposal provided by Sponsor multiple times to determine required variables . Linkedin profiles also helped me to confirm the variables. However there is no metadata available on linkedin therefore I used US labor department data to create initial data like Date of Birth, Number of Jobs changed and Number of Titles

Transcript

1. A DATASTORY Renuka Kulkarni-Kelapure
2. When I started the project….
3. Introduction and Expectations ´Two words about Sponsor – Curtis Foundation ´Assigned Tasks ´1)To create a Simulated Data Model Based on Linkedin ´2) Create a MYSQL DATABASE ´3)Analysis- ´A) Is success related to number of endorsements? ´B)Classification of Endorsements- Toxonomy creation ´C) Creation of Credibility Score ´D)Creation of Success Score
4. Why? Why ? Why?
5. The Beginnings……. Understanding the proposal • Define Success • Normalization of Titles • What Variables to Consider Research /Literature Review • How do we define variables ? • How to create based data model Coding • How to write loops ? • How to I assign multiple titles to one person • Figuring out logistics • How to avoid biases
6. Assumptions and Definition ´Age group – 22- 65 ( 1950- 1995) ´Apply to IT industry only ´Success is defined objectively , not subjectively in this model ´Job Titles are normalized based on the responsibilities of jobs. ´Titles are divided into 6 groups – 1 – Lowest ; 6 Highest ´Assignment of job titles is based on education. As per job requirement analysis most of the IT jobs need Undergraduate and above. ´Only US population is considered in this model. ´Created 3 levels of Endorsers.
7. Creation of Data Demographics • Generator to create names , DOB • Usmap- location • Derived Age, YOE, No of Jobs changed • No of titles – 6 • Randomly assigned – No of connections, • Education, Gender, Race • Salary is based on Current Titles , and Location Endorsers • Randomly Assigned YOE, endorsers Titles, No of Endorsers ID Titles • We have derived up to 5 previous titles . Previous titles are based on education and No of titles. • Titles are based in Education, YOE. Skill • Scrap data on Skills and Endorsements from linked in • Normalized the Skills based on Titles . • Assigned Skilled Randomly among each group based on current titles and Years of experience • No. of endorsements randomly assigned
8. One down two to go …DATABASE
9. Best DataBase Model
10. Descriptive Analysis Tables Rows Columns Demo 973 42 CurtisSkill2 8986 7 EndorSkill11 4689 8 EndorSkill21 4736 8 EndorSkill31 4648 8 Master Title 244 2 6 24276 72
11. Success Score What Variables are highly correlated? Success Score – (sum(Title Level+(Current_Salary/1000)+Fort une500))/Age
12. Credibility Score(CS) VS High Credibility Score(HCS) ´CS =Title level + year of experience+(median of total endorsements) ´HCS= CS x No. of similar Skills for which endorser has been endorsed x No. of endorsers who fullfill similar criteria
13. Do No. of endorsements Matter? No. of titles, No. of job Changed, and year of experience are strongly negatively correlated to Success Score. Success Score is strongly negatively correlated.
14. Regression- Model 1 Success Score Vs Total Endorsements The results are nonsignificant. That proves current endorsement system is broken
15. Multivariate Analysis
16. Future Pathways …. ´Testing Simulated Model against real life data model ´Creation of Neo4j graphical database ´Creation and Validation of High Credibility Score per person per skill ´Measuring the impact of score system on personal success ´ Use of system in future Skill-Gap Analysis
17. Thank You