CSIndexbr: A Brazilian Computer Science Index (Talk at CIn/UFPE)

Transcript

1. csindexbr.org, @csindexbr Marco Tulio Valente, ASERG/DCC/UFMG CSIndexbr: A Brazilian Computer Science Index
2. Scholarly Communication Tools 2 Kramer, Bianca; Bosman, Jeroen (2015): 101 Innovations in Scholarly Communication - the Changing Research Workflow. https://doi.org/10.6084/m9.figshare.1286826.v1
3. CSIndexbr is an experimental scholarly communication tool, with two goals: - Discovery e.g. What are the "best" Brazilian papers in my area? - Assessment e.g. What are the "best" Brazilian CS depts in my area? 3
4. Index of papers published by Brazilian CS professors in selected conferences and journals in the last five years (2014-today) - Transparent - Open - but "unofficial" 4
5. Primary data source: DBLP (dblp.org) - High-quality metadata about CS papers - Covers all relevant CS venues - Open-license - Very reliable API 5
6. Key decision: organization by research areas 6
7. Research Areas (21) 7 1. Software Engineering 2. Programming Languages 3. Human-Computer Interaction 4. Computer Networks 5. Distributed Systems 6. Computer Architecture & HPC 7. Hardware Design 8. Databases & Inf. Systems 9. Web & Information Retrieval 10. Data Mining & Mach. Learning 11. Artificial Intelligence 12. Algorithms & Complexity 13. Formal Methods & Logic 14. Operational Research 15. Security & Cryptography 16. Computer Vision 17. Computer Graphics 18. Robotics 19. CS Education 20. Bioinformatics 21. Computer Science (General)
8. Brazilian CS Professors (1,071) - Exactly 800 professors with indexed papers 8
9. Key contribution: Curated dataset of conferences and journals 9
10. Conferences (178) - 15 conferences / area (max) - Only full, main-track papers (10 pages) - short, tool, workshop etc papers are not indexed - Criteria: - submitted > 100 papers - acceptance < 30% - h5-index > 20 10
11. Exceptions: - Many areas: full papers < 10 pages - Computer Networks: 18 confs - Algorithms & Complexity: accept. ~ 40% - etc 11
12. Exceptions are highlighted in yellow in Stats (C) 12
13. Top-Conferences ( ) - 3 top-conferences / area (max) - submitted > 180 papers - h5-index > 30 13
14. Journals (175) - 15 journals / area (max) - Criteria: - Indexed by JCR - h5-index > 25 14
15. Top-Journals - 3 top-journals / area (max) - Criteria: - ACM Transactions or IEEE Transactions (or similar) 15
16. Multi-Journals - Publish papers in more than one area - Require manual division of the papers - 6 multi-journals: - Softw., Pract. Exper. (SE, PL, DS) - Journal of Systems and Software (SE, DS) - TACO (PL, Arch) - Sci. Comput. Program. (PL, SE, Formal) - Concurrency and Computation (DS, Arch) - Theor. Comput. Sci. (Algorith, Formal) 16
17. Goal 1: Discovery 17
18. Papers / Conference [always in the last 5 yrs] 18
19. Papers / Journal 19
20. Papers (Conferences & Journals) 20
21. Professors with Papers (in a Research Area) 21
22. Author Pages 22
23. Goal 2: Assessment 23
24. Department Rankings - 1.0: paper in top-conference or top-journal - 0.40: paper in journals - 0.33: paper in - conference - magazines - journals with short papers - mega-journals - journals with normalized-h5-index < 0.2 24
25. Dept Rankings: per Research Area 25
26. More details: FAQ https://csindexbr.org/faq.html 26
27. Beyond rankings: a repository for scientometrics studies on Brazilian scientific production in CS 27
28. Source code and data is public on GitHub https://github.com/aserg-ufmg/CSIndex 28
29. Documentation (in progress) https://github.com/aserg-ufmg/CSIndex 29
30. (1) h-index correlates with size 30
31. (2) Journals vs Conferences - 178 conferences, 175 journals - Papers - conferences: 676 - journals: 1,943 - ~3 times more papers in journals 31
32. 32 42% of the papers from 12 journals (7%)
33. 33 42% of the papers are from 12 journals (7%) All are commercial journals
34. 34 43% of the papers are from 12 journals (7%) All are commercial journals 11 journals from Elsevier
35. How common is this pattern (journals = 3 * conf)? 35
36. 36 Turing Awards (20): conferences vs journals Source: DBLP (February, 2019)
37. Other features: arXiv links & citations 37
38. Links to arXiv preprints (if available) 38
39. Only 5% of papers have preprints on arXiv 39
40. arXiv popularity (worldwide): 23% 40 Popularity of arXiv.org within Computer Science. Charles Sutton and Linan Gong, https://arxiv.org/pdf/1710.05225.pdf
41. Another feature: citations 41
42. CrossRef Citations - Crossref is an official DOI registration agency - They maintain a database of citations - used by ACM DL, IEEE DL, Dimensions etc - Has a public API (unlike Google Scholar) 42
43. CrossRef vs Google Scholar "Google Scholar unique citations have, on average, a much lower scientific impact than citations also found by WoS/Scopus". Google Scholar, Web of Science, and Scopus: a systematic comparison of citations in 252 subject categories, J. of Informetrics, 2018. https://arxiv.org/pdf/1808.05053.pdf 43
44. Deprecated feature: "Trending Papers" - Page with most cited papers (in all areas) - Reason for deprecation: - Areas with more papers, have more citations - Some papers attract more citations (systematic literature review, surveys, generic tools etc) 44
45. How to Contribute 45
46. How to Contribute - Indicating a missing Brazilian professor - Indicating a missing (and relevant) conference/journal - Indicating a missing paper: - But check before if it is a full paper - Also check if you don't have multiple DBLP pages - In case of multiple DBLP pages: - Mail "dblp@dagstuhl.de" asking the merge 46
47. Google Forms (for submitting issues) https://goo.gl/forms/kz3F1fZIKtubWYiu1 or from csindexbr.org 47
48. Implementation 48
49. Constraints and Requirements - Team: 2 developers (MT and Klerisson, UFU) - Zero budget - High availability - No (or very few) production bugs - Easy-to-use UI - Fast 49
50. Backend Module (Python) - Regenerates all data at each execution - Data format: csv - Executed in my laptop and pushed to GitHub - 3 hours to download authors' papers from DBLP - 20 minutes to regenerate all data - 90 minutes to update citations 50
51. Frontend Module (HTML and JavaScript) - Plots the csv files (using Google Charts library) - Almost no computation on browsers (performance) - Hosted at GitHub pages (0-budget & high availability) 51
52. Future Work 52
53. Backlog 1. Internal improvements, scripts, refactorings etc 2. Update conferences and journals statistics (2018) 3. Integration with CNPq (link to Lattes and Bolsa PQ) 4. Integration with Unpaywall (links to preprints) 5. "Global" depts ranking (all areas) 6. Adjust papers' scores by number of authors 7. Adjust dept's scores by number of professors 8. Extend data collection to more than 5 years 9. Other countries (2020?) 53
54. csindexbr.org @csindexbr Thanks 54