The GHTorrent dataset and toolsuite

Slide 1

Slide 1 text

The GHTorrent Dataset and Tool Suite Georgios Gousios Software Engineering Research Group TU Delft

Slide 2

Slide 2 text

  All data from Github

Slide 3

Slide 3 text

 Ready to be queried

Slide 4

Slide 4 text

ghtorrent.org

Slide 5

Slide 5 text

     Repositories

Slide 6

Slide 6 text

   Commits

Slide 7

Slide 7 text

    Pull requests

Slide 8

Slide 8 text

    Issues

Slide 9

Slide 9 text

    Users and Organizations

Slide 10

Slide 10 text

 Mirror event stream

Slide 11

Slide 11 text

<> PushEvent <> /users/:user ensure_user <> /repos/:user/:repo/ ensure_repo <> /repos/:user/:repo/commits ensure_commits ensure_user <> /:user/:repo/sha ensure_commit ensure_user <> /users/:user/ followers ensure_followers <> /repos/:user/:repo/ commits/:sha/comments ensure_commit_comments <> /users/:user/orgs ensure_orgs <> /orgs/:org/teams ensure_teams Recursive dependency retrieval

Slide 12

Slide 12 text

Build relational database to query

Slide 13

Slide 13 text

repositories users organizations issues /users/:user /user/repos /repos/:user/:repo/issues /orgs/:org { 88"type":8"User", 88"public_gists":80, 88"login":8"gousiosg", 88"followers":88, 88"name":8"Georgios8Gousios", 88"public_repos":84, 88"created_at":8..., 88"id":8386172, 88"following":84, } { . . . CoSQL database as cache

Slide 14

Slide 14 text

Periodic dumps of DBs online

Slide 15

Slide 15 text

Query relational DB online

Slide 16

Slide 16 text

$ gem install sqlite3 ghtorrent $ ght-retrieve-repo mojombo jekyll $ (edit config.yaml) Roll your own tools

Slide 17

Slide 17 text

Research !   

Slide 18

Slide 18 text

 Single developer identities

Slide 19

Slide 19 text

  Single developer identities

Slide 20

Slide 20 text

       Single developer identities

Slide 21

Slide 21 text

           Single developer identities

Slide 22

Slide 22 text

               Single developer identities

Slide 23

Slide 23 text

      Source tracking

Slide 24

Slide 24 text

        Source tracking

Slide 25

Slide 25 text

          Source tracking

Slide 26

Slide 26 text

            Source tracking

Slide 27

Slide 27 text

              Source tracking

Slide 28

Slide 28 text

                Source tracking

Slide 29

Slide 29 text

                  Source tracking

Slide 30

Slide 30 text

                    Source tracking

Slide 31

Slide 31 text

Network analysis

Slide 32

Slide 32 text

Distributed development Text Text TUD-SERG-2013-10 An Exploratory Study of the Pull- based Software Development model