Recent PAUSE Changes

Talk at PerlCon 2019 in Riga

Transcript

1. Recent PAUSE Changes Kenichi Ishigaki @charsbar PerlCon 2019 Aug 7, 2019
2. me • DBD::SQLite • JSON/JSON::PP • Perl::PrereqScanner::NotQuiteLite • CPANTS • CPAN::Groonga • PAUSE on Mojolicious
3. Perl Toolchain Summit (formerly known as Perl QA Hackathon) 2019 Sponsors Booking.com, cPanel, MaxMind, FastMail, ZipRecruiter, Cogendo, Elastic, OpenCage Data, Perl Services, Zoopla, Archer Education, OpusVL, Oetiker+Partner, SureVoIP, YEF
4. PAUSE has been a hot topic
5. • The Perl programming Authors Upload SErver • The back stage entrance to the CPAN • Written by Andreas König in 1995 • Generates indices of authors/packages What's PAUSE?
6. • Its UI was (last) overhauled in 1999 • It may look old-fashioned compared with other portals including MetaCPAN or modules.perl6.org • Things around us has been changing PAUSE has been serving us well for a long time
7. 0 500 1000 1500 2000 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 Active authors Authors with new releases PAUSE itself can't stop our downtrend 0 200 400 600 800 1000 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 New authors New active authors
8. Maybe we can help a little • to make it easier to transfer abandoned modules to someone • to show beginners how to make a release (PAUSE has some under "About PAUSE" section)
9. Growing pains • PAUSE used to list everything on a single page • This may not have been a problem before • Too many checkboxes for some authors (150 authors have or had 50+ distributions, and 50+ authors have or had 100+ dists now)
10. External PAUSE Tools but shouldn't it be better to fix PAUSE? (plus a few uploaders and scripts, etc) App::pause PERLANCAR App::PAUSE::CheckPerms NEILB App::PAUSE::cleanup YANICK App::PAUSE::Comaint MIYAGAWA Pcore::API::PAUSE ZDM WWW::PAUSE::CleanUpHomeDir ZOFFIX WWW::PAUSE::Simple PERLANCAR
11. PAUSE on Plack (2015~16) • Replaced mod_perl/Apache with Plack/Starman (2015) • Went into production (2016) • Occasional stability issue(s) (~2019?)
12. PAUSE on Mojolicious (2017~19) • Big rewrite for templates and plugins (2017) • Went into produciton from my mojo_wip branch (2018) • Fell back to PAUSE on Plack sometimes (2018) • Merged into master (2019)
13. Markdown rendering (2018) • Made it easier to add new documents • PAUSE Operating Model • Privacy Policy • More to come
14. From XHTML to HTML5 (2018) • No more table layout • Min.css just for grid layout • Not optimized yet New UI mockup by Babs Veloso (Not implemented yet)
15. JavaScript for Paging/Filtering (2018) • No time for a real pager • List.js
16. How many items in a page? • 100 items per page was too small for Mojolicious, DBIx::Class etc. • Increased to 300 for the time being • How can we help distributions with thousands of modules?
17. Permission management per distribution (2019) Transfer everything in a distribution with one check
18. No more partial transfer Filtering by "Starman" module was not enough because Starman contains a module under a different namespace (HTTP::Server::PSGI::Net::Server::PreFork)
19. Incomplete ownership • Distributions may be owned by multiple owners (because a new contributor releases a new module without specifying x-authority, or because of partial transfer) • If you want to be a new contributor, you (usually) need to ask all of the owners for permission
20. "Permission manager" permission • Formerly known as "Admin" permission • To let someone grant permissions to someone else without giving up your owner rights • Permission manager needs to know new modules added by others, but how? • Discussed but not implemented yet
21. reCAPTCHA (2018) • ID requests used to be approved by the PAUSE admins, but this process takes time • 25% of new users release their first distribution on the day of approval • David Golden introduced reCAPTCHA to eliminate this delay and admins' burden
22. Abused by spammer(s) It's disabled now
23. Exposed user status (2019) • For other CPAN services to remove/hide spammers' information • We may need "system" status as well for special users (ADOPTME etc)
24. GDPR: Right to be forgotten • You can't delete your account if you have uploaded something • You can empty most of your personal data by yourself • We started hiding data of deleted accounts (2019) • More to come
25. OAuth2 as authorization • Lee Johnson made a branch for this • For ACT users to allow ACT to use their personal data • ... but almost everything is exposed via indices now • Providing official API would be nice • (Ab)use for user verification for RT/MetaCPAN?
26. CSRF protection (2018) • Mojolicious::Plugin::WithCSRFProtection • Some of the pages are protected now
27. Multi-factor Auth • Recent account hijackings for other portals (rubygems, npm) • Not implemented yet but we should • As long as other CPAN clients/services support this • https://github.com/andk/pause/issues/292
28. Perl6 indices • Actually there are two indices (+ empty one) for Perl 6 • modules.perl6.org is making its own indices # CPAN/authors/p6dists.json.gz { "A/AK/AKIYM/Perl6/JSON-Hjson-0.0.1.tar.gz" : { "name" : "JSON::Hjson", "auth" : "AKIYM", "ver" : "0.0.1" }, ... } # CPAN/authors/p6provides.json.gz { "Math::Random" : [ "P/PS/PSIXDISTS/Perl6/Math-Random-0.000.001.tar.gz" ], "Math::FFT::Libfftw3::Exception" : [ "F/FR/FRITH/Perl6/Math-FFT-Libfftw3-0.1.1.tar.gz", "F/FR/FRITH/Perl6/Math-FFT-Libfftw3-0.1.2.tar.gz", "F/FR/FRITH/Perl6/Math-FFT-Libfftw3-0.1.3.tar.gz" ], ... }
29. Transferring Perl6 modules • Currently there's no permission management for Perl6 modules • Just author(s), instead of P5's first-come + contributors? • ADOPTME/HANDOFF/NEEDHELP/NOXFER?
30. Other changes • Ricardo Signes and Neil Bowers finished their efforts to make PAUSE indices case insensitive • Slaven Reziç fixed pause.cpan.org certification issue • Abandon primary permissions now means transfer to ADOPTME • Various test/logging improvements • Burp reports by Lee Johnson
31. Further reading Blog posts about PAUSE at PTS 2019 • http://blogs.perl.org/users/neilb/2019/05/pause-pts-2019.html • https://rjbs.manxome.org/rubric/entry/2116 • https://rjbs.manxome.org/rubric/entry/2117 • http://neilb.org/2019/05/01/fixing-pause-permissions.html • http://blogs.perl.org/users/kenichi_ishigaki/2019/05/perl-toolchain- summit-and-pause-permission-management-per-distribution.html PTS 2019 in general • http://act.qa-hackathon.org/pts2019/wiki?node=Blogs • http://act.qa-hackathon.org/pts2019/wiki?node=Results
32. Looking forward to your input https://github.com/andk/pause/issues
33. Thank you