cpm Shoichi Kaji 

Shoichi Kaji (SKAJI) author of cpm and mi6 

Agenda • 5 features of cpm • The internal of cpm — why cpm fast — • Toward cpm version 1.0 

5 features of cpm 

1. fast 

… and this is why I created cpm 

Let’s compare cpm with cpanm 

https://raw.githubusercontent.com/skaji/images/master/cpm/cpanm-Plack.gif 

https://raw.githubusercontent.com/skaji/images/master/cpm/cpm-Plack.gif 

cpm is 3x faster than cpanm 

2. self-contained 

You can use dependency-free
 self-contained cpm, which works fine with fresh perl 5.8.1+ 

› curl -sL https://git.io/cpm > cpm › chmod +x cpm › ./cpm -V cpm 0.983 (./cpm) This is a self-contained version, 0.983 

language: perl perl: - "5.30" install: - curl -sL https://git.io/cpm | perl - install -g script: - prove -l t It also is easy to use cpm in CI such as travis CI 

3. flexible resolvers 

• You can change cpm’s resolvers via —resolver option • Let’s say you have DarkPAN which contains your private distributions, but not whole CPAN distributions. • In fact, other CPAN clients do not work well for such case, but cpm does. 

› cpm install \ --resolver 02package,http://your-darkpan \ --resolver metadb \ Module1 Module2 ... resolve your darkpan first and if it fails,
 fall back to normal metadb resolver 

4. static install 

• Leon Timmermans has proposed a new concept for CPAN distribution installation called static-install • It is much simpler, safer, faster than traditional one (i.e, executing Makefile.PL) • cpm support static-install • I wrote a blog post about static install 

5. prebuilt 

• cpm keeps builds in ~/.perl-cpm and never fetch/build them again • This is (of cource!) inspired by Carmel • This makes cpm even faster! 

https://raw.githubusercontent.com/skaji/images/master/cpm/cpm-prebuilt.gif 

5 features of cpm • 1. fast • 2. self-contained • 3. flexible resolvers • 4. static install • 5. prebuilt 

The internal of cpm — why cpm fast — 

Programming paradigms • To make program fast, there are some programming paradigms • Multi Thread • I don’t think it is good idea to use thread in perl5. Oops. • Event Driven • This is a good choice. But once we adopt event-loop, we cannot use synchronous code anymore. This means that we cannot relay on cpanminus code. Oops. • Multi Process • Let’s use this 

Multi Process • Let’s use multi-process paradigm.
 So the next questions are: • Q1: How do we pass data from the master to workers and vice versa? • Q2: The master need to know that workers finish their jobs as soon as possible. How do we achieve this? worker worker worker master process 

IPC • Q1: How do we pass data between processes? • Idea1: files • Other process does not detect files are changed quickly. It appers inotify is "slow". • Idea2: TCP/IP • A Good choice. Because master and worker processes are in the same host, we don’t have to use TCP/IP necessarily. • Idea3: pipes • Let’s use this 
 We should prepare 2 pipes for master -> woker and woker-> master 

select • How does the master detect which workers are finished? • Workers will send results to the master via pipe, which also means the workers are finished. • So, if the master monitor pipes by select(2), it can detect which workers are finished quickly 

Wrap-up so far • We adopt multi process paradigm • Connect master with workers via 2 pipes • Master monitors pipes by select(2) so that it detects which workers are finished quickly worker worker worker master process pipe pipe pipe select 

Actual code of cpm The master "calculates" jobs Get ready (= finished) workers here (internally, we do select pipes!) Modularize connection between
 the master and workers as Parallel::Pipes module And send job to the ready worker This only 11-line code makes cpm fast! 

Toward cpm version 1.0 • Make distributions "first-class objects" • Traditionally we install CPAN modules into one specific directory. So, after install, we cannot see which distribution a module come from and it is hard to re-use distributions • On the other hand, MIYAGAWA has introduced a concept "central repositories" in his project Carmel • Let’s keep each distributuions separately • Once cpm implement it, for example, cpm can easily install only runtime dependencies 

Wrap up • cpm is a fast CPAN client • it also has some other iteresting features • cpm uses system calls fork/pipe/select effectvely so that it install CPAN modules fast • cpm 1.0 will treat distributions as "first-class objects" 

Thanks • PerlCon 2019 • Perl Toolchain Summit 2017, 2018, 2019 • and You ❤