Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

How to build traditional Perl interpreters.

AnaTofuZ
August 08, 2019

How to build traditional Perl interpreters.

AnaTofuZ

August 08, 2019
Tweet

More Decks by AnaTofuZ

Other Decks in Programming

Transcript

  1. my $self = shift; • Takahiro SHIMIZU (id:ANATOFUZ) • a.k.a

    “anagra” • I’m from Okinawa in Japan. • I belong to Okinawa.pm • A graduate student in Ryukyu-Univ. • My graduate study was MoarVM.
  2. This photo in Japan • This mean “You don't touch

    the camel" • … I was heading to PerlCon ...
  3. This talk is… • We usually use plenv, perlbrew, homebrew,

    apt and yum
 when install perl5. • These commands automatically execute shell scripts and execute make command.
  4. This talk is… • What if you have to install

    Perl 
 without using package manager commands? • I think to get Perl Source Code, do compile and 
 execute install. (copy to contain PATH directory) • These processes are called “builds".
  5. Install Perl “Install is also interesting when it comes to

    Perl” by Introduction Perl 
 (1994 Shinji Kono)
  6. Contents • How to build Perl • How to build

    and run Perl1.0 • Perl2.0 • Perl3.0 • Perl4.0 • Perl5.0 • How to build Perl6
  7. This talk is Perl • I will deal with Perl

    written by Larry in this talk. • Perl is written by C language.
  8. the historical Perl version is located • At the beginning

    of this talk, let’s think about
 where the source code of the
 historical Perl version is located.
  9. the historical Perl version is located • At the beginning

    of this talk, let’s think about
 where the source code of the
 historical Perl version is located.
  10. the historical Perl version is located • At the beginning

    of this talk, let’s think about
 where the source code of the
 historical Perl version is located.
  11. $git tag if-0.0603 if-0.0604 if-0.0605 perl-1.0 perl-1.0.15 perl-1.0.16 perl-2.0 perl-2.001

    perl-3.000 perl-3.044 perl-4.0.00 perl-4.0.36 perl-5.000 perl-5.000o
  12. How to get a traditional Perl Source Code • Easy

    to download from GitHub.
 • If you want to get commit logs.
 • If you do not want to get commit logs.
 git clone -b perl-1.0 —single-branch [email protected]:Perl/perl5.git wget https://github.com/Perl/perl5/archive/perl-1.0.tar.gz
  13. the tarball is feel the history. ~/w/p/o/p/t/perl5-perl-1.0 » ls -la

    total 816 drwxr-xr-x 44 anatofuz staff 1408 Dec 18 1987 ./ drwxr-xr-x 6 anatofuz staff 192 Jan 26 05:35 ../ -rwxr-xr-x 1 anatofuz staff 27329 Dec 18 1987 Configure* -rw-r--r-- 1 anatofuz staff 226 Dec 18 1987 EXTERN.h -rw-r--r-- 1 anatofuz staff 224 Dec 18 1987 INTERN.h -rw-r--r-- 1 anatofuz staff 5872 Dec 18 1987 MANIFEST -rw-r--r-- 1 anatofuz staff 3522 Dec 18 1987 Makefile.SH -rw-r--r-- 1 anatofuz staff 3386 Dec 18 1987 README -rw-r--r-- 1 anatofuz staff 84 Dec 18 1987 Wishlist -rw-r--r-- 1 anatofuz staff 47964 Dec 18 1987 arg.c -rw-r--r-- 1 anatofuz staff 5478 Dec 18 1987 arg.h -rw-r--r-- 1 anatofuz staff 3366 Dec 18 1987 array.c
  14. the tarball is feel the history. ~/w/p/o/p/t/perl5-perl-1.0 » ls -la

    total 816 drwxr-xr-x 44 anatofuz staff 1408 Dec 18 1987 ./ drwxr-xr-x 6 anatofuz staff 192 Jan 26 05:35 ../ -rwxr-xr-x 1 anatofuz staff 27329 Dec 18 1987 Configure* -rw-r--r-- 1 anatofuz staff 226 Dec 18 1987 EXTERN.h -rw-r--r-- 1 anatofuz staff 224 Dec 18 1987 INTERN.h -rw-r--r-- 1 anatofuz staff 5872 Dec 18 1987 MANIFEST -rw-r--r-- 1 anatofuz staff 3522 Dec 18 1987 Makefile.SH -rw-r--r-- 1 anatofuz staff 3386 Dec 18 1987 README -rw-r--r-- 1 anatofuz staff 84 Dec 18 1987 Wishlist -rw-r--r-- 1 anatofuz staff 47964 Dec 18 1987 arg.c -rw-r--r-- 1 anatofuz staff 5478 Dec 18 1987 arg.h -rw-r--r-- 1 anatofuz staff 3366 Dec 18 1987 array.c
  15. How to build latest Perl • These manual methods may

    not be aware of so much because brew, apt, and perlbrew etc. are convenient and usable these days.
  16. Traditional Perl build flow. • makedpened is actually executed at

    also perl5
 (it is executed automatically) • Basically it is almost the same as modern perl5!! • old perl (strictly around Perl3) depends manually on make
  17. How to create Configure • Handwriting✍
 (we must follow update

    OS)
 • Not use Configure
 (using Cmake, using other build tools) • create Configure from Some tools.
  18. How to create Configure • I think people often use

    so-called GNU tools such as autoconf and automake. • autoconf commands, create Configure using setting file called configure.in.
 • Created Configure create Makefile using makefile.in.
 (It is automake to automatically generate makefile.in)
  19. Other Languages • Ruby and Python have configure.in
 (using autoconf)


    • C compiler LLVM / clang does not rely on Configure
 because it uses cmake.
  20. Configure in Perl • Perl does not have a configure.in

    at from 1.0 • First of all, autoconf was not released in Perl 1.0 era. (Autoconf 1991 release, Perl 1987 release)
 • I wondered how Larry was creating the settings.
  21. A.「He creates dist toolkit by himself」 • Larry creates dist

    toolkit by himself. • dist is create Configure using setting files. • —> create package from codes. • —> sent the package to email. • Also, the maintenance right was handed over to other people immediately after publication.
  22. metaconfig • Perl script named metaconfig to generate configure.
 •

    Older brother of autoconf (also written in wikipedia) • Not metaconfig in the current GitHub Perl 5 repository.
  23. metaconfig process NFUBDPOpH Run packinit to put together the information

    of the program to be distributed QBDLBHF QBDLJOJU
  24. metaconfig process NFUBDPOpH Load MANIFEST (a list of source code

    Perl intend to use). Load .package QBDLBHF ."/*'&45 QBDLJOJU
  25. metaconfig process NFUBDPOpH Generate a file that arranges programs such

    as cp required at build time, 
 called wanted. QBDLBHF ."/*'&45 QBDLJOJU 8BOUFE
  26. metaconfig process NFUBDPOpH Create a file called Obsolete that summarizes

    the file changes QBDLBHF ."/*'&45 QBDLJOJU 8BOUFE 0CTPMFUF
  27. present day dist/metaconfig • The dist toolkit's metaconfig is plainly

    maintained. • But, Perl 4 flavored source code that can run on Perl 5. • Some dist toolkits are not maintained as Perl 5 
 (eg there is code left requiring getopts.pl)
  28. present day dist/metaconfig • metaconfig ins in the dist toolkit.

    • They are continuously maintained. • metaconfig can execute by Perl5!
 But, It is Perl4 flavor source code.
  29. a part of metaconfig source code sub main'profile { &auto_main'profile;

    } sub auto_main'profile { &main'dataload; } # Load the calling function from DATA segment and call it. This function is # called only once per routine to be loaded. sub main'dataload { local($__packname__) = (caller(1))[3]; $__packname__ =~ s/::/'/; local($__rpackname__) = $__packname__; local($__at__) = $@; $__rpackname__ =~ s/^auto_//; &perload'load_from_data($__rpackname__); local($__fun__) = "$__rpackname__"; $__fun__ =~ s/'/'load_/; eval "*$__packname__ = *$__fun__;"; # Change symbol table entry die $@ if $@; # Should not happen $@ = $__at__; # Restore value $@ had on entrance &$__fun__; # Call newly loaded function
  30. present-day dist/metaconfig • For historical Perl, Perl's configuration file (*

    .SH) required to run metaconfig can be used at that time, which is convenient.
 • Perl5's Configure is not just using dist's metaconfig, but it is becoming more and more often direct rewriting itself.
  31. Historical Perl rough structure • Written in C and yacc,

    there is no big difference between Perl 5 and other Perls. • perly.c have main function. • perl.c is a parsing phase generated by perl.y (YACC) • The names of perl.c and perly.c are replaced from Perl 5.
  32. Perl1.0 • 1987 release. The oldest existing Perl. • Older

    than IRC at historical. • Published at comp.sources.misc newsgroups. • Grammar definitions are done by yacc, 
 others are written in C
  33. the features of Perl1.0 • Perl1.0 has about two and

    a half data types • strings ( not scalar!) • arrays of strings • associative arrays [experimental]
  34. Perl1.0 has … • Replace • Basic regular expression •

    List operations (shift, unsift, pop, push) • @_ • String operators • chop • each • subroutine
  35. In Perl man page • Perl is a interpreted language

    optimized for scanning arbitrary text files, extracting information from those text files, and printing reports based on that information. It's also a good language for many system management tasks. •It combines (in the author's opinion, anyway) some of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. — by Perl1 man page
  36. example Perl1 Scripts • can for loop!!! #!/usr/bin/env perl @array

    = (1,2,3,4,5,6,7,8,9,10); for ( $i = 0; $i < $#array; $i++){ print $array[$i] ."\n"; }
  37. example Perl1 Scripts (subroutine) sub MAX { $max = pop(@_);

    while ($foo = pop(@_)) { $max = $foo if $max < $foo; } $max; } ... $bestday = do MAX($mon,$tue,$wed,$thu,$fri);
  38. measure the processing time with fizzbuzz example for ($i =

    0; $i < 10000_000; $i++){ if ($i % 3 == 0 ){ print "Fizz"; if ($i % 5 == 0){ print "Buzz"; } print "\n"; next; } if ($i % 5 == 0){ print "Buzz\n"; } print "$i\n"; }
  39. 100 order fizzbuzz • Perl1 is faster with calculate fizzbuzz

    for 0 to 100 than Perl5 1FSM 1FSM       
  40. 10 million order fizzbuzz • But, Perl5 is faster with

    10 million fizzbuzz than Perl1!!! 1FSM 1FSM       
  41. today Perl1.0 • PerlMonger has the habit of writing Perl

    1.0 patches regularly (100% test passing version exists) • I was implementation Perl1.0 Docker Images. • You can easy execute Perl1.0 when using Docker.
  42. Perl1.0 structure characteristics. • SV type advance, STR type is

    a simple configuration that only has char * • Naturally depends on the machine environment because it depends on the char type
  43. Perl1.0 structure Source Code • Written in C before published

    K&R C second edition
 (Perl1.0 is written in C before ANSI-C) • Since C at that time did not have a standard, the return type of the library function was different depending on the processing system. • These are controlled by the macro.
  44. Perl1.0 main function in C main(argc,argv,env) register int argc; register

    char **argv; register char **env; { register STR *str; register char *s; char *index(); linestr = str_new(80); str = str_make("-I/usr/lib/perl "); /* first
  45. Perl1.0 main function in C main(argc,argv,env) register int argc; register

    char **argv; register char **env; { register STR *str; register char *s; char *index(); linestr = str_new(80); str = str_make("-I/usr/lib/perl "); /* first There is no type in the main function. Receive environment in env. The keyword register hints to compiler that a given variable can be put in a register.
  46. Perl1.0 main function in C main(argc,argv,env) register int argc; register

    char **argv; register char **env; { register STR *str; register char *s; char *index(); linestr = str_new(80); str = str_make("-I/usr/lib/perl "); /* first In the first place, the declaration of arguments is so-called K & R style
  47. K&R style in Perl • It is rewritten in 1997

    by Nick Ing-Simmons, focusing on the miniperl part. • He corrected by protoize and handwriting.
  48. today Perl5 #ifdef NO_ENV_ARRAY_IN_MAIN extern char **environ; int main(int argc,

    char **argv) #else int main(int argc, char **argv, char **env) #endif { int exitstatus, i; #ifdef PERL_GLOBAL_STRUCT struct perl_vars *my_vars = init_global_struct(); # ifdef PERL_GLOBAL_STRUCT_PRIVATE int veto; main function is int type, written in perlmain.c
  49. Perl 1.0 process flow • First of all analyze the

    option • Load one file at a time with the files entered • YACC parsing while lexical analysis • I will just continue to run !!
  50. Perl2.0 is difficult… • The included Configure can be used

    as it is • but when make depend it is destroyed in relation to the header file of mac os ./config.h:172:19: warning: '/*' within block comment [-Wcomment] #/*undef TMINSYS /**/ ^ /usr/include/AvailabilityInternal.h:38846:2: error: #else after #else #else ^ /usr/include/AvailabilityInternal.h:38203:10: error: unterminated conditional directive #if __MAC_OS_X_VERSION_MIN_REQUIRED >= __MAC_10_13 ^ /usr/include/AvailabilityInternal.h:38116:10: error: unterminated conditional
  51. VS Configure<1> • Remake Configure itself using metaconfig that is

    maintained. • The newly created Configure will export files according to your environment, which will allow you to make. • You can execute make.
  52. VS Configure<2> • We try install gcc or llvm/clang. •

    If you try to build gcc from source code, 
 I think you need to use llvm / clang installed with brew. brew install gcc brew install llvm #should include PATH (at /usr/libexec/bin)
  53. fix Makefile • The Makefile was successfully generated, but it

    is in a situation where it can not be built subtly.
  54. fix Makefile • <built-in> and <command line> suddenly appear in

    Makefile • It can not be built because it does not exist as a header file…. arg.o: /usr/include/time.h arg.o: <built-in> arg.o: <command line>
  55. How to fix Makefile • The header file to be

    used is written to the Makefile after makedpend • So We want to change do not generate command-line etc. in makedepend's configuration file makedepend.SH
  56. makedepend.SH $egrep '^#include ' `cat .clist` `cat .hlist` >.deptmp $echo

    "Updating Makefile..." <.clist $sed -n \ -e '/\//{' \ -e 's|^\(.*\)/\(.*\)\.c|\2.o: \1/\2.c; '"$defrule \1/\2.c|p" \ -e d \ -e '}' \ -e 's|^\(.*\)\.c|\1.o: \1.c|p' >> Makefile.new <.hlist $sed -n 's|\(.*/\)\(.*\)|s= \2= \1\2=|p' >.hsed <.deptmp $sed -n 's|c:#include "\(.*\)".*$|o: \1|p' | \ $sed 's|^[^;]*/||' | \ $sed -f .hsed >> Makefile.new <.deptmp $sed -n 's|c:#include <\(.*\)>.*$|o: /usr/include/\1|p' \ >> Makefile.new <.deptmp $sed -n 's|h:#include "\(.*\)".*$|h: \1|p' | \ $sed -f .hsed >> Makefile.new
  57. makedepend.SH $egrep '^#include ' `cat .clist` `cat .hlist` >.deptmp $echo

    "Updating Makefile..." <.clist $sed -n \ -e '/\//{' \ -e 's|^\(.*\)/\(.*\)\.c|\2.o: \1/\2.c; '"$defrule \1/\2.c|p" \ -e d \ -e '}' \ -e 's|^\(.*\)\.c|\1.o: \1.c|p' >> Makefile.new <.hlist $sed -n 's|\(.*/\)\(.*\)|s= \2= \1\2=|p' >.hsed <.deptmp $sed -n 's|c:#include "\(.*\)".*$|o: \1|p' | \ $sed 's|^[^;]*/||' | \ $sed -f .hsed >> Makefile.new <.deptmp $sed -n 's|c:#include <\(.*\)>.*$|o: /usr/include/\1|p' \ >> Makefile.new <.deptmp $sed -n 's|h:#include "\(.*\)".*$|h: \1|p' | \ $sed -f .hsed >> Makefile.new I’m give up fixing it here?
  58. makedepend.SH $egrep '^#include ' `cat .clist` `cat .hlist` >.deptmp $echo

    "Updating Makefile..." <.clist $sed -n \ -e '/\//{' \ -e 's|^\(.*\)/\(.*\)\.c|\2.o: \1/\2.c; '"$defrule \1/\2.c|p" \ -e d \ -e '}' \ -e 's|^\(.*\)\.c|\1.o: \1.c|p' >> Makefile.new <.hlist $sed -n 's|\(.*/\)\(.*\)|s= \2= \1\2=|p' >.hsed <.deptmp $sed -n 's|c:#include "\(.*\)".*$|o: \1|p' | \ $sed 's|^[^;]*/||' | \ $sed -f .hsed >> Makefile.new <.deptmp $sed -n 's|c:#include <\(.*\)>.*$|o: /usr/include/\1|p' \ >> Makefile.new <.deptmp $sed -n 's|h:#include "\(.*\)".*$|h: \1|p' | \ $sed -f .hsed >> Makefile.new This code is sed…
  59. I remember •It combines (in the author's opinion, anyway) some

    of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. — by Perl1 man page
  60. I remember •It combines (in the author's opinion, anyway) some

    of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. — by Perl1 man page
  61. makedepend.SH $egrep '^#include ' `cat .clist` `cat .hlist` >.deptmp $echo

    "Updating Makefile..." <.clist $sed -n \ -e '/\//{' \ -e 's|^\(.*\)/\(.*\)\.c|\2.o: \1/\2.c; '"$defrule \1/\2.c|p" \ -e d \ -e '}' \ -e 's|^\(.*\)\.c|\1.o: \1.c|p' >> Makefile.new <.hlist $sed -n 's|\(.*/\)\(.*\)|s= \2= \1\2=|p' >.hsed <.deptmp $sed -n 's|c:#include "\(.*\)".*$|o: \1|p' | \ $sed 's|^[^;]*/||' | \ $sed -f .hsed >> Makefile.new <.deptmp $sed -n 's|c:#include <\(.*\)>.*$|o: /usr/include/\1|p' \ >> Makefile.new <.deptmp $sed -n 's|h:#include "\(.*\)".*$|h: \1|p' | \ $sed -f .hsed >> Makefile.new !!!!!!!!!!!!!!!!!!!!!!!!!!!
  62. makedepend.SH $egrep '^#include ' `cat .clist` `cat .hlist` >.deptmp $echo

    "Updating Makefile..." <.clist $sed -n \ -e '/\//{' \ -e 's|^\(.*\)/\(.*\)\.c|\2.o: \1/\2.c; '"$defrule \1/\2.c|p" \ -e d \ -e '}' \ -e 's|^\(.*\)\.c|\1.o: \1.c|p' >> Makefile.new <.hlist $sed -n 's|\(.*/\)\(.*\)|s= \2= \1\2=|p' >.hsed <.deptmp $sed -n 's|c:#include "\(.*\)".*$|o: \1|p' | \ $sed 's|^[^;]*/||' | \ $sed -f .hsed >> Makefile.new <.deptmp $sed -n 's|c:#include <\(.*\)>.*$|o: /usr/include/\1|p' \ >> Makefile.new <.deptmp $sed -n 's|h:#include "\(.*\)".*$|h: \1|p' | \ $sed -f .hsed >> Makefile.new We have a Perl1!!!!!!!!
  63. Forced resolution • But, We have a Per1!!!! • I

    use Perl1 “one liner”!!! • Of course you can do it with Perl5 !!! $./perl -ni -e 'print unless /(built-in)|(command line)/;’ Makefile
  64. Forced resolution • But, We have a Per1!!!! • I

    use Perl1 “one liner”!!! • Of course you can do it with Perl5 !!! $./perl -ni -e 'print unless /(built-in)|(command line)/;’ Makefile
  65. fix YACC • An error is generated by stabval for

    which nothing is described in YACC %type <ival> prog decl format %type <stabval> %type <cmdval> block lineseq line loop cond sideff nexpr else
  66. fix YACC • Comment out!!!!!!! %type <ival> prog decl format

    /*%type <stabval>*/ %type <cmdval> block lineseq line loop cond sideff nexpr else
  67. fix YACC • sprintf separates char pointer or int ...

    • Currently int type • It is not necessary because sprintf includes header files in the first place #ifdef CHARSPRINTF char *sprintf(); #else int sprintf(); #endif
  68. correct C language Source • unkown type name UIDTYPE…!? Really…!?

    ./perl.h:235:1: error: unknown type name 'UIDTYPE' UIDTYPE getuid(); ^ ./perl.h:236:1: error: unknown type name 'UIDTYPE' UIDTYPE geteuid(); ^ ./perl.h:237:1: error: unknown type name 'GIDTYPE' GIDTYPE getgid(); ^ ./perl.h:238:1: error: unknown type name 'GIDTYPE' GIDTYPE getegid();
  69. correct C language Source • Modified typed!!!!!!!! NAME geteuid, getuid

    -- get user identification SYNOPSIS #include <unistd.h> uid_t geteuid(void); uid_t getuid(void);
  70. correct C language Source • modify all typed to uid_t

    uid_t getuid(); uid_t geteuid(); uid_t getgid(); uid_t getegid();
  71. correct C language Source • conflicting types …!!! In file

    included from arg.c:10: ./perl.h:275:14: error: conflicting types for 'times' long time(), times(); ^
  72. correct C language Source • It is heartbreaking that errors

    increase.... arg.c:934:43: error: incomplete definition of type 'struct tms' apush(ary,str_nmake(((double)timesbuf.tms_utime)/HZ)); ~~~~~~~~^ ./perl.h:226:12: note: forward declaration of 'struct tms' EXT struct tms timesbuf; ^ arg.c:935:43: error: incomplete definition of type 'struct tms' apush(ary,str_nmake(((double)timesbuf.tms_stime)/HZ)); ~~~~~~~~^ ./perl.h:226:12: note: forward declaration of 'struct tms' EXT struct tms timesbuf; ^ arg.c:936:43: error: incomplete definition of type 'struct tms' apush(ary,str_nmake(((double)timesbuf.tms_cutime)/HZ));
  73. correct C language Source • morecoer is not defined as

    static, etc… malloc.c:155:1: error: static declaration of 'morecore' follows non-static declaration malloc.c:126:5: note: previous implicit declaration is here morecore(bucket); ^ malloc.c:164:5: error: non-void function 'morecore' should return a value [- Wreturn-type] return; ^ malloc.c:181:5: error: non-void function 'morecore' should return a value [- Wreturn-type] return; ^
  74. correct C language Source • I will put a word

    of magic! • If you do something with Gcc, it will automatically enter but not clang static morecore(register); static findbucket(union overhead*, int);
  75. Summary up to here In gcc, it is possible to

    build with the correction point introduced just now! (Let’s try!!)
  76. Perl4.0 • There is a problem of conflict with C

    library that function name includes • We have Perl 5 so replace to other function's name with using one-liner.
  77. Perl5.0 • Perl5.0 is most difficult build to historical Perl

    versions. • Even if Configure is recreated with metaconfig, 
 Makefile cannot be created.
  78. about Perl6 • The current Perl 6 is mainstream Rakudo

    • Rakudo is a three-stage configuration of VM + NQP (NotQuitPerl) + Perl 6 (made by NQP) • Traditional Perl 6 is probably like Pugs and Parrot
  79. Next • I have 3 projects related to traditional Perl.

    • rewrite Perl1 sourcecode C to Rust. • Implementing Inline::Perl1 as Perl5 CPAN. • Implementing scripts to run all Perl versions with one liner
 (PERL-ALL?)
  80. Conclusion • Perl5 is greatness. • We can build all

    Perl versions. • Let’s try build old Perl versions!
  81. How to build Perl6 .PBS#ZUF$PEF /21 NQP NBLF NPBS Pass

    the pre- bytecoded NQP interpreter to the MoarVM binary
  82. How to build Perl6 .PBS#ZUF$PEF /21 NQP NBLF NPBS 

    aORQ Provide as input NQP source code describing the NQP interpreter
  83. How to build Perl6 NQP NBLF NPBS  aORQ ORQ


    TUBHF ORQ
 TUBHF This is often called nqp