$30 off During Our Annual Pro Sale. View Details »

Why is building the Ruby environment hard?

SHIBATA Hiroshi
September 10, 2022

Why is building the Ruby environment hard?

The Rubyists are facing the build error with missing libraries like openssl, libyaml and libffi issues when they install the new versions every year. Other people are facing issues of nokogiri, rmagick and others when they develop Rails application.

Why we got these issues everyday? I describe the build failure case and their solution from the ruby-build maintainer's point of view. Finally, I introduce the plan for the imcompatible changes about build process in Ruby 3.2 and Rust support of next version of RubyGems.

SHIBATA Hiroshi

September 10, 2022
Tweet

More Decks by SHIBATA Hiroshi

Other Decks in Programming

Transcript

  1. 8IZJTCVJMEJOHUIF3VCZ
    FOWJSPONFOUIBSE
    )JSPTIJ4)*#"5"
    3VCZ,BJHJ

    View Slide

  2. Hiroshi SHIBATA @hsbt
    https://hsbt.org
    Executive Of
    fi
    cer VP of Engineering


    Technical Director


    at GMO Pepabo, Inc. @pepabo

    View Slide

  3. .ZXPSLJO3VCZ
    • 5IFNBJOUBJOFSPG3VCZ(FNT#VOEMFS 1TZDI +40/
    BOEPUIFST
    • "ENJOJTUSBUPSPGEFWFMPQNFOUQMBUGPSN
    • $PEF8FC.BJM
    • (JU)VC3FENJOF
    • (JU)VC"DUJPOT3VCZ$*
    • SCFOWSVCZCVJME

    View Slide

  4. ͳͥ3VCZͷϏϧυࣄ৘ͷ࿩Λ͢Δͷ͔
    • !ITCU͸3VCZ΍3VCZ(FNT SVCZCVJMEͷϝϯςφΛ͠
    ͍ͯΔؔ܎͔؀ڥߏஙʹ͍ͭͯ͸ҟৗʹৄ͍͠
    • ҰํͰɺଞͷਓʑΛݟΔͱ؀ڥߏஙͱ͍͏ͷ͸؆୯ͳ͜ͱͰ΋
    ͳ͘ɺԿ͔͕ϏϧυͰ͖ͳ͍ͱ͍͏͜ͱʹ਺࣌ؒۤઓ͍ͯ͠ΔΒ
    ͍͠
    • ؀ڥߏஙϚχΞͷཱ৔͔Β͜͜࠷ۙͷϏϧυΤϥʔͷ࿩୊Λर͍
    ͳ͕ΒɺҰਓͰ΋ϏϧυΤϥʔʹ࣌ؒΛඅ΍͢ਓΛݮΒ͍ͨ͠

    View Slide

  5. 3VCZͷΑ͋͘Δ
    ϏϧυΤϥʔͱͦͷղܾํ๏

    View Slide

  6. 3VCZͷΑ͋͘Δ
    ϏϧυΤϥʔͱͦͷղܾํ๏
    6CVOUVͰ
    Ϗϧυ͕Ͱ͖ͳ͍

    View Slide

  7. 6CVOUVͰϏϧυ͕Ͱ͖ͳ͍
    • ໰୊6CVOUVͰ͸0QFO44-ͷΈఏڙ͞ΕΔΑ͏ʹͳͬ
    ͨ🐧
    • 0QFO44-ʹରԠͨ͠3VCZͷPQFOTTMϥΠϒϥϦ͸3VCZ
    ʹͷΈ෇ଐ͢Δ
    • 3VCZ΍͸6CVOUVͰ͸Ϗϧυ͢Δ͜ͱ͕ ؆୯
    ʹ͸
    Ͱ͖ͳ͍
    • IUUQTCVHTSVCZMBOHPSHJTTVFT

    View Slide

  8. 6CVOUVͰϏϧυ͕Ͱ͖ͳ͍
    • ղܾ๏BSVCZCVJMEͰ͸ద౰ͳ0QFO44-ΛϏϧυͯ͠࢖͏
    Α͏ʹػೳ௥Ճ͕ͳ͞Εͨ
    • ैདྷ͸NBD04ͷ-JCSF44-޲͚ͷػೳ
    • IUUQTHJUIVCDPNSCFOWSVCZCVJMEQVMM
    • ղܾ๏C3VCZ΁0QFO44-ରԠͷίʔυΛ
    όοΫϙʔτ͢Δ
    • ػೳ௥Ճ΋͋Δ͚Ͳେৎ෉ Ͱఀ଺த

    View Slide

  9. 3VCZͷΑ͋͘Δ
    ϏϧυΤϥʔͱͦͷղܾํ๏
    3VCZ͕NBD04Ͱ
    ϏϧυͰ͖ͳ͍

    View Slide

  10. 3VCZ͕NBD04ͰϏϧυͰ͖ͳ͍
    • ໰୊3VCZ΍ͷϏϧυ࣌ʹGJEEMFͰΤϥʔ
    • ղܾ๏BGJEEMFͷΤϥʔΛ௚͢
    • ͔͠͠3VCZ͸ϝϯςφϯε͕ऴ͍ྃͯ͠ΔͷͰम
    ਖ਼͞Εͨ΋ͷ͕ϦϦʔε͞ΕΔ͜ͱ͸ͳ͍
    compiling escape.c


    closure.c:264:14: error: implicit declaration of function 'ffi_prep_closure' is invalid in C99 [-Werror,-Wimplicit-function-declaration]


    result = ffi_prep_closure(pcl, cif, callback, (void *)self);


    ^


    6 warnings and 1 error generated.

    View Slide

  11. 3VCZ͕NBD04ͰϏϧυͰ͖ͳ͍
    • ղܾ๏CA8OPFSSPSJNQMJDJUGVODUJPOEFDMBSBUJPOAΛ
    ࢖͏ɻ
    • ͱΓ͋͑ͣಈ͘ɺͱ͍͏ϨϕϧͳͷͰ஫ҙ
    • ղܾ๏DA%64&@''*@$-0463&@"--0$AΛ࢖͏
    • ͪ͜Β΋ؔ਺ͷݺ͹Εํ͕มΘͬͯͱΓ͋͑ͣϏϧυͰ͖
    ͯಈ͘ͱ͍͏΋ͷͳͷͰ஫ҙ

    View Slide

  12. 3VCZͷΑ͋͘Δ
    ϏϧυΤϥʔͱͦͷղܾํ๏
    NBD04 7FOUVSB
    Ͱ
    Ϗϧυ͕Ͱ͖ͳ͍

    View Slide

  13. NBD04 7FOUVSB
    ͰϏϧυ͕Ͱ͖ͳ͍
    • ໰୊݄͝ΖʹϦϦʔε༧ఆͷNBD04ͷ࣍ͷόʔδϣ
    ϯ7FOUVSBͰ͸3VCZ͕ϏϧυͰ͖ͳ͍🍎
    • IUUQTCVHTSVCZMBOHPSHJTTVFT
    (snip)


    linking shared-object -test-/arith_seq/extract.bundle


    Undefined symbols for architecture arm64:


    "_rb_arithmetic_sequence_extract", referenced from:


    _arith_seq_s_extract in extract.o


    "_rb_ary_new_capa", referenced from:


    _arith_seq_s_extract in extract.o


    "_rb_ary_store", referenced from:


    _arith_seq_s_extract in extract.o


    "_rb_define_singleton_method", referenced from:


    _Init_extract in extract.o


    "_rb_path2class", referenced from:


    _Init_extract in extract.o


    ld: symbol(s) not found for architecture arm64


    clang: error: linker command failed with exit code 1 (use -v to see invocation)

    View Slide

  14. NBD04 7FOUVSB
    ͰϏϧυ͕Ͱ͖ͳ͍
    • ղܾ๏LBUFJ͕࡞ͬͨɺ"QQMFͷMMWNͷํͷػೳมߋʹ௥ै͢
    Δमਖ਼ CVOEMF@MPBEFSΛ࢖͏
    ͰϏϧυͰ͖ΔΑ͏ʹͳͬͨ
    • IUUQTHJUIVCDPNSVCZSVCZQVMM
    • ʹ΋όοΫϙʔτ͠ͳ͍ͱϏϧυ͸͓ͦΒ͘ෆՄೳ
    • 13͸࡞ͬͨͷͰɺ7FOUVSBਖ਼ࣜϦϦʔε࣌ʹ͸ΈΜͳ࢖͑Δ
    Α͏ʹͳΔ͸ͣɻ
    • 3VCZ͸ϚʔδࡁΈͳͷͰେৎ෉

    View Slide

  15. NBD04 7FOUVSB
    ͰGPSL͕ͳΜ͔ม
    • AUFTU@QSPDFTTSCAͷGPSLͷςετ͕ಈ͔ͳ͍
    • ͓ͦΒ͘ࠓͷ··ͩͱ·ͱ΋ʹ։ൃ͸Ͱ͖ͳͦ͞͏
    • ؀ڥม਺ͱͯ͠AFYQPSU0#+$@%*4"#-&@*/*5*"-*;&@'03,@4"'&5::&4AͰͱ
    Γ͋͑ͣςετ͸ಈ͕͘
    • ܧଓௐࠪத
    TestProcess#test_daemon_noclose


    objc[10525]: +[NSPlaceholderMutableString initialize] may have been in progress in another thread when fork() was called.


    objc[10525]: +[NSPlaceholderMutableString initialize] may have been in progress in another thread when fork() was called.


    We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

    View Slide

  16. (FNͷΑ͋͘Δ
    ϏϧυΤϥʔͱͦͷղܾํ๏

    View Slide

  17. (FNͷΑ͋͘Δ
    ϏϧυΤϥʔͱͦͷղܾํ๏
    NZTRM͕ϏϧυͰ͖ͳ͍

    View Slide

  18. NZTRM͕ϏϧυͰ͖ͳ͍

    • ໰୊NZTRM@DPOGJH͕ݟ͔ͭΒͳ͍
    • ղܾ๏ANZTRM@DPOGJHAΛ࢖͏
    • ͜ͷखͷΤϥʔ͸QLHDPOGJHࢦఆͰղܾ͢Δ͜ͱ΋͋Δ
    (snip)


    checking for my_bool in mysql.h... no


    -----


    Setting rpath to /opt/homebrew/Cellar/mysql/8.0.30/lib


    -----


    $ gem install mysql2 -- --with-mysql-config=/opt/homebrew/bin/mysql_config


    View Slide

  19. NZTRM͕ϏϧυͰ͖ͳ͍

    • ໰୊MJCTTMͱMJC[TUEͰϦϯΫΤϥʔ
    • ղܾ๏AMEGMBHTAͱAXJUIDQQGMBHTAΛ࢖͏
    (snip)


    linking shared-object mysql2/mysql2.bundle


    ld: library not found for -lssl


    clang: error: linker command failed with exit code 1 (use -v to see invocation)


    $ gem i mysql2 -- --with-ldflags=-L(brew --prefix openssl)/lib -L(brew --prefix zstd)/lib \


    --with-cppflags=-I(brew --prefix openssl)/include


    View Slide

  20. NZTRM͕ͳͥϏϧυͰ͖ͳ͍ͷ͔
    • IPNFCSFXͷύεͱґଘϥΠϒϥϦͷ໰୊🍺
    • IPNFCSFX͕ఏڙ͢ΔNZTRMͷόΠφϦ͕ϦϯΫ͞ΕͯϥΠϒϥϦ
    PQFOTTM [TUE
    ͱSVCZ͕Ϗϧυ͞Εͨ࣌ʹϦϯΫ͞ΕΔϥΠϒϥϦ͕
    ෆҰக
    • .NBD04Ͱ͸APQUIPNFCSFXAʹ΋ͳͬͯΔͷ΋໽հ
    • IUUQTHJUIVCDPNSCFOWSVCZCVJMEQVMMJTTVFDPNNFOU
    • ͪͳΈʹ͜ΕΒͷ໰୊͸NZTRMͰࣗಈͰ൑ఆ͞ΕΔΑ͏ʹमਖ਼ࡁΈ

    View Slide

  21. CVOEMFEHFNT͕
    ϏϧυͰ͖ͳ͍
    (FNͷΑ͋͘Δ
    ϏϧυΤϥʔͱͦͷղܾํ๏

    View Slide

  22. CVOEMFEHFNT͕ϏϧυͰ͖ͳ͍
    • ໰୊3VCZͰCVOEMFEHFNTͷϏϧυʹࣦഊ͢Δ
    • IUUQTHJUIVCDPNSVCZSCTJTTVFT
    • ղܾࡦݪҼෆ໌
    • 3VCZ͔ΒCVOEMFEHFNTʹ$FYUFOTJPOͱͯ͠
    EFCVHͱSCT͕௥Ճ͞Ε͕ͨ؀ڥʹΑͬͯϏϧυͰ͖
    ͳ͍

    View Slide

  23. #VOEMFEHFNTͱ3VCZ(FNTͷ੍ݶ
    • 3VCZ(FNT͸ʮΠϯετʔϧࡁΈͷ3VCZʯΛ૝ఆͯ͠$֦ு
    ΛϏϧυ͢Δ
    • ͦͷͨΊʹΠϯετʔϧલɺ·ͨ͸్தʹ$֦ுͷHFNΛ
    Πϯετʔϧ͢Δͷ͸೉͍͠
    • ճආࡦ͸ݱঢ়ͳ͍͕ɺCVOEMFEHFNTΛεΩοϓͯࣗ͠෼Ͱ
    Πϯετʔϧ͢ΔͳͲखॱ͕ඞཁ ͳͷͰؤுΓ·͢

    • 3VCZʹ޲͚ͯͷܧଓ՝୊

    View Slide

  24. 3VCZͰى͜Γͦ͏ͳ
    ϏϧυΤϥʔͷ঺հ

    View Slide

  25. 3VCZͰى͜Γͦ͏ͳ
    ϏϧυΤϥʔͷ঺հ
    :+*5͕3VTUʹͳͬͨ

    View Slide

  26. :+*5͕3VTUʹͳͬͨ
    • ໰୊:+*5Λ࢖͏ͨΊʹ͸AFOBCMFZKJUAΦϓγϣϯΛ͚ͭͨ
    ্ͰSVTUD͕ඞཁ
    • ࢖͍͍ͨਓ͸SVTUDΛೖΕΔ
    • 6CVOUVͷBQUͰ͸SVTUDΛೖΕΔ
    ͜ͱ͕Ͱ͖ΔͷͰ؆୯
    • ղܾࡦSVCZCVJMEʹ͸SVTUDͷࣗಈ൑ఆΛߦͬͯඞཁͳΦϓ
    γϣϯΛ෇༩͢ΔΑ͏ʹͳ͍ͬͯΔͷͰศར

    View Slide

  27. SEQBSUZίʔυͷόϯυϧ໰୊
    3VCZͰى͜Γͦ͏ͳ
    ϏϧυΤϥʔͷ঺հ

    View Slide

  28. όϯυϧ͍ͯͨ͠SEQBSUZιʔείʔυͷ࡟আ
    • ໰୊:".-ϥΠϒϥϦͷQTZDIʹؚ·Ε͍ͯΔMJCZBNMͷιʔε
    ίʔυΛ࡟আͨͨ͠ΊɺMJCZBNM͕ͳ͍؀ڥͰ͸ϏϧυΤϥʔͱͳΔ
    • IUUQTCVHTSVCZMBOHPSHJTTVFT
    • ղܾࡦMJCZBNMΛΠϯετʔϧ͢Δ
    • 8JOEPXTWDQLH
    • NBD04IPNFCSFXPS.BD1PSUT
    • -JOVYBQU EOG QBDNBO

    View Slide

  29. SEQBSUZίʔυͷόϯυϧ໰୊
    • ݩʑ͸8JOEPXTͰͷϏϧυରࡦ
    • ೥WDQLHͷ͓͔͛Ͱ͜ͷ໰୊͸ղܾ͍ͯ͠Δ
    • 3VCZ ίϛολ
    ͷίϯτϩʔϧͷ֎Ͱมߋ΁ͷରԠίετ͸େ͖
    ͍
    • όϯυϧ͢Δίʔυͷߋ৽͸৬ਓʹΑΔMJCZBNMͷมߋΛ೺
    Ѳ্ͨ͠Ͱͷ3VCZ޲͚ͷύονΛखϚʔδ ITCUXPSL

    • ੬ऑੑ͕ൃݟ͞Εͨ৔߹͸3VCZຊମͷϦϦʔε΋ඞཁͱͳΔ

    View Slide

  30. ϏϧυΤϥʔΛ৐Γӽ͑ͨઌ΁

    View Slide

  31. ιϑτ΢ΣΞ͸Կ΋͠ͳ͍ͱյΕΔ
    • ʮԿ΋ͯ͠ͳ͍ͷʹյΕͨʯ
    • ιϑτ΢ΣΞΛಈ͔͢؀ڥ͸ઈ͑ͣมԽ͍ͯ͠Δ
    • 04ɺϛυϧ΢ΣΞɺϋʔυ΢ΣΞ
    • ੬ऑੑɺαϙʔτίετͳͲ؀ڥͷมԽ͔Β͸ಀΕΒΕͳ͍
    • ݱ୅ͷιϑτ΢ΣΞ͸ෳࡶոحͳ૊Έ߹ΘͤͷதͰͳΜͱ͔ಈ͍ͯ
    ͍Δ
    • ʮԿ΋͍ͯ͠ͳ͍͔ΒյΕΔʯʮͦΕ͸ͦ͏ʯ

    View Slide

  32. ιϑτ΢ΣΞ͕มԽʹదԠ͢Δ
    • 04΍ϥΠϒϥϦͳͲมԽ͢Δଆ͕มΘͬͨͱ͖ʹ·ͣ͸ಈ͔
    ͯ͠ΈΔ
    • มԽͨ͠ଆͷมߋΛ೺Ѳ͓ͯ͘͠ͱҙ֎ͱ೺ѲͰ͖Δมߋ
    Ͱ͋Δ͜ͱ͕ଟ͍
    • ؀ڥΛߏங͢Δ΋ͷશͯΛ೺Ѳ͢Δ͜ͱ͸ແཧͳͷͰɺม
    ߋ͢Δͷ͸Օॴʹͯ͠ɺͦΕͧΕৄ͍͠ਓΛͲΜͲΜཔΔ

    View Slide

  33. ίϛϡχςΟΛ׆༻͢Δ
    • Կ΋ͯ͠ͳ͍ͷʹյΕͨ࣌͸IUUQTCVHTSVCZ
    MBOHPSH΍(JU)VCͷ֤छϦϙδτϦͰใࠂ
    • Կ͔Λͯ͠յΕͨ࣌΋ใࠂ😇
    • SVCZKQTMBDL΍஍ҬϛʔτΞοϓͰฉ͍ͯΈΔ
    • ʮʙΛʙʹͯ͠ʙΛಈ͔ͯ͠ΈͨΜͰ͚͢Ͳɺಉ͜͡ͱ
    ΍ͬͨਓ͍·͢ ʯ

    View Slide

  34. ·ͱΊ

    View Slide

  35. 3VCZͷϏϧυ͸ ͪΐͬͱ
    ೉͍͠
    • 3VCZ͸αϝͷΑ͏ʹӭ͗ଓ͚ͯมԽ͠ଓ͚Δݴޠ🦈
    • ݴޠػೳతͳҙຯ͚ͩͰ͸ͳ͘؀ڥʹదԠ͠ଓ͚͍ͯΔ
    • ؀ڥ΁ͷదԠ͸ίϛολͷ౒ྗ͚ͩͰ͸ͳ͘ɺଟ͘ͷϢʔβʔͷڠ
    ྗͰ੒Γཱ͍ͬͯΔ
    • 4QFDJBMUIBOLTOPCV LBUFJ SIF [O[ LLVCVO FSFHPO
    • 3VCZͷະདྷʹ޲͚ͯΤίγεςϜ։ൃΛҰॹʹ΍͍͖ͬͯ·͠ΐ͏💎

    View Slide