Golangで作るFTPプロキシサーバ

Go Conference 2018 Springで話した内容です。
話し手は、Golangを利用して、FTPプロキシサーバを開発しています。その際に起こった問題や、Golangにおける複数ソケットを扱う際のテクニックを紹介します。

Transcript

1. ʙय़ͩʂ౦ژͩʂʂʂ߹ίϯͩʂʂʂʂ̍̍ʙ
   QZBNB(.01FQBCP *OD
   (P$POGFSFODF4QSJOH
   (PMBOHͰ࡞Δ
   '51ϓϩΩγαʔό
   

   View Slide

2. ϗεςΟϯάࣄۀ෦νʔϑςΫχΧϧϦʔυ
   
   ࢁԼ࿨඙!QZBNB
   

   View Slide

3. -JOVYͷ౷߹Ϣʔβʔ؅ཧ
   50.-ܗࣜ
   )551+40/
   -%"1΍%#ͷ؅ཧͷӡ༻ෛՙ
   ѻ͍΍͍͢ϑΥʔϚοτ
   ൚༻ੑɾ֦ுੑ
   

   View Slide

4. ϖύϘ෱Ԭ
   

   View Slide

5. ࠓ೔࿩͢͜ͱ
   wࠓɺ೥य़ɺ'51͕೤͍
   w'51ϓϩΩγαʔόΛ։ൃͨ͠ͱ͖ͷ5JQT
   

   View Slide

6. ͳͥࠓ'51͕೤͍ͷ͔ʁ
   

   View Slide

7. 4,:4
   

   View Slide

8. 4,:
   

   View Slide

9. 4
   ੈքతͳ ,
   ج४Ͱ :
   Ϡό͍
   

   View Slide

10. 4
    ੈքతͳ ,
    ج४Ͱ :
    Ϡό͍
    4FSWFS
    4FSWJDF
    4ZOD
    

    View Slide

11. 4
    ੈքతͳ ,
    ج४Ͱ :
    Ϡό͍
    4FSWFS
    4FSWJDF
    4ZOD
    

    View Slide

12. Ϩϯαόͷߏ੒্ͷ՝୊
    44) '51
    VTFS
    VTFSDPOUFOUT
    ෺ཧαʔό
    

    View Slide

13. Ϩϯαόͷߏ੒্ͷ՝୊
    VTFSDPOUFOUT
    44) '51
    VTFS
    VTFSDPOUFOUT
    44) '51
    ౰ࣾϨϯλϧαʔόαʔϏε͓͍ͯ͸෺ཧαʔόʹ͓٬༷ͷίϯςϯπ͕഑ஔ͞Ε͓ͯΓɺ
    ෛՙঢ়گͳͲΛؑΈͯɺαʔόҠઃ͢Δࡍʹɺ͓٬༷ͷ'51΍44)ͷ઀ଓઌ͕
    มΘͬͯ͠·͏ߏ੒্ͷ՝୊͕͋Δ
    Ҡઃ
    

    View Slide

14. 44)͸طʹ044͕ଘࡏ͢Δ
    wIUUQTHJUIVCDPNUHTTIQJQFS
    wϢʔβʔ໊ϕʔεͷ44)ϓϩΩγ
    VTFS"
    VTFS#
    TTIQJQFS
    4FSWFS"
    4FSWFS#
    Ϣʔβʔ໊ʹԠͯ͡ɺ઀ଓઌΛมߋ
    

    View Slide

15. QGUQΛόʔϯ࡞Δ͜ͱͰطଘͷ՝୊Λղܾ
    wIUUQTHJUIVCDPNQZBNBQGUQ
    wಈత'51ϓϩΩγ
    VTFS"
    VTFS#
    QGUQ
    4FSWFS"
    4FSWFS#
    ೚ҙͷϛυϧ΢ΣΞΛར༻ͯ͠઀ଓઌΛมߋ
    "1*
    .JEEMFXBSF-BZFS
    

    View Slide

16. '51 'JMF5SBOTGFS1SPUPDPM
    
    wฏจςΩετͷίϚϯυૢ࡞
    % ftp 127.0.0.1 2121
    Connected to 127.0.0.1.
    220 Welcome on ftpserver
    Name (127.0.0.1:yamashitakazuhiko): test
    331 OK
    Password:
    230 Password ok, continue
    ftp> ls
    227 Entering Passive Mode (127,0,0,1,224,164)
    150 Using transfer connection
    drw-rw-rw- 1 ftp ftp 4096 Mar 28 06:38 virtual
    226 Closing transfer connection
    

    View Slide

17. QGUQ͕໨ࢦ͢ੈք
    QGUQ
    PSJHJO
    DMJFOU
    64&3 64&3
    0,
    0,
    QGUQ͸֤ίϚϯυʹॲཧΛϑοΫͭͭ͠ɺϓϩΩγ
    ṖϑοΫ
    

    View Slide

18. 5$1ίωΫγϣϯΛ
    ॲཧ͢Δ
    

    View Slide

19. OFU-JTUFOͰίωΫγϣϯΛ͏͚Δ
    OFU-JTUFOFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    DPOO
    DPOO
    DPOO
    listener, _ := net.Listen(“tcp”,”127.0.0.1:21")
    for {
    connection, _ := listener.Accept()
    ɹ // ίωΫγϣϯΛgroutine΁
    }
    

    View Slide

20. ίωΫγϣϯ͝ͱʹίϚϯυΛಡΈࠐΈॲཧ
    reader = bufio.NewReader(connection)
    line, err := reader.ReadString('\n')
    => line: USER test
    ftp> open 127.0.0.1 2121
    ftp> user test
    CVpP3FBEFSΛར༻͢Δ͜ͱͰ
    ιέοτ͔Βͷಡࠐ͕ศརʹ
    

    View Slide

21. ಉ࣌઀ଓ਺؅ཧ͢Δ
    OFU-JTUFOFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    DPOO
    DPOO
    DPOO
    HPSPVUJOF։࢝࣌ʹΧ΢ϯτΞοϓ͠ɺ
    HPSPVUJOFͷऴྃ࣌ʹΧ΢ϯτμ΢ϯ͢Δ͜ͱͰ
    ಉ࣌઀ଓ਺Λ؅ཧ
    

    View Slide

22. *EMFλΠϜΞ΢τΛઃ͚Δ͜ͱͰࣗಈ੾அ
    OFU-JTUFOFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    DPOO
    DPOO
    DPOO
    OFU$POO4FU%FBEMJOFΛར༻͠ɺૢ࡞ͷͳ͍ΫϥΠΞϯτΛ
    λΠϜΞ΢τॲཧ
    

    View Slide

23. OFU$POO4FU%FBEMJOF
    for {
    connection.SetDeadline(
    ɹɹɹtime.Now().Add(
    ɹɹɹtime.Duration(time.Second.Nanoseconds() * int64(idleTimeout)))
    ɹ )
    // 1ߦͣͭಡΉ
    line, err := reader.ReadString('\n')
    // ड͚औͬͨίϚϯυͷॲཧ
    }
    %FBEMJOF౸ୡͨ͠৔߹ɺJPUJNFPVU
    

    View Slide

24. '51ϓϩΩγ
    OFU-JTUFOFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    HPSPVUJOF
    IBOEMFS
    DPOO
    DPOO
    DPOO
    PSJHJO
    PSJHJO
    PSJHJO
    $0.."/%
    $0.."/%
    $0.."/%
    $0.."/%
    $0.."/%
    $0.."/%
    HPSPVUJOF୯ҐͰϓϩΩγίωΫγϣϯΛுΔ
    

    View Slide

25. '51ίϚϯυΛॲཧ͢Δ
    

    View Slide

26. '51ʹ͸ଟ͘ͷίϚϯυ͕ଘࡏ
    w64&3ίϚϯυϨεϙϯε
    w'&"5ίϚϯυ/Ϩεϙϯε
    w1"47ίϚϯυϨεϙϯείωΫγϣϯʂʂʂ̍
    ίϚϯυ͝ͱͷڍಈͷҧ͍͔ΒϓϩΩγͳͷʹ
    ϓϩτίϧͷཧղ͕ඞཁ
    

    View Slide

27. View Slide

28. '51͸ίωΫγϣϯར༻͢Δ
    w'51͸ίϚϯυͷ΍ΓऔΓΛ͢Δ੍ޚίωΫγϣϯͱɺσʔλͷసૹ
    Λߦ͏సૹίωΫγϣϯͷ̎ͭΛར༻͢Δ
    QGUQ
    PSJHJO
    DMJFOU
    -*45 -*45
    ίϚϯυૢ࡞Ҏ֎ΛσʔλసૹίωΫγϣϯͰߦ͏
    EBUBDPOFDUJPO
    DPOUSPMDPOOFDUJPO
    pMFUYU
    pMFUYU
    

    View Slide

29. '51ͷσʔλసૹํࣜ͸ͭ
    w"DWJWFํࣜαʔό͔ΒσʔλసૹίωΫγϣϯΛுΔ
    w1BTTJWFํࣜΫϥΠΞϯτ͔ΒσʔλసૹίωΫγϣϯΛுΔ
    wͲͷϙʔτͰαʔό͕-JTUFOͯ͠Δ͔Λ1BTTJWFํࣜͷ৔߹͸ɺ1"47
    ίϚϯυͰ௨஌͢Δ
    DMJFOU QGUQ
    EBUBDPOFDUJPO DMJFOU QGUQ
    EBUBDPOFDUJPO
    "DUJWFํࣜ 1BTTJWFํࣜ
    

    View Slide

30. 1"47ίϚϯυ͸த਎Λॻ͖׵͑Δ
    &OUFSJOH1BTTJWF.PEF
    
    º
    
    QGUQ PSJHJO
    DMJFOU
    1"47 1"47
    
    
    

    View Slide

31. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ
    

    View Slide

32. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ σʔλίωΫγϣϯཁٻ
    

    View Slide

33. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ σʔλίωΫγϣϯཁٻ
    999Ͱ-JTUFO։࢝
    

    View Slide

34. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ σʔλίωΫγϣϯཁٻ
    999Ͱ-JTUFO։࢝
    :::Ͱ-JTUFO։࢝
    

    View Slide

35. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ σʔλίωΫγϣϯཁٻ
    999Ͱ-JTUFO։࢝
    :::Ͱ-JTUFO։࢝
    ϑΝΠϧసૹཁٻ
    

    View Slide

36. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ σʔλίωΫγϣϯཁٻ
    999Ͱ-JTUFO։࢝
    :::Ͱ-JTUFO։࢝
    ϑΝΠϧసૹཁٻ ϑΝΠϧసૹཁٻ
    

    View Slide

37. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ
    EBUBDPOFDUJPO
    σʔλίωΫγϣϯཁٻ
    999Ͱ-JTUFO։࢝
    :::Ͱ-JTUFO։࢝
    999UPQGUQ΁σʔλసૹ։࢝
    ϑΝΠϧసૹཁٻ ϑΝΠϧసૹཁٻ
    

    View Slide

38. σʔλίωΫγϣϯͷϓϩΩγ
    QGUQ PSJHJO
    DMJFOU
    σʔλίωΫγϣϯཁٻ
    DMJFOU͔Βݟͨ৔߹ɺQGUQ͸αʔόɺ
    ΦϦδϯ͔Βݟͨ৔߹ɺQGUQ͸ΫϥΠΞϯτͷڍಈ͕ٻΊΒΕΔ
    EBUBDPOFDUJPO EBUBDPOFDUJPO
    σʔλίωΫγϣϯཁٻ
    999Ͱ-JTUFO։࢝
    :::Ͱ-JTUFO։࢝
    :::UPDMJFOU΁σʔλసૹ։࢝ 999UPQGUQ΁σʔλసૹ։࢝
    ϑΝΠϧసૹཁٻ ϑΝΠϧసૹཁٻ
    

    View Slide

39. ɹɹEPXOMPBEHPSPVUJOF
    func proxy() {
    …
    // ϦϨʔ༻ͷgoroutineΛىಈ
    eg.Go(func() error { return relay(ctx, server, client) })
    eg.Go(func() error { return relay(ctx, client, server) })
    // ׬ྃ·Ͱ଴ͪ߹ΘͤΔ
    if err := eg.Wait(); err != nil {
    return err
    }
    }
    HSPVUJOFͰϓϩΩγ
    PSJHJO
    DMJFOU
    ɹɹVQMPBEHPSPVUJOF
    QGUQ
    

    View Slide

40. ී௨ʹ΍Δͱ͏·͍͔͘ͳ͍
    func relay(from, to net.Conn) error {
    buff := make([]byte, BUFFER_SIZE)
    for {
    select {
    case <-ctx.Done():
    return ctx.Err()
    default:
    if err := from.Read(buff); err != nil {
    return errr
    }
    if err := to.Write(b); err != nil {
    return errr
    }
    }
    }
    }
    Ξοϓϩʔυɺ΋͘͠͸μ΢ϯϩʔυͷऴྃͰ
    ྆ํίωΫγϣϯΛऴ͍ྃͨ͠ DUY%POF
    
    Ͱ΋ऴྃ͠ͳ͍
    

    View Slide

41. ɹɹEPXOMPBEHPSPVUJOF
    ϑΝΠϧΞοϓϩʔυͷॲཧ
    PSJHJO
    DMJFOU
    ɹɹVQMPBEHPSPVUJOF
    QGUQ
    3FBE 8SJUF
    3FBE
    8SJUF
    Ξοϓϩʔυ࣌ɺVQMPBEHSPVUJOF͸3FBE͕ൃੜ͢Δ͕ɺ
    EPXOMPBEHSPVUOF͸3FBE͕ൃੜ͠ͳ͍
    

    View Slide

42. 3FBE͸ϒϩοΩϯά
    func relay(from, to net.Conn) error {
    buff := make([]byte, BUFFER_SIZE)
    for {
    select {
    case <-ctx.Done():
    return ctx.Err()
    default:
    if err := from.Read(buff); err != nil {
    return errr
    }
    if err := to.Write(b); err != nil {
    return errr
    }
    }
    }
    }
    3FBEͰϒϩοΩϯά͞ΕΔ͕Ώ͑ʹɺ
    ࣍ͷϧʔϓ΁͍͔ͳ͍
    

    View Slide

43. ϊϯϒϩοΩϯά3FBE
    for {
    select {
    case <-ctx.Done():
    return ctx.Err()
    case err := <-errChan:
    return err
    // ಡࠐgoroutineΛىಈ͠ɺඇಉظͰಡΈࠐΉ
    case b := <-read:
    _, err := to.Write(b)
    if err != nil {
    return err
    }
    }
    }
    νϟϯωϧܦ༝ͰಡΈࠐΉ͜ͱʹΑΓ
    ϊϯϒϩοΩϯά΁
    

    View Slide

44. -FBLDPOUFYU8JUI$BODFM
    NBJO
    HPSPVUJOF
    HPSPVUJOF
    SFUVSO
    DUY%POF
    
    ctx, done := context.WithCancel(context.Background())
    defer done()
    eg, ctx := errgroup.WithContext(ctx)
    // ϦϨʔ༻ͷgoroutineΛىಈ
    eg.Go(func() error { return p.relay(ctx, server, client) })
    eg.Go(func() error { return p.relay(ctx, client, server })
    // ׬ྃ·Ͱ଴ͪ߹ΘͤΔ
    if err := eg.Wait(); err != nil {
    return err
    }
    EPOF
    
    

    View Slide

45. -FBLDPOOFDUJPO$MPTF
    HPGVOD
    \
    GPS\
    O FSSGSPN3FBE CV⒎
    
    JGFSSOJM\
    FSS$IBOFSS
    SFUVSO
    ^
    SFBECV⒎<O>
    ^
    ^
    
    ίωΫγϣϯΤϥʔ͕ൃੜɺ
    ϒϩοΫ͕ղআ͞ΕΔ
    for {
    select {
    case <-ctx.Done():
    from.Close()
    case err := <-errChan:
    return err
    case b := <-read:
    _, err := to.Write(b)
    if err != nil {
    return err
    }
    }
    }
    ૬ํͷసૹ͕ऴΘΓ࣍ୈ
    ίωΫγϣϯΛ
    Ϋϩʔζ
    Τϥʔνϟϯωϧܦ༝
    Ͱॲཧऴྃ
    

    View Slide

46. ࣮૷ґଘ
    

    View Slide

47. ϓϩτίϧʹ͸࣮૷ґଘ͕͖ͭ΋ͷ
    w3'$ͷղऍͷҧ͍ʹΑΓɺඍົʹ࣮૷͕ҟͳΔ
    wओཁͳαʔόɺΫϥΠΞϯτͷ࠷௿ݶͷಈ࡞ςετ͸͢΂͖
    

    View Slide

48. ·ͱΊ
    

    View Slide

49. 4
    ੈքతͳ ,
    ج४Ͱ :
    ΍͹͍
    

    View Slide

50. ϓϩτίϧαʔό͸ϓϩτίϧ࣮૷ɺ
    3'$Λͻͨ͢Β
    ಡΉͱ࡞ΕΔ
    

    View Slide

51. σʔλసૹϓϩτίϧ͸
    ϊϯϒϩοΩϯά໋
    

    View Slide

52. HPSPVUJOFϦʔΫͤͪ͞ΌͩΊɺ
    ໿ଋͩΑʁ
    

    View Slide

53. 5IBOLZPV
    ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU
    

    View Slide