身近に潜むtokenize 2016

YAPC Hachioji 2016 LT 資料です

Transcript

1. ਎ۙʹજΉ tokenize
   2016
   @moznion
   

   View Slide

2. @moznion
   

   View Slide

3. ͜͜Ͱݴ͏ tokenize
   - ͳΜ͔௕͍೚ҙͷจࣈྻ͕͋ͬͯ
   - ͦΕΛίϯϐϡʔλ͕ղऍ͠΍͢
   ͍୯Ґ (token) ʹͿͬͨ੾Δ
   

   View Slide

4. Έͳ͞Μ tokenize
   ͯ͠·͔͢
   

   View Slide

5. ༷ʑͳ΋ͷΛ token ʹ
   ෼͚͍͖ͯ·͠ΐ͏
   ෼͚Δඞཁ͕͋Γ·͢
   

   View Slide

6. ྫ͑͹ʁ
   

   View Slide

7. ༣ศ൪߸
   

   View Slide

8. ༣ศ൪߸
   ken_all.csv
   

   View Slide

9. ॅॴ
   

   View Slide

10. ॅॴ
    ૯຿লࢿྉ (pdf)
    

    View Slide

11. ి࿩൪߸
    

    View Slide

12. ి࿩൪߸
    ૯຿লࢿྉ (doc, pdf)
    

    View Slide

13. ςϯγϣϯ্͕ͬͯ

    ͖·͔ͨ͠ʁ
    

    View Slide

14. ͸͍ɼྑ͍Ͱ͢
    ͘͢͝ྑ͍Ͱ͢
    

    View Slide

15. ࢢ֎ہ൪ͷ࿩Λ͠·͠ΐ͏
    

    View Slide

16. 03-5321-1111
    

    View Slide

17. 03-5321-1111
    

    View Slide

18. Φοɼ؆୯ͦ͏Ͱ͢Ͷ
    

    View Slide

19. ࢢ֎ہ൪
    

    View Slide

20. ࢢ֎ہ൪
    ૯຿লࢿྉ (doc, pdf)
    

    View Slide

21. ࢢ֎ہ൪
    ૯຿লࢿྉ (doc, pdf)
    

    View Slide

22. View Slide

23. ͱΓ͋͑ͣத਎ݟͯΈ·͠ΐ͏
    

    View Slide

24. ൪߸۠ըίʔυ ൪߸۠ը ࢢ֎ہ൪ ࢢ಺ہ൪
    1
    ๺ւಓߐผࢢɺࡳຈࢢɺ
    ๺޿ౡࢢɺۭ஌܊ೆຈொ
    11 CDE
    

    View Slide

25. OKOK
    

    View Slide

26. ൪߸۠ըίʔυ ൪߸۠ը ࢢ֎ہ൪ ࢢ಺ہ൪
    3
    ๺ւಓ༦ுࢢʢ෋໺Λআ
    ͘ɻʣ
    123 DE
    

    View Slide

27. OKOK
    

    View Slide

28. ൪߸۠ըίʔυ ൪߸۠ը ࢢ֎ہ൪ ࢢ಺ہ൪
    32
    ๺ւಓҏୡࢢɺѾా܊
    ʢಎ但ބொٴͼ๛Ӝொʹ
    ݶΔɻʣɺ༗च܊
    142 DE
    

    View Slide

29. OKOK
    

    View Slide

30. ൪߸۠ըίʔυ ൪߸۠ը ࢢ֎ہ൪ ࢢ಺ہ൪
    93
    ੨৿ݝे࿨ాࢢɺࡾ୔
    ࢢɺ্๺܊ʢ౦๺ொʢѴ
    ๺ɺѴೆɺ্໺ɺେӜɺ
    ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށ
    ொٴͼ࿡ށொʹݶΔɻʣ
    176 DE
    

    View Slide

31. ੝Γ্͕ͬͯ·͍Γ·ͨ͠
    

    View Slide

32. ൪߸۠ըίʔυ ൪߸۠ը ࢢ֎ہ൪ ࢢ಺ہ൪
    363
    େࡕ෎஑ాࢢۭߓɺେࡕࢢʢ౦ॅ٢۠໼ాࣣஸ໨ٴͼฏ໺۠௕٢઒ล࢛ஸ໨Λআ
    ͘ɻʣɺ໳ਅࢢʢੴݪொɺઘொɺҰ൪ொɺେ૔ொɺ֞಺ொɺ܂࠽৽ொɺ޾෱ொɺणொɺӫ
    ொɺ
    খ࿏ொɺ৽ڮொɺ຤޿ொɺ݄ग़ொɺಊࢁொɺ఼ౡொɺதொɺ඿ொɺ଎ݟொɺ౦ాொɺਂా
    ொɺݹ઒ொɺຊொɺদੜொɺদ༿ொɺޚಊொɺ޲ౡொɺݩொɺ༄ాொٴͼ༄ொʹݶ
    Δɻʣɺ
    ਧాࢢɺઁ௡ࢢʢ๺ผ෎ொɺ৽ࡏՈɺਖ਼੃ɺਖ਼੃ຊொɺঙ԰ɺઍཬٰɺઍཬٰ৽ொɺઍཬ
    ٰ౦࢛ஸ໨ٴͼޒஸ໨ɺ੢Ұ௡԰ɺ඿ொɺ౦ਖ਼੃ɺ౦Ұ௡԰ɺ౦ผ෎ɺҰ௡԰ɺผ෎ɺ
    ࡾౡɺೆઍཬٰฒͼʹೆผ෎ொʹݶΔɻʣɺ๛தࢢɺ౦େࡕࢢʢѴொɺ஑ౡொɺ஑೭୺
    ொɺग़ӢҪொɺग़ӢҪຊொɺҴ༿ɺࠓถɺؠాொʢࡾஸ໨Λআ͘ɻʣɺӝੜಊҰஸ໨ɺՃ
    ೲɺ্ੴ੾ொɺ্࢛৚ொɺ্࿡ສࣉொɺ઒தɺ઒ాɺՏ಺ொɺਆాொɺتཬ઒ொɺ๺ੴ੾
    ொɺ๺ߵ஑ொɺ٬๥ொɺ೔Լொɺޒ৚ொɺߵ஑ொɺߵ஑ಙ҇ொɺߵ஑ຊொɺߵ஑ݩொɺݹ
    ຳྠɺࡩொɺ࢛৚ொɺౡ೭಺ɺԼ࿡ສࣉொɺত࿨ொɺ৽஑ౡொɺ৽ߵ஑ொɺ৽ொɺ৽ঙɺ
    ຤޿ொɺ֯ాɺળࠜࣉொɺୋ఼ொɺๅொɺཱՖொɺۄ۲ொ੢ɺۄ۲ொ౦ɺۄ۲ݩொɺ๛Ӝ
    ொɺௗډொɺதੴ੾ொɺத৽։ɺத໺ɺதߵ஑ொɺೆ૳ொɺ੢ੴ੾ொɺ੢ؠాҰஸ໨ɺ੢
    ߵ஑ொɺֹాொɺ෍ࢢொɺശ఼ொɺՖԂ੢ொɺՖԂ౦ொɺՖԂຊொɺ౦ੴ੾ொɺ౦ߵ஑
    ொɺ౦๛Ӝொɺ౦ࢁொɺඛߐɺඛ԰౦ɺත୻ࢁொɺຊঙதҰஸ໨ɺຊொɺদݪɺদݪೆɺ
    ਫ૸ɺೆߵ஑ொɺೆ࢛৚ொɺຳྠɺޚ޾ொɺݩொɺࢁखொɺ໻ੜொɺԣখ࿏ொɺԣປɺԣ
    ປ੢ɺԣປೆɺ٢ాɺ٢ాຊொɺ٢ాԼౡɺ٢ݪɺ࿡ສࣉொٴͼए૲ொΛআ͘ɻʣɺकޱ
    ࢢɺീඌࢢʢ஛ᔹɺ஛ᔹ੢ٴͼ஛ᔹ౦ʹݶΔɻʣɺฌݿݝೌ࡚ࢢ
    6 BCDE
    

    View Slide

33. େࡕ෎஑ాࢢۭߓɺେࡕࢢʢ౦ॅ٢۠໼ాࣣஸ໨ٴͼฏ໺۠௕٢઒ล࢛ஸ໨Λআ
    ͘ɻʣɺ໳ਅࢢʢੴݪொɺઘொɺҰ൪ொɺେ૔ொɺ֞಺ொɺ܂࠽৽ொɺ޾෱ொɺणொɺӫொɺ
    খ࿏ொɺ৽ڮொɺ຤޿ொɺ݄ग़ொɺಊࢁொɺ఼ౡொɺதொɺ඿ொɺ଎ݟொɺ౦ాொɺਂా
    ொɺݹ઒ொɺຊொɺদੜொɺদ༿ொɺޚಊொɺ޲ౡொɺݩொɺ༄ాொٴͼ༄ொʹݶΔɻʣɺ
    ਧాࢢɺઁ௡ࢢʢ๺ผ෎ொɺ৽ࡏՈɺਖ਼੃ɺਖ਼੃ຊொɺঙ԰ɺઍཬٰɺઍཬٰ৽ொɺઍཬ
    ٰ౦࢛ஸ໨ٴͼޒஸ໨ɺ੢Ұ௡԰ɺ඿ொɺ౦ਖ਼੃ɺ౦Ұ௡԰ɺ౦ผ෎ɺҰ௡԰ɺผ෎ɺ
    ࡾౡɺೆઍཬٰฒͼʹೆผ෎ொʹݶΔɻʣɺ๛தࢢɺ౦େࡕࢢʢѴொɺ஑ౡொɺ஑೭୺
    ொɺग़ӢҪொɺग़ӢҪຊொɺҴ༿ɺࠓถɺؠాொʢࡾஸ໨Λআ͘ɻʣɺӝੜಊҰஸ໨ɺՃ
    ೲɺ্ੴ੾ொɺ্࢛৚ொɺ্࿡ສࣉொɺ઒தɺ઒ాɺՏ಺ொɺਆాொɺتཬ઒ொɺ๺ੴ੾
    ொɺ๺ߵ஑ொɺ٬๥ொɺ೔Լொɺޒ৚ொɺߵ஑ொɺߵ஑ಙ҇ொɺߵ஑ຊொɺߵ஑ݩொɺݹ
    ຳྠɺࡩொɺ࢛৚ொɺౡ೭಺ɺԼ࿡ສࣉொɺত࿨ொɺ৽஑ౡொɺ৽ߵ஑ொɺ৽ொɺ৽ঙɺ
    ຤޿ொɺ֯ాɺળࠜࣉொɺୋ఼ொɺๅொɺཱՖொɺۄ۲ொ੢ɺۄ۲ொ౦ɺۄ۲ݩொɺ๛Ӝ
    ொɺௗډொɺதੴ੾ொɺத৽։ɺத໺ɺதߵ஑ொɺೆ૳ொɺ੢ੴ੾ொɺ੢ؠాҰஸ໨ɺ੢
    ߵ஑ொɺֹాொɺ෍ࢢொɺശ఼ொɺՖԂ੢ொɺՖԂ౦ொɺՖԂຊொɺ౦ੴ੾ொɺ౦ߵ஑
    ொɺ౦๛Ӝொɺ౦ࢁொɺඛߐɺඛ԰౦ɺත୻ࢁொɺຊঙதҰஸ໨ɺຊொɺদݪɺদݪೆɺ
    ਫ૸ɺೆߵ஑ொɺೆ࢛৚ொɺຳྠɺޚ޾ொɺݩொɺࢁखொɺ໻ੜொɺԣখ࿏ொɺԣປɺԣ
    ປ੢ɺԣປೆɺ٢ాɺ٢ాຊொɺ٢ాԼౡɺ٢ݪɺ࿡ສࣉொٴͼए૲ொΛআ͘ɻʣɺकޱ
    ࢢɺീඌࢢʢ஛ᔹɺ஛ᔹ੢ٴͼ஛ᔹ౦ʹݶΔɻʣɺฌݿݝೌ࡚ࢢ
    

    View Slide

34. ͸͍
    

    View Slide

35. ݟ·͢
    

    View Slide

36. ੨৿ݝे࿨ాࢢɺࡾ୔ࢢɺ্๺܊ʢ౦๺ொʢѴ๺ɺ
    Ѵೆɺ্໺ɺେӜɺ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށொٴͼ࿡ށொʹݶΔɻʣ
    

    View Slide

37. ϋϋʔϯ
    

    View Slide

38. શ֯ಡ఺Ͱ۟੾ΒΕͨ
    શׅ֯ހͷSࣜͩͳʁ
    

    View Slide

39. ੨৿ݝे࿨ాࢢɺࡾ୔ࢢɺ্๺܊ʢ౦๺ொʢѴ๺ɺ
    Ѵೆɺ্໺ɺେӜɺ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށொٴͼ࿡ށொʹݶΔɻʣ
    ʢ੨৿ݝे࿨ాࢢɺࡾ୔ࢢɺ্๺܊ʢ౦๺ொʢѴ
    ๺ɺѴೆɺ্໺ɺେӜɺ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށொٴͼ࿡ށொʹݶΔɻʣʣ
    

    View Slide

40. ʢ੨৿ݝे࿨ాࢢɺࡾ୔ࢢɺ্๺܊ʢ౦๺ொʢѴ
    ๺ɺѴೆɺ্໺ɺେӜɺ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށொٴͼ࿡ށொʹݶΔɻʣʣ
    ୯७ͳϦετ
    

    View Slide

41. ʢ੨৿ݝे࿨ాࢢɺࡾ୔ࢢɺ্๺܊ʢ౦๺ொʢѴ
    ๺ɺѴೆɺ্໺ɺେӜɺ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށொٴͼ࿡ށொʹݶ
    Δɻʣʣ
    ͜Ε͸വ਺
    

    View Slide

42. ʢ੨৿ݝे࿨ాࢢɺࡾ୔ࢢɺ্๺܊ʢ౦๺ொʢѴ
    ๺ɺѴೆɺ্໺ɺେӜɺ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށொٴͼ࿡ށொʹݶ
    Δɻʣʣ
    ߴ֊വ਺
    

    View Slide

43. ʢ੨৿ݝे࿨ాࢢɺࡾ୔ࢢɺ্๺܊ʢ౦๺ொʢѴ
    ๺ɺѴೆɺ্໺ɺେӜɺ্๺๺ɺ্๺ೆٴ
    ͼ৽ؗʹݶΔɻʣɺࣣށொٴͼ࿡ށொʹݶ
    Δɻʣʣ
    bool (cond)
    

    View Slide

44. φϧϗσΟ΢ε
    

    View Slide

45. ΍͍͖ͬͯ·͢
    

    View Slide

46. ࠓ·Ͱͷઆ໌ͱ͸ؔ܎ͷͳ͍࣮૷
    https://github.com/moznion/Number-Phone-
    JP-AreaCode
    

    View Slide

47. ͦͷ͏ͪ SࣜΛཧղ͢Δ࣮૷ʹͳΓ·͢
    

    View Slide

48. ࠷ޙʹ
    

    View Slide

49. ׭ެி͸ mysqldump ఏڙͯ͘͠Ε
    

    View Slide