Upgrade to Pro — share decks privately, control downloads, hide ads and more …

改行区切りのJSON

 改行区切りのJSON

JSON日の金曜日@新潟 9週目で発表した資料です。

D8eb514404fbca5bd485040edccdbe30?s=128

Matsui Masashi

October 14, 2017
Tweet

More Decks by Matsui Masashi

Other Decks in Technology

Transcript

  1. վߦ۠੾ΓͷJSON @circled9 2017-10-13 #friday13json 

  2. ࣗݾ঺հ.json { "name": "Matsui Masashi", "twitter": ["@circled9", "@ma_2_i"], "language": ["javascript",

    "ruby"], } 
  3. Agenda • ͸͡Ίʹ • େମڞ௨ͯ͠Δ͜ͱ • ֤ϑΥʔϚοτͷ঺հ • ·ͱΊ 

  4. ͸͡Ίʹ 

  5. վߦ۠੾ΓͷJSON • վߦΛ۠੾Γจࣈͱͯ͠ѻ͏ϑΥʔϚοτ • ֤ߦ͸validͳJSONͷ஋͕ॻ͔Ε͍ͯΔ • վߦ͠ͳ͍ͷͰɺՄಡੑ͸͋·Γॏࢹ͍ͯ͠ ͳ͍ 

  6. վߦ۠੾ΓͷJSONͷαϯϓϧ {"some":"thing"} {"foo":17,"bar":false,"quux":true} {"may":{"include":"nested","objects":["and","arrays"]}} 

  7. ͳͥվߦͰ۠੾Δͷ͔ • JSONΛύʔε͢Δʹ͸ɺϑΝΠϧͷ࠷ޙ·Ͱ ಡΈࠐ·ͳ͍ͱ͍͚ͳ͍ • վߦจࣈͰ۠੾Δ͜ͱʹΑͬͯɺϑΝΠϧΛ ෦෼తʹύʔεͰ͖ΔΑ͏ʹͳΔ 

  8. ϝϦοτ • ฒߦॲཧͰͷಡΈॻ͖͕͠΍͘͢ͳΔ • ֤ߦ͸ࠓ·ͰͷJSONͱมΘΒͳ͍ͷͰɺطଘ ͷࢿ࢈͕ੜ͔͠΍͍͢ • CSV/TSVͱൺֱ͢Δͱɺ֤ߦ͕දݱͰ͖Δ಺ ༰͕ॊೈʹͳΔ 

  9. େମڞ௨ͯ͠Δ͜ͱ 

  10. ڞ௨͢Δܾ·Γ͝ͱ • ۠੾Γจࣈ͸վߦจࣈ • ֤ߦ͸վߦΛؚ·ͳ͍validͳJSONͷ஋ • ΤϯίʔσΟϯά͸UTF-8 

  11. վߦจࣈ • LF (\n) ʹ͓͚ͯ͠͹ແ೉ • CR+LF (\r\n) Ͱ΋໰୊͸ͳ͍ •

    CR (\r) ͩͱ·͍ͣ͜ͱ͕͋Γͦ͏ • Mac OS 9ͳΜͯී௨࢖ͬͯ·ͤΜͶ 
  12. վߦจࣈ • CRΛ۠੾Γจࣈͱͯ͠ѻ͏͔͸ɺ֤ϑΥʔϚο τʹΑͬͯҟͳΔ • ͔͠͠ɺҎԼͷཧ༝͔ΒCR+LFͰ΋࣮࣭໰୊ͳ͍ • ۭߦ͸ແࢹ͢Δͷ͕௨ྫ • JSONͷύʔε࣌ʹۭനจࣈ͸আڈ͞ΕΔ

    
  13. ֤ϑΥʔϚοτͷ঺հ 

  14. ֤ϑΥʔϚοτͷ঺հ • NDJSON • LDJSON • JSON Lines • NDJ

    
  15. NDJSON • MediaType ͸ application/x-ndjson • ֦ுࢠ͸ .ndjson • CR͸վߦͱͯ͠ݟͳ͢

    
  16. LDJSON • NDJSONͷաڈͷݺͼ໊ • MediaType ͸ application/x-ldjson • ֦ுࢠ͸ .ldjson

    
  17. JSON Lines • MediaType ͸ಛʹݴٴͳ͠ • ֦ுࢠ͸ .jsonl • ѹॖͯ͠

    .jsonl.gz ΍ .jsonl.bz2 ͱͯ͠഑৴ ͢Δ͜ͱ΋ਪ঑ͯ͠Δ • CR͸վߦͱͯ͠ݟͳ͞ͳ͍ 
  18. NDJ • MediaType ͸ಛʹݴٴͳ͠ • ֦ுࢠ͸ .ndj • ࠷ॳͷ1ߦʹ //

    Ͱίϝϯτ͕ॻ͚Δ • CR͸վߦͱͯ͠ݟͳ͞ͳ͍ 
  19. ·ͱΊ 

  20. ·ͱΊ • վߦ۠੾ΓͷJSONͩͱฒߦॲཧͱ͔͕ḿΔ • վߦ۠੾ΓͷJSON΋͍͔ͭ͘ϑΥʔϚοτ͕ ͋Δ • ॻ͖ํͷ࢓༷͸େମಉ͡ 

  21. [EOF] 

  22. ࢀߟURL • NDJSON • http://specs.okfnlabs.org/ndjson/ • LDJSON • https://github.com/ndjson/ndjson-spec/commit/ c658c26ba15bea718228b5a3d54f0d4db22ec0ba

    • JSON Lines • http://jsonlines.org/ • NDJ • http://jimbojw.com/t/index.php?title=Newline_delimited_JSON