システムログ書式の構造に着目した システム異常検出手法の検討 /css2018-mizutani

Transcript

1. γεςϜϩάॻࣜͷߏ଄ʹண໨ͨ͠
   γεςϜҟৗݕग़ख๏ͷݕ౼
   ΫοΫύουגࣜձࣾ
   ਫ୩ਖ਼ܚ ίϯϐϡʔληΩϡϦςΟγϯϙδ΢Ϝ
   
   
   $
   ߈ܸݕ஌

2. ͸͡Ίʹ • ҟৗͳγεςϜϩάΛݕग़͢Δ͜ͱͰηΩϡϦςΟ Πϯγσϯτൃݟʹͭͳ͍͛ͨ • ϩάͷҟৗݕ஌͸ઌߦݚڀ͕ଟ਺͋Δ෼໺͕ͩ͋· Γ͏·͘ݱ৔Ͱӡ༻Ͱ͖͍ͯΔྫΛฉ͔ͳ͍ • ຊൃද͸࣮؀ڥͰܧଓతʹҟৗݕ஌Λճ͍ͯͨ͘͠ Ίཁ݅Λ੔ཧ͠ϓϩτλΠϓ͓Αͼ࣮ݧͯ͠Έͨ݁

   Ռͷใࠂ
   

3. എܠ • γεςϜϩάʹग़ݱ͢ΔʮҟৗʯΛൃݟ͍ͨ͠ • ҟৗɿ௨ৗ͸ग़ྗ͞Εͳ͍͸ͣͷϩά • ηΩϡϦςΟ؅ཧऀ͕ͨ·ͨ·ʮ໨ʯͰݟ͚ͭΔ͜ͱ ΋͋Δ͕͜ΕΛػցతʹݕग़͍ͨ͠ • ਖ਼ৗܥΛ͢΂ͯఆٛͰ͖Ε͹༰қ͕ͩ…

   • ѻ͏γεςϜͷछྨ͕ଟ͍ˍߏ੒มߋ΋ଟ͍ • ߏ੒มߋʹ൐ͬͯޡݕग़͞ΕΔͷ͸΍ΉΛಘͳ͍͕ม Խʹ௥ਵͯ͠΄͍͠
   

4. [Sun Sep 09 06:41:35.275151 2018] [:error] [pid 1517] [client 10.2.3.4:30626]

   script '/var/www/html/help.php' not found or unable to stat [Sun Sep 09 06:41:35.489685 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/java.php' not found or unable to stat [Sun Sep 09 06:41:35.699245 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/_query.php' not found or unable to stat [Sun Sep 09 06:41:35.903910 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/test.php' not found or unable to stat [Sun Sep 09 06:41:36.104791 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db_cts.php' not found or unable to stat [Sun Sep 09 06:41:36.305791 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db_pma.php' not found or unable to stat [Sun Sep 09 06:41:39.067416 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/logon.php' not found or unable to stat [Sun Sep 09 06:41:39.263892 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/help-e.php' not found or unable to stat [Sun Sep 09 06:41:39.469702 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/license.php' not found or unable to stat [Sun Sep 09 06:41:39.681337 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/log.php' not found or unable to stat [Sun Sep 09 06:41:39.875956 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/hell.php' not found or unable to stat [Sun Sep 09 06:41:40.072330 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/pmd_online.php' not found or unable to stat [Sun Sep 09 06:41:42.835995 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/x.php' not found or unable to stat [Sun Sep 09 06:41:43.050760 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/shell.php' not found or unable to stat [Sun Sep 09 06:41:43.248025 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/desktop.ini.php' not found or unable to stat [Sun Sep 09 06:41:43.450317 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/z.php' not found or unable to stat [Sun Sep 09 06:41:43.655400 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/lala.php' not found or unable to stat [Sun Sep 09 06:41:43.863698 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/lala-dpr.php' not found or unable to stat [Sun Sep 09 06:41:44.094330 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wpo.php' not found or unable to stat [Sun Sep 09 06:41:47.003542 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/text.php' not found or unable to stat [Sun Sep 09 06:41:47.201627 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wp-config.php' not found or unable to stat [Sun Sep 09 06:41:47.419065 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstik.php' not found or unable to stat [Sun Sep 09 06:41:47.625909 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstik2.php' not found or unable to stat [Sun Sep 09 06:41:47.836779 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstiks.php' not found or unable to stat [Sun Sep 09 06:41:48.032147 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstik-dpr.php' not found or unable to stat [Sun Sep 09 06:41:48.243196 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/lol.php' not found or unable to stat [Sun Sep 09 06:41:51.060698 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/uploader.php' not found or unable to stat [Sun Sep 09 06:41:51.279140 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmd.php' not found or unable to stat [Sun Sep 09 06:41:51.485373 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmx.php' not found or unable to stat [Sun Sep 09 06:41:51.696468 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmv.php' not found or unable to stat [Sun Sep 09 06:41:51.915964 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmdd.php' not found or unable to stat [Sun Sep 09 06:41:52.160011 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/knal.php' not found or unable to stat [Sun Sep 09 06:41:54.967426 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmd.php' not found or unable to stat [Sun Sep 09 06:41:54.967426 2018] [:error] [pid 1517] [client 192.168.1.2:12344] /bin/rm: cannot remove `/tmp/a.png’: No such file or directory [Sun Sep 09 06:41:55.161857 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/shell.php' not found or unable to stat [Sun Sep 09 06:41:55.396761 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/appserv.php' not found or unable to stat [Sun Sep 09 06:41:55.593244 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wuwu11.php' not found or unable to stat [Sun Sep 09 06:41:55.787654 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/xw.php' not found or unable to stat [Sun Sep 09 06:41:55.989777 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/xw1.php' not found or unable to stat [Sun Sep 09 06:41:56.175298 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/9678.php' not found or unable to stat [Sun Sep 09 06:41:59.016329 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wc.php' not found or unable to stat [Sun Sep 09 06:41:59.207458 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/xx.php' not found or unable to stat [Sun Sep 09 06:41:59.407406 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/s.php' not found or unable to stat [Sun Sep 09 06:41:59.606750 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/w.php' not found or unable to stat [Sun Sep 09 06:41:59.804259 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/sheep.php' not found or unable to stat [Sun Sep 09 06:41:59.992308 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/qaq.php' not found or unable to stat [Sun Sep 09 06:42:00.173038 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db.init.php' not found or unable to stat [Sun Sep 09 06:42:02.956986 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db_session.init.php' not found or unable to stat [Sun Sep 09 06:42:03.143240 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db__.init.php' not found or unable to stat ͜ͷϩά͸ΠϝʔδͰ͢

5. [Sun Sep 09 06:41:35.275151 2018] [:error] [pid 1517] [client 10.2.3.4:30626]

   script '/var/www/html/help.php' not found or unable to stat [Sun Sep 09 06:41:35.489685 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/java.php' not found or unable to stat [Sun Sep 09 06:41:35.699245 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/_query.php' not found or unable to stat [Sun Sep 09 06:41:35.903910 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/test.php' not found or unable to stat [Sun Sep 09 06:41:36.104791 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db_cts.php' not found or unable to stat [Sun Sep 09 06:41:36.305791 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db_pma.php' not found or unable to stat [Sun Sep 09 06:41:39.067416 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/logon.php' not found or unable to stat [Sun Sep 09 06:41:39.263892 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/help-e.php' not found or unable to stat [Sun Sep 09 06:41:39.469702 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/license.php' not found or unable to stat [Sun Sep 09 06:41:39.681337 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/log.php' not found or unable to stat [Sun Sep 09 06:41:39.875956 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/hell.php' not found or unable to stat [Sun Sep 09 06:41:40.072330 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/pmd_online.php' not found or unable to stat [Sun Sep 09 06:41:42.835995 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/x.php' not found or unable to stat [Sun Sep 09 06:41:43.050760 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/shell.php' not found or unable to stat [Sun Sep 09 06:41:43.248025 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/desktop.ini.php' not found or unable to stat [Sun Sep 09 06:41:43.450317 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/z.php' not found or unable to stat [Sun Sep 09 06:41:43.655400 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/lala.php' not found or unable to stat [Sun Sep 09 06:41:43.863698 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/lala-dpr.php' not found or unable to stat [Sun Sep 09 06:41:44.094330 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wpo.php' not found or unable to stat [Sun Sep 09 06:41:47.003542 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/text.php' not found or unable to stat [Sun Sep 09 06:41:47.201627 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wp-config.php' not found or unable to stat [Sun Sep 09 06:41:47.419065 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstik.php' not found or unable to stat [Sun Sep 09 06:41:47.625909 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstik2.php' not found or unable to stat [Sun Sep 09 06:41:47.836779 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstiks.php' not found or unable to stat [Sun Sep 09 06:41:48.032147 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/muhstik-dpr.php' not found or unable to stat [Sun Sep 09 06:41:48.243196 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/lol.php' not found or unable to stat [Sun Sep 09 06:41:51.060698 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/uploader.php' not found or unable to stat [Sun Sep 09 06:41:51.279140 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmd.php' not found or unable to stat [Sun Sep 09 06:41:51.485373 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmx.php' not found or unable to stat [Sun Sep 09 06:41:51.696468 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmv.php' not found or unable to stat [Sun Sep 09 06:41:51.915964 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmdd.php' not found or unable to stat [Sun Sep 09 06:41:52.160011 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/knal.php' not found or unable to stat [Sun Sep 09 06:41:54.967426 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/cmd.php' not found or unable to stat [Sun Sep 09 06:41:54.967426 2018] [:error] [pid 1517] [client 192.168.1.2:12344] /bin/rm: cannot remove `/tmp/a.png’: No such file or directory [Sun Sep 09 06:41:55.161857 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/shell.php' not found or unable to stat [Sun Sep 09 06:41:55.396761 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/appserv.php' not found or unable to stat [Sun Sep 09 06:41:55.593244 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wuwu11.php' not found or unable to stat [Sun Sep 09 06:41:55.787654 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/xw.php' not found or unable to stat [Sun Sep 09 06:41:55.989777 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/xw1.php' not found or unable to stat [Sun Sep 09 06:41:56.175298 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/9678.php' not found or unable to stat [Sun Sep 09 06:41:59.016329 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/wc.php' not found or unable to stat [Sun Sep 09 06:41:59.207458 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/xx.php' not found or unable to stat [Sun Sep 09 06:41:59.407406 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/s.php' not found or unable to stat [Sun Sep 09 06:41:59.606750 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/w.php' not found or unable to stat [Sun Sep 09 06:41:59.804259 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/sheep.php' not found or unable to stat [Sun Sep 09 06:41:59.992308 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/qaq.php' not found or unable to stat [Sun Sep 09 06:42:00.173038 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db.init.php' not found or unable to stat [Sun Sep 09 06:42:02.956986 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db_session.init.php' not found or unable to stat [Sun Sep 09 06:42:03.143240 2018] [:error] [pid 1517] [client 10.2.3.4:30626] script '/var/www/html/db__.init.php' not found or unable to stat  ͜ͷϩά͸ΠϝʔδͰ͢

6. ϩάͷҟৗݕ஌ʹؔ͢Δઌߦݚڀ • ϩάͷҟৗݕ஌ख๏ • άϧʔϓԽ͞Εͨϩά͔Β஋ͷग़ݱύλʔϯΛ΋ͱʹݕग़[1,2] • ϩάͷू߹͔ΒҟৗͱͳΔཁૉΛݕग़͢Δ[3] • ࣌ܥྻҟৗݕ஌ •

   ϩάͷछྨʢ㲈ϑΥʔϚοτʣͷࣗಈ൑ఆख๏ • AprioriΞϧΰϦζϜͷԠ༻[4] • Shortest Edit ScriptΛ࢖ͬͨख๏[5]
    [1] Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Commu- nications Security, CCS ’17, pp. 1285–1298, New York, NY, USA, 2017. ACM. [2] Siadati, H., & Memon, N. (2017). Detecting Structurally Anomalous Logins Within Enterprise Networks. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security - CCS ’17(pp. 1273–1284). [3] Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. (2009). Online system problem detection by mining patterns of console logs. In Proceedings - IEEE International Conference on Data Mining, ICDM. https://doi.org/10.1109/ICDM.2009.19 [4] Risto Vaarandi. A data clustering algorithm for mining patterns from event logs. in IEEE IPOM’03 Proceed- ings, pp. 119–126, 2003 [5] Min Du and Feifei Li. Spell: Streaming parsing of system event logs. In Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016, pp. 859–864, United States, 1 2017. Institute of Electrical and Electronics Engineers Inc.

7. ࣮ӡ༻؀ڥͰͷར༻Λߟྀͨ͠ཁ݅ • આ໌Մೳੑͷߴ͍ݕग़ • ݕग़݁Ռͷղऍ͕ӡ༻ͷෛ୲ʹ • ηΩϡϦςΟ୲౰ऀ͕༰қʹཧղͰ͖ΔϩδοΫ͕ඞཁ • ஞ࣍తʹॲཧͰ͖ΔੑೳͷΑ͍ΞϧΰϦζϜ •

   ࣮࣌ؒੑͷ࣮ݱ • ࣌ؒܭࢉྔͷ཈੍ʢೖྗ਺Nʹରͯ͠O(N)͕ཧ૝ʣ • ۭؒܭࢉྔͷ཈੍ʢೖྗ਺NͰ੒௕͠ͳ͍ͷ͕ཧ૝ʣ • पظతʹൃੜ͢Δִ͕͕ؒ௕͍ϩά΋͋Δʢ೔ɺिɺ݄୯Ґʣͨ ΊόονॲཧʹΑΔݕग़͸ආ͚͍ͨ=iterativeʹॲཧ͍ͨ͠ • ܭࢉࢿݯͳͲίετͷ໰୊
   

8. Ξϓϩʔν 1. ϩάͷେ෦෼͸printfͳͲͷϑΥʔϚοτจʹ୯७ͳ ஋ΛຒΊࠐΉ͚ͩͱׂΓ੾ͬͯɺϩάΛߏ଄Խ͢Δ 2. ߏ଄Խ͞ΕͨϩάΛॱ࣍ΫϥελϦϯάͯ͠ҟৗݕ ग़ʹར༻͢Δ • ΫϥελΛ஝ੵ͢Δʮֶशʯ •

   طଘΫϥελͱൺֱͯ͠ҟৗͳϩάΛݟ͚ͭΔʮݕग़ʯ ɹˠ جຊతͳखॱ͸྆ํಉ͡
   

9. τʔΫϯ෼ׂͱΫϥελϦϯά
    Dec 29 15:47:06 pylon sshd[27058]: Invalid user

   amavis from 111.205.93.154 m
   ʲೖྗʳ
   
   
   0 < th < 1
   
   ৽͍͠ΫϥελΛ࡞੒͢Δ͔Λܾఆ͢ΔͨΊͷ͖͍͠஋
   
   
   
   
   
   
   
   
   
   
   
   
   m
   
   ஞ࣍ೖྗ͞ΕΔϩάϝοηʔδ 
   ෼ׂ͞ΕͨτʔΫϯͱط ଘΫϥελͷத৺ͷϩάΛ ൺֱͯ͠Ұக཰ s Λܭࢉ ʢ࣍ϖʔδͰઆ໌ʣͯ͠
   s ≧ th
   
   τʔΫϯ௕͕Ұக
   ͔ͭ࠷΋T͕େ͖͍Ϋϥελ
   ʹ౷߹ 
   s < th
   ͷ৔߹͸৽ ͨʹΫϥελΛ࡞੒ ݕग़ϑΣΠζͰ͸৽ͨ ͳΫϥελ࡞੒Λҟৗ ͳϩάͷݕग़ͱ͢Δ طଘΫϥελ ৽نΫϥελ ͦΕͧΕʹத৺ͱͳΔϩά͕͋Δ

10. ϩάಉ࢜ͷҰக཰ͷܭࢉ
     Dec 29 15:47:50 sshd pylon [ 34512

    ] : Invalid user bobber from 10.1.x.x Dec 29 15:47:06 pylon sshd[27058]: Invalid user amavis from 111.205.93.154 Dec 29 15:47:06 sshd pylon [ 27058 ] : Invalid user amavis from 192.168.x.x ϩάΛۭനจࣈɾه߸Λϕʔεʹ
    ϩάϝοηʔδΛτʔΫϯʹ෼ׂ m ೖྗ͞Εͨϩά ൺֱର৅ͷϩάʢΫϥελͷத৺ͷϩάʣ ෆҰக ෆҰக ෆҰக ෆҰக Ұகͨ͠ϩάͷτʔΫϯ਺ ϩάશମͷτʔΫϯ௕ = s ʢ্هྫ͸ҰகτʔΫϯ਺͕ɺશମͷτʔΫϯ௕ͳͷͰҰக཰
    s 㲈
    ʣ

11. ࣮ݧ (1/3): ࣮૷͓ΑͼύϑΥʔϚϯεଌఆ • GoݴޠͰ࣮૷ • https://github.com/m-mizutani/logstruct • ύϑΥʔϚϯεධՁ •

    ۭؒܭࢉྔɾ࣌ؒܭࢉྔͱ΋ʹ ※ FʹϑΥʔϚοτͷू߹ɻϩάϝοηʔδ௕͸ඞͣҰఆҎԼʹͳΔલఏ
     σʔληοτ छྨ ϩά݅਺ ຊख๏ )FΒͷख๏<> P1 #MVF(FOF-    ඵ ඵ P2 )%'4    ඵ ඵ [1] P.He,J.Zhu,P.Xu,Z.Zheng,andM.R.Lyu. A Directed Acyclic Graph Approach to Online Log Parsing. ArXiv e-prints, June 2018. https://arxiv.org/abs/1806.04356 O(|F|) <latexit sha1_base64="c1uD42CJ1CSa9YUdi+zVue/IDOM=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquCHosCuLNCvYD2qVk02wbm02WJCuUbf+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305uZXVtfSO/Wdja3tndK+4fNLRMFKF1IrlUrQBrypmgdcMMp61YURwFnDaD4fXUbz5RpZkUD2YUUz/CfcFCRrCxUuOuPL4Zn3aLJbfizoCWiZeREmSodYtfnZ4kSUSFIRxr3fbc2PgpVoYRTieFTqJpjMkQ92nbUoEjqv10du0EnVilh0KpbAmDZurviRRHWo+iwHZG2Az0ojcV//PaiQkv/ZSJODFUkPmiMOHISDR9HfWYosTwkSWYKGZvRWSAFSbGBlSwIXiLLy+TxlnFcyve/XmpepXFkYcjOIYyeHABVbiFGtSBwCM8wyu8OdJ5cd6dj3lrzslmDuEPnM8fwi2OlA==</latexit> <latexit sha1_base64="c1uD42CJ1CSa9YUdi+zVue/IDOM=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquCHosCuLNCvYD2qVk02wbm02WJCuUbf+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305uZXVtfSO/Wdja3tndK+4fNLRMFKF1IrlUrQBrypmgdcMMp61YURwFnDaD4fXUbz5RpZkUD2YUUz/CfcFCRrCxUuOuPL4Zn3aLJbfizoCWiZeREmSodYtfnZ4kSUSFIRxr3fbc2PgpVoYRTieFTqJpjMkQ92nbUoEjqv10du0EnVilh0KpbAmDZurviRRHWo+iwHZG2Az0ojcV//PaiQkv/ZSJODFUkPmiMOHISDR9HfWYosTwkSWYKGZvRWSAFSbGBlSwIXiLLy+TxlnFcyve/XmpepXFkYcjOIYyeHABVbiFGtSBwCM8wyu8OdJ5cd6dj3lrzslmDuEPnM8fwi2OlA==</latexit> <latexit sha1_base64="c1uD42CJ1CSa9YUdi+zVue/IDOM=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquCHosCuLNCvYD2qVk02wbm02WJCuUbf+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305uZXVtfSO/Wdja3tndK+4fNLRMFKF1IrlUrQBrypmgdcMMp61YURwFnDaD4fXUbz5RpZkUD2YUUz/CfcFCRrCxUuOuPL4Zn3aLJbfizoCWiZeREmSodYtfnZ4kSUSFIRxr3fbc2PgpVoYRTieFTqJpjMkQ92nbUoEjqv10du0EnVilh0KpbAmDZurviRRHWo+iwHZG2Az0ojcV//PaiQkv/ZSJODFUkPmiMOHISDR9HfWYosTwkSWYKGZvRWSAFSbGBlSwIXiLLy+TxlnFcyve/XmpepXFkYcjOIYyeHABVbiFGtSBwCM8wyu8OdJ5cd6dj3lrzslmDuEPnM8fwi2OlA==</latexit> <latexit sha1_base64="c1uD42CJ1CSa9YUdi+zVue/IDOM=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquCHosCuLNCvYD2qVk02wbm02WJCuUbf+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305uZXVtfSO/Wdja3tndK+4fNLRMFKF1IrlUrQBrypmgdcMMp61YURwFnDaD4fXUbz5RpZkUD2YUUz/CfcFCRrCxUuOuPL4Zn3aLJbfizoCWiZeREmSodYtfnZ4kSUSFIRxr3fbc2PgpVoYRTieFTqJpjMkQ92nbUoEjqv10du0EnVilh0KpbAmDZurviRRHWo+iwHZG2Az0ojcV//PaiQkv/ZSJODFUkPmiMOHISDR9HfWYosTwkSWYKGZvRWSAFSbGBlSwIXiLLy+TxlnFcyve/XmpepXFkYcjOIYyeHABVbiFGtSBwCM8wyu8OdJ5cd6dj3lrzslmDuEPnM8fwi2OlA==</latexit> ※ ܭଌ݁Ռͷখ਺఺ୈ3ҐҎԼ੾Γࣺͯ ※ ຊख๏ͷܭଌ؀ڥɿ CPU 2.9 GHz Intel Core i7 / ϝϞϦ16GB / MacBookPro (High Sierra 10.13.6) ࣮ࡍͷܭଌʹΑΓҰఆͷੑೳ͕࣮ݱͰ͖͍ͯΔ͜ͱΛ֬ೝ

12. ࣮ݧ (2/3): ࣮؀ڥͰͷσʔλΛ༻͍࣮ͨݧ֓ཁ • ࣮ࡍͷϩάͰҙਤͨ͠ݕग़͕Ͱ͖Δ͔࣮ݧ • ໿1,000୆ͷAWS EC2Πϯελϯε͔Βsyslogʹग़ྗ͞ΕͨϩάΛར༻ • ͞Βʹ࿈ଓ͢Δ48࣌ؒͷϩάΛલޙ24࣌ؒͰ෼ׂ

    → D1, D2 • D1 Ͱϓϩηεछྨຖʹֶशͨ݁͠ՌΛ D2 ͷݕग़ʹར༻͠ҙਤͨ͠ݕग़ʹͳΔ͔Λݕূ • th = 0.65 ʹઃఆ
     ݕ஌਺ ϓϩηε
    छྨ਺ %ͷϩά਺
    ฏۉ஋ %ͷϩά਺
    தԝ஋ ओͳϓϩηε໊      BQBDIF
    ECVT
    JSRCBMBODF
    NZTRME
    QPTUpY
    OQUE
    ODTE
    QPTUHSFTRM
    STZTMPHE
    TZTUFNE
    [BCCJY ʙ      TTIE
    TTTE
    TV
    $30/
    CBTI
    EPDLFSE
    DISPOZE
    DPOTVM
    JOqVYE
    DMPVEJOJU ʙ       TVEP Ҏ্      LFSOFM
    EIDMJFOU
    KBWB
    EPDLFS ֓Ͷҙਤͨ݁͠Ռ͕ͩछྨͷϓϩηεͰϑΥʔϚοτ਺͕ऩଋͤͣ ऩଋ͍ͯ͠ͳ͍

13. ࣮ݧ (3/3): ࣮؀ڥͰͷσʔλΛ༻͍࣮ͨݧ݁Ռྫ • sshd • Received SIGHUP; restarting. •

    sssd • Configuration file: /etc/sssd/sssd.conf does not exist. • dockerd • time="2018-08-14T06:11:26.411627122Z" level=debug msg="[BUILDER] Command to be executed: [/bin/sh -c curl -sS https://dl.yarnpkg.com/ debian/pubkey.gpg | apt-key add - && echo "deb https:// dl.yarnpkg.com/debian/ stable main\" | tee /etc/apt/sources.list.d/ yarn.list]" • chronyd •Initial frequency 9.204 ppm •Source 2600:2600::199 online • influxd •SELECT mean(****) FROM ***.*** WHERE time >= now() - 30d GROUP BY time(1h), milestone fill(none) service=query
     ҟৗͳʢD1 ʹ͸ग़ݱͤͣD2 ʹ͸ग़ݱͨ͠ʣϩάͷྫ

14. %FNP

15. ߟ࡯ • ϑΥʔϚοτਪఆΞϧΰϦζϜͷվળ͓Αͼผख๏ ͱͷซ༻ͷݕ౼ • γϯϓϧͰӡ༻͠΍͍͢ΞϧΰϦζϜ͕ͩ೚ҙ௕ɾ೚ҙܗࣜͷϩάͷର Ԡ͕೉͍͠ • ಛఆܗࣜͷϩάΛআ֎͢ΔͳͲͷ޻෉͸ඞཁ •

    ݕग़݁Ռʹର͢ΔηΩϡϦςΟ৵֐ͷ൑ఆ • ҟৗͳϩάˠΠϯγσϯτͱ͸ݶΒͳ͍ • ผͷ৘ใͱͷ૊Έ߹ΘͤʹΑΔϦεΫ൑ఆͳͲߟ͑ΒΕΔ • ௕ظతͳӡ༻ʹ͓͚ΔϞσϧͷௐ੔ • ϑΥʔϚοτ಺ͷʮ೚ҙͷτʔΫϯʯ͕ଟ͘ͳΔͱҰக౓ܭࢉͷҙຯ͕ ͳ͘ͳΔ • Ұக౓ܭࢉͷࣜʹ޻෉͢Δ༨஍͕͋Δ
    

16. ·ͱΊ • େྔɾଟ༷ͳϩά͔Β࣮஍Ͱͷར༻Λ૝ఆͨ͠ҟৗݕग़ ख๏Λݕ౼ • ݁Ռͷઆ໌Մೳੑ΍ύϑΥʔϚϯεͳͲͷཁ݅੔ཧ • ౤ೖ͞ΕͨϩάͷϑΥʔϚοτΛਪఆ͠ɺͦͷϑΥʔϚοτ ͔Βҳ୤͢ΔϩάΛݕग़͢ΔγϯϓϧͳΞϧΰϦζϜͷݕ౼ •

    ΞϧΰϦζϜͷ࣮૷ͱ࣮ݧ • ύϑΥʔϚϯε͸ظ଴͢Δ݁Ռʹऩ·Δ • ҙਤ௨Γʹݕग़Ͱ͖ͨ΋ͷ΋͋Δ͕ɺద੾ʹϑΥʔϚοτԽ Ͱ͖ͳ͍छྨͷϩά΋͋ΔͨΊ໰୊ͷ෼ׂͳͲ͕ඞཁ
    

17. ຖ೔ͷྉཧΛָ͠Έʹ͢Δ Thank you