Upgrade to Pro — share decks privately, control downloads, hide ads and more …

不動産webサービスを強くする機械学習の使い方

A0b87642a5af9728e024b71582e36251?s=47 hiddy
July 29, 2016

 不動産webサービスを強くする機械学習の使い方

2016/07/28(木) 19:30〜
【不動産テック勉強会#1】人工知能時代に備えて不動産関連データについて色々語らう勉強会
※画像や物件名などについてはマスクしております。ご了承下さい。

A0b87642a5af9728e024b71582e36251?s=128

hiddy

July 29, 2016
Tweet

Transcript

  1. ෆಈ࢈XFCαʔϏεΛڧ͘͢Δ ػցֶशͷ࢖͍ํ ෆಈ࢈ςοΫษڧձ 5XJUUFS!IJEEZZ

  2. ͓໿ଋ ͜ͷ-5͸εϐʔΧʔݸਓͷ ݟղͰ͋Γɺॴଐ͢ΔاۀஂମΛ୅ද͢Δ΋ ͷͰ͸͋Γ·ͤΜɻ Disclaimer This LT expresses the viewpoints

    of ME and is not reviewed for correctness or accuracy by my company.
  3. ୭ʁ

  4. *%!IJEEZZ ৬ۀɿ1.ʢϓϩμΫτͷ΄͏ʣ ͓࢓ࣄɿΨνίʔσΟϯάҎ֎ ಛٕɿXFCαʔϏεͮ͘Γ  ɹͦΕඞཁʁͱݴ͍์ͭ ݴޠɿ3MFWFMͪΐͬ͜ͱSVCZSBJMT ઐ߈ɿܭྔܦࡁֶʢ541ʣ

  5. ͜Ε·Ͱ࡞ͬͨܞΘͬͨαʔϏε ohmy!Ո௞

  6. ௞ି෺݅৘ใαΠτΛ΍͍ͬͯ·͢

  7. σʔλ෼ੳɺϏδϡΞϥΠθʔγϣϯʹΑΓɺ Ϣʔβʔ͕෺݅બͼΛ͠΍͘͢͢Δ

  8. ࠓ೔͸-3௞ିͰͷ ػցֶशͷ ࢖͍ํΛ ͝঺հ͍ͨ͠

  9. ڪΔ΂͖෺͕݅ଘࡏ͢Δ

  10. None
  11. None
  12. None
  13. None
  14. ͜ͷ΁Μ͸Θ͔Γ΍͍͢

  15. ͜ͷ΁Μ͸Θ͔Γ΍͍͢ ೖྗϛε

  16. None
  17. ͜Ε͸ϗϯϞϊ

  18. None
  19. ͜Ε͸χηϞϊ

  20. Ո௞΍ؒऔΓɺ޿͞ͳͲͱ ͍Ζ͍Ζؔ࿈͍ͯ͠Δ

  21. Ո௞΍ؒऔΓɺ޿͞ͳͲͱ ͍Ζ͍Ζؔ࿈͍ͯ͠Δ ͸͘͡ͷ͕ΊΜͲ͍͘͞

  22. Ո௞΍ؒऔΓɺ޿͞ͳͲͱ ͍Ζ͍Ζؔ࿈͍ͯ͠Δ ͸͘͡ͷ͕ΊΜͲ͍͘͞ Ϟσϧͱ͔ߟ͑ͨ͘ͳ͍

  23. 0OFDMBTT47. TWN͞Μ͕ద౰ʹ֎Ε஋൑ఆͯ͘͠ΕΔ ڭࢣσʔλΛ༻ҙ͠ͳͯ͘ΠΠ Βͪ͘Μ

  24. One-class svm library(kernlab) # make data to one-class svm model

    DF <- data.frame(DF, class=1) outlier.svm <- ksvm(x=class ~bukken_shubetsu +struct +struct_all +madori +history_TOTAL +log_price_with_kanrihi, data=DF, type="one-svc",C=1000,scaled=TRUE,nu=0.01, kernel="rbfdot") #judge outlier DF$outlier <- predict(outlier.svm, DF, type = "response") ෺݅छผ ݐஙλΠϓ ઐ༗໘ੵ ؒऔΓ ங೥਺ ؅ཧඅࠐΈՈ௞ʢର਺Խʣ
  25. None
  26. None
  27. Կ΋ߟ͑ͣ ߦ͘Β͍Ͱ ֎Ε஋͸͚ͨ͡ʂ 0 ʾ˜ʽ 0ƂŖŘ̇

  28. ʢ໨తʣ ೖྗϛεσʔλͷ࡟ݮ ʢख๏ʣ "OPNBMZ%FUFDUJPO ʢํ๏ʣ 0OFDMBTT47.

  29. ͞ΒͳΔ໰୊

  30. None
  31. ࢛৞൒ɺ෩࿊ແɺτΠϨڞಉɺτΩϫ૳ͷΑ͏ͳ෺݅

  32. χʔζ͸ ͋Δ͔΋͠Εͳ͍

  33. χʔζ͸ ͋Δ͔΋͠Εͳ͍ ʢ͕ͩʣ

  34. ͋·Γʹଟ͘ ϦετϖʔδʹͰΔͱ ݟͨ໨͕ѱ͍ɻɻ

  35. ΑΖ͍͠ɺ ͳΒ͹ɺ ൑ఆثΛͭ͘Ζ͏

  36.  ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू ਓྗͰ໨ݟʂ ʢϋʔτΛڧ͘΋ͭ͜ͱʣ ˞෺݅ࣗମ͸ѱ͋͘Γ·ͤΜ

  37.  ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू  ಛघ෺݅Ͱ͸ͳ͍΋ͷΛ݅நग़ ͜͜͸Կ΋ߟ͑ͳͯ͘΋ΠΠ

  38. Boro detection svm library(kernlab) boro.svm <- ksvm( boro~bukken_shubetsu +struct +struct_all

    +madori_num +madori_type_num +history_TOTAL +log_price_with_kanrihi, data=DF, C=1.584893, scaled=TRUE,nu=0.01, kernel="rbfdot",kpar=list(sigma=10),cross=13)  ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू  ಛघ෺݅Ͱ͸ͳ͍΋ͷΛ݅நग़  47.ʹͯɺʮಛघ෺݅൑ఆثʯΛੜ੒ ෺݅छผ ݐஙλΠϓ ઐ༗໘ੵ ؒऔΓ ؒऔΓλΠϓ ؅ཧඅࠐΈՈ௞ʢର਺Խʣ ங೥਺ http://d.hatena.ne.jp/sleepy_yoshi/20120624/p1 Λνϡʔχϯάͷࢀߟʹ͠·ͨ͠
  39. None
  40. ಛघͳ෺͕݅ Ϧετ্Ґʹग़ͳͬͨ͘ʂ 0 ʾ˜ʽ 0ƂŖŘ̇

  41. ʢ໨తʣ Ϧετϖʔδͷ69վળ ʢख๏ʣ ̎஋෼ྨث ʢํ๏ʣ 47.

  42. ਺ࣈ΋͍͋͛ͨͰ͢ ઌੜɻ

  43. $73͕͋Βͳ͍͔ͳʔ ʢ͕͋Βͳ͍ʣ

  44. ͦ͏͔ɺ $7ʢ໰߹ͤʣ͞ΕΔ ͨ  ෺݅ʹ͸ͳʹ͔ಛ௃͕͋Δ ͷͰ͸ʁ

  45. 8FCϚʔέతΞϓϩʔνͩͱɺ ϢʔβʔηάϝϯτΛ͖Γɺ ରԠ͢Δ෺݅Λஸೡʹਫ਼౓ߴ͘ Ϩίϝϯυ͢Δͱ͜Ζ͕ͩɺɺɺ

  46. ࡶʹ σʔλυϦϒϯͰ $73վળ͍ͤͨ͞

  47. σʔλαΠΤϯςΟετϨϕϧද IUUQEIBUFOBOFKQTIBLF[P

  48. Ϩϕϧ Ϩϕϧͷਓୡ͸ूܭ෼ੳʹՃ͑ͯɺ࠷௿ݶͷػցֶश ΍౷ܭֶͷख๏Λ஌͍ͬͯ·͢ɻ47.΍ϥϯμϜϑΥϨ ετͳͲͷϝδϟʔͳख๏Λ֮͑ɺσʔλ෼ੳ͕໘ന͘ ͳͬͯ͘ΔࠒͰ͢ɻ͔͠͠ͳ͕Β3΍4144ͳͲͷઐ༻ ιϑτΛৗʹσϑΥϧτઃఆͷύϥϝʔλͰ෼ੳ͍ͯ͠ ͨΓɺಛ௃ྔબ୒΍લॲཧͷॏཁੑΛ؁͘ݟΔ܏޲͕͋ Γ·͢ɻ ͍ۙ͏ͪʹݱ࣮ͷσʔλ͸JSJTͷΑ͏ʹ؁͘ͳ͍͜ͱΛ ஌Δ͜ͱʹͳΔͰ͠ΐ͏ɻ

    
 ͋Γ͕ͪͳൃݴ ʮϥϯμϜϑΥϨετ࠷ڧʯ
  49. ͋Γ͕ͪͳൃݴ ʮϥϯμϜϑΥϨετ࠷ڧʯ

  50. ͋Γ͕ͪͳൃݴ ʮϥϯμϜϑΥϨετ࠷ڧʯ

  51. ϥϯμϜϑΥϨετͰ $7͞Ε΍͍͢෺݅Λ༧ଌ

  52. ϥϯμϜϑΥϨετͰ $7͞Ε΍͍͢෺݅Λ༧ଌ http://nakhirot.hatenablog.com/entry/20130704/1372874761 ΑΓൈਮ

  53. ϥϯμϜϑΥϨετͰ $7͞Ε΍͍͢෺݅Λ༧ଌ CVͨ͠ ෺݅σʔλ CV͠ͳ͔ͬͨ ෺݅σʔλ 3BEPN 'PSFTU ʢύλʔϯೝࣝʣ ࠓ೔ͷ

    ෺݅σʔλ $7ͦ͠͏ͳ ෺݅σʔλʂ Πϝʔδ ֶश ֶश ༧ଌ
  54. 3ͷSBOHFSQBDLBHFͳΒ QSPCBCJMJUZ͕ग़ྗՄೳ install.packages('Rcpp') install.packages('ranger') # make CV model CV.ranger <-

    ranger(formula = CV ~ walk_time1+struct+struct_all +level3+direction+madori_num+madori_type_num+price_with_kanrihi +history_TOTAL+station1+bukken_shubetsu+gyosha_no, data = DFtrain, num.trees=300, write.forest =TRUE, probability =TRUE, always.split.variables= "station1") # prediction of CV model cv.predict <- predict(CVmodel,DF) # draw probabilities cv.predict$predictions[,2]
  55. $71SPCBCJMJUZͷߴ͍ ॱʹϦετදࣔ

  56. $71SPCBCJMJUZͷߴ͍ ॱʹϦετදࣔ ޲্

  57. ʢ໨తʣ $73վળ ʢख๏ʣ ̎஋෼ྨʢ$7֬཰Λܭࢉʣ ʢํ๏ʣ 3BOEPN'PSFTU RͩͱrangerͳͲ৽͍࣮͠૷͕Φεεϝʂʂ

  58. ƅƁƅ Űŕ

  59. ʮਓ޻஌ೳ࣌୅ʹඋ͑ͯ ෆಈ࢈ؔ࿈σʔλʹ͍ͭ ͯ৭ʑޠΒ͏ษڧձʯ

  60. ਓ޻஌ೳͷఆٛ

  61. ਓ޻஌ೳͷఆٛ %FFQ-FBSOJOHͰ ͳΜ͔Ͱ͖ͳ͍͔ͳʔ

  62. %FFQ-FBSOJOH ͱ͍͑͹ը૾ղੳ

  63. ͜Ε·Ͱͷ෺݅αΠτʹ ͸ͳ͍ըظతػೳʂ

  64. ࿨ࣜτΠϨ൑ఆث

  65. ࿨ࣜτΠϨ൑ఆث ࿨ࣜτΠϨ͚ͩ͸ઈରʹݏͩʂ ͱݴ͏ਓ͸͖ͬͱ͍Δʹ͕͍ͪͳ͍

  66. H2O Deeplearning library(h2o) # Deep learningͰֶशͤ͞Δ localH2O <- h2o.init(ip =

    "localhost", port = 54321, startH2O = TRUE, nthreads=-1) res.dl <- h2o.deeplearning(x = 2:10001, y = 1, training_frame = as.h2o(target), activation = "TanhWithDropout", hidden=rep(160,5), epochs = 20) pred.dl <- h2o.predict(object=res.dl, newdata = as.h2o(target)) pred <- as.data.frame(pred.dl) # ਖ਼ղ཰Λ֬ೝ print(1-sum(abs(round(pred[,1]) - target[,1]))/length(target[,1]))  ࿨ࣜτΠϨը૾ΛਓྗͰ݅ఔ౓ऩू  ը૾αΠζΛἧ͑ͯɺάϨʔεέʔϧʹม׵  %FFQMFBSOJOHͰ࿨ࣜτΠϨ൑ఆثΛ࡞੒
  67. H2O Deeplearning ਖ਼ղ཰ɿ  ࿨ࣜτΠϨը૾ΛਓྗͰ݅ఔ౓ऩू  ը૾αΠζΛἧ͑ͯɺάϨʔεέʔϧʹม׵  %FFQMFBSOJOHͰ࿨ࣜτΠϨ൑ఆثΛ࡞੒

  68. ʢࠓ೔ͷ͓࿩ʣ ೖྗϛεσʔλ࡟আ ಛघ෺݅൑ఆ $73վળϑΟϧλ ࿨ࣜτΠϨ൑ఆث

  69. ΋ͬͱ෺݅બͼΛ ָ͍͠΋ͷʹͯ͠ ͍͖·͠ΐ͏ʂ

  70. ͋Γ͕ͱ͏͍͟͝·ͨ͠