不動産webサービスを強くする機械学習の使い方

by hiddy

Slide 1

Slide 1 text

ෆಈ࢈XFCαʔϏεΛڧ͘͢Δ ػցֶशͷ࢖͍ํ ෆಈ࢈ςοΫษڧձ 5XJUUFS!IJEEZZ

Slide 2

Slide 2 text

͓໿ଋ ͜ͷ-5͸εϐʔΧʔݸਓͷ ݟղͰ͋Γɺॴଐ͢ΔاۀஂମΛ୅ද͢Δ΋ ͷͰ͸͋Γ·ͤΜɻ Disclaimer This LT expresses the viewpoints of ME and is not reviewed for correctness or accuracy by my company.

Slide 3

Slide 3 text

୭ʁ

Slide 4

Slide 4 text

*%!IJEEZZ ৬ۀɿ1.ʢϓϩμΫτͷ΄͏ʣ ͓࢓ࣄɿΨνίʔσΟϯάҎ֎ ಛٕɿXFCαʔϏεͮ͘Γ ɹͦΕඞཁʁͱݴ͍์ͭ ݴޠɿ3MFWFMͪΐͬ͜ͱSVCZSBJMT ઐ߈ɿܭྔܦࡁֶʢ541ʣ

Slide 5

Slide 5 text

͜Ε·Ͱ࡞ͬͨܞΘͬͨαʔϏε ohmy!Ո௞

Slide 6

Slide 6 text

௞ି෺݅৘ใαΠτΛ΍͍ͬͯ·͢

Slide 7

Slide 7 text

σʔλ෼ੳɺϏδϡΞϥΠθʔγϣϯʹΑΓɺ Ϣʔβʔ͕෺݅બͼΛ͠΍͘͢͢Δ

Slide 8

Slide 8 text

ࠓ೔͸-3௞ିͰͷ ػցֶशͷ ࢖͍ํΛ ͝঺հ͍ͨ͠

Slide 9

Slide 9 text

ڪΔ΂͖෺͕݅ଘࡏ͢Δ

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

͜ͷ΁Μ͸Θ͔Γ΍͍͢

Slide 15

Slide 15 text

͜ͷ΁Μ͸Θ͔Γ΍͍͢ ೖྗϛε

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

͜Ε͸ϗϯϞϊ

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

͜Ε͸χηϞϊ

Slide 20

Slide 20 text

Ո௞΍ؒऔΓɺ޿͞ͳͲͱ ͍Ζ͍Ζؔ࿈͍ͯ͠Δ

Slide 21

Slide 21 text

Ո௞΍ؒऔΓɺ޿͞ͳͲͱ ͍Ζ͍Ζؔ࿈͍ͯ͠Δ ͸͘͡ͷ͕ΊΜͲ͍͘͞

Slide 22

Slide 22 text

Ո௞΍ؒऔΓɺ޿͞ͳͲͱ ͍Ζ͍Ζؔ࿈͍ͯ͠Δ ͸͘͡ͷ͕ΊΜͲ͍͘͞ Ϟσϧͱ͔ߟ͑ͨ͘ͳ͍

Slide 23

Slide 23 text

0OFDMBTT47. TWN͞Μ͕ద౰ʹ֎Ε஋൑ఆͯ͘͠ΕΔ ڭࢣσʔλΛ༻ҙ͠ͳͯ͘ΠΠ Βͪ͘Μ

Slide 24

Slide 24 text

One-class svm library(kernlab) # make data to one-class svm model DF <- data.frame(DF, class=1) outlier.svm <- ksvm(x=class ~bukken_shubetsu +struct +struct_all +madori +history_TOTAL +log_price_with_kanrihi, data=DF, type="one-svc",C=1000,scaled=TRUE,nu=0.01, kernel="rbfdot") #judge outlier DF$outlier <- predict(outlier.svm, DF, type = "response") ෺݅छผ ݐஙλΠϓ ઐ༗໘ੵ ؒऔΓ ங೥਺ ؅ཧඅࠐΈՈ௞ʢର਺Խʣ

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Կ΋ߟ͑ͣ ߦ͘Β͍Ͱ ֎Ε஋͸͚ͨ͡ʂ 0 ʾ˜ʽ 0ƂŖŘ̇

Slide 28

Slide 28 text

ʢ໨తʣ ೖྗϛεσʔλͷ࡟ݮ ʢख๏ʣ "OPNBMZ%FUFDUJPO ʢํ๏ʣ 0OFDMBTT47.

Slide 29

Slide 29 text

͞ΒͳΔ໰୊

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

࢛৞൒ɺ෩࿊ແɺτΠϨڞಉɺτΩϫ૳ͷΑ͏ͳ෺݅

Slide 32

Slide 32 text

χʔζ͸ ͋Δ͔΋͠Εͳ͍

Slide 33

Slide 33 text

χʔζ͸ ͋Δ͔΋͠Εͳ͍ ʢ͕ͩʣ

Slide 34

Slide 34 text

͋·Γʹଟ͘ ϦετϖʔδʹͰΔͱ ݟͨ໨͕ѱ͍ɻɻ

Slide 35

Slide 35 text

ΑΖ͍͠ɺ ͳΒ͹ɺ ൑ఆثΛͭ͘Ζ͏

Slide 36

Slide 36 text

ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू ਓྗͰ໨ݟʂ ʢϋʔτΛڧ͘΋ͭ͜ͱʣ ˞෺݅ࣗମ͸ѱ͋͘Γ·ͤΜ

Slide 37

Slide 37 text

ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू ಛघ෺݅Ͱ͸ͳ͍΋ͷΛ݅நग़ ͜͜͸Կ΋ߟ͑ͳͯ͘΋ΠΠ

Slide 38

Slide 38 text

Boro detection svm library(kernlab) boro.svm <- ksvm( boro~bukken_shubetsu +struct +struct_all +madori_num +madori_type_num +history_TOTAL +log_price_with_kanrihi, data=DF, C=1.584893, scaled=TRUE,nu=0.01, kernel="rbfdot",kpar=list(sigma=10),cross=13) ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू ಛघ෺݅Ͱ͸ͳ͍΋ͷΛ݅நग़ 47.ʹͯɺʮಛघ෺݅൑ఆثʯΛੜ੒ ෺݅छผ ݐஙλΠϓ ઐ༗໘ੵ ؒऔΓ ؒऔΓλΠϓ ؅ཧඅࠐΈՈ௞ʢର਺Խʣ ங೥਺ http://d.hatena.ne.jp/sleepy_yoshi/20120624/p1 Λνϡʔχϯάͷࢀߟʹ͠·ͨ͠

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

ಛघͳ෺͕݅ Ϧετ্Ґʹग़ͳͬͨ͘ʂ 0 ʾ˜ʽ 0ƂŖŘ̇

Slide 41

Slide 41 text

ʢ໨తʣ Ϧετϖʔδͷ69վળ ʢख๏ʣ ̎஋෼ྨث ʢํ๏ʣ 47.

Slide 42

Slide 42 text

਺ࣈ΋͍͋͛ͨͰ͢ ઌੜɻ

Slide 43

Slide 43 text

$73͕͋Βͳ͍͔ͳʔ ʢ͕͋Βͳ͍ʣ

Slide 44

Slide 44 text

ͦ͏͔ɺ $7ʢ໰߹ͤʣ͞ΕΔ ͨ ෺݅ʹ͸ͳʹ͔ಛ௃͕͋Δ ͷͰ͸ʁ

Slide 45

Slide 45 text

8FCϚʔέతΞϓϩʔνͩͱɺ ϢʔβʔηάϝϯτΛ͖Γɺ ରԠ͢Δ෺݅Λஸೡʹਫ਼౓ߴ͘ Ϩίϝϯυ͢Δͱ͜Ζ͕ͩɺɺɺ

Slide 46

Slide 46 text

ࡶʹ σʔλυϦϒϯͰ $73վળ͍ͤͨ͞

Slide 47

Slide 47 text

σʔλαΠΤϯςΟετϨϕϧද IUUQEIBUFOBOFKQTIBLF[P

Slide 48

Slide 48 text

Ϩϕϧ Ϩϕϧͷਓୡ͸ूܭ෼ੳʹՃ͑ͯɺ࠷௿ݶͷػցֶश ΍౷ܭֶͷख๏Λ஌͍ͬͯ·͢ɻ47.΍ϥϯμϜϑΥϨ ετͳͲͷϝδϟʔͳख๏Λ֮͑ɺσʔλ෼ੳ͕໘ന͘ ͳͬͯ͘ΔࠒͰ͢ɻ͔͠͠ͳ͕Β3΍4144ͳͲͷઐ༻ ιϑτΛৗʹσϑΥϧτઃఆͷύϥϝʔλͰ෼ੳ͍ͯ͠ ͨΓɺಛ௃ྔબ୒΍લॲཧͷॏཁੑΛ؁͘ݟΔ܏޲͕͋ Γ·͢ɻ ͍ۙ͏ͪʹݱ࣮ͷσʔλ͸JSJTͷΑ͏ʹ؁͘ͳ͍͜ͱΛ ஌Δ͜ͱʹͳΔͰ͠ΐ͏ɻ   ͋Γ͕ͪͳൃݴ ʮϥϯμϜϑΥϨετ࠷ڧʯ

Slide 49

Slide 49 text

͋Γ͕ͪͳൃݴ ʮϥϯμϜϑΥϨετ࠷ڧʯ

Slide 50

Slide 50 text

͋Γ͕ͪͳൃݴ ʮϥϯμϜϑΥϨετ࠷ڧʯ

Slide 51

Slide 51 text

ϥϯμϜϑΥϨετͰ $7͞Ε΍͍͢෺݅Λ༧ଌ

Slide 52

Slide 52 text

ϥϯμϜϑΥϨετͰ $7͞Ε΍͍͢෺݅Λ༧ଌ http://nakhirot.hatenablog.com/entry/20130704/1372874761 ΑΓൈਮ

Slide 53

Slide 53 text

ϥϯμϜϑΥϨετͰ $7͞Ε΍͍͢෺݅Λ༧ଌ CVͨ͠ ෺݅σʔλ CV͠ͳ͔ͬͨ ෺݅σʔλ 3BEPN 'PSFTU ʢύλʔϯೝࣝʣ ࠓ೔ͷ ෺݅σʔλ $7ͦ͠͏ͳ ෺݅σʔλʂ Πϝʔδ ֶश ֶश ༧ଌ

Slide 54

Slide 54 text

3ͷSBOHFSQBDLBHFͳΒ QSPCBCJMJUZ͕ग़ྗՄೳ install.packages('Rcpp') install.packages('ranger') # make CV model CV.ranger <- ranger(formula = CV ~ walk_time1+struct+struct_all +level3+direction+madori_num+madori_type_num+price_with_kanrihi +history_TOTAL+station1+bukken_shubetsu+gyosha_no, data = DFtrain, num.trees=300, write.forest =TRUE, probability =TRUE, always.split.variables= "station1") # prediction of CV model cv.predict <- predict(CVmodel,DF) # draw probabilities cv.predict$predictions[,2]

Slide 55

Slide 55 text

$71SPCBCJMJUZͷߴ͍ ॱʹϦετදࣔ

Slide 56

Slide 56 text

$71SPCBCJMJUZͷߴ͍ ॱʹϦετදࣔ ޲্

Slide 57

Slide 57 text

ʢ໨తʣ $73վળ ʢख๏ʣ ̎஋෼ྨʢ$7֬཰Λܭࢉʣ ʢํ๏ʣ 3BOEPN'PSFTU RͩͱrangerͳͲ৽͍࣮͠૷͕Φεεϝʂʂ

Slide 58

Slide 58 text

ƅƁƅ Űŕ

Slide 59

Slide 59 text

ʮਓ޻஌ೳ࣌୅ʹඋ͑ͯ ෆಈ࢈ؔ࿈σʔλʹ͍ͭ ͯ৭ʑޠΒ͏ษڧձʯ

Slide 60

Slide 60 text

ਓ޻஌ೳͷఆٛ

Slide 61

Slide 61 text

ਓ޻஌ೳͷఆٛ %FFQ-FBSOJOHͰ ͳΜ͔Ͱ͖ͳ͍͔ͳʔ

Slide 62

Slide 62 text

%FFQ-FBSOJOH ͱ͍͑͹ը૾ղੳ

Slide 63

Slide 63 text

͜Ε·Ͱͷ෺݅αΠτʹ ͸ͳ͍ըظతػೳʂ

Slide 64

Slide 64 text

࿨ࣜτΠϨ൑ఆث

Slide 65

Slide 65 text

࿨ࣜτΠϨ൑ఆث ࿨ࣜτΠϨ͚ͩ͸ઈରʹݏͩʂ ͱݴ͏ਓ͸͖ͬͱ͍Δʹ͕͍ͪͳ͍

Slide 66

Slide 66 text

H2O Deeplearning library(h2o) # Deep learningͰֶशͤ͞Δ localH2O <- h2o.init(ip = "localhost", port = 54321, startH2O = TRUE, nthreads=-1) res.dl <- h2o.deeplearning(x = 2:10001, y = 1, training_frame = as.h2o(target), activation = "TanhWithDropout", hidden=rep(160,5), epochs = 20) pred.dl <- h2o.predict(object=res.dl, newdata = as.h2o(target)) pred <- as.data.frame(pred.dl) # ਖ਼ղ཰Λ֬ೝ print(1-sum(abs(round(pred[,1]) - target[,1]))/length(target[,1])) ࿨ࣜτΠϨը૾ΛਓྗͰ݅ఔ౓ऩू ը૾αΠζΛἧ͑ͯɺάϨʔεέʔϧʹม׵ %FFQMFBSOJOHͰ࿨ࣜτΠϨ൑ఆثΛ࡞੒

Slide 67

Slide 67 text

H2O Deeplearning ਖ਼ղ཰ɿ ࿨ࣜτΠϨը૾ΛਓྗͰ݅ఔ౓ऩू ը૾αΠζΛἧ͑ͯɺάϨʔεέʔϧʹม׵ %FFQMFBSOJOHͰ࿨ࣜτΠϨ൑ఆثΛ࡞੒

Slide 68

Slide 68 text

ʢࠓ೔ͷ͓࿩ʣ ೖྗϛεσʔλ࡟আ ಛघ෺݅൑ఆ $73վળϑΟϧλ ࿨ࣜτΠϨ൑ఆث