Slide 18
Slide 18 text
Lasso ͷ EM ΞϧΰϦζϜ
Laplace Λ Gaussian scale mixture (GSM) Ͱදݱ͢Δ.
Lap(wj|0, 1/γ) =
γ
2
e−γ|wj | = N(wj|0, τ2
j
)Ga(τ2
j
|1,
γ2
2
)dτ2
j
͜ΕΛ༻͍Ε, ಉ࣌
p(y, w, τ, σ2|X) = N(y|Xw, σ2IN )N(w|0, Dτ )IG(σ2|aσ, bσ)
⎡
⎣
j
Ga(τ2
j
|1, γ2/2)
⎤
⎦
∝ (σ2)−N/2 exp −
1
2σ2
∥y − Xw∥2
2
|D−1/2
τ exp −
1
2
wT Dτ w (σ2)aσ+1
exp(−bσ/σ)
j
exp(−
γ2
2
τ2
j
)
ͨͩ͠,Dτ = diag(τ2
j
) Ͱ X ඪ४Խ,y centered ͞Ε͍ͯΔͷͰ offset ߲ແࢹՄೳ.
EM ΞϧΰϦζϜͰߟ͑Δ (Figueiredo, 2003)
E step: τ2
j
, σ2 Λਪఆ͢Δ
M step: w ʹؔͯ͠࠷దԽ͢Δ
࣮͜ͷ ˆ
w Lasso ਪఆྔͱಉ͡ʹͳΔ
Daisuke Yoneoka Sparse linear models October 20, 2014 18 / 21