no existence of conjugate prior. - [Chib 1993] demonstrated an auxiliary variable approach. In the conjugate priors are available to the conditional likelihood. - In this paper, three extensions are introduced. - To improve performance im probit regression simulation by joint updating. - Auxiliary approach for logistic regression. - Logistic models for multinomial response data. 2
regression model, - y i ∈ {0, 1}, i = 1, ..., n. - g(y) : a link funcion - η i : a linear predictor - x i = {x i1 , ..., x ip }, p covariate measurements - β : (p x 1) column vector, regression coeﬃcients - π(.) : a prior 3
= Φ(u), the cumulative distribution function of a standard normal distribution. 4 the stochastic auxiliary variable There is strong posterior correlation between β and z. This correlation is likely to cause slow mixing in the chain.
and z are updated jointly by the factorization. - Assume the prior, β ~ N(0, v) . - Ind(y, z) is an indicator function which truncates the multivariate normal distribution of z to the appropriate region. 6
multivariate truncated normal is known to be diﬃcult.However it's straightforward to Gibbs sample the distribution, - z -i : z with the i-th variable removed. - [Henderson, et.al, 1981] 7 S i denotes the i-th column of S = Vx' We can construct a Gibbs sampler from these expressions.
i ~ π(ε i ) to be the standard logistic distribution, then we obtain the logistic regression model. - As it stands we lose the conditional conjugacy for updating β . 8 ε i ~ Logistic-Distribution
a further set of variables, - KS : Kolmogorov-Smirnov distribution [Devroye 1986] . - ε i has a scale mixture of normal form with a marginal logistic distribution. - Sampling scheme : (β|z,λ) -> (z|β,λ) -> (λ|z,β) . - π(λ i | z i , β) does not have a standard form. See Appendix A4. 9
in joint updating. 1. π(z,β|y,λ) = π(z|y,λ) π(β|z,λ) or 2. π(z,λ|β,y) = π(z|β,y) π(λ|z,β) - In the latter case, the marginal densities for the z i 's are independent truncated logistic distributions. 10
regression model is deﬁned via, - M(.) : the single sample multinomial distribution. - β j : a separate set of coeﬃcients for each category. - It's usual to ﬁx one set of coeﬃcient, β Q to be zero. 11
likelihood of a set of coeﬃcients β j . - β -j = {β 1 , ..., β j-1 , β j+1 , ..., β Q } 12 The point here is that the L(β j |y,β -j ) has the form of a logistic regression on class indicator I(y i =j). This allows us to use the logistic sampling technique looping over the Q-1 classes. See : Appendix A5 and A3.