We study information matrices for statistical models by the $L^2$-Wasserstein metric. We call it the Wasserstein information matrix (WIM), which is an analog of the classical Fisher information matrix. Based on this matrix, we introduce Wasserstein score functions and study covariance operators in statistical models. Using them, we establish a Wasserstein-Cramer-Rao bound for estimation. Also, by a ratio of the Wasserstein and Fisher information matrices, we prove various functional inequalities within statistical models, including both the Log-Sobolev and Poincar{\'e} inequalities. These inequalities relate to a new efficiency property named Poincar{\'e} efficiency, introduced via a Wasserstein natural gradient of maximal likelihood estimation. Furthermore, online efficiency for Wasserstein natural gradient methods is also established. Several analytical examples and approximations of the WIM are presented, including location-scale families, independent families, and rectified linear unit (ReLU) generative models