Replicating Japanese CPI Using Scanner Data

https://stats.unece.org/ottawagroup/meeting/14
https://www.stat.go.jp/english/info/meetings/og2015/01agenda.html

Transcript

1. Satoshi Imai Tsutomu Watanabe Statistics Bureau of Japan University of Tokyo May 20, 2015
2. Scanner data based price index may be inconsistent with CPI • Offer prices may have different time series properties from transaction prices. – Price tag information collected by price collectors may not be the same as information contained in scanner data, which is based on actual transaction prices. – As shown by the literature on asset pricing in financial markets, quoted prices, such as prices quoted by stock market dealers, have different time series properties than transaction prices in terms of volatility, serial correlation, and so on. – In Japan, price collectors are instructed by the statistics bureau to ask its regular price when the target product is on sale. On the other hand, one can apply various filtering techniques to scanner data to estimate regular prices. However, it is likely that regular prices estimated this way may differ from regular prices obtained by price collectors • Differences in sampling procedures may yield different time series properties – Scanner data provides information on the number of customer visits for outlets and the quantities sold for products, which can be used when conducting outlet and product sampling. However, there is no guarantee that the set of outlets and the set of products chosen this way coincides with the one chosen based on the current procedure. 2
4. Items and weights in the CPI data 4
5. Methodology  Sampling – Outlet sampling – Product sampling – Price sampling • JSB price collectors collect prices on either Wednesday, Thursday, or Friday of the week which includes the 12th of the month. • JSB price collectors exclude “extra-low prices due to bargain, clearance, or discount sales, and quoted for less than eight days”  Quality adjustment – Quantity ratio method – Imputation method  Aggregation – Unweighted arithmetic mean of individual prices (i.e., Dutot index) for the lower level aggregation – Fixed base Laspeyres weighting for the upper level aggregation 5
6. 6 Number of Products that Meet the JSB Product Type Specifications ….. JSB Product Type Specifications for Butter Jul 1996 - Jan 2001 “Snow Brand Hokkaido Butter” Jan 2001 – present 200g. Packed in a paper container. Excluding unsalted butter.
7. Outlet and Product Sampling 7 : Number of customer visits to outlet i, which is included in the sample : Number of customer visits to outlet j, which is not included in the sample : Parameter governing the frequency of outlet replacements : Number of quantities sold for product i, which is included in the sample : Number of quantities sold for product j, which is not included in the sample : Parameter governing the frequency of product replacements
8. Replication using the CPI source data 95 96 97 98 99 100 101 102 Jan-2010 Apr-2010 Jul-2010 Oct-2010 Jan-2011 Apr-2011 Jul-2011 Oct-2011 Jan-2012 Apr-2012 Jul-2012 Oct-2012 Jan-2013 Apr-2013 Jul-2013 Oct-2013 Jan-2014 Apr-2014 Jul-2014 Price Level Actual Replicated using CPI source data -0.03 -0.02 -0.01 0.00 0.01 0.02 0.03 0.04 Jan-2011 Apr-2011 Jul-2011 Oct-2011 Jan-2012 Apr-2012 Jul-2012 Oct-2012 Jan-2013 Apr-2013 Jul-2013 Oct-2013 Jan-2014 Apr-2014 Jul-2014 Y/Y Inflation Actual Replicated using CPI source data 8
9. Replication using the scanner data 9 90 95 100 105 110 115 120 Jan-00 Jul-00 Jan-01 Jul-01 Jan-02 Jul-02 Jan-03 Jul-03 Jan-04 Jul-04 Jan-05 Jul-05 Jan-06 Jul-06 Jan-07 Jul-07 Jan-08 Jul-08 Jan-09 Jul-09 Jan-10 Jul-10 Jan-11 Jul-11 Jan-12 Jul-12 Jan-13 Jul-13 Jan-14 Jul-14 Price Level k_S=1; k_P=1 Actual -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 Jan-01 Jul-01 Jan-02 Jul-02 Jan-03 Jul-03 Jan-04 Jul-04 Jan-05 Jul-05 Jan-06 Jul-06 Jan-07 Jul-07 Jan-08 Jul-08 Jan-09 Jul-09 Jan-10 Jul-10 Jan-11 Jul-11 Jan-12 Jul-12 Jan-13 Jul-13 Jan-14 Jul-14 Y/Y Inflation k_S=1; k_P=1 Actual
10. Root Mean Square Error for the Difference between Scanner Data Based Inflation and CPI Inflation 10 1 2 3 4 5 0.0075 0.0080 0.0085 0.0090 0.0095 0.0100 0.0105 1 2 3 4 5 6 7 8 9 10
11. Replication with Inertia in Outlet and Product Replacement 11 90 95 100 105 110 115 120 Jan-00 Jul-00 Jan-01 Jul-01 Jan-02 Jul-02 Jan-03 Jul-03 Jan-04 Jul-04 Jan-05 Jul-05 Jan-06 Jul-06 Jan-07 Jul-07 Jan-08 Jul-08 Jan-09 Jul-09 Jan-10 Jul-10 Jan-11 Jul-11 Jan-12 Jul-12 Jan-13 Jul-13 Jan-14 Jul-14 Price Level k_S=5; k_P=10 Actual -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 Jan-01 Jul-01 Jan-02 Jul-02 Jan-03 Jul-03 Jan-04 Jul-04 Jan-05 Jul-05 Jan-06 Jul-06 Jan-07 Jul-07 Jan-08 Jul-08 Jan-09 Jul-09 Jan-10 Jul-10 Jan-11 Jul-11 Jan-12 Jul-12 Jan-13 Jul-13 Jan-14 Jul-14 Y/Y Inflation k_S=5; k_P=10 Actual
12. Mean and SD of Monthly Inflation By Item 12 -0.01 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 1042 1051 1071 1321 1333 1602 1621 1633 1641 1642 1643 1652 1654 1655 1656 1714 1721 1732 1761 1784 1871 1911 1921 1922 1931 1941 1951 2003 2021 4401 4412 4431 4441 4442 4451 4461 6101 6141 9124 9611 9621 9622 9623 9631 9641 9661 Item Code JSB Mean JSB SD POS Mean POS SD
13. Decomposition of Monthly Inflation into Extensive and Intensive Margins 13 Month on month Inflation for item c in month t Fraction of products in item c that experience price changes in month t Average size of price changes for those products in item c that experience price changes in month t Intensive margin Extensive margin
14. 14 0.00076 0.00001 0.00071 0.0000 0.0005 0.0010 0.0015 0.0020 var (inf) EM IM JSB 0.00170 0.00001 0.00142 0.0000 0.0005 0.0010 0.0015 0.0020 var (inf) EM IM POS 0.00094 0.00001 0.00071 0.0000 0.0005 0.0010 0.0015 0.0020 var (inf) EM IM Difference Intensive margin Extensive margin Diff in IM 0.00071 Due to Var (dP) 0.00002 Due to E (Fr)^2 0.00068
15. 15 0.0 0.2 0.4 0.6 0.8 1.0 Pr (same outlet) Pr (same product|same outlet) Pr (same price|same product, same outlet) Pr (same price, same product, same outlet) JSB POS
16. Decomposition of Inflation Volatility into Extensive and Intensive Margins 16 1.E-09 1.E-08 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 1.E-02 1042 1051 1071 1321 1333 1602 1621 1633 1641 1642 1643 1652 1654 1655 1656 1714 1721 1732 1761 1784 1871 1911 1921 1922 1931 1941 1951 2003 2021 4401 4412 4431 4441 4442 4451 4461 6101 6141 9124 9611 9621 9622 9623 9631 9641 9661 JSB Inflation Volatility Extensive Margin Intensive Margin 1.E-09 1.E-08 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 1.E-02 1042 1051 1071 1321 1333 1602 1621 1633 1641 1642 1643 1652 1654 1655 1656 1714 1721 1732 1761 1784 1871 1911 1921 1922 1931 1941 1951 2003 2021 4401 4412 4431 4441 4442 4451 4461 6101 6141 9124 9611 9621 9622 9623 9631 9641 9661 POS Inflation Volatility Extensive Margin Intensive Margin
17. Probability of No Price Adjustments 17 0.0 0.2 0.4 0.6 0.8 1.0 1042 1051 1071 1321 1333 1602 1621 1633 1641 1642 1643 1652 1654 1655 1656 1714 1721 1732 1761 1784 1871 1911 1921 1922 1931 1941 1951 2003 2021 4401 4412 4431 4441 4442 4451 4461 6101 6141 9124 9611 9621 9622 9623 9631 9641 9661 Pr(same outlet, same product, same price) JSB POS 1-1 POS 5-10 0.0 0.2 0.4 0.6 0.8 1.0 1042 1051 1071 1321 1333 1602 1621 1633 1641 1642 1643 1652 1654 1655 1656 1714 1721 1732 1761 1784 1871 1911 1921 1922 1931 1941 1951 2003 2021 4401 4412 4431 4441 4442 4451 4461 6101 6141 9124 9611 9621 9622 9623 9631 9641 9661 Pr(same price|same product, same outlet) JSB POS 1-1 POS 5-10 0.0 0.2 0.4 0.6 0.8 1.0 1042 1051 1071 1321 1333 1602 1621 1633 1641 1642 1643 1652 1654 1655 1656 1714 1721 1732 1761 1784 1871 1911 1921 1922 1931 1941 1951 2003 2021 4401 4412 4431 4441 4442 4451 4461 6101 6141 9124 9611 9621 9622 9623 9631 9641 9661 Pr(same product|same outlet) JSB POS 1-1 POS 5-10 0.0 0.2 0.4 0.6 0.8 1.0 1042 1051 1071 1321 1333 1602 1621 1633 1641 1642 1643 1652 1654 1655 1656 1714 1721 1732 1761 1784 1871 1911 1921 1922 1931 1941 1951 2003 2021 4401 4412 4431 4441 4442 4451 4461 6101 6141 9124 9611 9621 9622 9623 9631 9641 9661 Pr(same outlet) JSB POS 1-1 POS 5-10
18. 18 Distributions of price changes for individual products, dP 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 JSB POS 0.001 0.01 0.1 1 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 JSB POS
19. 0.001 0.01 0.1 1 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 2010 2011 2012 2013 2014 JSB Price Change Distributions by Year 19
20. Main findings of the paper  Scanner data based price index depends crucially on how often product substitution occurs. The deviation of scanner data based price index from actual CPI is not negligible when we choose parameter values for product substitution such that substitutions occur very frequently, while it becomes smaller as we pick parameter values such that substitutions occur only infrequently.  Scanner data based inflation differs significantly from actual CPI inflation in terms of volatility. We decompose the difference in the variance of monthly inflation into various factors to find that the difference in inflation volatility mainly stems from the difference in the frequency of price adjustments for individual products. Actual CPI inflation is less volatile since individual prices in the CPI data are stickier.  Small-sized price changes are less likely to occur in the CPI data than in the scanner data. Together with the fact that prices are stickier in the CPI data, this suggests that menu costs play a more important role in the CPI data. 20