Treatment of Outliers in Business Surveys: The Case of Short-term Economic Survey of Enterprises in Japan (Tankan)

Atsushi Ishikawa, Shunsuke Endo and Tetsuya Shiratori
Additional contact information
Atsushi Ishikawa: Bank of Japan
Shunsuke Endo: Bank of Japan
Tetsuya Shiratori: Bank of Japan

No 10-E-8, Bank of Japan Working Paper Series from Bank of Japan

Abstract: This paper discusses the appropriate treatment of outliers in Tankan. Under the current population estimate method, if one of the data which greatly diverges from the rest of the data (the so-called "outlier") does not represent the population, the estimates could also greatly diverge from the real value for the population. Since the real picture of the population when the outlier occurred is unknown, it is difficult to choose the optimum method through empirical analyses. Considering 1) the accuracy of Tankan, 2) the maintenance of its "easy to understand" quality, and 3) the reduction of its work load, we assume that outliers in Tankan are those which have extremely large impact on the rate of change from the previous year or the rate of revision from the previous survey of the estimated values for the population. The method of detecting and treating outliers is suitable for the characteristics of Tankan through empirical analyses of actual Tankan data. Specifically, detection method is based on the adjusted "range edit" method, which uses 1 percentile and 99 percentile, so that it can be applied to non-parametric data, for which the incidence of "0" (zero) is very high. The outliers thus detected are regarded as missing values and should be augmented by applying the missing value imputation, which is already being used for Tankan.

Keywords: TANKAN; outlier; stratified sampling; weight; stratum jumpers (search for similar items in EconPapers)
Date: 2010-09
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.boj.or.jp/en/research/wps_rev/wps_2010/data/wp10e08.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:boj:bojwps:10-e-8

Access Statistics for this paper

More papers in Bank of Japan Working Paper Series from Bank of Japan Contact information at EDIRC.
Bibliographic data for series maintained by Bank of Japan ().