In this paper, we study the size distortions of the KPSS test for stationarity when serial correlation is present and samples are small and medium-sized. It is argued that two distinct sources of the size distortions can be identified. The first source is the finite-sample distribution of the long-run variance estimator used in the KPSS test, while the second source of the size distortions is the serial correlation not captured by the long-run variance estimator due to a too narrow choice of truncation lag parameter. When the relative importance of the two sources is studied, it is found that the size of the KPSS test can be reasonably well controlled if the finite-sample distribution of the KPSS test statistic, conditional on the time-series dimension and the truncation lag parameter, is used. Hence, finite-sample critical values, that can be applied in order to reduce the size distortions of the KPSS test, are supplied. When the power of the test is studied, it is found that the price paid for the increased size control is a lower raw power against a non-stationary alternative hypothesis.