This paper studies the impact of decentralization on the shadow economy. We argue that decentralization may decrease the size of the shadow economy mainly through two transmission channels: (1) Decentralization enhancing public sector efficiency (efficiency effect), and (2) decentralization reducing the distance between bureaucrats and economic agents, which increases the probability of detection of shadow economic activities (deterrence effect). Using various measures of fiscal, political and government employment decentralization in a cross-section of countries, we find the deterrence effect to be of more importance. The deterrence effect is stronger, the lower the degree of institutional quality. Remarkably, we find no robust evidence of the efficiency effect.