We introduce a new 5-parameter family of distributions, the Asymmetric Exponential Power (AEP), able to cope with asymmetries and leptokurtosis and at the same time allowing for a continuous variation from non-normality to normality. We prove that the Maximum Likelihood (ML) estimates of the AEP parameters are consistent on the whole parameter space, and when sufficiently large values of the shape parameters are considered, they are also asymptotically efficient and normal. We derive the Fisher information matrix for the AEP and we show that it can be continuously extended also to the region of small shape parameters. Through numerical simulations, we find that this extension can be used to obtain a reliable value for the errors associated to ML estimates also for samples of relatively small size ( 100 observations). Moreover we find that at this sample size, the bias associated with ML estimates, although present, becomes negligible.