Construction of nonlinear time series models with a flexible probabilistic structure is an important challenge for statisticians. Applications of such a time series model include ecology, economics and finance. In this paper we consider a threshold model for all the first four conditional moments of a time series. The nonlinear structure in the conditional mean is specified by a threshold autoregression and that of the conditional variance by a threshold generalized autoregressive conditional heteroscedastic (GARCH) model. There are many options for the conditional innovation density in the modeling of the skewness and kurtosis such as the Gram-Charlier (GC) density and the skewed-t density. The Gram-Charlier (GC) density allows explicit modeling of the skewness and kurtosis parameters and therefore is the main focus of this paper. However, its performance is compared with that of Hansen's skewed-t distribution in the data analysis section of the paper. The regime-dependent feature for the first four conditional moments allows more flexibility in modeling and provides better insights into the structure of a time series. A Lagrange multiplier (LM) test is developed for testing for the presence of threshold structure. The test statistic is similar to the classical tests for the presence of a threshold structure but allowing for a more general regime-dependent structure. The new model and the LM test are illustrated using the Dow Jones Industrial Average, the Hong Kong Hang Seng Index and the Yen/US exchange rate.