We solve the optimal asset allocation problem using a mean variance approach. The original mean variance optimization problem can be embedded into a class of auxiliary stochastic linear-quadratic (LQ) problems using the method in Zhou and Li (2000) and Li and Ng (2000). We use a finite difference method with fully implicit timestepping to solve the resulting nonlinear Hamilton-Jacobi-Bellman (HJB) PDE, and present the solutions in terms of an efficient frontier and an optimal asset allocation strategy. The numerical scheme satisfies sufficient conditions to ensure convergence to the viscosity solution of the HJB PDE. We handle various constraints on the optimal policy. Numerical tests indicate that realistic constraints can have a dramatic effect on the optimal policy compared to the unconstrained solution.