We study a random matrix model for QCD at finite density via complex Langevin dynamics. This model has a phase transition to a phase with nonzero baryon density. We study the convergence of the algorithm as a function of the quark mass and the chemical potential and focus on two main observables: the baryon density and the chiral condensate. For simulations close to the chiral limit, the algorithm has wrong convergence properties when the quark mass is in the spectral domain of the Dirac operator. A possible solution of this problem is discussed.