Abstract |
Nonlinear least squares (NLS) problems arise in many applications. The common solvers
require to compute and store the corresponding Jacobian matrix explicitly, which is too expensive
for large problems. In this paper, we propose an effective Jacobian free method especially for
large NLS problems because of the novel combination of using automatic differentiation for J(x)v
and JT (x)v along with the preconditioning ideas that do not require forming the Jacobian matrix
J(x) explicitly. Together they yield a new and effective three-level iterative approach. In the
outer level, the dogleg/trust region method is employed to solve the NLS problem. At each
iteration of the dogleg method, we adopt the iterative linear least squares (LLS) solvers, CGLS
or BA-GMRES method, to solve the LLS problem generated at each step of the dogleg method
as the middle iteration. In order to accelerate the convergence of the iterative LLS solver, we
propose an inner iteration preconditioner based on the weighted Jacobi method. Compared to the
common dogleg solver and truncated Newton method, our proposed three level method need not
compute the gradient or Jacobian matrix explicitly, and is efficient in computational complexity
and memory storage. Furthermore, our method does not rely on the sparsity or structure pattern
of the Jacobian, gradient or Hessian matrix. Thus, it can be applied to solve any large general
NLS problem. Numerical experiments show that our proposed method is much superior to the
common trust region method and truncated Newton method. |