Docstrings

AbstractHessianApprox

A quasi-Newton Hessian approximation with a possibly custom structure and custom linear solves. See the module documentation for the extension API.

AugLagAlg(sub_alg = nothing)

Augmented Lagrangian meta-algorithm. sub_alg solves the subproblems after the relaxed constraints are folded into the objective; nothing selects the built-in QuasiNewtonAlg, which requires every constraint to be relaxed. Relaxing semidefinite constraints additionally requires NonconvexSemidefinite to be loaded (its extension provides the cone projections).

AugLagOptions(; kwargs...)

• sub_options: options for the sub-solver (or the built-in QuasiNewtonAlg).
• relaxed: a RelaxConstraintSelection (default: relax everything).
• penalty: :quadratic (classical ALM), :pbm (penalty-barrier-multiplier for inequalities, quadratic for equalities) or :mixed (PBM for inequalities, projected quadratic for semidefinite).
• ρ_init, ρ_max, ρ_incr, ρ_decr: initial/maximum penalty and the increase/decrease factors (two-way adaptation).
• max_iter: maximum outer iterations.
• ctol: feasibility tolerance.
• η₀, ω₀: initial feasibility/optimality targets of the subproblems.
• weights: per-constraint-group penalty weights (default: all ones).
• lsq_multipliers: initialize equality multipliers from the least-squares stationarity residual.
• keep_all: keep the trajectory of outer iterates in the result.

AugLagResult

The result of an AugLagAlg optimization: the GenericResult-style fields minimizer and minimum, the (currently nothing) sub-solver results, and trajectory — the vector of outer-loop iterates (x, f(x), ‖v(x)‖∞) when AugLagOptions(keep_all = true), nothing otherwise.

DenseBFGSApprox(form = :inverse)

Dense BFGS. With form = :inverse (default) the inverse Hessian approximation is stored and updated, so solve! is a plain matrix-vector product; with form = :hessian the direct Hessian is stored and solve! factorizes it.

DiagonalPreconditioner(d)

Diagonal Hessian scaling with entries d (all positive). solve! divides by the scaling, apply! multiplies by it.

IdentityPreconditioner()

Identity Hessian scaling: apply!/solve! copy their argument.

LBFGSApprox(memory; H0 = IdentityPreconditioner())

Limited-memory BFGS. Stores the last memory secant pairs and applies the approximation (and its inverse) with the two-loop recursion; H0 is the base operator used at the bottom of the recursion and serves as the preconditioner.

NewtonTRApprox()

Exact Hessian of the (augmented Lagrangian) objective at every iterate, differentiated through NonconvexCore.value_gradient. Used together with QuasiNewtonAlg(; linesearch = :trustregion) for the Newton trust-region method (Steihaug–Toint CG).

QuasiNewtonAlg(approx = LBFGSApprox(10); linesearch = :backtracking,
               max_iter = 200, tol = 1e-6, damping = true, Δ0 = 1.0)

A smooth box-constrained quasi-Newton optimizer: approx is any AbstractHessianApprox (L-BFGS with an optional preconditioner, dense or structured BFGS, or the exact-Hessian Newton trust region). linesearch is :backtracking (Armijo), :wolfe (strong Wolfe) or :trustregion (Steihaug–Toint with a projected step). damping enables Powell damping of the secant pairs for nonconvex objectives.

RelaxConstraintSelection(; eq = true, ineq = true, sd = true)

Selects which constraints are relaxed into the augmented Lagrangian. Each entry is true (relax all of that kind), false (keep all hard) or a vector of constraint indices (relax those only). Box constraints are always handled by the sub-solver.

StructuredBFGSApprox(B; form = :hessian)

BFGS with secants accumulated in a user-defined Hessian container B (e.g. a sparse matrix carrying a sparsity pattern, or any custom matrix type). The built-in update is the structure-preserving sparse BFGS (Fletcher): the dense BFGS update is restricted to the nonzero pattern of B, so the structure and the user's solve!/apply! implementations keep working. Custom matrix types should overload init!/update!/solve!/apply! directly.