Mirror descent

In mathematics mirror descent descent is an iterative optimization algorithm for finding a local minimum of a differentiable function.

It generalizes algorithms such as gradient descent and multiplicative weights.

History

Mirror descent was originally proposed by Nemirovski and Yudin in 1983.

Motivation

In gradient descent applied to a differentiable function $F$, one starts with a guess $\mathbf {x} _{0}$ for a local minimum of $F$, and considers the sequence $\mathbf {x} _{0},\mathbf {x} _{1},\mathbf {x} _{2},\ldots$ such that

$\mathbf {x} _{n+1}=\mathbf {x} _{n}-\gamma \nabla F(\mathbf {x} _{n}),\ n\geq 0.$

(assuming a constant learning rate)

We have a monotonic sequence

$F(\mathbf {x} _{0})\geq F(\mathbf {x} _{1})\geq F(\mathbf {x} _{2})\geq \cdots ,$

so, hopefully, the sequence $(\mathbf {x} _{n})$ converges to the desired local minimum.

This can be reformulated by noting that

$\mathbf {x} _{n+1}=\arg \min _{\mathbf {x} }\left(F(\mathbf {x} _{n})+\nabla F(\mathbf {x} _{n})^{T}(\mathbf {x} -\mathbf {x} _{n})+{\frac {1}{2\gamma }}\|\mathbf {x} -\mathbf {x} _{n}\|^{2}\right)$

In other words, $\mathbf {x} _{n+1}$ minimizes the first-order approximation to $F$ at $\mathbf {x} _{n}$ with added proximity term $\|\mathbf {x} -\mathbf {x} _{n}\|^{2}$.

This Euclidean distance term is a particular example of a Bregman distance. Using other Bregman distances will yield other algorithms such as Hedge which may be more suited to optimization over particular geometries.

Formulation

We are given convex function $f$ to optimize over convex set $K\subset \mathbb {R} ^{n}$, and given some norm $\|\cdot \|$ on $\mathbb {R} ^{n}$.

We are also given differentiable convex function $h\colon \mathbb {R} ^{n}\to \mathbb {R}$, $\alpha$-strongly convex with respect to the given norm. This is called the distance-generating function, and its gradient $\nabla h\colon \mathbb {R} ^{n}\to \mathbb {R} ^{n}$ is known as the mirror map.

Starting from initial $x_{0}\in K$, in each iteration of Mirror Descent:

• Map to the dual space: $\theta _{t}\leftarrow \nabla h(x_{t})$
• Update in the dual space using a gradient step: $\theta _{t+1}\leftarrow \theta _{t}-\eta _{t}\nabla f_{t}(x_{t})$
• Map pack to the primal space: $x'_{t+1}\leftarrow (\nabla h)^{-1}(\theta _{t+1})$
• Project back to the feasible region $K$: $x_{t+1}\leftarrow \min _{x\in K}D_{h}(x||x'_{t+1})$, where $D_{h}$ is the Bregman divergence

Extensions

Mirror descent in the online optimization setting is known as Online Mirror Descent (OMD).

See algo

• Gradient descent
• Multiplicative weight update method
• Hedge algorithm
• Bregman divergence

External links

• [1]
• [2]
• [3]