Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs
Abstract
We consider the decentralized optimization problem, where a network of $n$ agents aims to collaboratively minimize the average of their individual smooth and convex objective functions through peer-to-peer communication in a directed graph. To tackle this problem, we propose two accelerated gradient tracking methods, namely APD and APD-SC, for non-strongly convex and strongly convex objective functions, respectively. We show that APD and APD-SC converge at the rates $O\left(\frac{1}{k^2}\right)$ and $O\left(\left(1 - C\sqrt{\frac{\mu}{L}}\right)^k\right)$, respectively, up to constant factors depending only on the mixing matrix. APD and APD-SC are the first decentralized methods over unbalanced directed graphs that achieve the same provable acceleration as centralized methods. Numerical experiments demonstrate the effectiveness of both methods.
- Publication:
-
arXiv e-prints
- Pub Date:
- July 2021
- DOI:
- 10.48550/arXiv.2107.12065
- arXiv:
- arXiv:2107.12065
- Bibcode:
- 2021arXiv210712065S
- Keywords:
-
- Mathematics - Optimization and Control;
- Computer Science - Distributed;
- Parallel;
- and Cluster Computing;
- Computer Science - Machine Learning;
- Electrical Engineering and Systems Science - Signal Processing;
- Electrical Engineering and Systems Science - Systems and Control
- E-Print:
- SIAM Journal on Optimization, in press