Curvature-Exploiting Acceleration of Elastic Net Computations

Vien Mai, Mikael Johansson
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:4294-4303, 2019.

Abstract

This paper introduces an efficient second-order method for solving the elastic net problem. Its key innovation is a computationally efficient technique for injecting curvature information in the optimization process which admits a strong theoretical performance guarantee. In particular, we show improved run time over popular first-order methods and quantify the speed-up in terms of statistical measures of the data matrix. The improved time complexity is the result of an extensive exploitation of the problem structure and a careful combination of second-order information, variance reduction techniques, and momentum acceleration. Beside theoretical speed-up, experimental results demonstrate great practical performance benefits of curvature information, especially for ill-conditioned data sets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-mai19a, title = {Curvature-Exploiting Acceleration of Elastic Net Computations}, author = {Mai, Vien and Johansson, Mikael}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {4294--4303}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/mai19a/mai19a.pdf}, url = {https://proceedings.mlr.press/v97/mai19a.html}, abstract = {This paper introduces an efficient second-order method for solving the elastic net problem. Its key innovation is a computationally efficient technique for injecting curvature information in the optimization process which admits a strong theoretical performance guarantee. In particular, we show improved run time over popular first-order methods and quantify the speed-up in terms of statistical measures of the data matrix. The improved time complexity is the result of an extensive exploitation of the problem structure and a careful combination of second-order information, variance reduction techniques, and momentum acceleration. Beside theoretical speed-up, experimental results demonstrate great practical performance benefits of curvature information, especially for ill-conditioned data sets.} }
Endnote
%0 Conference Paper %T Curvature-Exploiting Acceleration of Elastic Net Computations %A Vien Mai %A Mikael Johansson %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-mai19a %I PMLR %P 4294--4303 %U https://proceedings.mlr.press/v97/mai19a.html %V 97 %X This paper introduces an efficient second-order method for solving the elastic net problem. Its key innovation is a computationally efficient technique for injecting curvature information in the optimization process which admits a strong theoretical performance guarantee. In particular, we show improved run time over popular first-order methods and quantify the speed-up in terms of statistical measures of the data matrix. The improved time complexity is the result of an extensive exploitation of the problem structure and a careful combination of second-order information, variance reduction techniques, and momentum acceleration. Beside theoretical speed-up, experimental results demonstrate great practical performance benefits of curvature information, especially for ill-conditioned data sets.
APA
Mai, V. & Johansson, M.. (2019). Curvature-Exploiting Acceleration of Elastic Net Computations. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:4294-4303 Available from https://proceedings.mlr.press/v97/mai19a.html.

Related Material