Saddle Points and Accelerated Perceptron Algorithms

Adams Wei Yu, Fatma Kilinc-Karzan, Jaime Carbonell
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1827-1835, 2014.

Abstract

In this paper, we consider the problem of finding a linear (binary) classifier or providing a near-infeasibility certificate if there is none. We bring a new perspective to addressing these two problems simultaneously in a single efficient process, by investigating a related Bilinear Saddle Point Problem (BSPP). More specifically, we show that a BSPP-based approach provides either a linear classifier or an ε-infeasibility certificate. We show that the accelerated primal-dual algorithm, Mirror Prox, can be used for this purpose and achieves the best known convergence rate of O(\sqrt\log n\overρ(A)) (O(\sqrt\log n\overε)), which is \emphalmost independent of the problem size, n. Our framework also solves kernelized and conic versions of the problem, with the same rate of convergence. We support our theoretical findings with an empirical study on synthetic and real data, highlighting the efficiency and numerical stability of our algorithms, especially on large-scale instances.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-yuc14, title = {Saddle Points and Accelerated Perceptron Algorithms}, author = {Yu, Adams Wei and Kilinc-Karzan, Fatma and Carbonell, Jaime}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1827--1835}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/yuc14.pdf}, url = {https://proceedings.mlr.press/v32/yuc14.html}, abstract = {In this paper, we consider the problem of finding a linear (binary) classifier or providing a near-infeasibility certificate if there is none. We bring a new perspective to addressing these two problems simultaneously in a single efficient process, by investigating a related Bilinear Saddle Point Problem (BSPP). More specifically, we show that a BSPP-based approach provides either a linear classifier or an ε-infeasibility certificate. We show that the accelerated primal-dual algorithm, Mirror Prox, can be used for this purpose and achieves the best known convergence rate of O(\sqrt\log n\overρ(A)) (O(\sqrt\log n\overε)), which is \emphalmost independent of the problem size, n. Our framework also solves kernelized and conic versions of the problem, with the same rate of convergence. We support our theoretical findings with an empirical study on synthetic and real data, highlighting the efficiency and numerical stability of our algorithms, especially on large-scale instances.} }
Endnote
%0 Conference Paper %T Saddle Points and Accelerated Perceptron Algorithms %A Adams Wei Yu %A Fatma Kilinc-Karzan %A Jaime Carbonell %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-yuc14 %I PMLR %P 1827--1835 %U https://proceedings.mlr.press/v32/yuc14.html %V 32 %N 2 %X In this paper, we consider the problem of finding a linear (binary) classifier or providing a near-infeasibility certificate if there is none. We bring a new perspective to addressing these two problems simultaneously in a single efficient process, by investigating a related Bilinear Saddle Point Problem (BSPP). More specifically, we show that a BSPP-based approach provides either a linear classifier or an ε-infeasibility certificate. We show that the accelerated primal-dual algorithm, Mirror Prox, can be used for this purpose and achieves the best known convergence rate of O(\sqrt\log n\overρ(A)) (O(\sqrt\log n\overε)), which is \emphalmost independent of the problem size, n. Our framework also solves kernelized and conic versions of the problem, with the same rate of convergence. We support our theoretical findings with an empirical study on synthetic and real data, highlighting the efficiency and numerical stability of our algorithms, especially on large-scale instances.
RIS
TY - CPAPER TI - Saddle Points and Accelerated Perceptron Algorithms AU - Adams Wei Yu AU - Fatma Kilinc-Karzan AU - Jaime Carbonell BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-yuc14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1827 EP - 1835 L1 - http://proceedings.mlr.press/v32/yuc14.pdf UR - https://proceedings.mlr.press/v32/yuc14.html AB - In this paper, we consider the problem of finding a linear (binary) classifier or providing a near-infeasibility certificate if there is none. We bring a new perspective to addressing these two problems simultaneously in a single efficient process, by investigating a related Bilinear Saddle Point Problem (BSPP). More specifically, we show that a BSPP-based approach provides either a linear classifier or an ε-infeasibility certificate. We show that the accelerated primal-dual algorithm, Mirror Prox, can be used for this purpose and achieves the best known convergence rate of O(\sqrt\log n\overρ(A)) (O(\sqrt\log n\overε)), which is \emphalmost independent of the problem size, n. Our framework also solves kernelized and conic versions of the problem, with the same rate of convergence. We support our theoretical findings with an empirical study on synthetic and real data, highlighting the efficiency and numerical stability of our algorithms, especially on large-scale instances. ER -
APA
Yu, A.W., Kilinc-Karzan, F. & Carbonell, J.. (2014). Saddle Points and Accelerated Perceptron Algorithms. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1827-1835 Available from https://proceedings.mlr.press/v32/yuc14.html.

Related Material