Kybernetika 60 no. 4, 535-552, 2024

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

Youcef Elhamam Hemici, Samia Khelladi and Djamel BenterkiDOI: 10.14736/kyb-2024-4-0535

Abstract:

The conjugate gradient method is one of the most effective algorithm for unconstrained nonlinear optimization problems. This is due to the fact that it does not need a lot of storage memory and its simple structure properties, which motivate us to propose a new hybrid conjugate gradient method through a convex combination of $\beta _{k}^{RMIL}$ and $\beta _{k}^{HS}$. We compute the convex parameter $\theta _{k}$ using the Newton direction. Global convergence is established through the strong Wolfe conditions. Numerical experiments show the superior efficiency of our algorithm to solve unconstrained optimization problem compared to other considered methods. Applied to image restoration problem, our algorithm is competitive with existing algorithms and performs even better when the level of noise in the image is significant.

Keywords:

unconstrained optimization, image restoration, conjugate gradient method, descent direction, line search

Classification:

65K05, 90C26, 90C30

References:

  1. N. Andrei: An unconstrained optimization test functions collection. Adv. Model. Optim. 10 (2008), 147-161.   DOI:10.1002/adem.200890003
  2. N. Andrei: Nonlinear conjugate gradient methods for unconstrained optimization. Springer Optimization and its Applications, Romania 2020.   CrossRef
  3. S. Ben Hanachi, B. Sellami and M. Belloufi: New iterative conjugate gradient method for nonlinear unconstrained optimization. RAIRO - Oper. Res. 56 (2022), 2315-2327.   DOI:10.1051/ro/2022109
  4. J. F. Cai, R. Chan and B. Morini: Minimization of an edge-preserving regularization functional by conjugate gradient type methods. Image Processing Based on Partial Differential Equations. Mathematics and Visualization. Springer, Berlin, Heidelberg 2007.   CrossRef
  5. R. H. Chan, C. W. Ho and M. Nikolova: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14 (2005), 10, 1479-1485.   DOI:10.1109/TIP.2005.852196
  6. Y. H. Dai and Y. Yuan: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10 (1999), 1, 177-182.   DOI:10.1137/S1052623497318992
  7. Y. H. Dai and Y. Yuan: An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103 (2001), 33-47.   CrossRef
  8. S. Delladji, M. Belloufi and B. Sellami: New hybrid conjugate gradient method as a convex combination of FR and BA methods. J. Inform. Optim. Sci. 42 (2021), 3, 591-602.   CrossRef
  9. S. Djordjevic: New hybrid conjugate gradient method as a convex combination of LS and FR methods. Acta Math. Scientia 39B (2019), 1, 214-228.   DOI:10.1007/s10473-019-0117-6
  10. S. Djordjevic: New hybrid conjugate gradient method as a convex combination of HS and FR methods. J. Appl. Math. Comput. 2 (2018), 9, 366-378.   CrossRef
  11. E. D. Dolan and J. J. Moré: Benchmarking optimization software with performance profiles. Math. Program. 91 (2002), 201-213.   DOI:10.1007/s101070100263
  12. R. Fletcher: Practical Methods of Optimization. Unconstrained Optimization. Wiley, New York 1987.   CrossRef
  13. R. Fletcher and C. M. Reeves: Function minimization by conjugate gradients. Comput. J. 7 (1964), 2, 149-154.   DOI:10.1093/comjnl/7.2.149
  14. W. W. Hager and H. Zhang: A survey of nonlinear conjugate gradient methods. Pacific J. Optim. 2 (2006), 35-58.   CrossRef
  15. M. R. Hestenes and E. Steifel: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49 (1952), 6, 409-436.   DOI:10.6028/jres.049.044
  16. X. Jiang, W. Liao, J. Yin and J. Jian: A new family of hybrid three-term conjugate gradient methods with applications in image restoration. Numer. Algor. 91 (2022), 161-191.   DOI:10.1007/s11075-022-01258-2
  17. Y. Liu and C. Storey: Efficient generalized conjugate gradient algorithm. Part 1: Theory. J. Optim. Theory Appl. 69 (1991), 1, 129-137.   DOI:10.1007/BF00940464
  18. G. Ma, H. Lin, W. Jin and D. Han: Two modified conjugate gradient methods for unconstrained optimization with applications in image restoration problems. J. Appl. Math. Comput. 68 (2022), 4733-4758.   DOI:10.1007/s12190-022-01725-y
  19. M. Malik, I. M. Sulaiman, A. B. Abubakar, G. Ardaneswari and A. Sukono: A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection. AIMS Math. 8 (2023), 1-28.   DOI:10.1155/2023/8851478
  20. P. Mtagulwa and P. Kaelo: A convergent modified HS-DY hybrid conjugate gradient method for unconstrained optimization problems. J. Inform. Optim. Sci. 40 (2019), 1, 97-113.   CrossRef
  21. E. Polak and G. Ribiere: Note sur la convergence des mé thodes de directions conjuguées. Rev. Française Inform. Recherche Opertionelle 16 (1969), 35-43.   CrossRef
  22. B. T. Polyak: The conjugate gradient method in extreme problems. U.S.S.R. Comput. Math. Phys. 9 (1969), 94-112.   CrossRef
  23. M. J. D. Powell: Restart procedures of the conjugate gradient method. Math. Program. 2 (1977), 241-254.   CrossRef
  24. M. Rivaie, M. Mustafa, W. J. Leong and M. Ismail: A new class of nonlinear conjugate gradient coefficients with global convergence properties. Appl. Math. Comput. 218 (2012), 22, 11323-11332.   DOI:10.1016/j.amc.2012.05.030
  25. M. Rivaie, M. Mustafa and A. Abdelrhaman: A new class of non linear conjugate gradient coefficients with exact and inexact line searches. Appl. Math. Comput. 268 (2015), 1152-1163.   DOI:10.1016/j.amc.2015.07.019
  26. B. Sellami and Y. Chaib: A new family of globally convergent conjugate gradient methods. Ann. Oper. Res. Springer 241 (2016), 497-513.   DOI:10.1007/s10479-016-2120-9
  27. B. Sellami and Y. Chaib: New conjugate gradient method for unconstrained optimization. RAIRO Oper. Res. 50 (2016), 1013-1026.   DOI:10.1051/ro/2015064
  28. X. Yang, Z. Luo and X. Dai: A global convergence of LS-CD hybrid conjugate gradient method. Adv. Numer. Anal. 2013 (2013), 5 pp.   CrossRef
  29. G. Zoutendijk: Nonlinear programming, computational methods. In: Integer and Nonlinear Programming (J. Abadie, ed.), 1970, pp. 37-86.   CrossRef