A Heuristic Fast Gradient Descent Method for Unimodal Optimization

The known gradient descent optimization methods applied to convex functions are using the gradient's magnitude in order to adaptively determine the current step size. The paper is presenting a new heuristic fast gradient descent (HFGD) approach, which uses the change in gradient's direction in order to adaptively determine the current step size. The new approach can be applied to solve classes of unimodal functions more general than the convex functions (e.g., quasi-convex functions), or as a local optimization method in multimodal optimization. Testing conducted on a testbed of 16 test functions showed an overall much better eciency and an overall better success rate of the proposed HFGD method when compared to other three known first order gradient descent methods.