{"type":"ScholarlyArticle","title":"Violation-Guided Learning for Constrained Formulations in Neural-Network Time-Series Predictions","authors":[{"name":"Benjamin W. Wah","slug":"benjamin-w-wah","given":"Benjamin W.","family":"Wah","affiliation":""},{"name":"Minglun Qian","slug":"minglun-qian","given":"Minglun","family":"Qian","affiliation":""}],"abstract":"Time-series predictions by artificial neural networks (ANNs) are traditionally formulated as unconstrained optimization problems. As an unconstrained formulation provides little guidance on search directions when a search gets stuck in a poor local minimum, we have proposed recently to use a constrained formulation in order to use constraint violations to provide additional guidance. In this paper, we formulate ANN learning with cross-validations for time-series predictions as a non-differentiable nonlinear constrained optimization problem. Based on our theory of Lagrange multipliers for discrete constrained optimization, we propose an efficient learning algorithm, called violation guided back-propagation (VGBP), that computes an approximate gradient using back-propagation (BP), that introduces annealing to avoid blind acceptance of trial points, and that applies a relax-and-tighten (R&T) strategy to achieve faster convergence. Extensive experimental results on well-known benchmarks, when compared to previous work, show one to two orders-of-magnitude improvement in prediction quality, while using less weights.","doi":null,"url":"https://ijcai.org/proceedings/2001/105","pdf":"https://ijcai.org/proceedings/2001/0105.pdf","markdown":"https://ijcai.org/proceedings/2001/105.md","bibtex":"https://ijcai.org/proceedings/2001/bibtex/105","volume":{"slug":"2001","title":"Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2001","url":"https://ijcai.org/proceedings/2001/","year":2001,"location":"Seattle, Washington, USA","dates":"","editors":"","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":105,"pages":{"first":771,"last":775},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}