---
title: "Violation-Guided Learning for Constrained Formulations in Neural-Network Time-Series Predictions"
authors:
  - "Benjamin W. Wah"
  - "Minglun Qian"
doi: ""
url: "https://ijcai.org/proceedings/2001/105"
pdf: "https://ijcai.org/proceedings/2001/0105.pdf"
venue: "Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence"
year: 2001
pages: "771-775"
track: ""
keywords:
---

# Violation-Guided Learning for Constrained Formulations in Neural-Network Time-Series Predictions

Benjamin W. Wah, Minglun Qian

*Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence*, pages 771-775.

## Abstract

Time-series predictions by artificial neural networks (ANNs) are traditionally formulated as unconstrained optimization problems. As an unconstrained formulation provides little guidance on search directions when a search gets stuck in a poor local minimum, we have proposed recently to use a constrained formulation in order to use constraint violations to provide additional guidance. In this paper, we formulate ANN learning with cross-validations for time-series predictions as a non-differentiable nonlinear constrained optimization problem. Based on our theory of Lagrange multipliers for discrete constrained optimization, we propose an efficient learning algorithm, called violation guided back-propagation (VGBP), that computes an approximate gradient using back-propagation (BP), that introduces annealing to avoid blind acceptance of trial points, and that applies a relax-and-tighten (R&T) strategy to achieve faster convergence. Extensive experimental results on well-known benchmarks, when compared to previous work, show one to two orders-of-magnitude improvement in prediction quality, while using less weights.