Proceedings

Sequence Labelling in Structured Domains with Hierarchical Recurrent Neural Networks

Download BibTeX Cite

Proceedings of the Twentieth International Joint Conference on Artificial Intelligence. Pages 774-778.

© 2007 International Joint Conferences on Artificial Intelligence

Abstract

Modelling data in structured domains requires establishing the relations among patterns at multiple scales. When these patterns arise from sequential data, the multiscale structure also contains a dynamic component that must be modelled, particularly, as is often the case, if the data is unsegmented. Probabilistic graphical models are the predominant framework for labelling unsegmented sequential data in structured domains. Their use requires a certain degree of a priori knowledge about the relations among patterns and about the patterns themselves. This paper presents a hierarchical system, based on the connectionist temporal classification algorithm, for labelling unsegmented sequential data at multiple scales with recurrent neural networks only. Experiments on the recognition of sequences of spoken digits show that the system outperforms hidden Markov models, while making fewer assumptions about the domain.

  1. [Bridle, 1990] John S. Bridle. Neurocomputing: Algorithms, architectures and applications, chapter Probabilistic in- terpretation of feedforward classification network outputs, pages 227-236. Springer-Verlag, 1990.
  2. [Graves and Schmidhuber, 2005] Alex Graves and Jürgen Schmidhuber. Framewise phoneme classification with bidirectional LSTM and other neural network architec- tures. Neural Networks, 18(5-6):602-610, June/July 2005.
  3. [Graves et al., 2005] Alex Graves, Santiago Fernández, and Jürgen Schmidhuber. Bidirectional LSTM networks for improved phoneme classification and recognition. In Pro- ceedings of the 2005 International Conference on Artifi- cial Neural Networks, Warsaw, Poland, 2005.
  4. [Graves et al., 2006] Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA, 2006.
  5. [Kumar and Hebert, 2005] Sanjiv Kumar and Martial Hebert. A hierarchical field framework for unified context-based classification. In Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China, 2005.
  6. [Lafferty et al., 2001] John Lafferty, Andrew McCallum, and Fernando Pereira. Conditional random fields: Proba- bilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning, 2001.
  7. [Leonard and Doddington, 1993] R. Gary Leonard and George Doddington. TIDIGITS. Linguistic Data Consortium, Philadelphia, 1993.
  8. [Nevill-Manning and Witten, 1997] Craig G. Nevill- Manning and Ian H. Witten. Identifying hierarchical structure in sequences: A linear-time algorithm. Journal of Artificial Intelligence Research, 7:67-82, 1997.
  9. [Rabiner, 1989] Lawrence R. Rabiner. A tutorial on hidden markov models and selected applications in speech recog- nition. Proceedings of the IEEE, 77(2):257-286, February 1989.
  10. [Schuster and Paliwal, 1997] M. Schuster and K. K. Paliwal. Bidirectional recurrent neural networks. IEEE Transac- tions on Signal Processing, 45:2673-2681, 1997.
  11. [Werbos, 1990] Paul J. Werbos. Backpropagation through time: What it does and how to do it. Proceedings of the IEEE, 78(10):1550-1560, 1990.
  12. [Young et al., 2005] Steve Young, Gunnar Evermann, Mark Gales, Thomas Hain, Dan Kershaw, Gareth Moore, Julian Odell, Dave Ollason, Dan Povey, Valtcho Valtchev, and Phil Woodland. The HTK Book version 3.3. Cambridge University Engineering Department, 2005.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards