Unsupervised Progressive Learning and the STAM Architecture

Smith, James; Taylor, Cameron; Baer, Seth; Dovrolis, Constantine

doi:10.24963/ijcai.2021/410

Unsupervised Progressive Learning and the STAM Architecture

James Smith, Cameron Taylor, Seth Baer, Constantine Dovrolis

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 2979-2987.

https://doi.org/10.24963/ijcai.2021/410

Keywords

Machine Learning: Unsupervised Learning
Machine Learning: Online Learning
Machine Learning: Incremental Learning

Abstract

We first pose the Unsupervised Progressive Learning (UPL) problem: an online

representation learning problem in which the learner observes a non-stationary

and unlabeled data stream, learning a growing number of features that persist

over time even though the data is not stored or replayed. To solve the UPL

problem we propose the Self-Taught Associative Memory (STAM) architecture.

Layered hierarchies of STAM modules learn based on a combination of online

clustering, novelty detection, forgetting outliers, and storing only prototypical

features rather than specific examples. We evaluate STAM representations using

clustering and classification tasks. While there are no existing learning scenarios

that are directly comparable to UPL, we compare the STAM architecture with two

recent continual learning models, Memory Aware Synapses (MAS) and Gradient

Episodic Memories (GEM), after adapting them in the UPL setting.

James Smith, Cameron Taylor, Seth Baer, and Constantine Dovrolis. 2021. Unsupervised Progressive Learning and the STAM Architecture. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 2979–2987. https://doi.org/10.24963/ijcai.2021/410

@inproceedings{ijcai2021p410,
  title     = {Unsupervised Progressive Learning and the STAM Architecture},
  author    = {Smith, James and Taylor, Cameron and Baer, Seth and Dovrolis, Constantine},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Prof. Zhi-Hua Zhou},
  pages     = {2979--2987},
  year      = {2021},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2021/410},
  url       = {https://doi.org/10.24963/ijcai.2021/410},
}

TY  - CPAPER
TI  - Unsupervised Progressive Learning and the STAM Architecture
AU  - Smith, James
AU  - Taylor, Cameron
AU  - Baer, Seth
AU  - Dovrolis, Constantine
T2  - Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2021
SP  - 2979
EP  - 2987
DO  - 10.24963/ijcai.2021/410
UR  - https://doi.org/10.24963/ijcai.2021/410
ER  -