{"type":"ScholarlyArticle","title":"Unsupervised Progressive Learning and the STAM Architecture","authors":[{"name":"James Smith","slug":"james-smith","given":"James","family":"Smith","affiliation":""},{"name":"Cameron Taylor","slug":"cameron-taylor","given":"Cameron","family":"Taylor","affiliation":""},{"name":"Seth Baer","slug":"seth-baer","given":"Seth","family":"Baer","affiliation":""},{"name":"Constantine Dovrolis","slug":"constantine-dovrolis","given":"Constantine","family":"Dovrolis","affiliation":""}],"abstract":"We first pose the Unsupervised Progressive Learning (UPL) problem: an online\r\nrepresentation learning problem in which the learner observes a non-stationary\r\nand unlabeled data stream, learning a growing number of features that persist\r\nover time even though the data is not stored or replayed. To solve the UPL\r\nproblem we propose the Self-Taught Associative Memory (STAM) architecture.\r\nLayered hierarchies of STAM modules learn based on a combination of online\r\nclustering, novelty detection, forgetting outliers, and storing only prototypical\r\nfeatures rather than specific examples. We evaluate STAM representations using\r\nclustering and classification tasks. While there are no existing learning scenarios\r\nthat are directly comparable to UPL, we compare the STAM architecture with two\r\nrecent continual learning models, Memory Aware Synapses (MAS) and Gradient\r\nEpisodic Memories (GEM), after adapting them in the UPL setting.","doi":"10.24963/ijcai.2021/410","url":"https://ijcai.org/proceedings/2021/410","pdf":"https://ijcai.org/proceedings/2021/0410.pdf","markdown":"https://ijcai.org/proceedings/2021/410.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/410","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Machine Learning","track":"Main Track","number":410,"pages":{"first":2979,"last":2987},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Online Learning","Machine Learning: Incremental Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}