{"type":"ScholarlyArticle","title":"Cost-effective Artificial Neural Networks","authors":[{"name":"Zahra Atashgahi","slug":"zahra-atashgahi","given":"Zahra","family":"Atashgahi","affiliation":"University of Twente , The Netherlands"}],"abstract":"Deep neural networks (DNNs) have gained huge attention over the last several years due to their promising results in various tasks. However, due to their large model size and over-parameterization, they are recognized as being computationally demanding. Therefore, deep learning models are not well-suited to applications with limited computational resources and battery life. Current solutions to reduce computation costs mainly focus on inference efficiency while being resource-intensive during training. This Ph.D. research aims to address these challenges by developing cost-effective neural networks that can achieve decent performance on various complex tasks using minimum computational resources during training and inference of the network.","doi":"10.24963/ijcai.2023/810","url":"https://ijcai.org/proceedings/2023/810","pdf":"https://ijcai.org/proceedings/2023/0810.pdf","markdown":"https://ijcai.org/proceedings/2023/810.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/810","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":810,"pages":{"first":7071,"last":7072},"keywords":["Machine Learning: ML: Learning sparse models","Machine Learning: ML: Feature extraction, selection and dimensionality reduction","Machine Learning: ML: Classification"],"videos":[],"references":[{"position":1,"raw":"[Atashgahi et al., 2022a] Zahra Atashgahi, Decebal Constantin Mocanu, Raymond Veldhuis, and Mykola Pechenizkiy. Memory- free online change-point detection: A novel neural network ap- proach. arXiv preprint arXiv:2207.03932, 2022.","title":"Memoryfree online change-point detection: A novel neural network approach","authors":"Atashgahi","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Atashgahi et al., 2022b] Zahra Atashgahi, Joost Pieterse, Shiwei Liu, Decebal Constantin Mocanu, Raymond Veldhuis, and Mykola Pechenizkiy. A brain-inspired algorithm for training highly sparse neural networks. Machine Learning (ECML-PKDD Journal track), 2022.","title":"A brain-inspired algorithm for training highly sparse neural networks","authors":"Atashgahi","year":"2022","venue":"ECML-PKDD Journal track","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Atashgahi et al., 2022c] Zahra Atashgahi, Ghada Sokar, Tim van der Lee, Elena Mocanu, Decebal Constantin Mocanu, Ray- mond Veldhuis, and Mykola Pechenizkiy. Quick and robust fea- ture selection: the strength of energy-efficient sparse training for autoencoders. Machine Learning (ECML-PKDD Journal track), 111(1), 2022.","title":"Quick and robust feature selection: the strength of energy-efficient sparse training for autoencoders","authors":"Atashgahi","year":"2022","venue":"Machine Learning (ECML-PKDD Journal track)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Atashgahi et al., 2022d] Zahra Atashgahi, Xuhao Zhang, Neil Kichler, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Raymond Veldhuis, and Decebal Constantin Mocanu. Feature selection with neuron evolution in sparse neural networks. In TMLR, 2022.","title":"Feature selection with neuron evolution in sparse neural networks","authors":"Atashgahi","year":"2022","venue":"TMLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Atashgahi et al., 2023] Zahra Atashgahi, Mykola Pechenizkiy, Raymond Veldhuis, and Decebal Constantin Mocanu. Adaptive sparsity level during training for efficient time series forecasting with transformers. arXiv preprint arXiv:2305.18382, 2023.","title":"Adaptive sparsity level during training for efficient time series forecasting with transformers","authors":"Zahra Atashgahi, Mykola Pechenizkiy, Raymond Veldhuis, Decebal Constantin, Mocanu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Liu et al., 2020] Shiwei Liu, Tim van der Lee, Anil Yaman, Zahra Atashgahi, Davide Ferraro, Ghada Sokar, Mykola Pechenizkiy, and Decebal C Mocanu. Topological insights into sparse neural networks. In Proceedings of ECML-PKDD, 2020.","title":"Topological insights into sparse neural networks","authors":"Shiwei Liu, Tim Van Der Lee, Anil Yaman, Zahra Atashgahi, Davide Ferraro, Ghada Sokar, Mykola Pechenizkiy, Mocanu","year":"2020","venue":"Proceedings of ECML-PKDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liu et al., 2021] Shiwei Liu, Tianlong Chen, Xiaohan Chen, Zahra Atashgahi, Lu Yin, Huanyu Kou, Li Shen, Mykola Pechenizkiy, Zhangyang Wang, and Decebal Constantin Mocanu. Sparse train- ing via boosting pruning plasticity with neuroregeneration. In NeurIPS, 2021.","title":"Sparse training via boosting pruning plasticity with neuroregeneration","authors":"Shiwei Liu, Tianlong Chen, Xiaohan Chen, Zahra Atashgahi, Lu Yin, Huanyu Kou, Li Shen, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin, Mocanu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2022] Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xi- aohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, and Decebal Constantin Mocanu. Deep en- sembling with no overhead for either training or testing: The all- round blessings of dynamic sparsity. In ICLR, 2022.","title":"Deep ensembling with no overhead for either training or testing: The allround blessings of dynamic sparsity","authors":"Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin, Mocanu","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Sokar et al., 2022] Ghada Sokar, Zahra Atashgahi, Mykola Pech- enizkiy, and Decebal Constantin Mocanu. Where to pay attention in sparse training for feature selection? In NeurIPS, 2022.","title":"Where to pay attention in sparse training for feature selection?","authors":"Ghada Sokar, Zahra Atashgahi, Mykola Pechenizkiy, Decebal Constantin, Mocanu","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Strubell et al., 2020] Emma Strubell, Ananya Ganesh, and An- drew McCallum. Energy and policy considerations for modern deep learning research. AAAI, 34, Apr. 2020.","title":"Energy and policy considerations for modern deep learning research","authors":"Emma Strubell, Ananya Ganesh, Andrew Mccallum","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}