{"type":"ScholarlyArticle","title":"AutoML for Outlier Detection with Optimal Transport Distances","authors":[{"name":"Prabhant Singh","slug":"prabhant-singh","given":"Prabhant","family":"Singh","affiliation":"Eindhoven University of Technology"},{"name":"Joaquin Vanschoren","slug":"joaquin-vanschoren","given":"Joaquin","family":"Vanschoren","affiliation":"Eindhoven University of Technology"}],"abstract":"Automated machine learning (AutoML) has been widely researched and adopted for supervised problems, but progress in unsupervised settings has been limited. We propose `\"LOTUS\", a novel framework to automate outlier detection based on meta-learning. Our premise is that the selection of the optimal outlier detection technique depends on the inherent properties of the data distribution. We leverage optimal transport to find the dataset with the most similar underlying distribution, and then apply the outlier detection techniques that proved to work best for that data distribution. We evaluate the robustness of our framework and find that it outperforms all state-of-the-art automated outlier detection tools. This approach can also be easily generalized to automate other unsupervised settings.","doi":"10.24963/ijcai.2023/843","url":"https://ijcai.org/proceedings/2023/843","pdf":"https://ijcai.org/proceedings/2023/0843.pdf","markdown":"https://ijcai.org/proceedings/2023/843.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/843","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Demo Track","track":"Demo Track","number":843,"pages":{"first":7175,"last":7178},"keywords":["Data Mining: DM: Anomaly/outlier detection","Machine Learning: ML: Automated machine learning"],"videos":[],"references":[{"position":1,"raw":"[Bahri et al., 2022] Maroua Bahri, Flavia Salutari, Andrian Putina, and Mauro Sozio. AutoML: state of the art with a focus on anomaly detection, challenges, and research di- rections. International Journal of Data Science and Ana- lytics, February 2022.","title":"AutoML: state of the art with a focus on anomaly detection, challenges, and research directions","authors":"Bahri","year":"2022","venue":"International Journal of Data Science and Analytics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Benavoli et al., 2014] Alessio Benavoli, Giorgio Corani, Francesca Mangili, Marco Zaffalon, and Fabrizio Ruggeri. A bayesian wilcoxon signed-rank test based on the dirich- let process. In Eric P. Xing and Tony Jebara, editors, Pro- ceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learn- ing Research, pages 1026-1034, Bejing, China, 22-24 Jun 2014. PMLR.","title":"A bayesian wilcoxon signed-rank test based on the dirichlet process","authors":"Alessio Benavoli, Giorgio Corani, Francesca Mangili, Marco Zaffalon, Fabrizio Ruggeri","year":"2014","venue":"Proceedings of the 31st International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Benavoli et al., 2017] Alessio Benavoli, Giorgio Corani, Janez Demšar, and Marco Zaffalon. Time for a change: a tutorial for comparing multiple classifiers through bayesian analysis. Journal of Machine Learning Research, 18(77):1-36, 2017.","title":"Time for a change: a tutorial for comparing multiple classifiers through bayesian analysis","authors":"Alessio Benavoli, Giorgio Corani, Janez Demšar, Marco Zaffalon","year":"2017","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cartella et al., 2021] Francesco Cartella, Orlando Anunciac ¸ão, Yuki Funabiki, Daisuke Yamaguchi, Toru Akishita, and Olivier Elshocht. Adversarial attacks for tabular data: Application to fraud detection and imbalanced data. ArXiv, abs/2101.08030, 2021.","title":"Adversarial attacks for tabular data: Application to fraud detection and imbalanced data","authors":"Francesco Cartella, Orlando Anunciac ¸ão, Yuki Funabiki, Daisuke Yamaguchi, Toru Akishita, Olivier Elshocht","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Datta et al., 2022] Debanjan Datta, S. Muthiah, John Sime- one, Amelia Meadows, and Naren Ramakrishnan. Scru- tinizing shipment records to thwart illegal timber trade. ArXiv, abs/2208.00493, 2022.","title":"Scrutinizing shipment records to thwart illegal timber trade","authors":"Debanjan Datta, S Muthiah, John Simeone, Amelia Meadows, Naren Ramakrishnan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feurer et al., 2015] Matthias Feurer, Aaron Klein, Katha- rina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. Efficient and robust automated machine learning. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28.","title":"Efficient and robust automated machine learning","authors":"Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, Frank Hutter","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Curran Associates, Inc., 2015.","title":null,"authors":"Curran Associates, Inc","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Feurer et al., 2020] Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. Auto-sklearn 2.0: Hands-free automl via meta-learning. arXiv:2007.04074 [cs.LG], 2020.","title":"Auto-sklearn 2.0: Hands-free automl via meta-learning","authors":"Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, Frank Hutter","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gijsbers and Vanschoren, 2021] Pieter Gijsbers and Joaquin Vanschoren. Gama: A general automated machine learning assistant. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track, 2021.","title":"Gama: A general automated machine learning assistant","authors":"Pieter Gijsbers, Joaquin Vanschoren","year":"2021","venue":"Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goix, 2016] Nicolas Goix. How to evaluate the quality of unsupervised anomaly detection algorithms?, 2016.","title":"How to evaluate the quality of unsupervised anomaly detection algorithms?","authors":"Nicolas Goix, Goix","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Han et al., 2022] Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. ADBench: Anomaly detection benchmark. In Thirty-sixth Conference on Neu- ral Information Processing Systems Datasets and Bench- marks Track, 2022.","title":"ADBench: Anomaly detection benchmark","authors":"Han","year":"2022","venue":"Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hutter et al., 2019] Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. Automated machine learning: Meth- ods, systems, challenges. Automated Machine Learning, 2019.","title":"Automated machine learning: Methods, systems, challenges","authors":"Hutter","year":"2019","venue":"Automated Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2020] Yuening Li, Daochen Zha, Na Zou, and Xia Hu. Pyodds: An end-to-end outlier detection system with automated machine learning. Companion Proceedings of the Web Conference 2020, 2020.","title":"Pyodds: An end-to-end outlier detection system with automated machine learning","authors":"Li","year":"2020","venue":"Companion Proceedings of the Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liang et al., 2022] Dong Liang, Jun Wang, Wenping Zhang, Yuqi Liu, Lei Wang, and Xiaoyong Zhao. Tabular data anomaly detection based on density peak clustering algo- rithm. 2022 International Conference on Big Data, In- formation and Computer Network (BDICN), pages 16-21, 2022.","title":"Tabular data anomaly detection based on density peak clustering algorithm","authors":"Dong Liang, Jun Wang, Wenping Zhang, Yuqi Liu, Lei Wang, Xiaoyong Zhao","year":"2022","venue":"International Conference on Big Data, Information and Computer Network (BDICN)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2021] Martin Q. Ma, Yue Zhao, Xiaorong Zhang, and Leman Akoglu. A large-scale study on unsupervised outlier model selection: Do internal strategies suffice? CoRR, abs/2104.01422, 2021.","title":"A large-scale study on unsupervised outlier model selection: Do internal strategies suffice?","authors":"Martin Q Ma, Yue Zhao, Xiaorong Zhang, Leman Akoglu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Marques et al., 2015] Henrique O. Marques, Ricardo J. G. B. Campello, Arthur Zimek, and Jorg Sander. On the in- ternal evaluation of unsupervised outlier detection. In Pro- ceedings of the 27th International Conference on Scien- tific and Statistical Database Management, SSDBM '15, New York, NY, USA, 2015. Association for Computing Machinery.","title":"On the internal evaluation of unsupervised outlier detection","authors":"Henrique O Marques, Ricardo J G B Campello, Arthur Zimek, Jorg Sander","year":"2015","venue":"Proceedings of the 27th International Conference on Scientific and Statistical Database Management, SSDBM '15","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peyré and Cuturi, 2019] Gabriel Peyré and Marco Cuturi. Computational optimal transport. Found. Trends Mach. Learn., 11:355-607, 2019.","title":"Computational optimal transport","authors":"Gabriel Peyré, Marco Cuturi","year":"2019","venue":"Found. Trends Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rakotoarison et al., 2022] Herilalaina Rakotoarison, Louisot Milijaona, Andry RASOANAIVO, Michele Sebag, and Marc Schoenauer. Learning meta-features for autoML. In International Conference on Learning Representations, 2022.","title":"Herilalaina Rakotoarison, Louisot Milijaona","authors":"Rakotoarison","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Scetbon and Cuturi, 2022] Meyer Scetbon and Marco Cu- turi. Low-rank optimal transport: Approximation, statis- tics and debiasing. NeurIPS 2022, abs/2205.12365, 2022.","title":"Low-rank optimal transport: Approximation, statistics and debiasing","authors":"Meyer Scetbon, Marco Cuturi","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Thornton et al., 2013] Chris Thornton, Frank Hutter, Hol- ger H. Hoos, and Kevin Leyton-Brown. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In 19th ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, page 847-855, 2013.","title":"Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms","authors":"Chris Thornton, Frank Hutter, H Holger, Kevin Hoos, Leyton-Brown","year":"2013","venue":"19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vanschoren, 2018] Joaquin Vanschoren. Meta-learning: A survey. ArXiv, abs/1810.03548, 2018.","title":"Joaquin Vanschoren. Meta-learning: A survey","authors":"Vanschoren","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2021] Chi Wang, Qingyun Wu, Markus Weimer, and Erkang Zhu. Flaml: A fast and lightweight automl library. In MLSys, 2021.","title":"Flaml: A fast and lightweight automl library","authors":"Wang","year":"2021","venue":"MLSys","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhao et al., 2019] Yue Zhao, Zain Nasrullah, and Zheng Li. Pyod: A python toolbox for scalable outlier detection. J. Mach. Learn. Res., 20:96:1-96:7, 2019.","title":"Pyod: A python toolbox for scalable outlier detection","authors":"Yue Zhao, Zain Nasrullah, Zheng Li","year":"2019","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2021] Yue Zhao, Ryan Rossi, and Leman Akoglu. Automatic unsupervised outlier model selection. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural In- formation Processing Systems, volume 34, pages 4489- 4502.","title":"Automatic unsupervised outlier model selection","authors":"Yue Zhao, Ryan Rossi, Leman Akoglu","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"Curran Associates, Inc., 2021.","title":null,"authors":"Curran Associates, Inc","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}