{"type":"ScholarlyArticle","title":"Fine-grained Generalization Analysis of Structured Output Prediction","authors":[{"name":"Waleed Mustafa","slug":"waleed-mustafa","given":"Waleed","family":"Mustafa","affiliation":""},{"name":"Yunwen Lei","slug":"yunwen-lei","given":"Yunwen","family":"Lei","affiliation":""},{"name":"Antoine Ledent","slug":"antoine-ledent","given":"Antoine","family":"Ledent","affiliation":""},{"name":"Marius Kloft","slug":"marius-kloft","given":"Marius","family":"Kloft","affiliation":""}],"abstract":"In machine learning we often encounter structured output prediction problems (SOPPs), i.e. problems where the output space admits a rich internal structure. Application domains where SOPPs naturally occur include natural language processing, speech recognition, and computer vision. Typical SOPPs have an extremely large label set, which grows exponentially as a function of the size of the output. Existing generalization analysis implies generalization bounds with at least a square-root dependency on the cardinality d of the label set, which can be vacuous in practice. In this paper, we significantly improve the state of the art by developing novel high-probability bounds with a logarithmic dependency on d. Furthermore, we leverage the lens of algorithmic stability to develop generalization bounds in expectation without any dependency on d. Our results therefore build a solid theoretical foundation for learning in large-scale SOPPs. Furthermore, we extend our results to learning with weakly dependent data.","doi":"10.24963/ijcai.2021/391","url":"https://ijcai.org/proceedings/2021/391","pdf":"https://ijcai.org/proceedings/2021/0391.pdf","markdown":"https://ijcai.org/proceedings/2021/391.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/391","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Machine Learning","track":"Main Track","number":391,"pages":{"first":2841,"last":2847},"keywords":["Machine Learning: Learning Theory","Machine Learning: Structured Prediction"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}