---
title: "Fine-grained Generalization Analysis of Structured Output Prediction"
authors:
  - "Waleed Mustafa"
  - "Yunwen Lei"
  - "Antoine Ledent"
  - "Marius Kloft"
doi: "10.24963/ijcai.2021/391"
url: "https://ijcai.org/proceedings/2021/391"
pdf: "https://ijcai.org/proceedings/2021/0391.pdf"
venue: "Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence"
year: 2021
pages: "2841-2847"
track: "Main Track"
keywords:
  - "Machine Learning: Learning Theory"
  - "Machine Learning: Structured Prediction"
---

# Fine-grained Generalization Analysis of Structured Output Prediction

Waleed Mustafa, Yunwen Lei, Antoine Ledent, Marius Kloft

*Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence*, pages 2841-2847. DOI: [10.24963/ijcai.2021/391](https://doi.org/10.24963/ijcai.2021/391)

## Abstract

In machine learning we often encounter structured output prediction problems (SOPPs), i.e. problems where the output space admits a rich internal structure. Application domains where SOPPs naturally occur include natural language processing, speech recognition, and computer vision. Typical SOPPs have an extremely large label set, which grows exponentially as a function of the size of the output. Existing generalization analysis implies generalization bounds with at least a square-root dependency on the cardinality d of the label set, which can be vacuous in practice. In this paper, we significantly improve the state of the art by developing novel high-probability bounds with a logarithmic dependency on d. Furthermore, we leverage the lens of algorithmic stability to develop generalization bounds in expectation without any dependency on d. Our results therefore build a solid theoretical foundation for learning in large-scale SOPPs. Furthermore, we extend our results to learning with weakly dependent data.