Multi-label metric learning serve as an effective strategy to facilitate multi-label classification, aiming to learn better similarity metrics from multi-label examples. Existing multi-label metric learning approaches learn consistent metrics across all multi-label instances in the label space. However, such consistent metric learning approaches are suboptimal as they neglect the nonlinear distribution characteristics of multi-label instances. In this paper, we present LSMM, a label-specific multi-metric learning framework for multi-label classification, where nonlinear distribution characteristics of multi-label examples are considered by learning label-specific multiple local metrics for different instances on the shoulder of a global one. Specifically, multi-label instances within each label space can be divided into several disjoint partitions through either semantic-based or cluster-based partition strategies, in each of which a local metric is trained to separate the instances locally. Besides, a global metric is introduced to implicitly exploit high-order label correlations across all labels. The combination of the global metric and label-specific local metrics is utilized to measure the semantic similarities between multi-label instances in each label space, under which similar intra-class instances are pushed closer and inter-class instances are pulled apart. Comprehensive experiments on benchmark multi-label data sets validate the superiority of LSMM in learning effective similarity metrics for multi-label classification.