{"type":"ScholarlyArticle","title":"Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Information","authors":[{"name":"Seungcheol Park","slug":"seungcheol-park","given":"Seungcheol","family":"Park","affiliation":""},{"name":"Sojin Lee","slug":"sojin-lee","given":"Sojin","family":"Lee","affiliation":""},{"name":"Jongjin Kim","slug":"jongjin-kim","given":"Jongjin","family":"Kim","affiliation":""},{"name":"Jinsik Lee","slug":"jinsik-lee","given":"Jinsik","family":"Lee","affiliation":""},{"name":"Hyunjik Jo","slug":"hyunjik-jo","given":"Hyunjik","family":"Jo","affiliation":""},{"name":"U Kang","slug":"u-kang","given":"U","family":"Kang","affiliation":""}],"abstract":"How can we accelerate large language models (LLMs) without sacrificing accuracy? The slow inference speed of LLMs hinders us to benefit from their remarkable performance in diverse applications. This is mainly because numerous sublayers are stacked together in LLMs. Sublayer pruning compresses and expedites LLMs via removing unnecessary sublayers. However, existing sublayer pruning algorithms are limited in accuracy since they naively select sublayers to prune, overlooking the different characteristics of each sublayer.\r\nIn this paper, we propose SPRINT (Sublayer Pruning with Latency and Tunability Information), an accurate sublayer pruning method for LLMs. SPRINT accurately selects a target sublayer to prune by considering 1) the amount of latency reduction after pruning and 2) the tunability of sublayers. SPRINT iteratively prunes redundant sublayers and swiftly tunes the parameters of remaining sublayers. Experiments show that SPRINT achieves the best accuracy-speedup trade-off, exhibiting up to 23.88%p higher accuracy on zero-shot commonsense reasoning benchmarks compared to existing pruning algorithms.","doi":"10.24963/ijcai.2025/913","url":"https://ijcai.org/proceedings/2025/913","pdf":"https://ijcai.org/proceedings/2025/0913.pdf","markdown":"https://ijcai.org/proceedings/2025/913.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/913","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Natural Language Processing","track":"Main Track","number":913,"pages":{"first":8213,"last":8221},"keywords":["Natural Language Processing: NLP: Language models","Natural Language Processing: NLP: Other"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}