{"type":"ScholarlyArticle","title":"Logic Distillation: Learning from Code Function by Function for Decision-making Tasks","authors":[{"name":"Dong Chen","slug":"dong-chen","given":"Dong","family":"Chen","affiliation":""},{"name":"Shilin Zhang","slug":"shilin-zhang","given":"Shilin","family":"Zhang","affiliation":""},{"name":"Fei Gao","slug":"fei-gao","given":"Fei","family":"Gao","affiliation":""},{"name":"Yueting Zhuang","slug":"yueting-zhuang","given":"Yueting","family":"Zhuang","affiliation":""},{"name":"Siliang Tang","slug":"siliang-tang","given":"Siliang","family":"Tang","affiliation":""},{"name":"Qidong Liu","slug":"qidong-liu","given":"Qidong","family":"Liu","affiliation":""},{"name":"Mingliang Xu","slug":"mingliang-xu","given":"Mingliang","family":"Xu","affiliation":""}],"abstract":"Large language models (LLMs) have garnered increasing attention owing to their powerful comprehension and generation capabilities. Generally, larger LLMs (L-LLMs) that require paid interfaces exhibit significantly superior performance compared to smaller LLMs (S-LLMs) that can be deployed on a variety of devices. Knowledge distillation (KD) aims to empower S-LLMs with the capabilities of L-LLMs, while S-LLMs merely mimic the outputs of L-LLMs, failing to get the powerful decision-making capability for new situations. Consequently, S-LLMs are helpless when it comes to continuous decision-making tasks that require logical reasoning. To tackle the identified challenges, we propose a novel framework called Logic Distillation (LD). Initially, LD employs L-LLMs to instantiate complex instructions into discrete functions and illustrates their usage to establish a function base. Subsequently, LD fine-tunes S-LLMs based on the function base to learn the logic employed by L-LLMs in decision-making. During testing, S-LLMs will yield decision-making outcomes, function by function, based on current states. Experiments demonstrate that with the assistance of LD, S-LLMs can achieve outstanding results in continuous decision-making tasks, comparable to, or even surpassing, those of L-LLMs. The code and data for the proposed method are provided for research purposes https://github.com/Anfeather/Logic-Distillation.","doi":"10.24963/ijcai.2025/816","url":"https://ijcai.org/proceedings/2025/816","pdf":"https://ijcai.org/proceedings/2025/0816.pdf","markdown":"https://ijcai.org/proceedings/2025/816.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/816","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":816,"pages":{"first":7338,"last":7346},"keywords":["Knowledge Representation and Reasoning: KRR: Learning and reasoning","Natural Language Processing: NLP: Language models","Multidisciplinary Topics and Applications: MTA: Game playing"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}