---
title: "Bayesian Experience Reuse for Learning from Multiple Demonstrators"
authors:
  - "Mike Gimelfarb"
  - "Scott Sanner"
  - "Chi-Guhn Lee"
doi: "10.24963/ijcai.2021/334"
url: "https://ijcai.org/proceedings/2021/334"
pdf: "https://ijcai.org/proceedings/2021/0334.pdf"
venue: "Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence"
year: 2021
pages: "2425-2431"
track: "Main Track"
keywords:
  - "Uncertainty in AI: Bayesian Networks"
  - "Machine Learning: Transfer, Adaptation, Multi-task Learning"
  - "Machine Learning: Deep Reinforcement Learning"
  - "Uncertainty in AI: Approximate Probabilistic Inference"
---

# Bayesian Experience Reuse for Learning from Multiple Demonstrators

Mike Gimelfarb, Scott Sanner, Chi-Guhn Lee

*Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence*, pages 2425-2431. DOI: [10.24963/ijcai.2021/334](https://doi.org/10.24963/ijcai.2021/334)

## Abstract

Learning from Demonstrations (LfD) is a powerful approach for incorporating advice from experts in the form of demonstrations. However, demonstrations often come from multiple sub-optimal experts with conflicting goals, rendering them difficult to incorporate effectively in online settings. To address this, we formulate a quadratic program whose solution yields an adaptive weighting over experts, that can be used to sample experts with relevant goals. In order to compare different source and target task goals safely, we model their uncertainty using normal-inverse-gamma priors, whose posteriors are learned from demonstrations using Bayesian neural networks with a shared encoder. Our resulting approach, which we call Bayesian Experience Reuse, can be applied for LfD in static and dynamic decision-making settings. We demonstrate its effectiveness for minimizing multi-modal functions, and optimizing a high-dimensional supply chain with cost uncertainty, where it is also shown to improve upon the performance of the demonstrators' policies.