Learning-Enabled Adaptation for Lifelong Deployment of Robots
- Robots can respond to unexpected changes in conditions during deployment through learning-enabled adaptation.
- Distribution shifts, where the input data distribution changes from the training distribution, can lead to incorrect predictions and unsafe outcomes.
Data Life Cycle for Lifelong Deployment
- The data life cycle for lifelong deployment involves input data, model predictions, uncertainty estimation, out-of-distribution detection, data labeling, and model fine-tuning.
- Out-of-distribution detection can be achieved using Bayesian methods such as the Scott algorithm, which estimates functional uncertainty and thresholds it to identify out-of-distribution inputs.
- To adapt to distribution shifts, a subset of out-of-distribution inputs are selected for labeling, and the model is fine-tuned or retrained using these labels.
- The performance-cost trade-off should be considered when selecting data for labeling, as more labeled data improves performance but incurs higher costs.
Diverse Subsampling for Efficient Model Adaptation
- Out-of-distribution (OOD) detection algorithms like SCOTT can identify inputs that are different from the training distribution.
- Subsampling a diverse set of OOD inputs for labeling can improve model performance more efficiently than randomly selecting inputs or selecting based solely on uncertainty.
- The proposed diverse subsampling (DS-SCOTT) algorithm uses information gain as a measure to select diverse inputs for labeling.
- DS-SCOTT outperforms random selection and uncertainty-based selection in terms of the trade-off between labeling cost and model performance improvement.
- DS-SCOTT can be used in a data life cycle to continuously improve model performance by requesting labels for a diverse set of OOD inputs.
- By selecting a diverse subset of 5% or 75% of the data, the model achieves performance comparable to labeling 100% of the data, while only labeling about 50% of it.
Addressing Forgetting and Distribution Shifts
- To address the problem of forgetting original data during continual learning, uncertainty estimates can be used to generate a regularization term that prevents forgetting.
- Runtime monitors can be constructed to distinguish between different classes of distribution shifts.
- Self-supervised methods can be used to find invariant features between in-distribution and out-distribution data, reducing the need for labeled data.
Applications and Implementation
- The method is particularly useful in domains prone to distribution shifts, such as autonomous driving, where factors like rain, lighting, and sensor degradation can affect perception modules.
- The diverse subset selection can be implemented entirely at the edge, and the authors are considering open-sourcing the code.
