Lars Lien Ankile

51 Vassar St

Cambridge, MA 02139

I’m a visiting researcher at the Improbable AI Lab at MIT CSAIL working on robot learning with Prof. Pulkit Agrawal.

Before this, I completed an M.Eng. of Data Science at Harvard University, where I completed my thesis work in the Improbable AI group, working on sample-efficient imitation learning. I also spent a year in the Data to Actionable Knowledge Lab at Harvard, working with Profs. Weiwei Pan and Finale Doshi-Velez on applying RL and Bayesian inference to model human decision-making for frictionful tasks in healthcare settings. Finally, I also spent a summer and fall interning with Prof. David Parkes and Matheus Ferreira in the EconCS Lab at Harvard working on detecting manipulation in multi-agent settings.

I did my undergrad at the Norwegian University of Science and Technology (NTNU) and did my thesis work on applying Deep Learning to econometric forecasting of complex and multivariate time series, supervised by Prof. Sjur Westgaard.

Research Interests

My research goal is to enable machines to learn in a human-like manner, primarily through observation of and interaction with the environment by developing methods merging imitation and reinforcement learning with robust policy representations. Through this work, I aim to build truly adaptive and flexible robots that can reliably learn and execute complex manipulation tasks in the physical world.

news

Nov 4, 2024	Our work on the scalable data platforms for robot learning, Dexhub and DART, led by Younghyo Park, is now available on arXiv! This introduces the AR-based teleoperation system DART, that enables scalable data collection without robots, and the Dexhub platform, which enables easy sharing and collaboration on robot learning datasets.
Sep 1, 2024	The paper led by Allen Ren at Princeton, Diffusion Policy Policy Optimization, is now available on arXiv! Here, we introduce DPPO, an algorithmic framework and set of best practices for diretly fine-tuning diffusion-based policies (as opposed to finetuning a residual model as in ResIP).
Jul 23, 2024	Our latest work, From Imitation to Refinement is now available on arXiv! In this work, we take the lessons from Juicer, and show the limitations of imitation learning for tasks requiring precise control, and propose a simple-yet-effective way to finetune pre-trained diffusion models for such tasks using residual models.
Jun 30, 2024	Data-Efficient Imitation Learning for Robotic Assembly got accepted to IROS 2024 October 14-18! Please reach out if you are going and want to chat about imitation learning, robotics, or anything else!
Apr 10, 2024	Our recent work on Data-Efficient Imitation Learning for Robotic Assembly is available to read on arXiv! In this work, we show how one can learn long assemblies (~2500 timesteps) with only <50 demonstrations using diffusion policies and data augmentation strategies.

selected publications

Robot Learning with Super-Linear Scaling

Marcel Torne, Arhan Jain, Jiayi Yuan, and 5 more authors

2024

Abs HTML PDF

Scaling robot learning requires data collection pipelines that scale favorably with human effort. In this work, we propose Crowdsourcing and Amortizing Human Effort for Real-to-Sim-to-Real(CASHER), a pipeline for scaling up data collection and learning in simulation where the performance scales superlinearly with human effort. The key idea is to crowdsource digital twins of real-world scenes using 3D reconstruction and collect large-scale data in simulation, rather than the real-world. Data collection in simulation is initially driven by RL, bootstrapped with human demonstrations. As the training of a generalist policy progresses across environments, its generalization capabilities can be used to replace human effort with model generated demonstrations. This results in a pipeline where behavioral data is collected in simulation with continually reducing human effort. We show that CASHER demonstrates zero-shot and few-shot scaling laws on three real-world tasks across diverse scenarios. We show that CASHER enables fine-tuning of pre-trained policies to a target scenario using a video scan without any additional human effort.
DexHub and DART: Towards Internet Scale Robot Data Collection

Younghyo Park, Jagdeep Singh Bhatia, Lars Ankile, and 1 more author

2024

Abs HTML PDF

The quest to build a generalist robotic system is impeded by the scarcity of diverse and high-quality data. While real-world data collection effort exist, requirements for robot hardware, physical environment setups, and frequent resets significantly impede the scalability needed for modern learning frameworks. We introduce DART, a teleoperation platform designed for crowdsourcing that reimagines robotic data collection by leveraging cloud-based simulation and augmented reality (AR) to address many limitations of prior data collection efforts. Our user studies highlight that DART enables higher data collection throughput and lower physical fatigue compared to real-world teleoperation. We also demonstrate that policies trained using DART-collected datasets successfully transfer to reality and are robust to unseen visual disturbances. All data collected through DART is automatically stored in our cloud-hosted database, DexHub, which will be made publicly available upon curation, paving the path for DexHub to become an ever-growing data hub for robot learning.
From Imitation to Refinement–Residual RL for Precise Visual Assembly

Lars Ankile, Anthony Simeonov, Idan Shenfeld, and 2 more authors

2024

Abs HTML PDF Code

Behavior cloning (BC) currently stands as a dominant paradigm for learning real-world visual manipulation. However, in tasks that require locally corrective behaviors like multi-part assembly, learning robust policies purely from human demonstrations remains challenging. Reinforcement learning (RL) can mitigate these limitations by allowing policies to acquire locally corrective behaviors through task reward supervision and exploration. This paper explores the use of RL fine-tuning to improve upon BC-trained policies in precise manipulation tasks. We analyze and overcome technical challenges associated with using RL to directly train policy networks that incorporate modern architectural components like diffusion models and action chunking. We propose training residual policies on top of frozen BC-trained diffusion models using standard policy gradient methods and sparse rewards, an approach we call ResiP (Residual for Precise manipulation). Our experimental results demonstrate that this residual learning framework can significantly improve success rates beyond the base BC-trained models in high-precision assembly tasks by learning corrective actions. We also show that by combining ResiP with teacher-student distillation and visual domain randomization, our method can enable learning real-world policies for robotic assembly directly from RGB images. Find videos and code at \urlhttps://residual-assembly.github.io.
Diffusion Policy Policy Optimization

Allen Z Ren, Justin Lidard, Lars L Ankile, and 6 more authors

2024

Abs HTML PDF Code

We introduce Diffusion Policy Policy Optimization, DPPO, an algorithmic framework including best practices for fine-tuning diffusion-based policies (e.g. Diffusion Policy) in continuous control and robot learning tasks using the policy gradient (PG) method from reinforcement learning (RL). PG methods are ubiquitous in training RL policies with other policy parameterizations; nevertheless, they had been conjectured to be less efficient for diffusion-based policies. Surprisingly, we show that DPPO achieves the strongest overall performance and efficiency for fine-tuning in common benchmarks compared to other RL methods for diffusion-based policies and also compared to PG fine-tuning of other policy parameterizations. Through experimental investigation, we find that DPPO takes advantage of unique synergies between RL fine-tuning and the diffusion parameterization, leading to structured and on-manifold exploration, stable training, and strong policy robustness. We further demonstrate the strengths of DPPO in a range of realistic settings, including simulated robotic tasks with pixel observations, and via zero-shot deployment of simulation-trained policies on robot hardware in a long-horizon, multi-stage manipulation task. Website with code: diffusion-ppo.github.io
JUICER: Data-Efficient Imitation Learning for Robotic Assembly

Lars Ankile, Anthony Simeonov, Idan Shenfeld, and 1 more author

2024

Abs HTML PDF Code

While learning from demonstrations is powerful for acquiring visuomotor policies, high-performance imitation without large demonstration datasets remains challenging for tasks requiring precise, long-horizon manipulation. This paper proposes a pipeline for improving imitation learning performance with a small human demonstration budget. We apply our approach to assembly tasks that require precisely grasping, reorienting, and inserting multiple parts over long horizons and multiple task phases. Our pipeline combines expressive policy architectures and various techniques for dataset expansion and simulation-based data augmentation. These help expand dataset support and supervise the model with locally corrective actions near bottleneck regions requiring high precision. We demonstrate our pipeline on four furniture assembly tasks in simulation, enabling a manipulator to assemble up to five parts over nearly 2500 time steps directly from RGB images, outperforming imitation and data augmentation baselines.