The current dominant paradigm of imitation learning relies on strong supervision of expert actions for learning both what to and how to imitate.

We propose an alternative paradigm wherein an agent first explores the world without any expert supervision and then distills its own experience into a goal-conditioned skill policy using a novel forward consistency loss formulation. In our framework, the role of the human expert is only to communicate goals (i.e., what to imitate) during inference. The learned policy is then employed to mimic the expert (i.e., how to imitate) after observing just a visual demonstration. Our method is "zero-shot" in the sense that the agent never has access to expert actions either during training or for task demonstration at inference.

We evaluate our zero-shot imitator in two real-world settings: complex rope manipulation using a Baxter robot and navigation in previously unseen office environments using a TurtleBot.
Through further experiments in VizDoom simulation, we provide evidence that better mechanisms for exploration lead to learning a more capable policy which in turn improves end task performance.

We have released the TensorFlow based implementation on the github page. Try our code!

We pursue an alternative paradigm wherein an agent first explores the world without any expert supervision and then distills its experience into a goal-conditioned skill policy with a novel forward consistency loss. The key insight is the intuition that, for most tasks, reaching the goal is more important than how it is reached.

Rope manipulation training data was re-used from ICRA17 paper on "Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation", available here.

Left half: View as seen by TurtleBot
Right half: Goal image (or landmark image) shown to TurtleBot
Note that all these results are in unseen novel environment.

@inproceedings{pathakICLR18zeroshot,
    Author = {Pathak, Deepak and
    Mahmoudieh, Parsa and Luo, Guanghao and
    Agrawal, Pulkit and Chen, Dian and
    Shentu, Yide and Shelhamer, Evan and
    Malik, Jitendra and Efros, Alexei A. and
    Darrell, Trevor},
    Title = {Zero-Shot Visual Imitation},
    Booktitle = {ICLR},
    Year = {2018}
}