Learning to Control Self-Assembling Morphologies:
A Study of Generalization via Modularity

Deepak Pathak *
Chris Lu *
Trevor Darrell
Phillip Isola
Alexei A. Efros
UC Berkeley
UC Berkeley
UC Berkeley
MIT
UC Berkeley
* equal contribution
To appear at NeurIPS, 2019 (Spotlight)
Winner of the Virtual Creatures Competition at GECCO 2019

[Download Paper]
[GitHub Code]


Contemporary sensorimotor learning approaches typically start with an existing complex agent (e.g., a robotic arm), which they learn to control. In contrast, this paper investigates a modular co-evolution strategy: a collection of primitive agents learns to dynamically self-assemble into composite bodies while also learning to coordinate their behavior to control these bodies. Each primitive agent consists of a limb with a motor attached at one end. Limbs may choose to link up to form collectives. When a limb initiates a link-up action and there is another limb nearby, the latter is magnetically connected to the 'parent' limb's motor. This forms a new single agent, which may further link with other agents. In this way, complex morphologies can emerge, controlled by a policy whose architecture is in explicit correspondence with the morphology. We evaluate the performance of these dynamic and modular agents in simulated environments. We demonstrate better generalization to test-time changes both in the environment, as well as in the agent morphology, compared to static and monolithic baselines.


Self-Assembling Agents

This work investigates the joint learning of control and morphology in self-assembling agents. We illustrate our dynamic agents in two environments / tasks: standing up and locomotion. For each of these, we generate several new environment for evaluating zero-shot generalization without any finetuning.




Source Code and Environment

We have released the PyTorch based implementation and environment on the github page. Try our code!
[GitHub]


Paper and Bibtex

[Paper]  [ArXiv]

Citation
 
Deepak Pathak, Chris Lu, Trevor Darrell, Phillip Isola, Alexei A. Efros. Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity
In arXiv 2019.

[Bibtex]
@inproceedings{pathak19assemblies,
  Author = {Pathak, Deepak and
  Lu, Chris and Darrell, Trevor and
  Isola, Phillip and Efros, Alexei A.},
  Title = {Learning to Control Self-
  Assembling Morphologies: A Study of
  Generalization via Modularity},
  Booktitle = {arXiv preprint arXiv:1902.05546},
  Year = {2019}
}


Acknowledgements

We would like to thank Igor Mordatch, Chris Atkeson, Abhinav Gupta and the members of BAIR for fruitful discussions and comments. This work was supported in part by Berkeley DeepDrive, and the Valrhona reinforcement learning fellowship. DP is supported by the Facebook graduate fellowship.