Understanding Multimodality in Generative Behavioral Cloning

Understanding Multimodality in Generative Behavioral Cloning. By Lorenzo Mazza and five others. Published on the thirtieth of September. Its code repository is worked out: about eight hundred and fifty source files, with a readme and a licence.

Behavioral cloning gets hard when one observation admits several valid actions. An expert might go left around an obstacle one time and right the next, and a policy that averages the two goes straight into it. This work asks how generative behavioral-cloning policies represent that kind of multimodal expert behavior, and its answer is that the bottleneck depends on how the model is parameterized. For latent-variable policies, keeping the demonstrated modes apart requires action-conditioned information in the latent representation. Regularize the posterior too hard toward the prior and that information is suppressed, so the policy can no longer tell the modes apart. Regularize more weakly, or in aggregate, and the mode information survives, but the difficulty moves: now the prior used at deployment has to cover the latent regions that matter.

Action-space generative policies hit a different wall. Their multimodality is constrained by how smooth the transport from base space to action space is. A map with a small Lipschitz constant cannot place substantial probability on many well-separated modes. To cover many modes, then, the map needs either sharp transitions in base space or bridge regions in action space that lie off the support of the data.

The authors test these mechanisms in two settings: synthetic multimodal navigation, and a bimodal manipulation task on a physical robot. The experiments in both support the account.

The same analysis turns up a less comfortable result about standard robotic simulation benchmarks. Conditional multimodality there is limited, and deterministic regression stays competitive. That bounds what those benchmarks can say about multimodal policies, since a setting with little multimodality in it cannot reward a model for handling it.

Two model families, two bottlenecks. For latent policies the question is whether mode information survives regularization and whether the prior then reaches it. For action-space policies it is whether the transport is allowed to be sharp enough. And underneath both sits a question about the test bed itself: whether the benchmark contains the multimodality the method was built for.