Few-shot Learning of Homogeneous Human Locomotion Styles

At Pacific Graphics this year I will be presenting work on Few-shot Modelling of Homogeneous Human Locomotion Styles. In this work we experimented with training neural networks for character control to model new styles of locomotion (e.g. old, duck footed) when given only a short reference clip of any one style.

Advantages of our method are that learning a new style takes very little training time, uses only limited data and learns without forgetting, that is, for a small increase in the size of the model, a new style is learned without decreasing the quality of the previously learned styles. Furthermore, we show a higher level of generalization than other popular systems such as motion matching or the phase-functioned neural network; whilst the PFNN does generalise well to new terrain, both these methods cannot generate motions that are very different from the training data, such as turning motions if only trained on straight locomotion. We also demo our model with many more styles than most previous work.

On the other hand the results of our method are not of production quality (unlike the aforementioned alternate methods) and we currently remain limited to learning homogeneous motions, that is, if shown only walking we cannot generate running. The model also does not make use of cross style learning, so when we learn a new style this does not make it easier to learn a similar style in the future.

Update: This work went on to win the Best Student Paper Award - Pacific Graphics 2018.

Paper, Bibtex, Code

Abstract: Using neural networks for learning motion controllers from motion capture data is becoming popular due to the natural and smooth motions they can produce, the wide range of movements they can learn and their compactness once they are trained. Despite these advantages, these systems require large amounts of motion capture data for each new character or style of motion to be generated, and systems have to undergo lengthy retraining, and often reengineering, to get acceptable results. This can make the use of these systems impractical for animators and designers and solving this issue is an open and rather unexplored problem in computer graphics. In this paper we propose a transfer learning approach for adapting a learned neural network to characters that move in different styles from those on which the original neural network is trained. Given a pretrained character controller in the form of a Phase-Functioned Neural Network for locomotion, our system can quickly adapt the locomotion to novel styles using only a short motion clip as an example. We introduce a canonical polyadic tensor decomposition to reduce the amount of parameters required for learning from each new style, which both reduces the memory burden at runtime and facilitates learning from smaller quantities of data. We show that our system is suitable for learning stylized motions with few clips of motion data and synthesizing smooth motions in real-time.