{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T02:08:30Z","timestamp":1773972510279,"version":"3.50.1"},"reference-count":60,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2021,6,4]],"date-time":"2021-06-04T00:00:00Z","timestamp":1622764800000},"content-version":"vor","delay-in-days":34,"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/http\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computer Graphics Forum"],"published-print":{"date-parts":[[2021,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Equipping characters with diverse motor skills is the current bottleneck of physics\u2010based character animation. We propose a Deep Reinforcement Learning (DRL) framework that enables physics\u2010based characters to learn and explore motor skills from reference motions. The key insight is to use loose space\u2010time constraints, termed spacetime bounds, to limit the search space in an early termination fashion. As we only rely on the reference to specify loose spacetime bounds, our learning is more robust with respect to low quality references. Moreover, spacetime bounds are hard constraints that improve learning of challenging motion segments, which can be ignored by imitation\u2010only learning. We compare our method with state\u2010of\u2010the\u2010art tracking\u2010based DRL methods. We also show how to guide style exploration within the proposed framework.<\/jats:p>","DOI":"10.1111\/cgf.142630","type":"journal-article","created":{"date-parts":[[2021,6,4]],"date-time":"2021-06-04T16:37:32Z","timestamp":1622824652000},"page":"251-263","update-policy":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Learning and Exploring Motor Skills with Spacetime Bounds"],"prefix":"10.1111","volume":"40","author":[{"given":"Li\u2010Ke","family":"Ma","sequence":"first","affiliation":[{"name":"Simon Fraser University"},{"name":"Tsinghua University"}]},{"given":"Zeshi","family":"Yang","sequence":"additional","affiliation":[{"name":"Simon Fraser University"}]},{"given":"Xin","family":"Tong","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia"}]},{"given":"Baining","family":"Guo","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia"}]},{"given":"KangKang","family":"Yin","sequence":"additional","affiliation":[{"name":"Simon Fraser University"}]}],"member":"311","published-online":{"date-parts":[[2021,6,4]]},"reference":[{"key":"e_1_2_8_2_2","unstructured":"AmayaK. BruderlinA. CalvertT.: Emotion from motion. InGraphics interface(1996) vol. 96 pp.222\u2013229. 2"},{"issue":"8","key":"e_1_2_8_3_2","first-page":"1405","article-title":"Trajectory optimization for full-body movements with complex contacts","volume":"19","author":"Al Borno M.","year":"2013","journal-title":"TVCG"},{"key":"e_1_2_8_4_2","doi-asserted-by":"crossref","unstructured":"AgrawalS. ShenS. van dePanneM.: Diverse motion variations for physics-based character animation. InProceedings of the 12th ACM SIGGRAPH\/Eurographics Symposium on Computer Animation(2013) pp.37\u201344. 2","DOI":"10.1145\/2485895.2485907"},{"issue":"4","key":"e_1_2_8_5_2","article-title":"Task-based locomotion","volume":"35","author":"Agrawal S.","year":"2016","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_6_2","doi-asserted-by":"crossref","unstructured":"AristidouA. ZengQ. StavrakisE. YinK. Cohen-OrD. ChrysanthouY. ChenB.: Emotion control of unstructured dance movements. InProceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation(2017). 2 4 11","DOI":"10.1145\/3099564.3099566"},{"issue":"6","key":"e_1_2_8_7_2","article-title":"DReCon: data-driven responsive control of physics-based characters","volume":"38","author":"Bergamin K.","year":"2019","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_8_2","doi-asserted-by":"crossref","unstructured":"BrandM. HertzmannA.: Style machines. InProceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques(2000) pp.183\u2013192. 2","DOI":"10.1145\/344779.344865"},{"key":"e_1_2_8_9_2","unstructured":"Bullet: Bullet physics library 2015.https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/http\/bulletphysics.org. 6 8"},{"issue":"4","key":"e_1_2_8_10_2","article-title":"Generalized biped walking control","volume":"29","author":"Coros S.","year":"2010","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964954"},{"key":"e_1_2_8_12_2","unstructured":"ClavetS.: Motion matching and the road to next-gen animation. InGCD(2016). 2"},{"key":"e_1_2_8_13_2","doi-asserted-by":"crossref","unstructured":"DingK. LiuL. van dePanneM. YinK.: Learning reduced-order feedback policies for motion skills. InProc. ACM SIGGRAPH \/ Eurographics Symposium on Computer Animation(2015). 2 5","DOI":"10.1145\/2786784.2786802"},{"key":"e_1_2_8_14_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01134.x"},{"key":"e_1_2_8_15_2","doi-asserted-by":"publisher","DOI":"10.1080\/10867651.1998.10487493"},{"key":"e_1_2_8_16_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511524646"},{"issue":"4","key":"e_1_2_8_17_2","article-title":"Phase-functioned neural networks for character control","volume":"36","author":"Holden D.","year":"2017","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073315"},{"key":"e_1_2_8_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2767002"},{"issue":"4","key":"e_1_2_8_20_2","article-title":"A deep learning framework for character motion synthesis and editing","volume":"35","author":"Holden D.","year":"2016","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_21_2","unstructured":"HeessN. TBD. SriramS. LemmonJ. MerelJ. WayneG. TassaY. ErezT. WangZ. EslamiS. et al.: Emergence of locomotion behaviours in rich environments.ArXiv abs\/1707.02286(2017). 2"},{"issue":"1","key":"e_1_2_8_22_2","article-title":"Generalizing motion edits with gaussian processes","volume":"28","author":"Ikemoto L.","year":"2009","journal-title":"ACM Transctions on Graphics"},{"issue":"4","key":"e_1_2_8_23_2","article-title":"Synthesis of biologically realistic human motion using joint torque actuation","volume":"38","author":"Jiang Y.","year":"2019","journal-title":"ACM Transactions on Graphics (Proc. SIGGRAPH 2019)"},{"key":"e_1_2_8_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566605"},{"key":"e_1_2_8_25_2","volume-title":"Relativity: the special and the general theory","author":"Lawson R. W.","year":"1960"},{"key":"e_1_2_8_26_2","unstructured":"LillicrapT. P. HuntJ. J. PritzelA. HeessN. ErezT. TassaY. SilverD. WierstraD.: Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971(2015). 2"},{"issue":"4","key":"e_1_2_8_27_2","article-title":"Data-driven biped control","volume":"29","author":"Lee Y.","year":"2010","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566596"},{"issue":"3","key":"e_1_2_8_29_2","article-title":"Guided learning of control graphs for physics-based characters","volume":"35","author":"Liu L.","year":"2016","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_30_2","doi-asserted-by":"crossref","unstructured":"LeeY. WamplerK. BernsteinG. Popovi\u0107J. Popovi\u0107Z.: Motion fields for interactive character locomotion. InACM SIGGRAPH Asia 2010 Papers(2010). 2","DOI":"10.1145\/1882262.1866160"},{"issue":"4","key":"e_1_2_8_31_2","article-title":"Continuous character control with low-dimensional embeddings","volume":"31","author":"Levine S.","year":"2012","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_32_2","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12571"},{"issue":"4","key":"e_1_2_8_33_2","article-title":"Sampling-based contact-rich motion control","volume":"29","author":"Liu L.","year":"2010","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366173"},{"key":"e_1_2_8_35_2","doi-asserted-by":"crossref","unstructured":"MinJ. LiuH. ChaiJ.: Synthesis and editing of personalized stylistic human motion. InProceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games(2010) pp.39\u201346. 2","DOI":"10.1145\/1730804.1730811"},{"key":"e_1_2_8_36_2","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13832"},{"issue":"4","key":"e_1_2_8_37_2","article-title":"DeepMimic: Example-guided deep reinforcement learning of physics-based character skills","volume":"37","author":"Peng X. B.","year":"2018","journal-title":"ACM Transctions on Graphics"},{"issue":"4","key":"e_1_2_8_38_2","article-title":"DeepLoco: Dynamic locomotion skills using hierarchical deep reinforcement learning","volume":"36","author":"Peng X. B.","year":"2017","journal-title":"ACM Transactions on Graphics (Proc. SIGGRAPH 2017)"},{"key":"e_1_2_8_39_2","volume-title":"Techniques of differential topology in relativity","author":"Penrose R.","year":"1987"},{"issue":"6","key":"e_1_2_8_40_2","article-title":"SFV: Reinforcement learning of physical skills from videos","volume":"37","author":"Peng X. B.","year":"2018","journal-title":"ACM Transctions on Graphics"},{"issue":"6","key":"e_1_2_8_41_2","article-title":"Learning predict-and-simulate policies from unorganized human motion data","volume":"38","author":"Park S.","year":"2019","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_42_2","unstructured":"PyTorch:Pytorch 2018.https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/pytorch.org\/. 6"},{"key":"e_1_2_8_43_2","volume-title":"Reinforcement learning: An introduction","author":"Sutton R. S.","year":"2018"},{"key":"e_1_2_8_44_2","unstructured":"ShapiroA. CaoY. FaloutsosP.: Style components. InProceedings of Graphics Interface 2006(2006) pp.33\u201339. 2"},{"key":"e_1_2_8_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276510"},{"key":"e_1_2_8_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015754"},{"key":"e_1_2_8_47_2","unstructured":"SchulmanJ. MoritzP. LevineS. JordanM. AbbeelP.: High-dimensional continuous control using generalized advantage estimation.CoRR abs\/1506.02438(2015). 5"},{"key":"e_1_2_8_48_2","unstructured":"SchulmanJ. WolskiF. DhariwalP. RadfordA. KlimovO.: Proximal policy optimization algorithms.CoRR abs\/1707.06347(2017). 5"},{"key":"e_1_2_8_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2011.30"},{"key":"e_1_2_8_50_2","doi-asserted-by":"crossref","unstructured":"WangJ. M. FleetD. J. HertzmannA.: Multifactor gaussian process models for style-content separation. InProceedings of the 24th International Conference on Machine Learning(2007) pp.975\u2013982. 2","DOI":"10.1145\/1273496.1273619"},{"issue":"4","key":"e_1_2_8_51_2","article-title":"A scalable approach to control diverse behaviors for physically simulated characters","volume":"39","author":"Won J.","year":"2020","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356499"},{"key":"e_1_2_8_53_2","doi-asserted-by":"crossref","unstructured":"XieZ. LingH. Y. KimN. H. van dePanneM.: Allsteps: Curriculum-driven learning of stepping stone skills. InProceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation(2020). 6","DOI":"10.1111\/cgf.14115"},{"issue":"4","key":"e_1_2_8_54_2","article-title":"Realtime style transfer for unlabeled heterogeneous human motion","volume":"34","author":"Xia S.","year":"2015","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_55_2","unstructured":"YinK. ClineM. B. PaiD. K.: Motion perturbation based on simple neuromotor control models. InProceedings of Pacific Graphics(2003). 2 5"},{"issue":"4","key":"e_1_2_8_56_2","article-title":"Optimal feedback control for character animation using an abstract model","volume":"29","author":"Ye Y.","year":"2010","journal-title":"ACM Transctions on Graphics"},{"issue":"3","key":"e_1_2_8_57_2","article-title":"SIMBICON: Simple biped locomotion control","volume":"26","author":"Yin K.","year":"2007","journal-title":"ACM Transctions on Graphics"},{"issue":"4","key":"e_1_2_8_58_2","article-title":"Spectral style transfer for human motion between independent actions","volume":"35","author":"Yumer M. E.","year":"2016","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201397"},{"issue":"4","key":"e_1_2_8_60_2","article-title":"Mode-adaptive neural networks for quadruped motion control","volume":"37","author":"Zhang H.","year":"2018","journal-title":"ACM Transctions on Graphics"},{"key":"e_1_2_8_61_2","doi-asserted-by":"crossref","unstructured":"ZhangZ. WangJ.: MLLE: Modified locally linear embedding using multiple weights. InAdvances in neural information processing systems(2007) pp.1593\u20131600. 10","DOI":"10.7551\/mitpress\/7503.003.0204"}],"container-title":["Computer Graphics Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/cgf.142630","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/cgf.142630","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/cgf.142630","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,4]],"date-time":"2023-11-04T14:37:43Z","timestamp":1699108663000},"score":1,"resource":{"primary":{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/10.1111\/cgf.142630"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5]]},"references-count":60,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["10.1111\/cgf.142630"],"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.1111\/cgf.142630","archive":["Portico"],"relation":{},"ISSN":["0167-7055","1467-8659"],"issn-type":[{"value":"0167-7055","type":"print"},{"value":"1467-8659","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5]]},"assertion":[{"value":"2021-06-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}