forked from academicpages/academicpages.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
b653d32
commit 69cd8da
Showing
4 changed files
with
78 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| title: "Decoupling Contact for Fine-Grained Motion Style Transfer" | ||
| collection: publications | ||
| permalink: /publication/Decoupling_Contact | ||
| excerpt: '[Xiangjun Tang](https://yuyujunjun.github.io/), [Linjun Wu](https://fivezerojun.github.io/), [He Wang](https://drhewang.com/), **[Yiqian Wu](https://onethousandwu.com/)**, Bo Hu, Songnan Li, Xu Gong, Yuchen Liao, Qilong Kou, [Xiaogang Jin*](http://www.cad.zju.edu.cn/home/jin)' | ||
| date: 2024-07-28 | ||
| venue: "ACM SIGGRAPH Asia 2024 (Conference Track)" | ||
| paperurl: 'http://www.cad.zju.edu.cn/home/jin/SigA20241/High_quality_Controllable_Motion_Style_Transfer.pdf' | ||
| citation: '' | ||
| code: '' | ||
| video: 'http://www.cad.zju.edu.cn/home/jin/SigA20241/demo.mp4' | ||
| supplementary_materials: 'http://www.cad.zju.edu.cn/home/jin/SigA20241/supplemental_material.pdf' | ||
| project_page: 'http://www.cad.zju.edu.cn/home/jin/SigA20241/Decoupling_Contact.htm' | ||
| year: '2024' | ||
| first_author: 'no' | ||
| teaser: 'http://oneThousand1000.github.io/images/publications/Decoupling_Contact.jpg' | ||
| --- | ||
|
|
||
|  | ||
|
|
||
|
|
||
| > **Decoupling Contact for Fine-Grained Motion Style Transfer** | ||
| > | ||
| > ACM SIGGRAPH Asia 2024 (Conference Track) | ||
| > | ||
| > [Xiangjun Tang](https://yuyujunjun.github.io/), [Linjun Wu](https://fivezerojun.github.io/), [He Wang](https://drhewang.com/), [Yiqian Wu](https://onethousandwu.com/), Bo Hu, Songnan Li, Xu Gong, Yuchen Liao, Qilong Kou, [Xiaogang Jin*](http://www.cad.zju.edu.cn/home/jin) | ||
| [](http://www.cad.zju.edu.cn/home/jin/SigA20241/Decoupling_Contact.htm) | ||
| [](http://www.cad.zju.edu.cn/home/jin/SigA20241/High_quality_Controllable_Motion_Style_Transfer.pdf) | ||
| [](http://www.cad.zju.edu.cn/home/jin/SigA20241/demo.mp4) | ||
|
|
||
| <b>Abstract:</b> | ||
|
|
||
| Motion style transfer changes the style of a motion while retaining its content and is useful in computer animations and games. Contact is an essential component of motion style transfer that should be controlled explicitly in order to express the style vividly while enhancing motion naturalness and quality. However, it is unknown how to decouple and control contact to achieve fine-grained control in motion style transfer. In this paper, we present a novel style transfer method for fine-grained control over contacts while achieving both motion naturalness and spatial temporal variations of style. Based on our empirical evidence, we propose controlling contact indirectly through the hip velocity, which can be further decomposed into the trajectory and contact timing, respectively. To this end, we propose a new model that explicitly models the correlations between motions and trajectory/contact timing/style, allowing us to decouple and control each separately. Our approach is built around a motion manifold, where hip controls can be easily integrated into a Transformer-based decoder. It is versatile in that it can generate motions directly as well as be used as post-processing for existing methods to improve quality and contact controllability. In addition, we propose a new metric that measures a correlation pattern of motions based on our empirical evidence, aligning well with human perception in terms of motion naturalness. Based on extensive evaluation, our method outperforms existing methods in terms of style expressivity and motion quality. | ||
|
|
||
|
|
||
| Recommended citation: | ||
| ``` | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| --- | ||
| title: "StyleTex: Style Image-Guided Texture Generation for 3D Models" | ||
| collection: publications | ||
| permalink: /publication/StyleTex | ||
| excerpt: '[Zhiyu Xie](), [Yuqing Zhang](https://zzzyuqing.github.io/), [Xiangjun Tang](https://yuyujunjun.github.io/), **[Yiqian Wu](https://onethousandwu.com/)**, Dehan Chen, Gongsheng Li, [Xiaogang Jin*](http://www.cad.zju.edu.cn/home/jin) ' | ||
| date: 2024-07-28 | ||
| venue: "ACM SIGGRAPH Asia 2024 (Conference Track)" | ||
| paperurl: '' | ||
| citation: '' | ||
| code: '' | ||
| video: '' | ||
| supplementary_materials: '' | ||
| project_page: '' | ||
| year: '2024' | ||
| first_author: 'no' | ||
| teaser: 'http://oneThousand1000.github.io/images/publications/styletex.png' | ||
| --- | ||
|
|
||
|
|
||
| > **StyleTex: Style Image-Guided Texture Generation for 3D Models** | ||
| > | ||
| > ACM SIGGRAPH Asia 2024 (Conference Track) | ||
| > | ||
| > [Zhiyu Xie](), [Yuqing Zhang](https://zzzyuqing.github.io/), [Xiangjun Tang](https://yuyujunjun.github.io/), [Yiqian Wu](https://onethousandwu.com/), Dehan Chen, Gongsheng Li, [Xiaogang Jin*](http://www.cad.zju.edu.cn/home/jin) | ||
|  | ||
|
|
||
| <b>Abstract:</b> | ||
|
|
||
| Style-guided texture generation aims to generate a texture that is harmonious with both the style of the reference image and the geometry of the input mesh, given a reference style image and a 3D mesh with its text description. Although diffusion-based 3D texture generation methods, such as distillation sampling, have numerous promising applications in stylized games and films, it requires addressing two challenges: 1) decouple style and content completely from the reference image for 3D models, and 2) align the generated texture with the color tone, style of the reference image, and the given text prompt. To this end, we introduce StyleTex, an innovative diffusion-model-based framework for creating stylized textures for 3D models. Our key insight is to decouple style information from the reference image while disregarding content in diffusion-based distillation sampling. Specifically, given a reference image, we first decompose its style feature from the image CLIP embedding by subtracting the embedding's orthogonal projection in the direction of the content feature, which is represented by a text CLIP embedding. Our novel approach to disentangling the reference image's style and content information allows us to generate distinct style and content features. We then inject the style feature into the cross-attention mechanism to incorporate it into the generation process, while utilizing the content feature as a negative prompt to further dissociate content information. Finally, we incorporate these strategies into StyleTex to obtain stylized textures. We utilize Interval Score Matching to address over-smoothness and over-saturation, in combination with a geometry-aware ControlNet that ensures consistent geometry throughout the generative process. The resulting textures generated by StyleTex retain the style of the reference image, while also aligning with the text prompts and intrinsic details of the given 3D mesh. Quantitative and qualitative experiments show that our method outperforms existing baseline methods by a significant margin. | ||
|
|
||
|
|
||
|
|
||
|
|
||
| Recommended citation: | ||
| ``` | ||
| ``` |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.