近期计划是什么？| What are the short-term plans?

[hengtuibabai]

停更了吗？近期计划是什么？
感觉离好用还差一点点了啊，停更的话太可惜了！

Has it stopped? What are the short-term plans?
I feel like just a little short of being useful. It would be a pity if I stopped the watch!

[vinthony]

hi， 最近再肝paper，可能会在月底或者6月份开始继续更新。目前已经完成的部分和正在做的部分，比如:

• 支持size参数可以控制输出图的分辨率，训了一个512x512的模型face render还在测试之中，期望可以干掉人脸增强器。
• mannually的控制crop的区域，可以得到更加自定义的结果，WEBUI。
• WEBUI更多功能，比如refpose等。
• 接受了两种不同的自动crop方式，能够复现v0.0.1版本的效果。
• 减少了一些依赖文件,比如dlib，用现在的方法更便于安装。
• 加速模型和优化checkpoints，可能会不需要下载那么大的模型。
• 更加解耦的mappingnet的训练，可能可以支持只针对头进行动作。
• A simpler facerender model (or the TensorRT support) will be included to support faster generation: A light-weight FaceRender is support for more than 100x faster rendering on Mac OS. #457
• text-geneartion-webui
• anime-generator.
• OpenTalker WEBUI, 集成Sadtalker和video-retalking的技术，做到输入图像和视频进行驱动和编辑。
• Fix API problem， 是否支持API调用的方式生成视频? #379，[bug] [gradio] Web API not working #374 , 这是个特别牛的项目，请问SD那边能用api调用吗? #251 ,
• FPS, Frame Per Second Option request. #294,

[vinthony]

any new suggestions are also welcome : )

[2793145003]

之前看到 有人用 Diffusion model，用 Diffusion model 的话也许能做到原图分辨率？（不懂瞎说）

[vinthony]

diffusion-based 估计会很慢，不过face vid2vid也很慢就是了。质量上我从paper里没看到很大的差距因为他的diffusion不是SD，是自己训的。这里有一个备选是 先用sadtalker生成keypoints，再用pre-trained 的controlnet-face去解。

[lliang2003]

OpenTalker WEBUI, 集成Sadtalker和video-retalking的技术，做到输入图像和视频进行驱动和编辑。

[canghaiyunfan]

请问如何实现只针对头进行动作？

[Niutonian]

I love your project,
I think a good way to control ref_pose and ref_eyebllink would be awesome,
Another thing that I would love to see implemented is "Idle animation", in case no audio is played it goes through a preset sequence of blink and move without looking too weird

[MoroseYu]

不好停更啊，很期待~

[vinthony]

cool! thanks for your advise! will woking on it.

[2793145003]

之后会解决脸色苍白双眼无神的问题吗？（还是我用的方法不对？
quick_demo里的full3，size=512，出来的结果脸会变得很白（去掉gfpgan也很白）
眼睛也一直虚着，去掉still之后可以眨眼但又没法贴回原图

[vinthony]

I love your project, I think a good way to control ref_pose and ref_eyebllink would be awesome, Another thing that I would love to see implemented is "Idle animation", in case no audio is played it goes through a preset sequence of blink and move without looking too weird

We have update this feature in #386, however, more work need to be done to make it better.

[vinthony]

这个我也发现了，可能是和训练数据有关。我找时间再训练一下模型。

之后会解决脸色苍白双眼无神的问题吗？（还是我用的方法不对？ quick_demo里的full3，size=512，出来的结果脸会变得很白（去掉gfpgan也很白） 眼睛也一直虚着，去掉still之后可以眨眼但又没法贴回原图

[zyl280505776]

挺好的，期待持续迭代

[jyzd111]

大佬厉害，我用的国外的网站生成的和你效果差不多，你的还能自己调整参数。期待早日上2D，

[grazder]

what are your plans on cartoon images? like in makeittalk

[vinthony]

A lightweight facerender is added for generation, which might be working in real-time on GPU and 100x faster on Macbook. See the discussion #457.

[vinthony]

what are your plans on cartoon images? like in makeittalk

will try to add something like: https://github.com/pkhungurn/talking-head-anime-3-demo

[Kedreamix]

你好，我想问一下，有计划开源训练的代码么，也就是每一part的训练代码

[FranM2030]

I've noticed that you're able to provide a head pose reference video, could we do the same for a half body. Provide a reference video for half body that drives the upper body movement along with the head? Just he head is a bit limited.

[xyyyuuan]

amd的显卡是不是只能跑在cpu上，amd不动

[ifredom]

感谢大兄弟，就差一点就很好用了，有空更更，不要鸽啦

[skyliwq]

加油加油

[warycat]

加油加油，这个东西太有用了，我要把它集成到我的app里。

[Tybost]

Still crossing my fingers for anime head support. ;)

[slavakurilyak]

When can we expect the next release? The last release was made 168 days ago.

[creepcat-gh]

大佬厉害，确实非常好用

[zjy-2020]

大佬，网盘下载的压缩包李没有.pth文件和BFM, hub文件了吗
就是checkpoints/auido2exp_00300-model.pth，checkpoints/auido2pose_00140-model.pth， checkpoints/epoch_20.pth等文件。
没有话程序老是报错

[rucieryi369]

会有微调的代码吗？谢谢

[denvey]

OpenTalker WEBUI

请问这个在哪里呀

[XayerMorgan]

Any plans for continued development. I was using this for a while, but suddenly it stopped working. Can't find the root cause, because automatic1111 and foocus work great. Standalone or automatic1111 give same errors. Would love to see this continue, and it's a remarkable piece of work.

[oisilener1982]

Sadly Devs are very busy in other projects :(

[ck1123456]

您好，请问我提供中文音频，生成的视频不是自己提供的声音，求指教

[ck1123456]

您好，请问我提供中文音频，生成的视频不是自己提供的声音，求指教

[oisilener1982]

When can we expect the next release? The last release was made 168 days ago.

Most likely there will no further updates. The last update was 9 months ago. With the coming release of EMO and VASA-1 sadtalker might really be dead, however they are not released yet

The other alternative is v-express by tencent but sadtalker is way better,

[NinoNeumann]

Now there might be something even better—hallo (https://github.com/fudan-generative-vision/hallo), and the exciting part is that this project is open-source.

[oisilener1982]

Hallo is taking so long to render :( We need to upgrade to Rtx 5090 once it is released

[gg22mm]

看好这个项目，期待升级更牛的版本出来。 ~~ 虽然目前在图片生成视频已是 number on

[gg22mm]

Now there might be something even better—hallo (https://github.com/fudan-generative-vision/hallo), and the exciting part is that this project is open-source.

这说的这个不支持中文搞个毛线

[gg22mm]

hi， 最近再肝paper，可能会在月底或者6月份开始继续更新。目前已经完成的部分和正在做的部分，比如:

* [x]  支持size参数可以控制输出图的分辨率，训了一个512x512的模型face render还在测试之中，期望可以干掉人脸增强器。

* [x]  mannually的控制crop的区域，可以得到更加自定义的结果，WEBUI。

* [x]  WEBUI更多功能，比如refpose等。

* [x]  接受了两种不同的自动crop方式，能够复现v0.0.1版本的效果。

* [x]  减少了一些依赖文件,比如dlib，用现在的方法更便于安装。

* [x]  加速模型和优化checkpoints，可能会不需要下载那么大的模型。

* [ ]  更加解耦的mappingnet的训练，可能可以支持只针对头进行动作。

* [x]  A simpler facerender model (or the TensorRT support) will be included to support faster generation: [A light-weight FaceRender is support for more than 100x faster rendering on Mac OS. #457](https://github.com/OpenTalker/SadTalker/discussions/457)

* [ ]  text-geneartion-webui

* [ ]  anime-generator.

* [ ]  OpenTalker WEBUI, 集成Sadtalker和video-retalking的技术，做到输入图像和视频进行驱动和编辑。

* [ ]  Fix API problem， [是否支持API调用的方式生成视频? #379](https://github.com/OpenTalker/SadTalker/issues/379)，[[bug] [gradio] Web API not working #374](https://github.com/OpenTalker/SadTalker/issues/374) , [这是个特别牛的项目，请问SD那边能用api调用吗? #251](https://github.com/OpenTalker/SadTalker/issues/251) ,

* [ ]  FPS, [Frame Per Second Option request. #294](https://github.com/OpenTalker/SadTalker/issues/294),

我感觉以后升级的话（可以往这方面发展，这样就不会有头部与身体衔接问题）：
1、直接生成全省图，现在的逻辑是先生成半身图如：

2、再经过full.才放大（就会有衔接问题，头部与身体会些错位，有抖动错位）

4.3_full.mp4

3、最后经过enhanced提升效果