Skip to content

Commit

Permalink
windows one click demo
Browse files Browse the repository at this point in the history
  • Loading branch information
vinthony committed Apr 17, 2023
1 parent 3d09abf commit 3bea177
Show file tree
Hide file tree
Showing 5 changed files with 269 additions and 33 deletions.
81 changes: 52 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,35 +87,47 @@ https://user-images.githubusercontent.com/4397546/231495639-5d4bb925-ea64-4a36-a
- [ ] Audio-driven Anime Avatar.
- [ ] training code of each componments.

## ⚙️ Installation ([中文windows教程](https://www.bilibili.com/video/BV1Dc411W7V6/)|[日本語コース](https://br-d.fanbox.cc/posts/5685086?utm_campaign=manage_post_page&utm_medium=share&utm_source=twitter) )
## ⚙️ Installation

#### Installing Sadtalker on Linux:
Tutorials from communities: [中文windows教程](https://www.bilibili.com/video/BV1Dc411W7V6/) | [日本語コース](https://br-d.fanbox.cc/posts/5685086?utm_campaign=manage_post_page&utm_medium=share&utm_source=twitter)

```bash
git clone https://github.com/Winfredy/SadTalker.git
### Linux:

1. Installing [anaconda](https://www.anaconda.com/), python and git.

2. Creating the env and install the requirements.
```bash
git clone https://github.com/Winfredy/SadTalker.git

cd SadTalker

cd SadTalker
conda create -n sadtalker python=3.8

conda create -n sadtalker python=3.8
conda activate sadtalker

conda activate sadtalker
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113

pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
conda install ffmpeg

conda install ffmpeg
pip install -r requirements.txt

pip install -r requirements.txt
### tts is optional for gradio demo.
### pip install TTS

### tts is optional for gradio demo.
### pip install TTS
```
### Windows ([中文windows教程](https://www.bilibili.com/video/BV1Dc411W7V6/)):

```
1. Install [Python 3.10.6](https://www.python.org/downloads/windows/), checking "Add Python to PATH".
2. Install [git](https://git-scm.com/download/win).
3. Install `ffmpeg`, following [this instruction](https://www.wikihow.com/Install-FFmpeg-on-Windows).
4. Download our SadTalker repository, for example by running `git clone https://github.com/Winfredy/SadTalker.git`.
5. Run `start.bat` from Windows Explorer as normal, non-administrator, user, a gradio WebUI demo will be started.

More tips about installnation on Windows and the Docker file can be founded [here](docs/install.md)
### Macbook:

#### [Sd-Webui-Extension](docs/webui_extension.md).
More tips about installnation on Macbook and the Docker file can be founded [here](docs/install.md)

#### Download Trained Models
## Download Trained Models

You can run the following script to put all the models in the right place.

Expand All @@ -132,10 +144,17 @@ Other alternatives:

**百度云盘**: we provided the downloaded model in [checkpoints, 提取码: sadt.](https://pan.baidu.com/s/1nXuVNd0exUl37ISwWqbFGA?pwd=sadt) And [gfpgan, 提取码: sadt.](https://pan.baidu.com/s/1kb1BCPaLOWX1JJb9Czbn6w?pwd=sadt)



<details><summary>Model Details</summary>

The final folder will be shown as:

<img width="331" alt="image" src="https://user-images.githubusercontent.com/4397546/232511411-4ca75cbf-a434-48c5-9ae0-9009e8316484.png">

<details><summary>Model Details</summary>

Model explains:

| Model | Description
| :--- | :----------
|checkpoints/auido2exp_00300-model.pth | Pre-trained ExpNet in Sadtalker.
Expand All @@ -148,12 +167,26 @@ The final folder will be shown as:
|checkpoints/shape_predictor_68_face_landmarks.dat | Face landmark model used in [dilb](http://dlib.net/).
|checkpoints/BFM | 3DMM library file.
|checkpoints/hub | Face detection models used in [face alignment](https://github.com/1adrianb/face-alignment).
|gfpgan/weights | Face detection and enhanced models used in `facexlib` and `gfpgan`.


</details>

## 🔮 Quick Start ([Best Practice](docs/best_practice.md))

#### Animating Portrait Image from default config.
### Online: WebUI Demos | [Huggingface](https://huggingface.co/spaces/vinthony/SadTalker) | [SDWebUI-Colab](https://colab.research.google.com/github/camenduru/stable-diffusion-webui-colab/blob/main/video/stable/stable_diffusion_1_5_video_webui_colab.ipynb) | [Colab](https://colab.research.google.com/github/Winfredy/SadTalker/blob/main/quick_demo.ipynb)

Local [Autiomatic1111 stable-diffusion webui](docs/webui_extension.md) installation

Local gradio demo similar to our [hugging-face demo](https://huggingface.co/spaces/vinthony/SadTalker) can be run by:

```bash
## you need manually install TTS(https://github.com/coqui-ai/TTS) via `pip install tts` in advanced.
python app.py
```


### Animating Portrait Image from default config.

```bash
python inference.py --driven_audio <audio.wav> --source_image <video.mp4 or picture.png> --enhancer gfpgan
Expand All @@ -163,7 +196,7 @@ The results will be saved in `results/$SOME_TIMESTAMP/*.mp4`.
More examples and configuration and tips can be founded in the [ >>> best practice documents <<<](docs/best_practice.md).


#### Full body/image Generation
### Full body/image Generation

Using `--still` to generate a natural full body video. You can add `enhancer` to improve the quality of the generated video.

Expand All @@ -176,16 +209,6 @@ python inference.py --driven_audio <audio.wav> \
--enhancer gfpgan
```

#### Local Gradio demo.
A local gradio demo similar to our [hugging-face demo](https://huggingface.co/spaces/vinthony/SadTalker) can be run by:

```bash

## you need manually install TTS(https://github.com/coqui-ai/TTS) via `pip install tts` in advanced.

python app.py
```


## 🛎 Citation

Expand Down
197 changes: 197 additions & 0 deletions launcher.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
# this scripts installs necessary requirements and launches main program in webui.py
# borrow from : https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/launch.py
import subprocess
import os
import sys
import importlib.util
import shlex
import platform
import json

python = sys.executable
git = os.environ.get('GIT', "git")
index_url = os.environ.get('INDEX_URL', "")
stored_commit_hash = None
skip_install = False
dir_repos = "repositories"
script_path = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))

if 'GRADIO_ANALYTICS_ENABLED' not in os.environ:
os.environ['GRADIO_ANALYTICS_ENABLED'] = 'False'


def check_python_version():
is_windows = platform.system() == "Windows"
major = sys.version_info.major
minor = sys.version_info.minor
micro = sys.version_info.micro

if is_windows:
supported_minors = [10]
else:
supported_minors = [7, 8, 9, 10, 11]

if not (major == 3 and minor in supported_minors):

raise (f"""
INCOMPATIBLE PYTHON VERSION
This program is tested with 3.10.6 Python, but you have {major}.{minor}.{micro}.
If you encounter an error with "RuntimeError: Couldn't install torch." message,
or any other error regarding unsuccessful package (library) installation,
please downgrade (or upgrade) to the latest version of 3.10 Python
and delete current Python and "venv" folder in WebUI's directory.
You can download 3.10 Python from here: https://www.python.org/downloads/release/python-3109/
{"Alternatively, use a binary release of WebUI: https://github.com/AUTOMATIC1111/stable-diffusion-webui/releases" if is_windows else ""}
Use --skip-python-version-check to suppress this warning.
""")


def commit_hash():
global stored_commit_hash

if stored_commit_hash is not None:
return stored_commit_hash

try:
stored_commit_hash = run(f"{git} rev-parse HEAD").strip()
except Exception:
stored_commit_hash = "<none>"

return stored_commit_hash


def run(command, desc=None, errdesc=None, custom_env=None, live=False):
if desc is not None:
print(desc)

if live:
result = subprocess.run(command, shell=True, env=os.environ if custom_env is None else custom_env)
if result.returncode != 0:
raise RuntimeError(f"""{errdesc or 'Error running command'}.
Command: {command}
Error code: {result.returncode}""")

return ""

result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, env=os.environ if custom_env is None else custom_env)

if result.returncode != 0:

message = f"""{errdesc or 'Error running command'}.
Command: {command}
Error code: {result.returncode}
stdout: {result.stdout.decode(encoding="utf8", errors="ignore") if len(result.stdout)>0 else '<empty>'}
stderr: {result.stderr.decode(encoding="utf8", errors="ignore") if len(result.stderr)>0 else '<empty>'}
"""
raise RuntimeError(message)

return result.stdout.decode(encoding="utf8", errors="ignore")


def check_run(command):
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
return result.returncode == 0


def is_installed(package):
try:
spec = importlib.util.find_spec(package)
except ModuleNotFoundError:
return False

return spec is not None


def repo_dir(name):
return os.path.join(script_path, dir_repos, name)


def run_python(code, desc=None, errdesc=None):
return run(f'"{python}" -c "{code}"', desc, errdesc)


def run_pip(args, desc=None):
if skip_install:
return

index_url_line = f' --index-url {index_url}' if index_url != '' else ''
return run(f'"{python}" -m pip {args} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}")


def check_run_python(code):
return check_run(f'"{python}" -c "{code}"')


def git_clone(url, dir, name, commithash=None):
# TODO clone into temporary dir and move if successful

if os.path.exists(dir):
if commithash is None:
return

current_hash = run(f'"{git}" -C "{dir}" rev-parse HEAD', None, f"Couldn't determine {name}'s hash: {commithash}").strip()
if current_hash == commithash:
return

run(f'"{git}" -C "{dir}" fetch', f"Fetching updates for {name}...", f"Couldn't fetch {name}")
run(f'"{git}" -C "{dir}" checkout {commithash}', f"Checking out commit for {name} with hash: {commithash}...", f"Couldn't checkout commit {commithash} for {name}")
return

run(f'"{git}" clone "{url}" "{dir}"', f"Cloning {name} into {dir}...", f"Couldn't clone {name}")

if commithash is not None:
run(f'"{git}" -C "{dir}" checkout {commithash}', None, "Couldn't checkout {name}'s hash: {commithash}")


def git_pull_recursive(dir):
for subdir, _, _ in os.walk(dir):
if os.path.exists(os.path.join(subdir, '.git')):
try:
output = subprocess.check_output([git, '-C', subdir, 'pull', '--autostash'])
print(f"Pulled changes for repository in '{subdir}':\n{output.decode('utf-8').strip()}\n")
except subprocess.CalledProcessError as e:
print(f"Couldn't perform 'git pull' on repository in '{subdir}':\n{e.output.decode('utf-8').strip()}\n")


def run_extension_installer(extension_dir):
path_installer = os.path.join(extension_dir, "install.py")
if not os.path.isfile(path_installer):
return

try:
env = os.environ.copy()
env['PYTHONPATH'] = os.path.abspath(".")

print(run(f'"{python}" "{path_installer}"', errdesc=f"Error running install.py for extension {extension_dir}", custom_env=env))
except Exception as e:
print(e, file=sys.stderr)


def prepare_environment():
global skip_install

torch_command = os.environ.get('TORCH_COMMAND', "pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117")
requirements_file = os.environ.get('REQS_FILE', "requirements.txt")

commit = commit_hash()

print(f"Python {sys.version}")
print(f"Commit hash: {commit}")

if not is_installed("torch") or not is_installed("torchvision"):
run(f'"{python}" -m {torch_command}', "Installing torch and torchvision", "Couldn't install torch", live=True)

run_python("import torch; assert torch.cuda.is_available(), 'Torch is not able to use GPU; add --skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check'")

run_pip(f"install -r \"{requirements_file}\"", "requirements for SadTalker WebUI (may take longer time in first time)")


def start():
print(f"Launching SadTalker Web UI")
from app import sadtalker_demo
demo = sadtalker_demo()
demo.launch(share=True)

if __name__ == "__main__":
prepare_environment()
start()
5 changes: 2 additions & 3 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
numpy==1.23.4
face_alignment==1.3.5
imageio==2.19.3
imageio-ffmpeg==0.4.7
librosa==0.9.2 #
numba
resampy==0.3.1
pydub==0.25.1
scipy==1.5.3
scipy==1.10.1
kornia==0.6.8
tqdm
yacs==0.1.8
pyyaml
joblib==1.1.0
scikit-image==0.19.3
basicsr==1.4.2
facexlib==0.2.5
facexlib==0.3.0
gradio
gfpgan
dlib-bin
Expand Down
2 changes: 1 addition & 1 deletion src/utils/croper.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ def crop(self, img_np_list, still=False, xsize=512): # first frame for all vi
img_np = img_np_list[0]
lm = self.get_landmark(img_np)
if lm is None:
return None
raise 'can not detect the landmark from source image'
rsize, crop, quad = self.align_face(img=Image.fromarray(img_np), lm=lm, output_size=xsize)
clx, cly, crx, cry = crop
lx, ly, rx, ry = quad
Expand Down
17 changes: 17 additions & 0 deletions start.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
@echo off

IF NOT EXIST venv (
python -m venv venv
) ELSE (
echo venv folder already exists, skipping creation...
)
call .\venv\Scripts\activate.bat

set PYTHON="venv\Scripts\Python.exe"
echo venv %PYTHON%

%PYTHON% Launcher.py

echo.
echo Launch unsuccessful. Exiting.
pause

0 comments on commit 3bea177

Please sign in to comment.