Coordinate_based_MLPs

Intro

Experiments of coordinate-based MLPs based on Pytorch-lightning

graph LR
	1[x]-->3[P.E.]-->4[Linear layers + activation<br>256]-->5[Linear layers + activation<br>256]-->6[Linear layers + activation<br>256]-->7[Linear layers + Sgmoid<br>256]
	2[y]-->3[P.E.]
	7-->8[R]
	7-->9[G]
	7-->10[B]

Positional Encoding	Equation
Fourier feature mapping	$\gamma(v) = [..., \text{cos}(2\pi \sigma^{\frac{j}{m}}v), \text{sin}(2\pi \sigma^{\frac{j}{m}}v), ...]^T \ \text{for j in } [0, m-1]$
Fourier feature mapping (Gaussian distribution)	$\gamma(v) = [..., \text{cos}(2\pi Bv), \text{sin}(2\pi Bv), ...]^T \text{where each entry in B} \in \mathbb{R}^{m\times d} \text{is sampled from }\mathcal{N}(0, \sigma^2)$

Activation function	Equation
ReLU	$\text{max}(0,x)$
Siren	$\text{sin}(w_0\times W_x + b)$
Gaussian	$e^{\frac{-0.5x^2}{a^2}}$
Quadratic	$\frac{1}{1+(ax)^2}$
Multi Quadratic	$\frac{1}{\sqrt{1+(ax)^2}}$
Laplacian	$e^{\frac{-\lvert x \rvert}{a}}$
Super-Gaussian	$\left [ e^{\frac{-0.5x^2}{a^2}}\right ]^b$
ExpSin	$e^{-\text{sin}(ax)}$

How to run?

Data preparation

build a directory "data/"
make sure your own images put in "data/"
Data used in My Experiment: Pluto image: NASA

Run single experiment

# raw MLPs with RuLU activation function without positional encoding
python train.py --arch=relu --use_pe=False --exp_name=raw_mlps_800*800_1024

Run all experiments at once

# run exp with defualt setting: Image_wh=800*800 batch_size=1024
bash exp.sh
# run exp with setting: Image_wh=800*800 batch_size=800*800
bash exp_640000.sh

Experiment logs

ReLU P.E.(Fourier Mapping)

Without Positional Encoding

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	33.262	21.601

With Positional Encoding

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	22.405	16.957

ReLU Fourier Mapping( Gaussian distribution)

2π

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	31.523	24.383

20π

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	30.004	25.222

200π

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	8.585	7.146

Siren activation

As Siren dependent on quality of initialization, in this experiment I didn't initialize it specially, so the outputs are bad.

omega_0 = 30

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	8.767	22.326

Gaussian activation

a=0.1 with P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	8.36	8.61

a=0.1 without P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	30.677	23.899

Super-Gaussian activation

a=0.1 b=2 with P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	8.347	9.494

a=0.1 b=2 without P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	30.644	24.183

ExpSin activation

a=0.1 with P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	9.589	9.259

a=01 without P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	24.268	9.204

Laplacian activation

a=0.1 with P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	8.273	9.268

a=0.1 without P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	31.283	24.006

Quadratic activation

a=10 with P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	8.417	8.572

a=10 without P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	31.222	23.861

Multi-Quadratic activation

a=20 with P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	8.448	8.846

a=20 without P.E.

	Image size 800*800, batch size 1024	Image size 800800, batch size 800800

MAX_PSNR	30.355	23.944

Related papers

Acknowledgment

Thanks for kwea123's wonderful live stream and his repo

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
images		images
README.md		README.md
dataset.py		dataset.py
exp.sh		exp.sh
exp_640000.sh		exp_640000.sh
metrics.py		metrics.py
models.py		models.py
opt.py		opt.py
requirements.txt		requirements.txt
train.py		train.py

fusheng-ji/Coordinate_based_MLPs

Folders and files

Latest commit

History

Repository files navigation

Coordinate_based_MLPs

Intro

How to run?

Data preparation

Run single experiment

Run all experiments at once

Experiment logs

ReLU P.E.(Fourier Mapping)

Without Positional Encoding

With Positional Encoding

ReLU Fourier Mapping( Gaussian distribution)

2π

20π

200π

Siren activation

omega_0 = 30

Gaussian activation

a=0.1 with P.E.

a=0.1 without P.E.

Super-Gaussian activation

a=0.1 b=2 with P.E.

a=0.1 b=2 without P.E.

ExpSin activation

a=0.1 with P.E.

a=01 without P.E.

Laplacian activation

a=0.1 with P.E.

a=0.1 without P.E.

Quadratic activation

a=10 with P.E.

a=10 without P.E.

Multi-Quadratic activation

a=20 with P.E.

a=20 without P.E.

Related papers

Acknowledgment

About

Resources

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages