Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial implicit/truncated backward modes #29

Merged
merged 14 commits into from
Jan 19, 2022
Merged

Initial implicit/truncated backward modes #29

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
da2b4ff
Initial WIP commit of implicit/truncated backward modes
bamos Dec 12, 2021
0894d6a
spacing
bamos Dec 12, 2021
9fd8e0b
add numdifftools requirement
bamos Dec 12, 2021
ada3fc5
fix mypy and GPU issues
bamos Jan 11, 2022
565129b
import BackwardMode as part of the main thesus module
bamos Jan 11, 2022
fd7d42e
add ValueError messages
bamos Jan 11, 2022
656587d
add comments to backward_modes and add it to examples/README
bamos Jan 11, 2022
cd5a311
Remove error_increase_induces
bamos Jan 11, 2022
58c5610
move converged_indices from the info back into the optimizaiton loop
bamos Jan 11, 2022
64ecfc6
Merge branch 'main' into diff
bamos Jan 11, 2022
f9cb599
fix gradient scaling for #39
bamos Jan 19, 2022
41e7b46
update backward tests
bamos Jan 19, 2022
9e61d13
add type hints/remove unused track_best_solution
bamos Jan 19, 2022
f58345f
remove erroneous update
bamos Jan 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update backward tests
  • Loading branch information
@bamos
bamos committed Jan 19, 2022
commit 41e7b46afc5ab20fe320736517eb4197e0789aa6
23 changes: 11 additions & 12 deletions theseus/optimizer/nonlinear/tests/test_backwards.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,17 +90,16 @@ def fit_x(data_x_np):
].squeeze()
assert torch.allclose(da_dx_numeric, da_dx_full, atol=1e-3)

# TODO: Debug a little more
# updated_inputs, info = theseus_optim.forward(
# theseus_inputs,
# track_best_solution=True, verbose=False,
# backward_mode=th.BackwardMode.IMPLICIT,
# )
# da_dx_implicit = torch.autograd.grad(
# updated_inputs['a'], data_x,
# retain_graph=True)[0].squeeze()
# EPS=1e-3
# assert torch.allclose(da_dx_numeric, da_dx_implicit, atol=EPS)
updated_inputs, info = theseus_optim.forward(
theseus_inputs,
track_best_solution=True,
verbose=False,
backward_mode=th.BackwardMode.IMPLICIT,
)
da_dx_implicit = torch.autograd.grad(
updated_inputs["a"], data_x, retain_graph=True
)[0].squeeze()
assert torch.allclose(da_dx_numeric, da_dx_implicit, atol=1e-4)

updated_inputs, info = theseus_optim.forward(
theseus_inputs,
Expand All @@ -112,7 +111,7 @@ def fit_x(data_x_np):
da_dx_truncated = torch.autograd.grad(
updated_inputs["a"], data_x, retain_graph=True
)[0].squeeze()
assert torch.allclose(da_dx_numeric, da_dx_truncated, atol=1e-1)
assert torch.allclose(da_dx_numeric, da_dx_truncated, atol=1e-4)


test_backwards()