Skip to content

Commit

Permalink
list of papers and readme update (facebookresearch#663)
Browse files Browse the repository at this point in the history
  • Loading branch information
mhmukadam authored Sep 16, 2024
1 parent e160076 commit 3077acb
Show file tree
Hide file tree
Showing 3 changed files with 34 additions and 4 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,20 @@

<p align="center">
<a href="https://arxiv.org/abs/2207.09442">Paper</a> •
<a href="https://ai.facebook.com/blog/theseus-a-library-for-encoding-domain-knowledge-in-end-to-end-ai-models/">Blog</a> •
<a href="https://youtu.be/zIbPHXegNn8">Video</a> •
<a href="https://x.com/mukadammh/status/1549773856738615297">Twitter</a> •
<a href="https://sites.google.com/view/theseus-ai/">Webpage</a> •
<a href="https://github.com/facebookresearch/theseus/tree/main/tutorials">Tutorials</a> •
<a href="https://theseus-ai.readthedocs.io/">Docs</a>
<a href="https://github.com/facebookresearch/theseus/tree/main/tutorials">Tutorials</a>
</p>

Theseus is an efficient application-agnostic library for building custom nonlinear optimization layers in PyTorch to support constructing various problems in robotics and vision as end-to-end differentiable architectures.

![](https://raw.githubusercontent.com/facebookresearch/theseus/main/docs/source/img/theseuslayer.png)
![](https://raw.githubusercontent.com/facebookresearch/theseus/main/docs/source/img/theseuslayer.gif)

Differentiable nonlinear optimization provides a general scheme to encode inductive priors, as the objective function can be partly parameterized by neural models and partly with expert domain-specific differentiable models. The ability to compute gradients end-to-end is retained by differentiating through the optimizer which allows neural models to train on the final task loss, while also taking advantage of priors captured by the optimizer.

See [list of papers](https://github.com/facebookresearch/theseus/blob/main/papers.md) published using Theseus for examples across various application domains.

-----

## Current Features
Expand Down
Binary file added docs/source/img/theseuslayer.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 28 additions & 0 deletions papers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Non-exhaustive list of papers using Theseus

* (2024 Jul) [\[2407.18569\] PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learning](https://arxiv.org/abs/2407.18569)
* (2024 Jul) [\[2407.20223\] Correspondence-Free SE(3) Point Cloud Registration in RKHS via Unsupervised Equivariant Learning](https://arxiv.org/abs/2407.20223)
* (2024 May) [\[2405.19309\] SDPRLayers: Certifiable Backpropagation Through Polynomial Optimization Problems in Robotics](https://arxiv.org/abs/2405.19309)
* (2024 Feb) [\[2402.05421\] DiffTOP: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learning](https://arxiv.org/abs/2402.05421)
* (2024 Jan) [\[2401.15036\] Distributed Simultaneous Localisation and Auto-Calibration using Gaussian Belief Propagation](https://arxiv.org/abs/2401.15036)
* (2024 Jan) [\[2401.10685\] Towards End-to-End GPS Localization with Neural Pseudorange Correction](https://arxiv.org/abs/2401.10685)
* (2023 Dec) [\[2312.04563\] Visual Geometry Grounded Deep Structure From Motion](https://arxiv.org/abs/2312.04563)
* (2023 Dec) [\[2312.13469\] Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation](https://arxiv.org/abs/2312.13469)
* (2023 Dec) [Learning‐enabled decision‐making for autonomous driving: framework and methodology](https://dr.ntu.edu.sg/handle/10356/172842)
* (2023 Dec) [\[2312.09138\] Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments](https://arxiv.org/abs/2312.09138)
* (2023 Oct) [Monocular 3D Object Detection with Viewpoint-Invariant Inter-Object Estimation for Better Contextual Behavior Understanding](https://ippc-iros23.github.io/papers/minghan_zhu.pdf)
* (2023 Oct) [\[2310.06903\] Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimization](https://arxiv.org/abs/2310.06903)
* (2023 Sep) [Equivariant and Geometry-Aware 3D Perception for Robots and Automated Vehicles](https://deepblue.lib.umich.edu/handle/2027.42/177794)
* (2023 Sep) [Revisiting Monocular Visual Odometry from Downward Facing Cameras](https://etd.ohiolink.edu/acprod/odb_etd/ws/send_file/send?accession=osu1682013681124081&disposition=inline)
* (2023 Aug) [\[2308.16539\] On a Connection between Differential Games, Optimal Control, and Energy-based Models for Multi-Agent Interactions](https://arxiv.org/abs/2308.16539)
* (2023 Jun) [GameFormer Planner: A Learning-enabled Interactive Prediction and Planning Framework for Autonomous Vehicles](https://opendrivelab.com/e2ead/AD23Challenge/Track_4_AID.pdf)
* (2023 May) [Exploring Safe Reinforcement Learning for Sequential Decision Making](https://www.ri.cmu.edu/app/uploads/2023/05/MSR_thesis_Fan_Yang.pdf)
* (2023 May) [\[2305.09527\] Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares](https://arxiv.org/abs/2305.09527)
* (2022 Nov) [\[2211.02647\] Neural Grasp Distance Fields for Robot Manipulation](https://arxiv.org/abs/2211.02647)
* (2022 Oct) [\[2210.14210\] MidasTouch: Monte-Carlo inference over distributions across sliding touch](https://arxiv.org/abs/2210.14210)
* (2022 Sep) [\[2209.15428\] PyPose: A Library for Robot Learning with Physics-based Optimization](https://arxiv.org/abs/2209.15428)
* (2022 Sep) [Taking an Electoral Photograph with Neural Networks](https://pub.tik.ee.ethz.ch/students/2022-FS/SA-2022-23.pdf)
* (2022 Sep) [\[2209.03855\] SE(3)-DiffusionFields: Learning cost functions for joint grasp and motion optimization through diffusion](https://arxiv.org/abs/2209.03855)
* (2022 Jul) [\[2207.10422\] Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving](https://arxiv.org/abs/2207.10422)

Please submit a PR to get your paper that uses Theseus added to this list.

0 comments on commit 3077acb

Please sign in to comment.