LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking

💡 Notice

Applying LORS to AdaMixer. The code is implemented as a python class called FactorizedAdaptiveMixing, located in the file AdaMixer_with_LORS/mmdet/models/roi_heads/bbox_heads/factorized_adaptive_mixing_operator.py.
Applying LORS to Transformer(Deit-Tiny). The code consists of python class MultiheadAttention_LORS and FFN_LORS, implemented in the file DeiT-Tiny_with_LORS/mmcls/models/backbones/lors_vit.py.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
.asset		.asset
AdaMixer_with_LORS		AdaMixer_with_LORS
DeiT-Tiny_with_LORS		DeiT-Tiny_with_LORS
README.md		README.md
requirements.txt		requirements.txt