This repository contains the official codes for MLS-BRN (CVPR 2024), our multi-level supervised building reconstruction network that can flexibly utilize training samples with different annotation levels.
- We design MLS-BRN, a multi-level supervised building reconstruction network, which consists of new tasks and modules to enhance the relation between different components of a building instance and alleviate the demand on 3D annotations.
- We propose a multi-level training strategy that enables the training of MLS-BRN with different supervision levels to further improve the 3D reconstruction performance.
- We extend the monocular building reconstruction datasets to more cities. Comprehensive experiments under different settings demonstrate the potential of MLS-BRN in large-scale cross-city scenarios.
Please check out our paper for further details.
We inherit the environement of BONAI, and here is a reference to deploy it:
# create & activate environment
conda create -n mlsbrn python=3.8
conda activate mlsbrn
# install pytorch-1.11.0
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113
# install dependency packages
pip install mmcv-full==1.7.0 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.11.0/index.html
cd MLS-BRN/
pip install -v -e .
pip install yapf==0.40.1
# install wwtool package for evaluate code
git clone https://github.com/jwwangchn/wwtool.git
cd wwtool
python setup.py develop
# install bstool package for evaluate code
git clone https://github.com/Hoteryoung/bstool.git
cd bstool
git pull origin modify_for_loft-foa-fro
git checkout modify_for_loft-foa-fro
python setup.py developPlease download BONAI and our proposed dataset，then put the datasets into one directory and specify the directory as data_root variable in configs/_base_/datasets/bonai_instance_hfm_ssl.py.
The config files are defined in configs/_base_/models/bonai_loft_foahfm_r50_fpn_basic.py and configs/_base_/schedules/schedule_2x_bonai.py. We provide shell scripts for training and test in tools/.
To train or test the model in different environments, modify the given shell script and config files accordingly.
Note: you need to specify the dataset as CITY variable in tools/dist_test.sh when testing.
cd MLS-BRN/
# for non-slurm system
# train
./tools/dist_train.sh loft_foahfm_ssl loft_foahfm_r50_fpn_2x_bonai_ssl
# resume training from a checkpoint
./tools/dist_train.sh loft_foahfm_ssl loft_foahfm_r50_fpn_2x_bonai_ssl --resume-from='path to checkpoint'
# test & evaluate, <timestamp> refers to the timestamp of the training results folder in ./work_dirs/
./tools/dist_test.sh loft_foahfm_r50_fpn_2x_bonai_ssl <timestamp>
# for slurm system
# train
./tools/slurm_train.sh loft_foahfm_ssl loft_foahfm_r50_fpn_2x_bonai_ssl
# test & evaluate
./tools/slurm_test.sh loft_foahfm_r50_fpn_2x_bonai_ssl <timestamp>If you use our dataset, codebase or models in your research, please consider cite.
@InProceedings{Li_2024_CVPR,
author = {Li, Weijia and Yang, Haote and Hu, Zhenghao and Zheng, Juepeng and Xia, Gui-Song and He, Conghui},
title = {3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024},
pages = {27728-27737}
}