This is an experimental Tensorflow implementation of MV3D - a ConvNet for object detection with Lidar and Mono-camera. And this work based on the code of hengck23
For details about MV3D please refer to the paper Multi-View 3D Object Detection Network for Autonomous Driving by Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, Tian Xia.
-
Requirements for Tensorflow 1.0 (see: Tensorflow)
-
Python packages you might not have:
cython,python-opencv,easydict,mayavi (for visualization)
- For training the end-to-end version of Faster R-CNN with VGG16 and ResNet50, 8G of GPU memory is sufficient (using CUDNN)
- 使用https://repo.continuum.io/archive/Anaconda3-4.3.1-Linux-x86_64.sh ；ref : https://www.continuum.io/downloads#linux install
conda create -n tensorflow python=3.5
source activate tensorflow
conda install -c menpo opencv3=3.2.0
conda install matplotlib simplejson pandas
cuda install: https://developer.nvidia.com/cuda-downloads
按照python35 tensorflow gpu: ref https://www.tensorflow.org/install/install_linux 安装这里要求的cudnn版本 https://developer.nvidia.com/rdp/cudnn-download
pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.0.1-cp35-cp35m-linux_x86_64.whl
conda install -c menpo mayavi
conda install pyqt=4
pip install easydict
pip install pillow
roi_pooling.so不同机器需重新编译；
cd $MV3D/net/roipooling_op/
目录下有make.sh 编译;查看 make.sh 文件；使用 /usr/local/cuda/bin/nvcc 80版本nvcc编译;- Downloads KITTI object datasets.
% Specify KITTI data path so that the structure is like
% {kitti_dir}/object/training/image_2
% /image_3
% /calib
% /lidar_bv
% /velodyne
% {kitti_dir}/object/testing/image_2
% /image_3
% /calib
% /lidar_bv
% /velodyne- Make Lidar top View data %Edit your data path:
vim $MV3D/net/common.py:
kitti_dir = "{kitti_dir}/object/training"
train_data_root= 'your path to save the top view data' %Make data
cd $MV3D/data
python generate_top_view_data.py% Generate groundtruth file
cd $MV3D/data
python generate_gt.py
- Download pre-trained ImageNet models Download the pre-trained ImageNet models tensorflow checkpoint vgg16 and ResNet V1 50
mv resnet_v1_50.ckpt $MV3D/outputs/check_points/resnet_v1_50.ckpt
mv vgg_16.ckpt $MV3D/outputs/check_points/vgg_16.ckpt- Run script to train model
cd $MV3D
python ./tools/train_ResNet_vgg_double_up_c.pyDEVICE is either cpu/gpu
Image and corresponding Lidar map
part.2: Didi Udacity Challenge 2017 — Car and pedestrian Detection using Lidar and RGB