Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundle adjustment Not converged #8

Closed
mwsunshine opened this issue Jun 4, 2019 · 5 comments
Closed

Bundle adjustment Not converged #8

mwsunshine opened this issue Jun 4, 2019 · 5 comments

Comments

@mwsunshine
Copy link

hi, authors, many thanks for your impressive work and open-source code!

I have followed your instructions and got some amazing outputs(videos). however, for some other input images test cases, I got some errors.

By using imgs2poses.py, the bundle adjustment report in the file colmap_output.txt says No convergence and all the other images could not register neither.

I got 24 input images(the other test case 20 images), and there should be plenty of features within the images.

the colmap_output.txt is like this:

==============================================================================
Feature extraction

Processed file [1/24]
Name: 01.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7604
Processed file [2/24]
Name: 02.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7460
Processed file [3/24]
Name: 03.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7671
Processed file [4/24]
Name: 04.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7636
Processed file [5/24]
Name: 05.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 8000
Processed file [6/24]
Name: 06.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 8334
Processed file [7/24]
Name: 07.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7943
Processed file [8/24]
Name: 08.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7749
Processed file [9/24]
Name: 09.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 8346
Processed file [10/24]
Name: 10.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 8143
Processed file [11/24]
Name: 11.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7826
Processed file [12/24]
Name: 12.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 5609
Processed file [13/24]
Name: 13.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 6984
Processed file [14/24]
Name: 14.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7445
Processed file [15/24]
Name: 15.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 6511
Processed file [16/24]
Name: 16.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7683
Processed file [17/24]
Name: 17.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 8220
Processed file [18/24]
Name: 18.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 8505
Processed file [19/24]
Name: 19.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 9363
Processed file [20/24]
Name: 20.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 6798
Processed file [21/24]
Name: 21.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7544
Processed file [22/24]
Name: 22.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 5691
Processed file [23/24]
Name: 23.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7068
Processed file [24/24]
Name: 24.jpg
Dimensions: 1440 x 1080
Camera: #1 - SIMPLE_RADIAL
Focal Length: 1728.00px
Features: 7409
Elapsed time: 0.041 [minutes]

==============================================================================
Exhaustive feature matching

Matching block [1/1, 1/1] in 6.299s
Elapsed time: 0.109 [minutes]

==============================================================================
Loading database

Loading cameras... 1 in 0.000s
Loading matches... 276 in 0.020s
Loading images... 24 in 0.027s (connected 24)
Building correspondence graph... in 0.108s (ignored 0)

Elapsed time: 0.003 [minutes]

==============================================================================
Initializing with image pair #10 and #5

==============================================================================
Global bundle adjustment

iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 3.135909e+03 0.00e+00 1.41e+05 0.00e+00 0.00e+00 1.00e+04 0 1.75e-03 6.96e-03
1 3.892343e+03 -7.56e+02 0.00e+00 4.13e+01 -3.03e-01 5.00e+03 1 3.41e-03 1.04e-02
2 2.438515e+03 6.97e+02 1.06e+05 3.85e+01 2.86e-01 4.64e+03 1 4.46e-03 1.49e-02
3 6.548220e+02 1.78e+03 4.14e+03 5.68e+00 9.99e-01 1.39e+04 1 4.03e-03 1.89e-02
4 6.482664e+02 6.56e+00 1.20e+03 5.05e+00 9.94e-01 4.17e+04 1 3.28e-03 2.22e-02
5 6.452768e+02 2.99e+00 1.27e+03 1.75e+01 8.18e-01 5.61e+04 1 3.35e-03 2.56e-02
6 6.423903e+02 2.89e+00 1.93e+03 2.31e+01 5.51e-01 5.62e+04 1 3.35e-03 2.90e-02
7 6.377616e+02 4.63e+00 1.60e+03 2.26e+01 7.00e-01 6.00e+04 1 3.31e-03 3.23e-02
8 6.337074e+02 4.05e+00 1.75e+03 2.35e+01 6.49e-01 6.17e+04 1 3.68e-03 3.60e-02
9 6.294490e+02 4.26e+00 1.83e+03 2.36e+01 6.75e-01 6.44e+04 1 6.69e-03 4.27e-02
10 6.254248e+02 4.02e+00 1.99e+03 2.40e+01 6.59e-01 6.66e+04 1 3.82e-03 4.66e-02
11 6.214221e+02 4.00e+00 2.10e+03 2.43e+01 6.64e-01 6.90e+04 1 5.85e-03 5.25e-02
12 6.175485e+02 3.87e+00 2.23e+03 2.46e+01 6.58e-01 7.12e+04 1 6.20e-03 5.88e-02
13 6.137530e+02 3.80e+00 2.33e+03 2.48e+01 6.57e-01 7.35e+04 1 3.55e-03 6.23e-02
14 6.100624e+02 3.69e+00 2.43e+03 2.51e+01 6.53e-01 7.57e+04 1 3.46e-03 6.58e-02
15 6.064668e+02 3.60e+00 2.52e+03 2.53e+01 6.52e-01 7.78e+04 1 4.29e-03 7.01e-02
16 6.029737e+02 3.49e+00 2.61e+03 2.54e+01 6.49e-01 7.99e+04 1 3.60e-03 7.38e-02
17 5.995826e+02 3.39e+00 2.68e+03 2.56e+01 6.47e-01 8.20e+04 1 3.33e-03 7.71e-02
18 5.962960e+02 3.29e+00 2.74e+03 2.57e+01 6.45e-01 8.41e+04 1 3.59e-03 8.07e-02
19 5.931150e+02 3.18e+00 2.79e+03 2.57e+01 6.43e-01 8.61e+04 1 3.38e-03 8.41e-02
20 5.900405e+02 3.07e+00 2.84e+03 2.58e+01 6.41e-01 8.81e+04 1 3.50e-03 8.76e-02
21 5.870732e+02 2.97e+00 2.88e+03 2.58e+01 6.40e-01 9.00e+04 1 3.42e-03 9.11e-02
22 5.842132e+02 2.86e+00 2.90e+03 2.58e+01 6.38e-01 9.20e+04 1 3.44e-03 9.45e-02
23 5.814602e+02 2.75e+00 2.93e+03 2.57e+01 6.37e-01 9.39e+04 1 3.67e-03 9.82e-02
24 5.788136e+02 2.65e+00 2.94e+03 2.56e+01 6.35e-01 9.58e+04 1 6.73e-03 1.05e-01
25 5.762724e+02 2.54e+00 2.94e+03 2.55e+01 6.34e-01 9.77e+04 1 3.61e-03 1.09e-01
26 5.738352e+02 2.44e+00 2.94e+03 2.54e+01 6.33e-01 9.96e+04 1 6.27e-03 1.15e-01
27 5.715003e+02 2.33e+00 2.94e+03 2.52e+01 6.32e-01 1.01e+05 1 3.35e-03 1.18e-01
28 5.692657e+02 2.23e+00 2.92e+03 2.50e+01 6.31e-01 1.03e+05 1 3.17e-03 1.21e-01
29 5.671292e+02 2.14e+00 2.91e+03 2.48e+01 6.30e-01 1.05e+05 1 3.16e-03 1.25e-01
30 5.650882e+02 2.04e+00 2.88e+03 2.45e+01 6.29e-01 1.07e+05 1 4.22e-03 1.29e-01
31 5.631401e+02 1.95e+00 2.86e+03 2.42e+01 6.29e-01 1.09e+05 1 6.12e-03 1.35e-01
32 5.612820e+02 1.86e+00 2.83e+03 2.39e+01 6.28e-01 1.11e+05 1 7.77e-03 1.43e-01
33 5.595109e+02 1.77e+00 2.79e+03 2.36e+01 6.27e-01 1.13e+05 1 3.59e-03 1.46e-01
34 5.578237e+02 1.69e+00 2.75e+03 2.33e+01 6.27e-01 1.14e+05 1 3.35e-03 1.50e-01
35 5.562172e+02 1.61e+00 2.71e+03 2.29e+01 6.26e-01 1.16e+05 1 3.34e-03 1.53e-01
36 5.546881e+02 1.53e+00 2.67e+03 2.25e+01 6.26e-01 1.18e+05 1 3.27e-03 1.56e-01
37 5.532332e+02 1.45e+00 2.62e+03 2.21e+01 6.25e-01 1.20e+05 1 3.31e-03 1.60e-01
38 5.518493e+02 1.38e+00 2.57e+03 2.17e+01 6.25e-01 1.22e+05 1 3.53e-03 1.63e-01
39 5.505330e+02 1.32e+00 2.52e+03 2.13e+01 6.25e-01 1.24e+05 1 3.65e-03 1.67e-01
40 5.492811e+02 1.25e+00 2.47e+03 2.08e+01 6.25e-01 1.26e+05 1 3.54e-03 1.71e-01
41 5.480904e+02 1.19e+00 2.42e+03 2.04e+01 6.25e-01 1.28e+05 1 3.46e-03 1.74e-01
42 5.469577e+02 1.13e+00 2.37e+03 1.99e+01 6.25e-01 1.30e+05 1 3.32e-03 1.77e-01
43 5.458800e+02 1.08e+00 2.31e+03 1.95e+01 6.25e-01 1.32e+05 1 3.22e-03 1.81e-01
44 5.448544e+02 1.03e+00 2.26e+03 1.90e+01 6.26e-01 1.34e+05 1 3.22e-03 1.84e-01
45 5.438778e+02 9.77e-01 2.21e+03 1.85e+01 6.26e-01 1.36e+05 1 3.36e-03 1.87e-01
46 5.429474e+02 9.30e-01 2.15e+03 1.80e+01 6.27e-01 1.39e+05 1 3.38e-03 1.91e-01
47 5.420605e+02 8.87e-01 2.10e+03 1.76e+01 6.27e-01 1.41e+05 1 3.25e-03 1.94e-01
48 5.412145e+02 8.46e-01 2.05e+03 1.71e+01 6.28e-01 1.43e+05 1 3.35e-03 1.97e-01
49 5.404067e+02 8.08e-01 2.00e+03 1.66e+01 6.30e-01 1.46e+05 1 3.28e-03 2.00e-01
50 5.396346e+02 7.72e-01 1.95e+03 1.62e+01 6.31e-01 1.48e+05 1 3.31e-03 2.04e-01
51 5.388960e+02 7.39e-01 1.90e+03 1.57e+01 6.32e-01 1.51e+05 1 3.36e-03 2.07e-01
52 5.381884e+02 7.08e-01 1.85e+03 1.52e+01 6.34e-01 1.54e+05 1 3.94e-03 2.11e-01
53 5.375097e+02 6.79e-01 1.80e+03 1.48e+01 6.36e-01 1.57e+05 1 3.37e-03 2.15e-01
54 5.368576e+02 6.52e-01 1.76e+03 1.43e+01 6.38e-01 1.61e+05 1 3.58e-03 2.18e-01
55 5.362300e+02 6.28e-01 1.72e+03 1.39e+01 6.41e-01 1.65e+05 1 3.37e-03 2.22e-01
56 5.356248e+02 6.05e-01 1.68e+03 1.35e+01 6.44e-01 1.69e+05 1 3.36e-03 2.25e-01
57 5.350400e+02 5.85e-01 1.64e+03 1.31e+01 6.47e-01 1.73e+05 1 3.40e-03 2.28e-01
58 5.344735e+02 5.66e-01 1.60e+03 1.27e+01 6.50e-01 1.78e+05 1 9.00e-03 2.37e-01
59 5.339232e+02 5.50e-01 1.56e+03 1.23e+01 6.54e-01 1.83e+05 1 4.21e-03 2.42e-01
60 5.333872e+02 5.36e-01 1.53e+03 1.19e+01 6.58e-01 1.89e+05 1 3.32e-03 2.45e-01
61 5.328631e+02 5.24e-01 1.50e+03 1.16e+01 6.63e-01 1.96e+05 1 3.29e-03 2.48e-01
62 5.323487e+02 5.14e-01 1.48e+03 1.13e+01 6.69e-01 2.04e+05 1 3.31e-03 2.52e-01
63 5.318415e+02 5.07e-01 1.45e+03 1.11e+01 6.75e-01 2.13e+05 1 3.36e-03 2.55e-01
64 5.313387e+02 5.03e-01 1.44e+03 1.09e+01 6.82e-01 2.24e+05 1 3.43e-03 2.58e-01
65 5.308372e+02 5.02e-01 1.42e+03 1.08e+01 6.89e-01 2.36e+05 1 3.42e-03 2.62e-01
66 5.303331e+02 5.04e-01 1.41e+03 1.07e+01 6.98e-01 2.52e+05 1 3.46e-03 2.65e-01
67 5.298219e+02 5.11e-01 1.40e+03 1.08e+01 7.08e-01 2.72e+05 1 3.34e-03 2.69e-01
68 5.292984e+02 5.24e-01 1.40e+03 1.11e+01 7.18e-01 2.96e+05 1 3.34e-03 2.72e-01
69 5.287569e+02 5.42e-01 1.39e+03 1.17e+01 7.27e-01 3.27e+05 1 3.51e-03 2.76e-01
70 5.281945e+02 5.62e-01 1.36e+03 1.25e+01 7.28e-01 3.61e+05 1 3.52e-03 2.79e-01
71 5.276208e+02 5.74e-01 1.26e+03 1.37e+01 7.07e-01 3.88e+05 1 3.42e-03 2.83e-01
72 5.270656e+02 5.55e-01 1.02e+03 1.48e+01 6.45e-01 3.98e+05 1 3.34e-03 2.86e-01
73 5.265535e+02 5.12e-01 6.44e+02 1.55e+01 5.55e-01 3.98e+05 1 3.34e-03 2.89e-01
74 5.261001e+02 4.53e-01 5.15e+02 1.61e+01 4.48e-01 3.98e+05 1 3.31e-03 2.93e-01
75 5.257127e+02 3.87e-01 6.49e+02 1.66e+01 3.38e-01 3.85e+05 1 3.49e-03 2.96e-01
76 5.252750e+02 4.38e-01 7.28e+02 1.67e+01 3.32e-01 3.71e+05 1 3.56e-03 3.00e-01
77 5.248302e+02 4.45e-01 7.83e+02 1.67e+01 3.14e-01 3.52e+05 1 3.46e-03 3.03e-01
78 5.243386e+02 4.92e-01 8.03e+02 1.65e+01 3.33e-01 3.40e+05 1 3.41e-03 3.07e-01
79 5.238930e+02 4.46e-01 8.30e+02 1.65e+01 3.04e-01 3.20e+05 1 3.34e-03 3.10e-01
80 5.233664e+02 5.27e-01 8.10e+02 1.60e+01 3.56e-01 3.13e+05 1 3.35e-03 3.13e-01
81 5.229668e+02 4.00e-01 8.73e+02 1.61e+01 2.86e-01 2.90e+05 1 3.40e-03 3.17e-01
82 5.223966e+02 5.70e-01 8.34e+02 1.54e+01 4.04e-01 2.88e+05 1 3.46e-03 3.20e-01
83 5.220552e+02 3.41e-01 8.91e+02 1.56e+01 2.72e-01 2.63e+05 1 3.39e-03 3.24e-01
84 5.214732e+02 5.82e-01 8.03e+02 1.46e+01 4.51e-01 2.63e+05 1 3.38e-03 3.27e-01
85 5.211545e+02 3.19e-01 8.47e+02 1.49e+01 2.93e-01 2.46e+05 1 3.42e-03 3.30e-01
86 5.206662e+02 4.88e-01 7.80e+02 1.42e+01 4.33e-01 2.45e+05 1 3.44e-03 3.34e-01
87 5.203502e+02 3.16e-01 8.10e+02 1.44e+01 3.17e-01 2.34e+05 1 3.44e-03 3.37e-01
88 5.199311e+02 4.19e-01 7.68e+02 1.40e+01 4.09e-01 2.32e+05 1 3.48e-03 3.41e-01
89 5.196121e+02 3.19e-01 7.85e+02 1.41e+01 3.36e-01 2.24e+05 1 3.37e-03 3.44e-01
90 5.192324e+02 3.80e-01 7.57e+02 1.38e+01 3.94e-01 2.22e+05 1 3.54e-03 3.48e-01
91 5.189100e+02 3.22e-01 7.64e+02 1.38e+01 3.51e-01 2.17e+05 1 3.48e-03 3.51e-01
92 5.185548e+02 3.55e-01 7.44e+02 1.37e+01 3.85e-01 2.14e+05 1 3.51e-03 3.55e-01
93 5.182307e+02 3.24e-01 7.43e+02 1.36e+01 3.63e-01 2.10e+05 1 3.47e-03 3.58e-01
94 5.178916e+02 3.39e-01 7.29e+02 1.35e+01 3.82e-01 2.07e+05 1 5.19e-03 3.63e-01
95 5.175684e+02 3.23e-01 7.24e+02 1.35e+01 3.72e-01 2.04e+05 1 3.98e-03 3.68e-01
96 5.172400e+02 3.28e-01 7.13e+02 1.34e+01 3.81e-01 2.01e+05 1 7.22e-03 3.75e-01
97 5.169195e+02 3.20e-01 7.12e+02 1.33e+01 3.79e-01 1.98e+05 1 7.04e-03 3.82e-01
98 5.165985e+02 3.21e-01 7.13e+02 1.32e+01 3.83e-01 1.96e+05 1 3.49e-03 3.85e-01
99 5.162817e+02 3.17e-01 7.14e+02 1.31e+01 3.84e-01 1.93e+05 1 3.45e-03 3.89e-01
100 5.159661e+02 3.16e-01 7.16e+02 1.31e+01 3.86e-01 1.91e+05 1 3.44e-03 3.92e-01

Bundle adjustment report

Residuals : 4080

Parameters : 3067
Iterations : 101
Time : 0.392774 [s]
Initial cost : 0.876701 [px]
Final cost : 0.355615 [px]
Termination : No convergence

=> Filtered observations: 123
=> Filtered images: 0

==============================================================================
Registering image #4 (3)

=> Image sees 702 / 4523 points
=> Could not register, trying another image.

==============================================================================
Registering image #9 (3)

=> Image sees 694 / 4947 points
=> Could not register, trying another image.

==============================================================================
Registering image #8 (3)

=> Image sees 654 / 4674 points
=> Could not register, trying another image.

==============================================================================
Registering image #3 (3)

=> Image sees 646 / 4530 points
=> Could not register, trying another image.

==============================================================================
Registering image #6 (3)

=> Image sees 673 / 4529 points
=> Could not register, trying another image.

==============================================================================
Registering image #11 (3)

=> Image sees 644 / 4900 points
=> Could not register, trying another image.

==============================================================================
Registering image #7 (3)

=> Image sees 616 / 4430 points
=> Could not register, trying another image.

==============================================================================
Registering image #16 (3)

=> Image sees 559 / 4688 points
=> Could not register, trying another image.

==============================================================================
Registering image #14 (3)

=> Image sees 561 / 4622 points
=> Could not register, trying another image.

==============================================================================
Registering image #2 (3)

=> Image sees 522 / 4383 points
=> Could not register, trying another image.

==============================================================================
Registering image #17 (3)

=> Image sees 493 / 4716 points
=> Could not register, trying another image.

==============================================================================
Registering image #18 (3)

=> Image sees 453 / 4233 points
=> Could not register, trying another image.

==============================================================================
Registering image #1 (3)

=> Image sees 420 / 3982 points
=> Could not register, trying another image.

==============================================================================
Registering image #12 (3)

=> Image sees 429 / 3419 points
=> Could not register, trying another image.

==============================================================================
Registering image #15 (3)

=> Image sees 430 / 3892 points
=> Could not register, trying another image.

==============================================================================
Registering image #21 (3)

=> Image sees 373 / 3995 points
=> Could not register, trying another image.

==============================================================================
Registering image #13 (3)

=> Image sees 366 / 4086 points
=> Could not register, trying another image.

==============================================================================
Registering image #20 (3)

=> Image sees 347 / 3149 points
=> Could not register, trying another image.

==============================================================================
Registering image #19 (3)

=> Image sees 316 / 3682 points
=> Could not register, trying another image.

==============================================================================
Registering image #22 (3)

=> Image sees 289 / 3132 points
=> Could not register, trying another image.

==============================================================================
Registering image #23 (3)

=> Image sees 213 / 3458 points
=> Could not register, trying another image.

==============================================================================
Registering image #24 (3)

=> Image sees 157 / 3211 points
=> Could not register, trying another image.

==============================================================================
Retriangulation

=> Merged observations: 0
=> Completed observations: 0
=> Retriangulated observations: 0

==============================================================================
Global bundle adjustment

iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 4.660303e+02 0.00e+00 6.94e+02 0.00e+00 0.00e+00 1.00e+04 0 1.29e-03 6.86e-03
1 4.542867e+02 1.17e+01 1.86e+01 2.67e+01 9.90e-01 3.00e+04 1 3.47e-03 1.04e-02
2 4.539819e+02 3.05e-01 4.54e+00 1.73e+01 9.57e-01 9.00e+04 1 3.31e-03 1.37e-02
3 4.537922e+02 1.90e-01 5.62e+01 4.26e+01 9.58e-01 2.70e+05 1 3.37e-03 1.71e-02
4 4.535130e+02 2.79e-01 3.76e+02 1.21e+02 4.99e-01 2.70e+05 1 3.38e-03 2.05e-02
5 4.529053e+02 6.08e-01 3.06e+02 1.19e+02 7.51e-01 3.09e+05 1 3.11e-03 2.36e-02
6 4.523693e+02 5.36e-01 3.21e+02 1.34e+02 6.79e-01 3.24e+05 1 3.11e-03 2.67e-02
7 4.517444e+02 6.25e-01 2.76e+02 1.40e+02 7.35e-01 3.61e+05 1 3.07e-03 2.98e-02
8 4.511257e+02 6.19e-01 2.48e+02 1.54e+02 7.11e-01 3.91e+05 1 3.08e-03 3.29e-02
9 4.504388e+02 6.87e-01 1.74e+02 1.65e+02 7.43e-01 4.41e+05 1 3.15e-03 3.61e-02
10 4.497332e+02 7.06e-01 1.10e+02 1.85e+02 7.26e-01 4.86e+05 1 3.07e-03 3.91e-02
11 4.489893e+02 7.44e-01 1.22e+02 2.01e+02 7.11e-01 5.25e+05 1 3.11e-03 4.23e-02
12 4.483014e+02 6.88e-01 3.84e+02 2.16e+02 6.16e-01 5.31e+05 1 3.05e-03 4.53e-02
13 4.476780e+02 6.23e-01 6.34e+02 2.17e+02 5.08e-01 5.31e+05 1 3.06e-03 4.84e-02
14 4.471637e+02 5.14e-01 8.62e+02 2.16e+02 3.73e-01 5.23e+05 1 3.13e-03 5.16e-02
15 4.466950e+02 4.69e-01 1.05e+03 2.12e+02 2.91e-01 4.87e+05 1 3.02e-03 5.46e-02
16 4.460323e+02 6.63e-01 1.09e+03 1.97e+02 3.63e-01 4.77e+05 1 3.07e-03 5.77e-02
17 4.455845e+02 4.48e-01 1.20e+03 1.93e+02 2.46e-01 4.22e+05 1 3.13e-03 6.08e-02
18 4.446908e+02 8.94e-01 1.07e+03 1.70e+02 4.59e-01 4.22e+05 1 3.15e-03 6.40e-02
19 4.443183e+02 3.72e-01 1.17e+03 1.70e+02 2.30e-01 3.64e+05 1 4.25e-03 6.83e-02
20 4.434147e+02 9.04e-01 9.67e+02 1.47e+02 5.21e-01 3.64e+05 1 3.41e-03 7.17e-02
21 4.430443e+02 3.70e-01 1.04e+03 1.47e+02 2.82e-01 3.36e+05 1 5.73e-03 7.75e-02
22 4.424412e+02 6.03e-01 9.57e+02 1.36e+02 4.34e-01 3.36e+05 1 5.30e-03 8.28e-02
23 4.420824e+02 3.59e-01 1.01e+03 1.36e+02 2.92e-01 3.13e+05 1 5.48e-03 8.83e-02
24 4.415397e+02 5.43e-01 9.43e+02 1.27e+02 4.24e-01 3.12e+05 1 3.33e-03 9.16e-02
25 4.411907e+02 3.49e-01 9.89e+02 1.26e+02 3.04e-01 2.95e+05 1 3.33e-03 9.50e-02
26 4.407024e+02 4.88e-01 9.31e+02 1.19e+02 4.13e-01 2.93e+05 1 3.36e-03 9.84e-02
27 4.403605e+02 3.42e-01 9.67e+02 1.19e+02 3.17e-01 2.79e+05 1 3.44e-03 1.02e-01
28 4.399180e+02 4.43e-01 9.22e+02 1.14e+02 4.02e-01 2.77e+05 1 3.45e-03 1.05e-01
29 4.395810e+02 3.37e-01 9.49e+02 1.13e+02 3.30e-01 2.67e+05 1 3.15e-03 1.08e-01
30 4.391754e+02 4.06e-01 9.17e+02 1.09e+02 3.92e-01 2.64e+05 1 3.24e-03 1.12e-01
31 4.388422e+02 3.33e-01 9.34e+02 1.08e+02 3.42e-01 2.56e+05 1 3.12e-03 1.15e-01
32 4.384659e+02 3.76e-01 9.13e+02 1.05e+02 3.85e-01 2.53e+05 1 7.35e-03 1.22e-01
33 4.381364e+02 3.29e-01 9.23e+02 1.04e+02 3.53e-01 2.47e+05 1 3.25e-03 1.26e-01
34 4.377831e+02 3.53e-01 9.09e+02 1.02e+02 3.80e-01 2.43e+05 1 3.47e-03 1.29e-01
35 4.374583e+02 3.25e-01 9.14e+02 1.01e+02 3.63e-01 2.38e+05 1 3.45e-03 1.32e-01
36 4.371228e+02 3.35e-01 9.05e+02 9.91e+01 3.79e-01 2.35e+05 1 7.07e-03 1.40e-01
37 4.368037e+02 3.19e-01 9.06e+02 9.81e+01 3.70e-01 2.31e+05 1 5.73e-03 1.45e-01
38 4.364820e+02 3.22e-01 9.01e+02 9.68e+01 3.79e-01 2.28e+05 1 3.20e-03 1.49e-01
39 4.361697e+02 3.12e-01 9.01e+02 9.58e+01 3.77e-01 2.24e+05 1 6.12e-03 1.55e-01
40 4.358587e+02 3.11e-01 8.98e+02 9.48e+01 3.82e-01 2.21e+05 1 5.48e-03 1.60e-01
41 4.355537e+02 3.05e-01 8.97e+02 9.40e+01 3.82e-01 2.19e+05 1 2.78e-03 1.63e-01
42 4.352514e+02 3.02e-01 8.95e+02 9.32e+01 3.85e-01 2.16e+05 1 2.82e-03 1.66e-01
43 4.349534e+02 2.98e-01 8.94e+02 9.26e+01 3.86e-01 2.13e+05 1 2.91e-03 1.69e-01
44 4.346585e+02 2.95e-01 8.93e+02 9.19e+01 3.89e-01 2.11e+05 1 4.07e-03 1.73e-01
45 4.343670e+02 2.91e-01 8.93e+02 9.14e+01 3.91e-01 2.09e+05 1 3.17e-03 1.76e-01
46 4.340786e+02 2.88e-01 8.92e+02 9.10e+01 3.93e-01 2.07e+05 1 2.92e-03 1.79e-01
47 4.337931e+02 2.86e-01 8.92e+02 9.06e+01 3.95e-01 2.05e+05 1 2.99e-03 1.82e-01
48 4.335103e+02 2.83e-01 8.92e+02 9.02e+01 3.97e-01 2.03e+05 1 2.97e-03 1.85e-01
49 4.332302e+02 2.80e-01 8.92e+02 9.00e+01 3.99e-01 2.01e+05 1 2.95e-03 1.88e-01
50 4.329527e+02 2.78e-01 8.92e+02 8.98e+01 4.01e-01 2.00e+05 1 3.44e-03 1.91e-01
51 4.326775e+02 2.75e-01 8.93e+02 8.96e+01 4.03e-01 1.98e+05 1 3.06e-03 1.94e-01
52 4.324046e+02 2.73e-01 8.93e+02 8.95e+01 4.05e-01 1.97e+05 1 3.12e-03 1.98e-01
53 4.321340e+02 2.71e-01 8.94e+02 8.95e+01 4.07e-01 1.96e+05 1 2.97e-03 2.01e-01
54 4.318654e+02 2.69e-01 8.94e+02 8.95e+01 4.09e-01 1.95e+05 1 3.02e-03 2.04e-01
55 4.315990e+02 2.66e-01 8.95e+02 8.95e+01 4.11e-01 1.94e+05 1 2.95e-03 2.07e-01
56 4.313345e+02 2.64e-01 8.96e+02 8.96e+01 4.13e-01 1.93e+05 1 2.95e-03 2.10e-01
57 4.310720e+02 2.63e-01 8.97e+02 8.98e+01 4.15e-01 1.92e+05 1 2.99e-03 2.13e-01
58 4.308113e+02 2.61e-01 8.97e+02 9.00e+01 4.17e-01 1.91e+05 1 3.05e-03 2.16e-01
59 4.305526e+02 2.59e-01 8.98e+02 9.02e+01 4.20e-01 1.90e+05 1 2.96e-03 2.19e-01
60 4.302956e+02 2.57e-01 8.99e+02 9.05e+01 4.22e-01 1.89e+05 1 3.01e-03 2.22e-01
61 4.300404e+02 2.55e-01 8.99e+02 9.08e+01 4.24e-01 1.89e+05 1 2.96e-03 2.25e-01
62 4.297870e+02 2.53e-01 9.00e+02 9.11e+01 4.26e-01 1.88e+05 1 3.00e-03 2.28e-01
63 4.295353e+02 2.52e-01 9.00e+02 9.15e+01 4.29e-01 1.87e+05 1 2.96e-03 2.31e-01
64 4.292854e+02 2.50e-01 9.00e+02 9.19e+01 4.31e-01 1.87e+05 1 2.95e-03 2.34e-01
65 4.290372e+02 2.48e-01 9.00e+02 9.24e+01 4.34e-01 1.87e+05 1 3.02e-03 2.37e-01
66 4.287907e+02 2.46e-01 9.00e+02 9.29e+01 4.36e-01 1.86e+05 1 2.99e-03 2.40e-01
67 4.285459e+02 2.45e-01 9.00e+02 9.34e+01 4.39e-01 1.86e+05 1 2.94e-03 2.43e-01
68 4.283029e+02 2.43e-01 8.99e+02 9.40e+01 4.42e-01 1.86e+05 1 3.09e-03 2.46e-01
69 4.280616e+02 2.41e-01 8.98e+02 9.46e+01 4.45e-01 1.85e+05 1 2.98e-03 2.49e-01
70 4.278220e+02 2.40e-01 8.97e+02 9.52e+01 4.48e-01 1.85e+05 1 3.08e-03 2.52e-01
71 4.275843e+02 2.38e-01 8.95e+02 9.59e+01 4.51e-01 1.85e+05 1 3.04e-03 2.55e-01
72 4.273483e+02 2.36e-01 8.93e+02 9.66e+01 4.54e-01 1.85e+05 1 2.98e-03 2.58e-01
73 4.271141e+02 2.34e-01 8.91e+02 9.73e+01 4.58e-01 1.85e+05 1 3.05e-03 2.61e-01
74 4.268818e+02 2.32e-01 8.88e+02 9.80e+01 4.62e-01 1.85e+05 1 3.48e-03 2.64e-01
75 4.266514e+02 2.30e-01 8.85e+02 9.88e+01 4.66e-01 1.85e+05 1 3.24e-03 2.68e-01
76 4.264229e+02 2.28e-01 8.81e+02 9.96e+01 4.70e-01 1.84e+05 1 3.06e-03 2.71e-01
77 4.261964e+02 2.27e-01 8.77e+02 1.00e+02 4.74e-01 1.84e+05 1 3.12e-03 2.74e-01
78 4.259719e+02 2.25e-01 8.72e+02 1.01e+02 4.79e-01 1.84e+05 1 3.03e-03 2.77e-01
79 4.257495e+02 2.22e-01 8.66e+02 1.02e+02 4.83e-01 1.84e+05 1 3.04e-03 2.80e-01
80 4.255292e+02 2.20e-01 8.60e+02 1.03e+02 4.88e-01 1.84e+05 1 3.05e-03 2.83e-01
81 4.253111e+02 2.18e-01 8.53e+02 1.04e+02 4.94e-01 1.84e+05 1 3.07e-03 2.86e-01
82 4.250953e+02 2.16e-01 8.45e+02 1.05e+02 4.99e-01 1.84e+05 1 3.12e-03 2.89e-01
83 4.248818e+02 2.13e-01 8.37e+02 1.05e+02 5.05e-01 1.84e+05 1 3.06e-03 2.92e-01
84 4.246707e+02 2.11e-01 8.28e+02 1.06e+02 5.11e-01 1.84e+05 1 3.28e-03 2.96e-01
85 4.244622e+02 2.09e-01 8.19e+02 1.07e+02 5.18e-01 1.84e+05 1 3.00e-03 2.99e-01
86 4.242562e+02 2.06e-01 8.09e+02 1.08e+02 5.25e-01 1.84e+05 1 3.17e-03 3.02e-01
87 4.240529e+02 2.03e-01 7.98e+02 1.09e+02 5.31e-01 1.85e+05 1 3.08e-03 3.05e-01
88 4.238524e+02 2.01e-01 7.87e+02 1.10e+02 5.38e-01 1.85e+05 1 3.13e-03 3.08e-01
89 4.236546e+02 1.98e-01 7.76e+02 1.11e+02 5.45e-01 1.85e+05 1 3.05e-03 3.11e-01
90 4.234597e+02 1.95e-01 7.65e+02 1.12e+02 5.52e-01 1.85e+05 1 3.24e-03 3.14e-01
91 4.232676e+02 1.92e-01 7.53e+02 1.13e+02 5.59e-01 1.85e+05 1 3.09e-03 3.17e-01
92 4.230783e+02 1.89e-01 7.42e+02 1.14e+02 5.66e-01 1.86e+05 1 3.04e-03 3.21e-01
93 4.228919e+02 1.86e-01 7.32e+02 1.15e+02 5.73e-01 1.86e+05 1 7.14e-03 3.28e-01
94 4.227082e+02 1.84e-01 7.21e+02 1.17e+02 5.79e-01 1.87e+05 1 4.19e-03 3.32e-01
95 4.225271e+02 1.81e-01 7.12e+02 1.18e+02 5.85e-01 1.88e+05 1 3.00e-03 3.35e-01
96 4.223486e+02 1.79e-01 7.03e+02 1.20e+02 5.91e-01 1.89e+05 1 2.78e-03 3.38e-01
97 4.221724e+02 1.76e-01 6.95e+02 1.21e+02 5.96e-01 1.90e+05 1 2.81e-03 3.41e-01
98 4.219985e+02 1.74e-01 6.88e+02 1.23e+02 6.01e-01 1.92e+05 1 2.96e-03 3.44e-01
99 4.218266e+02 1.72e-01 6.81e+02 1.25e+02 6.05e-01 1.94e+05 1 2.90e-03 3.46e-01
100 4.216566e+02 1.70e-01 6.75e+02 1.27e+02 6.09e-01 1.96e+05 1 4.07e-03 3.51e-01

Bundle adjustment report

Residuals : 3588

Parameters : 2698
Iterations : 101
Time : 0.350847 [s]
Initial cost : 0.360397 [px]
Final cost : 0.34281 [px]
Termination : No convergence

=> Merged observations: 0
=> Completed observations: 0
=> Filtered observations: 897
=> Changed observations: 0.500000

==============================================================================
Global bundle adjustment

=> Merged observations: 0
=> Completed observations: 0
=> Filtered observations: 0
=> Changed observations: -nan

==============================================================================
Global bundle adjustment

=> Merged observations: 0
=> Completed observations: 0
=> Filtered observations: 0
=> Changed observations: -nan

==============================================================================
Global bundle adjustment

=> Merged observations: 0
=> Completed observations: 0
=> Filtered observations: 0
=> Changed observations: -nan

==============================================================================
Global bundle adjustment

=> Merged observations: 0
=> Completed observations: 0
=> Filtered observations: 0
=> Changed observations: -nan
=> Filtered images: 0

Elapsed time: 0.022 [minutes]

@bmild
Copy link
Collaborator

bmild commented Jun 4, 2019

It looks like COLMAP is successfully computing a lot of features in your images but that there might not be very much overlap between your input images (400-800 features matched out of ~4000). If the field of view overlaps more between subsequent images, you may have better luck.

I've noticed that COLMAP frequently fails when the inputs are "outward facing", the camera rotating like you are taking a 360 degree panorama, as opposed to a more "inward facing" motion with the camera always pointing toward a central object, since the outward facing views have less overlap.

Because our pose estimation is just a light wrapper around COLMAP, we don't have much control over whether it successfully reconstructs the camera positions, you might want to take a look at the advice here:
https://colmap.github.io/tutorial.html
https://colmap.github.io/faq.html
Or the issues posted in the COLMAP Github repo:
https://github.com/colmap/colmap

If you want to try adjusting the command line arguments yourself for COLMAP's bundle adjustment. you can add or modify some arguments here:
https://github.com/Fyusion/LLFF/blob/master/llff/poses/colmap_wrapper.py#L59

@mwsunshine
Copy link
Author

It looks like COLMAP is successfully computing a lot of features in your images but that there might not be very much overlap between your input images (400-800 features matched out of ~4000). If the field of view overlaps more between subsequent images, you may have better luck.

I've noticed that COLMAP frequently fails when the inputs are "outward facing", the camera rotating like you are taking a 360 degree panorama, as opposed to a more "inward facing" motion with the camera always pointing toward a central object, since the outward facing views have less overlap.

Because our pose estimation is just a light wrapper around COLMAP, we don't have much control over whether it successfully reconstructs the camera positions, you might want to take a look at the advice here:
https://colmap.github.io/tutorial.html
https://colmap.github.io/faq.html
Or the issues posted in the COLMAP Github repo:
https://github.com/colmap/colmap

If you want to try adjusting the command line arguments yourself for COLMAP's bundle adjustment. you can add or modify some arguments here:
https://github.com/Fyusion/LLFF/blob/master/llff/poses/colmap_wrapper.py#L59

THANK YOU for your fast response!
I took the pictures utilizing my iphone and the motion of my phone is almost like what your tif (photoing a motorcycle at the homepage)does: shifting in the same plane and taking pictures at 6 cols * 4 rows positions.

I am also curious why there is not much overlap between my inputs because I try to keep the main object in sight for all images.
How much percent of the overlap should work in your opinion?
Furthermore , I will look into the link you share and thanks again!

@mwsunshine
Copy link
Author

hi, May I ask another question regarding to the model itself? As I look through your paper, common questions come to my mind as what are the inputs, outputs and loss of the model. I think the input is 5 PSVs and output is one rendered new image. however, I cannot find the definition of the loss(should be my fault of course). Could you please show me what is the loss? Is the ground truth set as the center image within the 5 input images or another new target image?

@bmild
Copy link
Collaborator

bmild commented Jun 13, 2019

The input is 5 PSVs and the output is 1 MPI, which is then used to render a new target image (not the center image). So for each training step you need 6 images, 5 for input and 1 for supervision.

The loss is mentioned in section 5.2, it's the same VGG loss used by Zhou et al. in the Stereo Magnification paper. You can find the definition here:
https://github.com/google/stereo-magnification/blob/master/stereomag/mpi.py#L258

@mwsunshine
Copy link
Author

Thank you for your clear explanation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants