Extreme memory usage and bug with FP16 and darkflow_utils

[abagshaw]

When using YOLOv2 COCO or VOC on tensornets (TF 1.7, CPU, Win 10) the memory usage with the input as tf.float32 is massive (3GB +) which is more than twice what darkflow uses for the same model. Not quite sure why.

When trying to use tf.float16 with YOLOv2, memory usage is significantly reduced (could be due to a bug that so little memory is used actually...), but there seems to be a problem with the Cython utils ported from darkflow handling the FP16 output data.

I get the following error:

Traceback (most recent call last):
File "blah\src\test.py", line 15, in
boxes = model.get_boxes(preds, img.shape[1:3])
File "C:\Users\abags\AppData\Local\Programs\Python\Python36\lib\site-packages\tensornets\references\yolos.py", line 189, in _get_boxes
return get_v2_boxes(opts('yolov2'), *args, **kwargs)
File "C:\Users\abags\AppData\Local\Programs\Python\Python36\lib\site-packages\tensornets\references\yolo_utils.py", line 94, in get_v2_boxes
results = yolov2_box(opts, outs[0].copy())
File "tensornets\references\darkflow_utils\get_boxes.pyx", line 108, in tensornets.references.darkflow_utils.get_boxes.yolov2_box
ValueError: Does not understand character buffer dtype format string ('e')

[taehoonlee]

Thank you for the report @abagshaw!

1. The memory issue: It is hard for me to support for Windows. I am so sorry. In my machine (CentOS 7, Python 2.7), YOLOv2COCO consumes 300M and 1.3G with tf.float16 and tf.float32, respectively.
2. The Cython util issue has been resolved by this patch.

[abagshaw]

@taehoonlee Thanks for getting that fixed. One very important point I forgot to mention is that I'm using Python 3.6.5 (not Python 2) so that may actually be the source of the memory problems. I might poke around and see if I can find the issue.

[taehoonlee]

OK, @abagshaw. I am going to investigate Python 3.6.

[taehoonlee]

@abagshaw, In order to investigate the memory issue, I tested 12 combinations and summarize the memory consumptions (MB) as follows. In my tests, tf.float32 looks optimized on TF 1.8, and there was no significant difference in the other combinations. I think that 3GB+ consumption may be happened due to Windows.

config	tf.float32	tf.float16
TF 1.5 + Python 2.7	1331	355
TF 1.5 + Python 3.6	1337	315
TF 1.7 + Python 2.7	1328	359
TF 1.7 + Python 3.6	1333	324
TF 1.8 + Python 2.7	1121	354
TF 1.8 + Python 3.6	1136	332

[abagshaw]

@taehoonlee Thanks so much for all that work. Yes, I suppose this must be a Windows problem - oh well 😄