Scribd
Upload
16 views9 pages

Modul 11 Optimasi Model Deep Learning

Uploaded by

yosuamarb
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
Download as docx, pdf, or txt
16 views9 pages

Modul 11 Optimasi Model Deep Learning

Uploaded by

yosuamarb
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
Download as docx, pdf, or txt
Download as docx, pdf, or txt
You are on page 1/ 9

1

Modul Pembelajaran

P152110007- Deep Learning

Optimasi model deep learning


Abstrak Sub-CPMK

Mahasiswa mampu menjelaskan Memahami optimasi model deep learning (CPMK-


mengenai optimasi model deep 4)
learning.

Pendahuluan

Deep learning has been evolving at a very fast pace and its application can be found in
every aspect of human life at the moment. With thousands of models being released every
month and capabilities improving with each one of them it has become essential to
optimize them such that they can be put to practical use. More often than not modelers
find it difficult to productionalize their models owing to hardware limitations or low
throughput rate caused by unoptimized models.

There are various methods one can use to optimize deep learning models, eg:
Quantization, where one performs computations and stores tensors at lower bit widths
than floating point precision. As a result, this allows for a more compact model
representation and the use of high-performance vectorized operations on many hardware
platforms thereby, saving us a lot of inference computation cost and maintaining inference
accuracies. There are other methods as well such as pruning, knowledge distillation, etc.

However, all these methods require changes in network architecture or training


pipeline followed by fine-tuning which might be time-consuming.

Here comes OpenVino, which is an open-source toolkit for optimizing and deploying
deep learning models developed by Intel. It has the potential to improve model
performance and efficiency on various hardware platforms. One can also deploy
OpenVino converted models to cloud instances such as AWS C5 which have dedicated
Intel CPUs and would be highly efficient for running inferences on these optimized
models.

Source: OpenVino docs

In this module, I would walk you through an example to convert the Pytorch CycleGAN
model to OpenVino and perform inference with the same and also observe how it helps in
improving latency. We will first convert the Pytorch model to Onnx and then to OpenVino.

2023 Deep Learning


2 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/
For the purpose of this article, we’ll be using CycleGAN’s pre-trained Yosemite summer-
to-winter model and also their inference pipeline.

Conversion Pipeline

Deep Learning Model Optimization, using OpenVino

Pytorch to Onnx conversion pipeline

### Assuming you have cloned the Cycle GAN repo from github:
https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
### run download_cyclegan_model.sh script to download pretrained
summer2winter_yosemite_pretrained model (instructions on github page)
### Create a jupyter notebook and follow the below code snippets

from models import create_model


from options.test_options import TestOptions
import torch
from polygraphy.backend.onnx import fold_constants
import onnx

opt = TestOptions.parse()
opt.netG = 'resnet_9blocks'
opt.norm = 'instance'
opt.checkpoints_dir = './checkpoints' ### directory storing the pre-
trained model
opt.name = 'summer2winter_yosemite_pretrained'

model = create_model(opt)
model.setup(opt)

### conversion to onnx


dummy_inp = torch.rand(1, 3, 256, 256)
onnx_filepath = './summer2winter_yosemite.onnx'
torch.onnx.export(model, dummy_inp, onnx_filepath, opset_version=10,
input_names = ['inp_img'])

### optional step to do Constant folding which involves pre-computing


expressions that do not depend on runtime information
converted_onnx_model = onnx.load('./summer2winter_yosemite.onnx')
constant_folded_model = fold_constants(onnx_model)
onnx.save(constant_folded_model, './summer2winter_yosemite_folded.onnx')

2023 Deep Learning


3 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/
### This will give you converted onnx model, which can itself be used
for on-device deployment

OpenVino installation — docs

python3 -m venv openvino_env

source openvino_env/bin/activate

python -m pip install --upgrade pip

pip install openvino-dev==2022.1.0. ### tested on this version currently

OpenVino test installation

mo -h ### This will return the help message for Model Optimizer if
installation finished successfully

Converting Onnx to OpenVino

### To convert, run the following in your terminal (you can use either
onnx model or the folded onnx model)
### input_model - specify your input onnx file
### data_type - use fp16 to convert model to fp16 precision for better
performance
### output_dir - directory where your openvino files will be saved

mo --input_model summer2winter_yosemite_folded.onnx --data_type fp16 --


output_dir "./summer2winter_yosemite_openvino"

2023 Deep Learning


4 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/
Once executed, it will create the below 3 files in your directory

summer2winter_yosemite_folded.bin - this contains the model weights ~23


Mb
summer2winter_yosemite_folded.mapping - ~15 Kb
summer2winter_yosemite_folded.xml - this contrains the model structure
~175 Kb

Now that we have both our Pytorch and OpenVino models, let’s perform inference on the

below sample image and compare their runtimes and qualitative outputs

Input Image: Yosemite Summer (Source)

### Pytorch Model Inference

from util.util import tensor2im


from data.base_dataset import get_transform
from PIL import Image
import numpy as np

input_image_filename = 'Yosemite_summer.jpg'
input_image = Image.open(input_image_filename).convert('RGB')
transformation = get_transform(opt, grayscale = False)
transformed_img = transformation(input_image).unsqueeze(0)

if opt.eval:

2023 Deep Learning


5 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/
model.eval()
with torch.no_grad():
output_image = model.netG_A(transformed_img)

output_image = tensor2im(output_image) ### final Pytorch result image

### OpenVino Model Inference


from openvino.inference_engine import IECore, IENetwork
from openvino.runtime import Core

ie_obj = IECore()

openvino_network = ie_obj.read_network(model =
'summer2winter_yosemite_folded.xml', weights =
'summer2winter_yosemite_folded.bin')
executable_network = ie_obj.load_network(network = openvino_network,
device='CPU')

openvino_output = executable_network.infer({'inp_img': transformed_img})

arr_key = list(openvino_output.keys())[0] ### get key for putput image


openvino_output_img =
torch.from_numpy(np.asarray(openvino_output[arr_key]))
openvino_output_image = tensor2im(openvino_output_img) ### final
OpenVino result image

Plotting the resultant images for quality comparison

import matplotlib.pyplot as plt

plt.figure(figsize = (30, 30))


plt.subplot(1, 2, 1)
plt.gca().set_title('Pytorch Output', fontsize = 'xx-large')
plt.imshow(output_image)

plt.subplot(1, 2, 2)
plt.gca().set_title('OpenVino Output', fontsize = 'xx-large')
plt.imshow(openvino_output_image)

2023 Deep Learning


6 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/
Ouptut Image comparison for Pytorch and Openvino (FP16)

From the above images, we can see that there are almost no discernible differences
between the two output images. Also, on comparing the inference runtimes for
the Pytorch and OpenVino models for 19 runs it is observed that
the summer2winter_yosemite CycleGAN OpenVino model runs 50% faster as compared to
the Pytorch one, which is a significant gain if we are trying to integrate this model into an
application for a real-world use case.

Conclusion

OpenVino is an open-source toolkit introduced by Intel for deep learning model

optimization. It has introduced a novel intermediate graph representation format known as

IR along with an operation set for representing the deep learning network which is typically

2023 Deep Learning


7 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/
represented as a directed graph. It uses an XML file and a binary file representing the

graph or IR. To learn more about IR and Opsets you can refer to their official docs here.

OpenVino has definitely made the path to productionalizing a deep learning model much

more efficient. Not only on various hardware devices, we can also deploy these models on

cloud instances for a seamless application experience.

2023 Deep Learning


8 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/
Referensi
 https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-
networks-the-eli5-way-3bd2b1164a53
 https://www.d2l.ai/chapter_convolutional-modern/index.html
 https://www.trivusi.web.id/2022/04/algoritma-cnn.html

2023 Deep Learning


9 Dr.Hadi Santoso, S.Kom., M.Kom
Biro Bahan Ajar E-learning dan MKCU
http://pbael.mercubuana.ac.id/

You might also like