Computer Vision/05_Inference_Server.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "05_Inference_Server.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "JisHBvhVNUzu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#default_exp deployment"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CvAHuMB5LXfZ",
        "colab_type": "text"
      },
      "source": [
        "# Server Implementation\n",
        "\n",
        "Let's now take what we had before and run inference based on a list of filenames. We'll make a quick script to get the ball rolling for how we want everything to do using `nbdev` again"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "apSJf-DQA7Qg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#hide\n",
        "#Run once per session\n",
        "!pip install fastai2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MApJ2nLzNQW8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#hide\n",
        "from nbdev.showdoc import *"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_nOK-lvHNM0q",
        "colab_type": "text"
      },
      "source": [
        "We'll want the libraries we've used"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U9Dpl8xQHQFp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#export\n",
        "from fastai2.vision.all import *\n",
        "from fastai2.basics import *"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GLXtXePwN0l6",
        "colab_type": "text"
      },
      "source": [
        "Including our new `style_transfer.py` file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PhSk005RN3vy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#export\n",
        "from style_transfer import *"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J420YY-TN6Cy",
        "colab_type": "text"
      },
      "source": [
        "Let's grab our original style image"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kLOWaG5DjJNP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "learn = load_learner('myModel', cpu=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y-FnBcIOkgIk",
        "colab_type": "text"
      },
      "source": [
        "And now we can make and prepare our dataloader with a filename!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X5UdTww6kA6O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dset = Datasets('cat.jpg', tfms=[PILImage.create])\n",
        "dl = dset.dataloaders(after_item=[ToTensor()], after_batch=[IntToFloatTensor(), Normalize.from_stats(*imagenet_stats)], bs=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PQSPUZVekEbu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "t_im = dl.one_batch()[0]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wKLECBr1kksf",
        "colab_type": "text"
      },
      "source": [
        "And get our raw output. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0XkjV7w8kGnk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with torch.no_grad():\n",
        "  res = learn.model(t_im)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JG0ozHTbkmCJ",
        "colab_type": "text"
      },
      "source": [
        "Let's wrap this into a function"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NSdvp2fQkn8C",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#export\n",
        "def get_learner(fn, cpu=False):\n",
        "  return load_learner(fn, cpu=cpu)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d1lBQQ9-kyGW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#export\n",
        "def make_datasets(learn, fns, bs=1):\n",
        "  cuda = next(learn.model.parameters()).is_cuda\n",
        "  dset = Datasets(fns, tfms=[PILImage.create])\n",
        "  if cuda: \n",
        "    after_batch = [IntToFloatTensor(), Normalize.from_stats(*imagenet_stats)] \n",
        "    dl = dset.dataloaders(after_item=[ToTensor()], after_batch=after_batch, bs=1)\n",
        "  else: \n",
        "    after_batch = [Normalize.from_stats(*imagenet_stats, cuda=False)]\n",
        "    dl = dset.dataloaders(after_item=[ToTensor()], after_batch=after_batch, bs=1, device='cpu')\n",
        "  return dl"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "abiGY-2pK7Is",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#export\n",
        "from torchvision.utils import save_image"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3inkYqDCQv82",
        "colab_type": "text"
      },
      "source": [
        "We can write a quick `save_im` function to save all our outputed tensors to images"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ODgjWcfHQa6V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#export\n",
        "def save_im(imgs:list, path):\n",
        "  \"Save a n*c*w*h `Tensor` into seperate images\"\n",
        "  [save_image(im, f'{path}/{i}.png') for i, im in enumerate(imgs)]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s6ZQCmGUQ0Zr",
        "colab_type": "text"
      },
      "source": [
        "Now let's put it all together"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mM-KXOtMPwFE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#export\n",
        "def inference(pkl_name, fnames:list, path:Path, cpu:bool=True):\n",
        "  \"Grab inference on a model, filenames, and a path to save it to\"\n",
        "  path = path/'results'\n",
        "  path.mkdir(parents=True, exist_ok=True)\n",
        "  learn = get_learner(pkl_name, cpu)\n",
        "  if len(fnames) > 1:\n",
        "    dls = []\n",
        "    for fname in fnames:\n",
        "      dls.append(make_datasets(learn, fnames, 1))\n",
        "  else:\n",
        "    dls = [make_datasets(learn, fnames, 1)]\n",
        "  res = []\n",
        "  for b in dls:\n",
        "    t_im = b.one_batch()[0]\n",
        "    with torch.no_grad():\n",
        "      out = learn.model(t_im)\n",
        "    res.append(out)\n",
        "  save_im(res, path)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r8vYeOHRRA48",
        "colab_type": "text"
      },
      "source": [
        "And try it out!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vPVmJs16nY7S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fnames = ['cat.jpg'] * 5"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dfc3ZdseKjn0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "inference('myModel', fnames, path=Path(''))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9n0kvdR0Sa6h",
        "colab_type": "text"
      },
      "source": [
        "Lastly let's make a `.py` file again to run it off of"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bsp5kNXcRFSN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#hide\n",
        "from nbdev.imports import *\n",
        "from nbdev.export import reset_nbdev_module, notebook2script\n",
        "\n",
        "create_config('myLib', user='muellerzr', path='.', cfg_name='settings.ini')\n",
        "cfg = Config(cfg_name='settings.ini')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E2ykuLxiSgsY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#hide\n",
        "reset_nbdev_module()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rioWZFfBSibJ",
        "colab_type": "code",
        "outputId": "1e44ac22-38a7-4838-9e12-46511a31d177",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#hide\n",
        "from nbdev.export import notebook2script\n",
        "notebook2script('05_Inference_Server.ipynb')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Converted 05_Inference_Server.ipynb.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3Mswv2kCSuYz",
        "colab_type": "text"
      },
      "source": [
        "And we're done!"
      ]
    }
  ]
}