diff --git a/ch06/README.md b/ch06/README.md
new file mode 100644
index 00000000..ecb97a84
--- /dev/null
+++ b/ch06/README.md
@@ -0,0 +1,55 @@
+Python Machine Learning - Code Examples
+
+
+## Chapter 6: Learning Best Practices for Model Evaluation and Hyperparameter Tuning
+
+### Chapter Outline
+
+- Streamlining workflows with pipelines
+ - Loading the Breast Cancer Wisconsin dataset
+ - Combining transformers and estimators in a pipeline
+- Using k-fold cross-validation to assess model performance
+ - The holdout method
+ - K-fold cross-validation
+- Debugging algorithms with learning and validation curves
+ - Diagnosing bias and variance problems with learning curves
+ - Addressing over- and underfitting with validation curves
+- Fine-tuning machine learning models via grid search
+ - Tuning hyperparameters via grid search
+ - Algorithm selection with nested cross-validation
+- Looking at different performance evaluation metrics
+ - Reading a confusion matrix
+ - Optimizing the precision and recall of a classification model
+ - Plotting a receiver operating characteristic
+ - Scoring metrics for multiclass classification
+- Dealing with class imbalance
+- Summary
+
+### A note on using the code examples
+
+The recommended way to interact with the code examples in this book is via Jupyter Notebook (the `.ipynb` files). Using Jupyter Notebook, you will be able to execute the code step by step and have all the resulting outputs (including plots and images) all in one convenient document.
+
+
+
+
+
+Setting up Jupyter Notebook is really easy: if you are using the Anaconda Python distribution, all you need to install jupyter notebook is to execute the following command in your terminal:
+
+ conda install jupyter notebook
+
+Then you can launch jupyter notebook by executing
+
+ jupyter notebook
+
+A window will open up in your browser, which you can then use to navigate to the target directory that contains the `.ipynb` file you wish to open.
+
+**More installation and setup instructions can be found in the [README.md file of Chapter 1](../ch01/README.md)**.
+
+**(Even if you decide not to install Jupyter Notebook, note that you can also view the notebook files on GitHub by simply clicking on them: [`ch06.ipynb`](ch06.ipynb))**
+
+In addition to the code examples, I added a table of contents to each Jupyter notebook as well as section headers that are consistent with the content of the book. Also, I included the original images and figures in hope that these make it easier to navigate and work with the code interactively as you are reading the book.
+
+
+
+
+When I was creating these notebooks, I was hoping to make your reading (and coding) experience as convenient as possible! However, if you don't wish to use Jupyter Notebooks, I also converted these notebooks to regular Python script files (`.py` files) that can be viewed and edited in any plaintext editor.
\ No newline at end of file
diff --git a/ch06/ch06.ipynb b/ch06/ch06.ipynb
new file mode 100644
index 00000000..35ccf00e
--- /dev/null
+++ b/ch06/ch06.ipynb
@@ -0,0 +1,1704 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*Python Machine Learning 3rd Edition* by [Sebastian Raschka](https://sebastianraschka.com), Packt Publishing Ltd. 2019\n",
+ "\n",
+ "Code Repository: https://github.com/rasbt/python-machine-learning-book-3rd-edition\n",
+ "\n",
+ "Code License: [MIT License](https://github.com/rasbt/python-machine-learning-book-3rd-edition/blob/master/LICENSE.txt)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Python Machine Learning - Code Examples"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Chapter 6 - Learning Best Practices for Model Evaluation and Hyperparameter Tuning"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Sebastian Raschka \n",
+ "last updated: 2019-06-14 \n",
+ "\n",
+ "CPython 3.7.3\n",
+ "IPython 7.5.0\n",
+ "\n",
+ "numpy 1.16.4\n",
+ "pandas 0.24.2\n",
+ "matplotlib 3.1.0\n",
+ "sklearn 0.21.1\n"
+ ]
+ }
+ ],
+ "source": [
+ "%load_ext watermark\n",
+ "%watermark -a \"Sebastian Raschka\" -u -d -v -p numpy,pandas,matplotlib,sklearn"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "*The use of `watermark` is optional. You can install this IPython extension via \"`pip install watermark`\". For more information, please see: https://github.com/rasbt/watermark.*"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " 0 \n",
+ " 1 \n",
+ " 2 \n",
+ " 3 \n",
+ " 4 \n",
+ " 5 \n",
+ " 6 \n",
+ " 7 \n",
+ " 8 \n",
+ " 9 \n",
+ " ... \n",
+ " 22 \n",
+ " 23 \n",
+ " 24 \n",
+ " 25 \n",
+ " 26 \n",
+ " 27 \n",
+ " 28 \n",
+ " 29 \n",
+ " 30 \n",
+ " 31 \n",
+ " \n",
+ " \n",
+ " \n",
+ " \n",
+ " 0 \n",
+ " 842302 \n",
+ " M \n",
+ " 17.99 \n",
+ " 10.38 \n",
+ " 122.80 \n",
+ " 1001.0 \n",
+ " 0.11840 \n",
+ " 0.27760 \n",
+ " 0.3001 \n",
+ " 0.14710 \n",
+ " ... \n",
+ " 25.38 \n",
+ " 17.33 \n",
+ " 184.60 \n",
+ " 2019.0 \n",
+ " 0.1622 \n",
+ " 0.6656 \n",
+ " 0.7119 \n",
+ " 0.2654 \n",
+ " 0.4601 \n",
+ " 0.11890 \n",
+ " \n",
+ " \n",
+ " 1 \n",
+ " 842517 \n",
+ " M \n",
+ " 20.57 \n",
+ " 17.77 \n",
+ " 132.90 \n",
+ " 1326.0 \n",
+ " 0.08474 \n",
+ " 0.07864 \n",
+ " 0.0869 \n",
+ " 0.07017 \n",
+ " ... \n",
+ " 24.99 \n",
+ " 23.41 \n",
+ " 158.80 \n",
+ " 1956.0 \n",
+ " 0.1238 \n",
+ " 0.1866 \n",
+ " 0.2416 \n",
+ " 0.1860 \n",
+ " 0.2750 \n",
+ " 0.08902 \n",
+ " \n",
+ " \n",
+ " 2 \n",
+ " 84300903 \n",
+ " M \n",
+ " 19.69 \n",
+ " 21.25 \n",
+ " 130.00 \n",
+ " 1203.0 \n",
+ " 0.10960 \n",
+ " 0.15990 \n",
+ " 0.1974 \n",
+ " 0.12790 \n",
+ " ... \n",
+ " 23.57 \n",
+ " 25.53 \n",
+ " 152.50 \n",
+ " 1709.0 \n",
+ " 0.1444 \n",
+ " 0.4245 \n",
+ " 0.4504 \n",
+ " 0.2430 \n",
+ " 0.3613 \n",
+ " 0.08758 \n",
+ " \n",
+ " \n",
+ " 3 \n",
+ " 84348301 \n",
+ " M \n",
+ " 11.42 \n",
+ " 20.38 \n",
+ " 77.58 \n",
+ " 386.1 \n",
+ " 0.14250 \n",
+ " 0.28390 \n",
+ " 0.2414 \n",
+ " 0.10520 \n",
+ " ... \n",
+ " 14.91 \n",
+ " 26.50 \n",
+ " 98.87 \n",
+ " 567.7 \n",
+ " 0.2098 \n",
+ " 0.8663 \n",
+ " 0.6869 \n",
+ " 0.2575 \n",
+ " 0.6638 \n",
+ " 0.17300 \n",
+ " \n",
+ " \n",
+ " 4 \n",
+ " 84358402 \n",
+ " M \n",
+ " 20.29 \n",
+ " 14.34 \n",
+ " 135.10 \n",
+ " 1297.0 \n",
+ " 0.10030 \n",
+ " 0.13280 \n",
+ " 0.1980 \n",
+ " 0.10430 \n",
+ " ... \n",
+ " 22.54 \n",
+ " 16.67 \n",
+ " 152.20 \n",
+ " 1575.0 \n",
+ " 0.1374 \n",
+ " 0.2050 \n",
+ " 0.4000 \n",
+ " 0.1625 \n",
+ " 0.2364 \n",
+ " 0.07678 \n",
+ " \n",
+ " \n",
+ "
\n",
+ "
5 rows × 32 columns
\n",
+ "
"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {
+ "image/png": {
+ "width": 500
+ }
+ },
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "Image(filename='images/06_01.png', width=500) "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ "execution_count": 10,
+ "metadata": {
+ "image/png": {
+ "width": 500
+ }
+ },
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "Image(filename='images/06_02.png', width=500) "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {
+ "image/png": {
+ "width": 500
+ }
+ },
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "Image(filename='images/06_03.png', width=500) "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Fold: 1, Class dist.: [256 153], Acc: 0.935\n",
+ "Fold: 2, Class dist.: [256 153], Acc: 0.935\n",
+ "Fold: 3, Class dist.: [256 153], Acc: 0.957\n",
+ "Fold: 4, Class dist.: [256 153], Acc: 0.957\n",
+ "Fold: 5, Class dist.: [256 153], Acc: 0.935\n",
+ "Fold: 6, Class dist.: [257 153], Acc: 0.956\n",
+ "Fold: 7, Class dist.: [257 153], Acc: 0.978\n",
+ "Fold: 8, Class dist.: [257 153], Acc: 0.933\n",
+ "Fold: 9, Class dist.: [257 153], Acc: 0.956\n",
+ "Fold: 10, Class dist.: [257 153], Acc: 0.956\n",
+ "\n",
+ "CV accuracy: 0.950 +/- 0.014\n"
+ ]
+ }
+ ],
+ "source": [
+ "import numpy as np\n",
+ "from sklearn.model_selection import StratifiedKFold\n",
+ " \n",
+ "\n",
+ "kfold = StratifiedKFold(n_splits=10,\n",
+ " random_state=1).split(X_train, y_train)\n",
+ "\n",
+ "scores = []\n",
+ "for k, (train, test) in enumerate(kfold):\n",
+ " pipe_lr.fit(X_train[train], y_train[train])\n",
+ " score = pipe_lr.score(X_train[test], y_train[test])\n",
+ " scores.append(score)\n",
+ " print('Fold: %2d, Class dist.: %s, Acc: %.3f' % (k+1,\n",
+ " np.bincount(y_train[train]), score))\n",
+ " \n",
+ "print('\\nCV accuracy: %.3f +/- %.3f' % (np.mean(scores), np.std(scores)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CV accuracy scores: [0.93478261 0.93478261 0.95652174 0.95652174 0.93478261 0.95555556\n",
+ " 0.97777778 0.93333333 0.95555556 0.95555556]\n",
+ "CV accuracy: 0.950 +/- 0.014\n"
+ ]
+ }
+ ],
+ "source": [
+ "from sklearn.model_selection import cross_val_score\n",
+ "\n",
+ "scores = cross_val_score(estimator=pipe_lr,\n",
+ " X=X_train,\n",
+ " y=y_train,\n",
+ " cv=10,\n",
+ " n_jobs=1)\n",
+ "print('CV accuracy scores: %s' % scores)\n",
+ "print('CV accuracy: %.3f +/- %.3f' % (np.mean(scores), np.std(scores)))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {
+ "image/png": {
+ "width": 600
+ }
+ },
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "Image(filename='images/06_04.png', width=600) "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd5xb1Z3//9dR11RPscfjOuOGweNeQ7Uh2ECykFCCSeghpAHJZsMG9geBULIJJSFAypKEGhLjhVC/lEDwhKVjG2NTXLEBV1ymSDPqOr8/jq7KjGY8tkczkv15Ph56SLr36upIY9+3zrnnnqO01gghhBD5xtbfBRBCCCGykYASQgiRlySghBBC5CUJKCGEEHlJAkoIIURecvR3AXpLdXW1rqury/n7tLW1UVxcnPP36U2FWGYozHJLmftGIZYZCrPcfVHmZcuW7dJaD+y4/KAJqLq6OpYuXZrz92lsbGTu3Lk5f5/eVIhlhsIst5S5bxRimaEwy90XZVZKfZJtuTTxCSGEyEsSUEIIIfKSBJQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfKSBJQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfKSBJQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfKSBJQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfKSBJQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfKSBJQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfJSzgJKKXWvUupzpdT7XaxXSqk7lVLrlVIrlVLT0tZdoJRal7hdkKsyWmIxeOYZuPFGcx+L5fodC7NMQgjRlxw53Pf9wN3Ag12sPxkYm7jNBn4PzFZKVQLXATMADSxTSj2ltW7KRSFjMViwAN56C9raoLgYZs+GF14Auz0X71iYZRJCiL6Ws4DSWr+ilKrrZpPTgAe11hp4Uyk1QClVC8wFXtRa7wFQSr0InAT8LRflfO45EwR+v3nu98O//gUnnABDh3befseOw/njH3NRkpQtW+C11yAaTZXp9ddh8WI455zcvneh0NoEeTSauikFTic4HOZeqf4upRCFQ2tNXMez3pqDzcR1nFg8RjQezbh5HB6Glw/PSZmUyYfcSATUM1rrhizrngF+obV+NfH8n8BPMAHl0VrflFh+LRDQWt+WZR+XApcC1NTUTF+0aNE+l/HBB0dy//11aJ1+NNOUlkYoLY122l5rjcrxkc/nc+DzOYHO7zNwYJBRo9oYNcrPqFFtjBnjZ/jwAHZ7139Hv99PSUlJDkvc+7Q25S4uLkFriMfJuLf+2cZisGxZFRs2lDBqlJ8ZM3Zjt5v1Npu5KZW6T7/lQiF+11LmvpPLcmu09SD5OHmvtXmcWKfRWZepDsccjSYajOLwmLpM+nqFSr7G6/QeUNnnzZu3TGs9o+PyXDbx7U22Q4TuZnnnhVrfA9wDMGPGDD137tx9LoTfD//7v6kaFEBxseL++12ccoqr0/ZvvNHIF76w7++zL559Fs4/3zTvWdxumD8fQiEP69Z5WLq0KnleyuWC+no47DAYPx4mTICJE6G21qx7991GZs6ci91uahd2e//VLtJrPNEoRCIQDJpbKGRuVm0oEGikpGQuWpsyp5ffbjdh9c1vwnvvQSAAXi9Mngx//rNZ3/G9wASXUube7QaPx7zO6zXfldOZqoXZ9uMMbWNjI/vz77A/SZn7zr6WW2tNKBYiFA3RFm6jLdJGNB4lEo8kazNx4p2DpUPFQymFTdmwKRuKtMdKYVf25ONsNq7YSP2U+qzrYvEYwWiQqbVTe/yZ9kV/BtRmIL1eOAzYmlg+t8PyxlwV4uSTzfmdjud7Tjst+/kepcxBLZdOOw3mzOlcpscfNzWG9nbYvh3efRfWrIF16+Djj02z4FNPpfYzcCCMHg1Dh47i9dfN47o687msg7HbbR67XOZxegjsS5jF450DIRTKDJ5QKFXzSedwpN6ruDgVDE1NUFRkPuu2beZmPd6+3XzuLVtS+2lvhzffhAsugKOOMp939GgYMQKy/Wi1ArK93Ty2wgvMY5fL/K09HlMOtzvVfOh07l+ACdGVuI4TioYIxUL4w35aQ634w/5k2NiUDZfdlQwXp8PZbbAcDPozoJ4CLlNKLcJ0kmjRWm9TSr0A/FwpVZHYbj5wda4KYbebzgfPPQcrVsCUKSa0+rMzQndlssJlwABTYwoGTYjt3g0tLbBrlwmrTz4xB/A1a2Dp0mE89pjZt9MJY8bAuHEwdqx5PHo0lJebkIFUDcO6dzhSQWbdx2KZNZ9o59ZQbLZU0Dkc5kCf/n8pGoXPP+8cPFYYbdlyJM3Nnfc7YAAMHmzK0ZHWsHo1vPNOapnTCSNHpgLL+sz19ebzuN3Z/w7WOa6mJvO9pjctgvlMVu2rqMiUJx4334cVuEJkE4vHCMVCBKNBfCEfvpCPtmhbsq3IYXPgsrsod5cf1AG0NzkLKKXU3zA1oWql1GZMzzwngNb6D8CzwCnAeqAduCixbo9S6kbAOsTcYHWYyBW7Hb78ZXPLFz0pk1KpA2R1tTk4trebQNu92zwG2LHj/4jHj2P9ehNYa9aYThdPPpnaV3W1aR4cNy7VVDhqVCqMYjHTDPrssyYAxo+HY44xB/eios4H43jcHNS7qv1s2wY7d6ZC0VJcbJomBw+G4cN3MXbsEGprU8tqa83nBViyBH70o9TnBFOWW2+FWbNMUG/YkLqtWQMvvph6T5sNhg3LDK3Ro83nLilJ/SDYW4A1N5vPGo+bwH7vvVSwFxdDWZm5t8LwED7eHJKi8SjBaJBgJEg4Fua9He8RjATNORylcdqcuOwuBrgHHNJhlE0ue/F1298s0Xvv+12suxe4NxflOpjZbObAWlICQ4akmq9aWjTV1ebgboWK12tqBlZgrVljguehh8zrwBxgR40yYTR2rAmnTZvMQdjjMQf1b387VQtKD5/PP0/tx+J2p4LmyCNTgWPd19ZCaWlq+40b11JfP6TLz3vssTBpkgkEq0yTJpnldrs5DzdxYuZrQiHzGazQWr/eBNmrr2aWt7Y2FVjpt4qK1DbZAqy52dTwwARYOAxbt6auY1PK/H3Kysy9FVrSXHhwCMfChKIhAtEAraFWfCEf4Vg4uT4Wj+FQDiq8Fd3sRVj6s4lP5JjTaZruXC6YOtUcnP1+E0xNTeZgefjhMH16qrksGjUHcCuw1qyBt9/OPLcFplPCqlVw2WWp9xo0yBzYp06lU61n8GBzcN/XH4gdz23FYqmmR4Bf/9o05338sQnRKVPA58tsmkzvcu52mxriYYdlvk80Cp99lhlcGzaYrv2BQGq7qqrOTYWjR5vzfdZ7xGLwyivw4YdwxBGpwARTrnAYduww58+sz2LVtEpLU503JLTyl9bahFHMdF7whU0zXTRu2rqVUrjsLtwON8Wu4uTrmm3NOO3O/ip2wZGAOkRYnTs8nszmQJ/PNAc2JS6DdjpNR4oxY+BLX0q9/vbb4Y9/7NzJYeFC+P73zT7354BqhU4kYu6t8zxKmXVtbabMpaWm1ufxZF7r5HCYDiUW6xyQFcYtLaZW0/F8WsfrpBwOc06qvh6++MXM/W3dmtlUuGGDGd3D50ttV1pqvrPq6sNYu9bUJMPhzj0Llep83ktr8/l37TK1T6usXq/5gZEeWnJeq+9ZPemC0SBt4bZk54W4Nr3nrDAqchZht8kfqDdJQB2i0psDa2tTzYHNzbBnT+rgax0Yp00zB8yO53vmzjU1p2y0TgWQ1WOuYwcMl8vst6Qk1dHA6iX3xhumdrevn8s6LzdggDnHZIVWMGhCq7WVZOcLpVLNdNku7rXOUw0bBscdl/nZdu7sXON6/fXqjMsDrJ6FF10ERx9tQmzMGHMRuBU2SqV6UqYLh82Ph+3bU9+Xx5OqaVk/OBzyv7hXRWIR2iPttIZaaQm20BZtQ2lzzY/dZsdtd1PmLjvkzxfFYtD4Ygmr3ivnlGNz07lM/mkLINUcWF5uerxZvQP37DEH84kTTXPVBx+YdV6vOd8zZ445CFshBJkhZJ3vsmo/Lldm7acvmrHSQ8s6h9QxtKyaVvr3YYVltuOQUiaYBw2CL3whtfymmzbzl7/UZ9Q0tYb33zeXDVjcbnN+z2oqtJoLR4xIBU620IpETG33888zr+cqLTXBZYWWU1qReiwWjyUDaXdgN4GIadN12By4He686LwQi8ErLxXz4So3R0wMcewX2zLCoCfjLXS3zd7WWa0b6eX51sKhrFruIdBu46Hf52Y4NgkokZV1oKuqMgfzQMD0+nv8cXPuaexYc62RVfsoKkodGK3wcTjyt8dax9AaPjwztHw+U9NqaUn9x9xbaAGMGePPWtO8/XaYMSOztrV+vbmW7ZlnUts6naaZMf0c15gx5kdDeu0yXTSaaqoFU16nMxVa1jVc2brlH2ysv1V391pr2sLt+CN+9rTvoTXcCihs2PA6PLh0Ba8uKWb1KjeHTQhx5FwTBhrMqAsZ++r83Oolmj76ifU8HIZPP818bVyn9mudc42EFaGgIhCw0eZX3P2LwXy20U04rHC6NEOHhzn7ol2ASrxOEY+Z18e19TixLvHYlCf7dqmymsfxmEqWRwfdxBxlxGOJ7eKKz3c4+GBFEfGY+Y/g95sfYM8917u9oSWgxF7ZbOYkfnGxOd9k1ZQOtqalvdW0rNBKP6dlhZZ18J8xYzeTJ3ce3cLqKDFlirmla2tLdYlfv97cPvgAnn8+dfCz201IWaE1dmzmtVwOhwkiSyxm9mudW7TOv1nns9KFw+a6uY7XeVk6HuSzre9uXXfLo1HToeSjj0yHnaOPzgx/68CZ/hzMD4BlyzIP/l1LnEOKtdES2UNbtJlYPI7NpnDZvLhsAwCFUhCLwk0/Gcr61V5CQYXboxkzPsA1t2xBKRMcwaAiErYRCipCQRuhkCJs3YdshMNmeTjx3FoeCirC/lJC8VLCIUXIWpZ4bShtH92JhBWbNnj45TXDut1uf9nsGrtNoxJDhdlUKTaHDZsy62w2CAZMuKVrazPXbUpAiX51sAVTd7oKLesC5dZWc7N6RWoNv/mN6fm4bp056Kb34sumuDh7l/hgEDZuTIXWhg1mn//8Z6rbus1mmgU71rhGjTKBlR5a4bBpgvnwQ9OL0aoBR6Op2pelqxpidzXifV0Xi8F3vmOaP60wnzgR/ud/Ut9X+nm69HvrR5P13PqbBIOJzj9tEVrag+zx+dnl8xMMQChoIxbyEA2PIBS0m0BIhEwwcf/5Ngcfr3ej42bHwYDi/XeLOPfkMUSj+9ce7XLHcXs0bnccl9OBt9iO26PxeuOUD9B4PIn1Ho078dha5nLHWfpGMa/+s4SMUeCU5uTTWlhwWotpJldgUyZUlNLJDjm2xBidJnQSL7XpxNiUOm28Suu1mT8ItAZH20aixfXJ719rWP5mMXfcVEswkCpTcXHnH18H6hA61OxdIBLAbrPjsh8CbSFiv9lsqYN/x9B64w1zXurYY02TnlKZ3d6tW0/OEXk8JuAOPzxzeThsLgVIbyrcsMHURKxruZQyHTGs0Bo1Cv72N7Oddc1YQwPcfTe0tzuIRFJNTNb5hng8s5mq463jcutcRcd1Xb1+5UrzizucuEyovR2WL4drrzUddwIBcwsGU/ft7ea+tXUWsVhqm3CYDpyJW2nHFUkOhwkEjzcVDL4WGzpLbWzchCBTZwaS26UHicvdOVis5y6XTp5n1RrCn2/EVlmf0RzY8RxP+nBbAB5vjKWvF2eEgdutmTKrjZGjQ0DmuVwrdABTC0oLd2vQZBNoqe2S26sOr1ewewMMHZk5yPLIkW388/8FeH+Fh2DARnGxYvZs01GiN0lApfm46WN8YR9ep5dqbzVl7jLpOpomHAvTHmmnJdiSvN7DGhfMpmzdPrdOMlvdctMfWwNdWo+t9VprgtFgp/XZtk1/3B+s0LKa4iDVfd66tbebWyCQOTix9fqedh5xucyIH+PGZS6PRMz5jfQa1/r1ZozGjhdNBwLm+rHZswGOPtCP32siEXj6afM9WB1rrBqs9XzAABg4yE9FlROHKwLOdmyuIG5PDI8HiovsFHttJjC8mYHh9sTxJO6ztQS89X/F/Pf/l1kz8Hg1Z53fxMwj27KOrG8FbvrfzLrezfrerU5DDm1miXU4wW4DuwMciQu+rfEdbbZUsNjtMG5cG0ueD7BquZdAu8JbpJk8LcB532zrk8sOmh2pi8/TPfD4Fhpf9PL+SjsnHzNEevHlWkzHKHWVotFs829jc+tmAAZ4BlDpraTEVXjD+x8Iq7ttS7CFpmATwWgQAKfdDFKZHK4/wRrUMmMof1JD/SsUKJJddoHkMq11xmOAQDTAyu0re7QtCtCm55XD5jAjNNtsqcfKllznsDkygjR9pOeOoz0fyGCc1kgT1uDClZWpdda1T9YtFEqFV1tb5gzKPa19OZ2pC4cXLEgtj0bhl7+EB7NMHXrssXDEEeuorh6bMTVJsomow5Ql1mO7Pfvyvb02/bZsGdx2m6kRWbxesyz9WjTzHWhCsSDt0TZaw0207VhHtKIKhcJt9+KydzEeVdp3mF5TiUZNgFjhYjlsYhtjxgdY95GXcEjhcptzUBOmtBGLpToFpY+ub4272JPvYeMKqB/XdTm7ct+jW3jlpWI+et/N4Q2de/H1B7sd5s73M+f4IFNrux7x5UBIQGXhsruSzXzWr/iPmz4GoD3Szsd7PqayqJIiZ9FB1RwYjUeT3W33BPZkdLf1ODx9PjxLs62ZAd4sP926YAVh+kRrwWgw+RjIWJdeI0uGHiZgk7W4xHw5VrAppTJDMC347MpOLB6jKdCUEXzWdnabHbuym/ft4toni1X7sn6FBwL7X/tyOMzQUo8+2rl34de/DnV1W6ivH9vj77m3HHaYOZ/WsUPJvHlmfTgWJhBrozXUREukiWgsSlyDAw9O7LjiFcmecSGd2USWfq2dtSwZ8k5wOlK1lvQpXJSCB5/YwuuNxaz50M3hE0PMPbH/w+DY+49iV2AXuIF15lbtrea1b77WvwXLMQmovVDKTMZlTcjVpJpoDjWzK7ALrTUep4eB3oGUuksLrjnQuv7DF/KxJ7CHtoi5wtRus/dLIB0oK3Cs5sXesi/BF4wFWbtnrRkINHF0VEolA1CjzVQJNmcy2Jx2Z3L06vQant1mx+G14S6yUzbAPDc1132vfc2caTogrFrVuXfhp592/Lzd31uPe9Ljb2/3v/mNaYJcswZGjY4xZVYb6z9vpTm8m0g8iAacyonbXoRN2fFY05wo8BZl1irTazFWDSb9+b448UttnPiltr1vmCPReJTt/u1sad3CVt9WE04d7Ars4u8f/Z0ydxmDSwbTMMjMC+sLmdMUDlvhH94L/xP0MaVURlNfOBZmi28L8VZzoEpvDvQ4PP1+gV+6WDxGIBpIBpI/bH6K25QNr9NbcIHUV/Yl+JpsTVR4uv8etdbEdIy4jhOOhwnGUmEXi8eSzZXJGUtVKuzA1PDtNjsuW+Le7cLldVCSWE7cTixqIxY19+GQnWC7jd/9zs6SJbB2ram9zJljeiBGo7BnT3Ke1cSJcHNxjrKZ5lplS4Rt4jkqsV6Z19hsVs1TZ9yTfJ4IcxUnThyIJ9bHmX6iZtIXg4R1G9ihzGFnkMuDx1GRrNVYYWPZuCJ1rq8QhWNhtvq2stW3lS2+LQQjQc6bfB4Alz97OS9tfCn546c7V//TzER05PAjue+0+wA4bdFpbPFtochZRImrhDJ3GXPr5nLlkVcC8MvXfold2Sl1lVLiLqHMVcboytEcMfAIALb7t1PiKqHYWdzt8euoPx/VKThrimvY/uPt+/6FdEEC6gB11xzosDmo9FZS4a3ol+bAuI4TiASIxqN8tPMjWkOtQKJW6PAywNP/V8gfipRSONT+/dfT2tTkYjpGJB4hGAviC/uSAae17lRjU3aFLtFQAsctdHAcNuLxeDI8aG1DDXs7Y05W6xyfNcd1em1QJZYl57/W5iyhUiqxTJsaVuK8oy0R8PYOHVus97GaTV32njfn5rv2SLsJn9YtbPFtYWf7Tn4w+wcAXNd4HYveX5Sxfbm7PBlQR444ktGVoxlaNpShpeY2/y/zs77PS+e9RGuoNWMA2m9P/zY723dmTHpY7i5Prn9u3XPsat9FJJ7qOXP2hLO5Yd4NxOIxjrv/OMD8cC1xleDFy/nx87lk2iWEoiGuXXItZe6yrLW6HW079vMby04Cqhd1bA6MxWM0BZv4vO1zAIqcRVR5qyjzlOF1eHu9OVBrTSAaoC3cxp7AHlpCLVijLkfiEQmkg4A1Rbed/fu3E4vHksFh1QybbXuk9ryPWkOtydqP1Qx3xewrKHIWcedbd/Lbd36bsb3T5uSSqZcAcOSwIxlYNNCET9lQhpQOoaa4JrntOQ3dzlSUYXj58E7Lzm44u9vXNF7YCEAoGsIX9tEaasXrMMcsjeameTeZqULCPvxhP1u3baW2pBaAtkgbS7cuxRf2dbX7XiUBlUN2mz1rc+BnrZ+hlKLcXU6Vt4piV/F+NQdaNba2cBtNwSaag83JEZbdDndyNs5mWzMeR47nqRcFoZDOkfaXbE1XHoeH577xHENKh/DXVX/lZ//6Waf1CxsWUjegjjnD5uBxeBhSOoQhpUMYWjqUgcUDk03EC8YsYAEL2BfV3upOZar2Vu/Hp0txO9y4HW6qi1L7cdgcnDXhrIztNq7YSP04c6FupbeSly94GYDD7u4wZ00OSED1oWzNgRuaNgDmH0ZVURUDPAModhZnnTPGGvY/PZBiifFG3A43pe7SXu8gIMTBJhqPss2/jW2+bcnzQFt9W1nYsJCGQQ1Zm66C0SBbfVsZUjqE6bXT+c8j/zNZ+xlaOpRKb2XyB+asobOYNXRWr5b5YO+t1xUJqH6SrTlwT2APO/ymDddqDix1lxKNR2kKNNEUbDIXyCpw292UuEokkIToIBKLsKFpA1t9W00I+U0AnTruVObVz2PNrjWcvvj0jNdUF1Vz3Mjjkj3hspkxZAYAh1UfxmHVua895Ltstbr0psreIAGVJ7pqDrR6B8qEaEIYcR3ng9YPeH/N+2zzb0sG0dz6uZzTcA6+sI/TFp2W3N5pczKkdAhHDzcjZowcMJKbj7852QRXW1KL29H9hb6is9e++RqxeIxgNMjU2qk5eQ8JqDyV3hwoxMEq2/meam81t86/lc9aPjPNb34TQFMHT+U/jvwPAK5adRWRlaYX2gDPAIaUDiESM88rPBXccdIdDCkxAVRVVJXR0lDiKuHMI87so08oDoQElBCi10ViEXxhH76QL9nVGQVfGGZmd3x41cNsbNrY5QWo1zVex6ctn2JXdgaXDGZI6RBK3WbgV5uycXPDzTRMbGBwyWCKXcUZr1dKcfKY/R+1NBcdEsT+kYASBa2rX+CH6knl7nT1Xf1l+l86bRuKhmgJtSQDxhf2EYwGmT/aXI/z5JonWb5teUYAeRwe7v/K/QB85/99h1c/fTVjn/UD6nn+3OcBeH7986zetbrLsv72lN9S5i5jYNHArM3ak8onUV9Zv0+fv6fk307+kIASBa2rX+AAm5o34XV4KXWX4nV4D6lrwAKRALsDu5MXa/rCvi6/q6tWXUV0bRRfyMez33gWh83Bz1/9eaeLSZ02J6u+uwqlFEu3LOXFj1+kzF1GqbuUMndZxgnysyeczby6ecmRDMrcZRkjbDz4lQdRSnXZVXlc1X6MqCoOOhJQomDd9MpN3a4/+9GzaQ42A5ihXdylfHX8V7nq6KsA+NELP8Lj8FDmLkseSCcOmsjU2qlorVmzew2lLnPwLXYV92mPybZwG9v820wNJdyKP+THF/Zx4qgTqSqq4u0tb/PIB4/gD/kz1j/01YcYXj6ch1c9zK2v39qj9wrFQwz0DmRk+UgisQgOm4Mvjf0S46vHU+ZKBVCpKzW30o3H38iNx9/Y5T6tmlZXDqUfC2L/SUCJgrDNt42XN73MB59/wM9P+DnAXscqu37u9bQEWzLOhVi/zGPxGGt3r03WLtojZpjvi6dezNTaqbRF2jJ6ginMGIzfm/k9Lp56MU2BJq7+59XJ2oEVcCPbRnLun8/tVFspcZVw2czLOLbuWEZXjGb1rtXc9fZd+EK+ZPl8YR+/XvBrjhx+JK9++ipXPH9Fp880pnIMVUVVNAWaWLljJaWuUkrdpYwcMJJSV2lygNBjRhxDhaeCUrdZX+oq5YzFZ2T9nn49+dfUT8lsLsvFtTzZyPke0R0JKJG3tga28sxbz/Dyppf5cOeHANSV19EcbGaAZwA/Pe6nPLzq4S5f392JcrvNzjNffyb5PBo3TVzW+Q6X3cVdJ99lAiwRHq2hVsZUjgHMhZvb/dtZu3ttMmA0mstGX5a1Kc0f9vOL135BhbeC0RWjicajfNryKaWuUmqKaxhdMZpSdylV3ioAJg+ezK8X/DoZfKWuUkpcJVR6zaRSC8YsYMGYrkcjKJRrdeR8j+iOBJTIG6FoiDc3v8lh1YcxuGQwa/1r+d3a3zF18FSuPPJKjq8/nlEVozJe01u/wB02R8Z4dC67q9tmqtrSWp5Y+ETyeVzHaY+089mqz7h7w91ZX/POt95JXuvWMKiBp895usv9Dy4ZzCljT9nXj9Etqa2IQiMBJfrVnsAeGjc18vLGl3nts9doj7Tzk6N+wsVTL2ZO5Rxeu/i1ZK0hm3z5BW6N/Oyxdz3mYZm7rA9L1FlX39XGFRv7uCRC9IwElOhTWmvaIm2UuEpoC7dx7H3HEolHqCmu4bTDTuP4+uOZPXQ2AB67p9twEuJQkj7lyaFCAkrkXDQeZfm25by88WVe3vgyNcU1PHT6QxS7ivnZ3J8xvno8Rww84qD5jydNaT2XPnlj+hQgKsvcUYXImr8rruPJWZnbI+3JubuSszKrxBxcXe0HjQ3Ti9T6rjQap92Jy+7CaXMW/HeVjQSUyKl7lt3Dn5b/iZZQC06bkznD5nDiqBOT6884InvPskKWL82O/ckKnlg8RjQeJa7jROPRjIOoNbmi2+7GruzEiKHjqYN2TMcSs/diJkJMUChi8RjNgeYOEyeSMbli8n3Q2J6j/EcAACAASURBVJQta/B1FYpWWFihkj4h5L6GisPmwG6z47A5sClbsrdl+s2mbHu9WWUOx8KEoqHkfE7+sJ/mUHNyFmZUaqi0Qp/2vbBLL/LKltYtLNm0hCUbl/CrBb+i3FNOhaeC4+uP5/j64zly+JEZA+KKwrO34ElvhnLb3bjsLoqdxbgdbjwOT/KAbB2we3IAtUJBY6ab12heW/8akwZPSj7Pdp/+mvSQ6eoWi8eIEycejxPDTGNjZvp1YVf2AwoVy+f2zxldOfqA/gZW+JS6S6kuNjXzuI4ngysYNbMs+0N+fDEzsaBCYbPZkq8tlFkQJKDEAdnh38Hf3v8bL298mTW71wBmSJstvi2Ue8o5a8JZnSZAE/knPXjSAyg9eKxf5z0JHruy91qTkzWLcMYylEzCmcambHgcHjwOD+WUU4MZ1SMWjxGKmdpWe6QdX8hHW7jNTNuTkM/NhBJQoseyjeVW4amgJdTC9Nrp/OSonzCvbh71FbkZI03sP601kXiEcCxMJBZBKZWs7cTiMVpCLcngKbIX4XF4cNvdOO3OjNpObwaPyD27zU6RrYgiZ1HGZRSRWIRQLEQ4FjbX+YV8tIRaMmrATpuz32dUkIAS3dJac9fbd/Hhzg+zXoDaFGzi7UveptxT3g+lE9lYzT3hWDh53gQFxY5iqrxVyRmbreB5c/2bzBwyU4LnEOK0O5Ozdls9ZbXWppkwFiIQCeAP+/GH/ckJU8EEnhVcfTE3nQSUwB/2s3b3WlbvWp281RTXcNcpd6GU4sUNL6LRXb5ewqn/RGKRZBhZTXB2ZSa/rPZWU+wyzXB7O+8g4SSUUrgdbtwOtxn8N9FM2LS6iUk1kwjFQrSH2/FH/PhCvuQ5u1w2tUpAHUK01mzxbWH1rtXsaNvBNyZ+A4DvP/t93tz8JmAuJh1fPT5jmJwnFj6B3WbvcuRpkXvWr9twLJxx/sDj9FDuLqfUXWqa5RJhJERv8jq9eJ1eBngGJJdF41FC0VC3P14PlATUQSoYDeK2u1FK8dLnL9H490bW7FqDL2x69ThtTs464ixcdheXTruUCyZfwPjq8dSW1Hb6NV0I08wHIgGi8Sg2ZcNus5t7ZS+IsncUi8cymujA/LotcZUwyDOIEleJ+aVrdxfk5xMHB4fNgcOV2wiRgDoINAebeW/He6zZtYaPdn3E6l2r2dS8icYLGqkpqaE92k4sHuNL48wUCuOrxjOualzyl/ZRI47q0fvk4wWo0XiU1lArJa4SqoqqiMQipjNANEwwFjRNX0qBptMvPat3WH+GmhVEVscFtDk/UOIqYXDJYLxOb7LzgjTDiUONBFSeytZjrspbxb2n3cvqXatZs2sNZ044k9EVo3nlk1e48sUrARhaOpTDqg/jpDEnJc85nDrkVH4w5QcHXKZ8uwDVF/IR0zFGVYxiYNHALg/g6V2n0++jsSjheJhwNEwkHiESi3QKtY4XgKZf47IvoZbecSEWN9fYoKDIUUSlt5ISZ4kJI4e74C+uFKK3yP+EPJWtx9zuwO7kHEUuu4uptVMZXTGao0cczV+++hcOqz6s3wck7QtW19iqoipGlo/E7XB3u73dZseOHXpYMbIu7MwWaumdEqzaWnqoxeIxM0lih1ENrBEEqrxVpuOC3ZyMLpQLJoXoDxJQeaIl2MK7299l2dZlLNu2rMvtbp9/O+Orx1M3oC75S7vSW0nl0IN/UFWtNc3BZpw2J+Orx2dc19GbrKa//Qm1V9a9wsRBE5OhZlM23A53Xl4EKUS+y2lAKaVOAn6D+W/+J631LzqsHwncCwwE9gDnaq03J9bFgFWJTT/VWp+ay7L2ta2+rTQFmpgwaALReJS5D8ylPdKOw+ZgwsAJXb7uy+O+3IelzB+BSID2SDtDS4cypGxI3jWDWaGmUHid3v4ujhAHhZz9L1dK2YHfAicCm4F3lFJPaa0/TNvsNuBBrfUDSqnjgf8GzkusC2itp+SqfH1t/Z71vLXlLZZvXc6ybcvY5t9Gw6AGHvvaYzhsDq477jpqS2qZVDMJr9MrXboTYvEYraFWipxFTKqZRLGruL+LJIToI7n8GToLWK+1/hhAKbUIOA1ID6gjgH9PPF4CPMFBIBgNsmrHKlbvWs15k03e3vnWnbyw4QUGFQ9ixpAZXFJ7CTOGzEi+5ivjv5Kxj3zsMdfXYvEYvrCPugF1DCweKOdrhDjEqI5D0/fajpU6EzhJa31J4vl5wGyt9WVp2/wVeEtr/Rul1OnAY0C11nq3UioKrACiwC+01p3CSyl1KXApQE1NzfRFixYdUJkD0cBe56AJtYdwF3U+Kb/ev57GnY180PoB6/zriGpzMeWiWYsY4BrAZ+2f4bQ5qXHX9Pm5iK7KnK+SA5cGY5SWlnY7pUG+8fv9lJQU1ojtUua+U4jl7osyz5s3b5nWekbH5bmsQWU7qnRMwx8DdyulLgReAbZgAglghNZ6q1JqFPCyUmqV1npDxs60vge4B2DGjBl67ty5B1TglTtWYlf25BhV2Xz87sc4RjlYtnUZy7ct5+KpF1M3oI4VH63gyZVP0jCogQvHXsj02ulMHTw1eSK/nv4bQHXjio3UT8n/AVy11rQEW7Db7IyqGMV7b73Hgf5N+1pjY6OUuQ8UYpmhMMvdn2XOZUBtBoanPR8GbE3fQGu9FTgdQClVApyhtW5JW4fW+mOlVCMwFcgIqL60uXUzt71+G29/+ja7X9sNQKmrlBPqT6BuQB0njz2Zk8eeLFMA7CerE8SQ0iEMLRuad50ghBB9L5dHgXeAsUqpekzNaCHw9fQNlFLVwB6tdRy4GtOjD6VUBdCutQ4ltjkKuCVXBR1822B2tO3IWFbkLGJa7TTmjpzLeZPPo9hZzHs73mNS+SSOPeJYptdOZ2zV2OR5EQmm/ROLx2gJtlDkKmJizUSZ0FAIkZSzgNJaR5VSlwEvYLqZ36u1/kApdQOwVGv9FDAX+G+llMY08X0/8fLDgf9RSsUBG+Yc1Ied3qSXdAwngPZIO5+3fZ4cIaDCW8GSC5aY5rKJ+d9cVgj8YT+RWIS6AXUMKhkknSCEEBly2o6itX4WeLbDsp+mPX4UeDTL614HJuaybD3x9DlP93cRDkrWSBCV3kpGDhwptU8hRFbS0C/6jNaa1lArNmXjsKrDqPBWyOgKQoguSUCJPhGMBmkLtzG4ZDDDyoZ121NSCCHAnN855NUU13RadqhdFJsrsXiMpkATGs3EmonUV9RLOAkhekRqUMD2H28HenYdlOg5qxPEiPIR1JTUSCcIIcQ+kYASvS4Si+AL+xjgGUDdwDrpBCGE2C8SUKLXWJ0gFIqxlWOp9FZKJwghxH6TgBK9IhQN4Q/7pROEEKLXSECJAxLXcVqDrbidbhoGNVDqLu3vIgkhDhISUGK/WZ0ghpcNp6akJjnqhhBC9AYJKLFPtNZE4hF8IdMJ4vDqw2UGWSFETuw1oBLj6T2stW7qg/KIfqS1JhqPEo1HiekY0XgUNOjELClKKRQKj8PDuKpx0glCCJFTPalBDcZM174cM9r4CzpXsxyKnLHCxwoeK3wgFUA2ZcPtcON1evHYPXidXpx2Jw6bA6fN3EsznhCir+w1oLTW1yilrgXmAxdhJhhcDPy54wSCon90DJ9YPEZcx5Mz0UbjUVpCLSZ8HF7cdjcehwe3w43D5si4CSFEvujREUlrrZVS24HtmBlvK4BHlVIvaq3/M5cFPNQlpz+Px4jEIyZ8iIMmNRW6Ihk65e5yPA4PLrvL1HzsTt5c/yazhs7q3w8ihBD7qCfnoK4ALgB2AX8CrtRaR5RSNmAdIAGVI9YAq0XOItx2N6XuUrwObzJ8rACyK7ucCxJCHHR6UoOqBk7XWn+SvlBrHVdKfTk3xRLNgWbcDjeTaiZR7Cru7+IIIUSf60lAPQvssZ4opUqBI7TWb2mtP8pZyQ5R0XiUlmALg4oHMXLASDkvJIQ4ZPVkeOnfA/60522JZaKXBSIB/CE/YyrHMLpytISTEOKQ1pMjoErvVp5o2pMjZy/SWid72TXUNFDkLOrvIgkhRL/rSQ3qY6XUFUopZ+L2A+DjXBfsUBGNR9kT2EN1UTUNAyWchBDC0pOA+g5wJLAF2AzMBi7NZaEOFe2RdvxhP+OqxjGqYpRcBCuEEGl6cqHu58DCPijLIUNrTXOoGa/Dy8RBE2UsOyGEyKIn10F5gG8CE4Dk1Kha64tzWK6DViQWoTXUSm1JLcPLh0utSQghutCTJr6HMOPxLQD+BQwDfLks1MHKH/bTFmnjsKrDqKuok3ASQohu9CSgxmitrwXatNYPAF8CJua2WAcXrTXNgWZcdheTaiZRWVTZ30USQoi815Pu4pHEfbNSqgEzHl9dzkp0kAnHwvhCPoaVDWNo2VBsqie/CYQQQvQkoO5RSlUA1wBPASXAtTkt1UHCH/YTi8c4vPpwBngH9HdxhBCioHQbUIkBYVsTkxW+Aozqk1IVOK01zcFmSl2ljB44GrfD3d9FEkKIgtNte5PWOg5c1kdlOSiEY2H2BPYwtGwo4weOl3ASQoj91JMmvheVUj8GHsGMwweA1npP1y85NPlCPrTWTBg0gTJ3WX8XRwghClpPAsq63un7acs00tyXFNdxWoItlLnLGF05Gpfd1d9FEkKIgteTkSTq+6IghSocC+MP+RlRPoLa0lqZOFAIIXpJT0aSOD/bcq31g71fnMLSGmpFoZgwaAKl7tL+Lo4QQhxUetLENzPtsQc4AVgOHLIBFddxmgJNVHorqa+olyY9IYTIgZ408V2e/lwpVY4Z/uiQFNdxmoPN1A2oY3DJYGnSE0KIHNmfiQfbgbG9XZBC0BpsRWtNw6AGSlwl/V0cIYQ4qPXkHNTTmF57YK6bOgJYnMtC5ZtYPEZzsJmqoip8Tp+EkxBC9IGe1KBuS3scBT7RWm/OUXnyTjAapD3czqiKUQwqHsRWtvZ3kYQQ4pDQk4D6FNimtQ4CKKW8Sqk6rfWmnJYsD7QEW3DanEysmUixq7i/iyOEEIeUngyt/b9APO15LLHsoBWLx9jdvpsKbwUNNQ0STkII0Q96UoNyaK3D1hOtdVgpddD2qw5Gg0TjUUZXjmZg0UDppSeEEP2kJzWonUqpU60nSqnTgF25K1L/sSkbdpudiTUTGVQ8SMJJCCH6UU8C6jvAfymlPlVKfQr8BPh2T3aulDpJKbVGKbVeKXVVlvUjlVL/VEqtVEo1KqWGpa27QCm1LnG7oKcf6EDUDaijYVADRc6ivng7IYQQ3ejJhbobgDlKqRJAaa19PdmxUsoO/BY4EdgMvKOUekpr/WHaZrcBD2qtH1BKHQ/8N3CeUqoSuA6Ygenivizx2qZ9+XD7SrqPCyFE/thrDUop9XOl1ACttV9r7VNKVSilburBvmcB67XWHyfOYS0CTuuwzRHAPxOPl6StXwC8qLXekwilF4GTevKBhBBCHByU1rr7DZR6V2s9tcOy5VrraXt53ZnASVrrSxLPzwNma60vS9vmr8BbWuvfKKVOBx4DqoGLAI/W+qbEdtcCAa31bR3e41LgUoCamprpixYt6slnPiB+v5+SksKqaRVimaEwyy1l7huFWGYozHL3RZnnzZu3TGs9o+PynvTisyul3FrrEJjroICeTBObrYdBxzT8MXC3UupCzJTyWzAXA/fktWit7wHuAZgxY4aeO3duD4p1YBobG+mL9+lNhVhmKMxyS5n7RiGWGQqz3P1Z5p4E1F+Afyql7ks8vwh4oAev2wwMT3s+DDKHYdBabwVOB0ic4zpDa92ilNoMzO3w2sYevKcQQoiDxF7PQWmtbwFuAg7HnDN6HhjZg32/A4xVStUnrptaCDyVvoFSqlopZZXhauDexOMXgPmJ810VwPzEMiGEEIeInnQzB9iOGU3iDMx8UB/t7QVa6yhwGSZYPgIWa60/UErdkHZd1VxgjVJqLVAD3Jx47R7gRkzIvQPckFgmhBDiENFlE59Sahym1nMOsBt4BNOpYl5Pd661fhZ4tsOyn6Y9fhR4tIvX3kuqRiWEEOIQ0905qNXA/wH/prVeD6CU+vc+KZUQQohDXndNfGdgmvaWKKX+qJQ6gey964QQQohe12VAaa0f11qfDYzH9KD7d6BGKfV7pdT8PiqfEEKIQ1RPevG1aa0f1lp/GdPdewXQaVw9IYQQojf1tBcfYHrXaa3/R2t9fK4KJIQQQsA+BpQQQgjRVySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRlySghBBC5CUJKCGEEHlJAkoIIURekoASQgiRl3IaUEqpk5RSa5RS65VSV2VZP0IptUQp9a5SaqVS6pTE8jqlVEAptSJx+0MuyymEECL/OHK1Y6WUHfgtcCKwGXhHKfWU1vrDtM2uARZrrX+vlDoCeBaoS6zboLWekqvyCSGEyG+5rEHNAtZrrT/WWoeBRcBpHbbRQFnicTmwNYflEUIIUUByGVBDgc/Snm9OLEt3PXCuUmozpvZ0edq6+kTT37+UUsfksJxCCCHykNJa52bHSp0FLNBaX5J4fh4wS2t9edo2P0qU4Xal1BeAPwMNgBMo0VrvVkpNB54AJmitWzu8x6XApQA1NTXTFy1alJPPks7v91NSUpLz9+lNhVhmKMxyS5n7RiGWGQqz3H1R5nnz5i3TWs/otEJrnZMb8AXghbTnVwNXd9jmA2B42vOPgUFZ9tUIzOju/aZPn677wpIlS/rkfXpTIZZZ68Ist5S5bxRimbUuzHL3RZmBpTrLcT2XTXzvAGOVUvVKKRewEHiqwzafAicAKKUOBzzATqXUwEQnC5RSo4CxifASQghxiMhZLz6tdVQpdRnwAmAH7tVaf6CUugGTlk8B/wH8USn175gOExdqrbVS6ljgBqVUFIgB39Fa78lVWYUQQuSfnAUUgNb6WUznh/RlP017/CFwVJbXPQY8lsuyCSGEyG8ykoQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfKSBJQQQoi8JAElhBAiL0lACSGEyEsSUEIIIfJSTi/UFUIcuiKRCJs3byYYDPb6vsvLy/noo496fb+5Vojl7s0yezwehg0bhtPp7NH2ElBCiJzYvHkzpaWl1NXVoZTq1X37fD5KS0t7dZ99oRDL3Vtl1lqze/duNm/eTH19fY9eI018QoicCAaDVFVV9Xo4icKklKKqqmqfatQSUEKInJFwEun29d+DBJQQQoi8JAElhMgLsRg88wzceKO5j8UObH/Nzc387ne/26/XnnLKKTQ3N3e7zU9/+lNeeuml/dq/6BnpJCGE6HexGCxYAG+9BW1tUFwMs2fDCy+A3b5/+7QC6nvf+16W94th72bHzz77bJfrLDfccMP+FawfRaNRHI7COexLDUoIkXM//CHMndv1bcoUWLIE/H7Q2twvWWKWZ9v+lFO8/PCH3b/nVVddxYYNG5gyZQpXXnkljY2NzJs3j69//etMnDgRgK985StMnz6dCRMmcM899yRfW1dXx65du9i0aROHH3443/rWt5gwYQLz588nEAgAcOGFF/Loo48mt7/uuuuYNm0aEydOZPXq1QDs3LmTE088kWnTpvHtb3+bCRMmsGvXrk5l/e53v8uMGTOYMGEC1113XXL5O++8w5FHHsnkyZOZNWsWPp+PWCzGj3/8YyZOnMikSZO46667MsoMsHTpUubOnQvA9ddfz6WXXsr8+fM5//zz2bRpE8cccwzTpk1j2rRpvP7668n3u+WWW5g4cSKTJ09Ofn/HHHNMcv26deuYPn169198LyqcKBVCHLT8fojHM5fF42Z5VdX+7fMXv/gF77//PitWrACgsbGRt99+m/fffz/Zzfnee++lsrKSQCDAzJkzOeOMM6jq8Ibr1q3jb3/7G3/84x/52te+xmOPPca5557b6f2qq6tZvnw5v/vd77jtttv405/+xM9+9jOOP/54rr76ap5//vmMEEx38803U1lZSSwW44QTTmDlypWMHz+es88+m0ceeYSZM2fS2tqK1+vlnnvuYePGjbz77rs4HA727Nn7ZOPLli3j1Vdfxev10t7ezosvvojH42HdunWcc845LF26lOeee44nnniCt956i6KiIvbs2UNlZSVlZWWsWLGCKVOmcN9993HhhRfu419i/0lACSFy7o47ul//zDNwzjkmkCwlJXDXXfDlL3fe3ucL7Ne1ObNmzcq4BufOO+/k8ccfB+Czzz5j3bp1nQKqvr6eKVOmADB9+nQ2bdqUdd+nn356cpu///3vALz66qvJ/Z900kkMGDAg62sXL17MPffcQzQaZdu2bXz44YcopaitrWXmzJkAlJWVAfDSSy/xne98J9lUV1lZudfPfeqpp+L1egFzAfVll13GihUrsNvtrF27Nrnfiy66iKKiooz9nn/++dx333386le/4pFHHuHtt9/e6/v1FgkoIUS/O/lkc86p4zmok0/u3fcpLi5OPm5sbOSll17ijTfeoKioiLlz52a9Rsftdicf2+32ZBNfV9vZ7Xai0ShgLk7dm40bN3LbbbfxzjvvUFFRwYUXXkgwGERrnbVbdlfLHQ4H8UQ1tOPnSP/cv/71r6mpqeG9994jHo/j8Xi63e9pp53GLbfcwvHHH8/06dM7BXguyTkoIUS/s9tNh4i//Q1uuMHcH0gHCYDS0lJ8Pl+X61taWqioqKCoqIjVq1fz5ptv7v+bdeHoo49m8eLFAPzjH//I2jOwtbWV4uJiysvL2bFjB8899xwA48ePZ+vWrbzzzjuAGdEhGo0yf/58/vCHPyRD0Griq6urY9myZQA89thjXZappaWF2tpabDYbDz30ELFEd8n58+dz77330t7enrFfj8fDggUL+O53v8tFF110wN/JvpCAEkLkBbvdNOddc425P5BwAqiqquKoo46ioaGBK6+8stP6k046iWg0yqRJk7j22muZM2fOgb1hFtdddx3/+Mc/mDZtGs899xyDBw/u1DQ5efJkpk6dyoQJE7j44os56qijAHC5XDzyyCNcfvnlTJ48mRNPPJFgMMgll1zCiBEjmDRpEpMnT+avf/1r8r1+8IMfcMwxx3TbQ/F73/seDzzwAHPmzGHt2rXJ2tVJJ53EqaeeyowZM5gyZQq33XZb8jXf+MY3UEoxf/783v6Kuqe1Pihu06dP131hyZIlffI+vakQy6x1YZZbypzy4Ycf5mS/Wmvd2tqas333pmAwqCORiNZa69dff11PnDixn0u071pbW/Wtt96qr7nmml7ZX7Z/F8BSneW4LueghBAiRz799FO+9rWvEY/Hcblc3Hnnnf1dpH329a9/nU8++YSXX365z99bAkoIIXJk7NixvPvuu8nn3Z0Ty1d//etf+20EdjkHJYQQIi9JQAkhhMhLElBCCCHykgSUEEKIvCQBJYQQCSUlJQBs3bqVM888M+s2c+fOZenSpd3u54477khe8Ao9m75DdCa9+IQQeWHwbYPZ0bYjY1lNcQ3bf7y9z8syZMiQ5Ejl++OOO+7g3HPPTY5rZ03fUSi9+KzrkGy2/q3DSA1KCNEn5t4/t9Ptd++YCQXbI+2dwglILtvVvivjdacsPmWv7/eTn/wkY8LC66+/nttvvx2/388JJ5yQnBrjySef7PTaTZs20dDQAEAgEGDhwoVMmjSJs88+O2MsvmzTZNx5551s3bqVefPmMW/ePCBzKoxf/epXNDQ00NDQwB2JUXS7m9Yj3dNPP83s2bOZOnUqX/ziF9mxw3w/fr+fiy66KDkFhzXU0fPPP8+0adOYPHkyJ5xwQvJ7SB8loqGhgU2bNiXL8L3vfY9p06bx2Wef8d3vfpfjjjuuR9OAHHPMMcmR4wGOOuooVq5cude/U3ekBiWEOCgtXLiQH/7wh8kJCxcvXszzzz+Px+Ph8ccfp6ysjF27djFnzhxOPfXUrAOlAvz+97+nqKiIlStXsnLlSqZNm5Zcl22ajCuuuIJf/epXLFmyhOrq6ox9vfvuu9x333289dZbaK2ZPXs2xx13HBUVFT2a1uPoo4/mzTffRCnFn/70J2655RZuv/12brzxRsrLy1m1ahUATU1N7Ny5k29961u88sor1NfX92hajjVr1nDfffclg/3mm2/G6XRSVFS012lALrnkEu6//37uuOMO1q5dSygUYtKkST3/g2UhASWE6BONFzZ2ua7IWdTta6uLqjNe35OmsqlTp/L555+zdetWdu7cSUVFBSNGjCASifBf//VfvPLKK9hsNrZs2cKOHTsYPHhw1v288sorXHHFFQBMmjQp46CbbZqM7g7Kb7zxBl/96leT49+dfvrp/N///R+nnnpqj6b12Lx5M2effTbbtm0jHA4npw556aWXWLRoUXK7iooKnn76aY499tjkNj2ZlmPkyJEZYxIuXryYP/zhD8Tj8b1OA3LWWWdx4403cuutt3Lvvff2yrxRElBCiIPWmWeeyaOPPsr27dtZuHAhAA8//DA7d+5k2bJlOJ1O6urqsk6zkS5b7aqraTK6o7uZfqMn03pcfvnl/OhHP+LUU0+lsbGR66+/PrnfjmXMtgwyp+WAzKk50qflsD7fyy+/zIgRI/Y6DUhRUREnnngiTz75JIsXL95rR5KekHNQQoi8UFNc06Nl+2LhwoUsWrSIRx99NNkrr6WlhUGDBuF0OlmyZAmffPJJt/s49thjefjhhwF4//337ZoHqgAADctJREFUk+dVupomA7qe6uOoo47iiSeeoL29nba2Nh5//PGMKdX3pqWlhaFDhwLwwAMPJJfPnz+fu+++O/m8qamJL3zhC/zrX/9i48aNQOa0HMuXLwdg+fLlyfUd7es0IACXXHIJV1xxBTNnzuxRjW1vpAYlhMgLueitN2HCBHw+H0OHDqW2thYwU0f827/9W3JaifHjx3e7D2sepEmTJjFlyhRmzZoFZE6TMWrUqOQ0GQCXXnopJ598MrW1tSxZsiS5fMqUKVx44YXJfVxyySVMnTq1y1l6O7r++us566yzGDp0KHPmzEmGyzXXXMP3v/99GhoasNvtXHfddZx++uncc889nH766cTjcQYNGsSLL77IGWecwYMPPsiUKVOYOXMm48aNy/pe1uebNWsWY8aMyToNSCAQwOv18tJLL1FSUsL06dMpKyvrvXmjsg1xXog3mW6ja4VYZq0Ls9xS5hSZbqOzQiz3vpR5y5YteuzYsToWi3W5zb5MtyFNfEIIIQ7Ygw8+yOzZs7n55pt77fopaeITQghxwM4//3zOP//8Xt2n1KCEEDmju+m1Jg49+/rvQQJKCJETHo+H3bt3S0gJwITT7t278Xg8PX6NNPEJIXJi2LBhbN68mZ07d/b6voPB4D4d6PJFIZa7N8vs8XgYNmxYj7eXgBJC5ITT6UyOYtDbGhsbmTp1ak72nUuFWO7+LHNOm/iUUicppdYopdYrpa7Ksn6EUmqJUupdpdRKpdQpaeuuTrxujVJqQS7LKYQQIv/krAallLIDvwVOBDYD7yilntJaf5i22TXAYq3175VSRwDPAnWJxwuBCcAQ4CWl1DitdSxX5RVCCJFfclmDmgWs11p/rLUOA4uA0zpso4GyxONyYGvi8WnAIq11SGu9EVif2J8QQohDRC7PQQ0FPkt7vhmY3WGb64F/KKUuB4qBL6a99s0Orx3a8Q2UUpcClyae+pVSaw682HtVDezqg/fpTYVYZijMckuZ+0YhlhkKs9x9UeaR2RbmMqCyTa7Ssb/pOcD9WuvblVJfAB5SSjX08LVore8B7jngku4DpdRSrfWMvnzPA1WIZYbCLLeUuW8UYpmhMMvdn2XOZUBtBoanPR9GqgnP8k3gJACt9RtKKQ8mrXvyWiGEEAexXJ6DegcYq5SqV0q5MJ0enuqwzafACQBKqcMBD7Azsd1CpZRbKVUPjAXezmFZhRBC5Jmc1aC01lGl1GXAC4AduFdr/YFS6gbMyLVPAf8B/FEp9e+YJrwLEyPbfqCUWgx8CESB7+dRD74+bVLsJYVYZijMckuZ+0YhlhkKs9z9VmYlw5AIIYTIRzIWnxBCiLwkASWEECIvSUDthVJqk1JqlVJqhVJqaWJZpVLqRaXUusR9RT+X8V6l1OdKqffTlmUtozLuTAwjtVIpNS2Pyny9UmpL4rtekW9DXymlhieG5vpIKfWBUuoHieV5+113U+Z8/649Sqm3lVLvJcr9s8TyeqXUW4nv+pFEBywSHaoeSZT7LaVUXR6V+X6l1Ma073pKYnm///tIK7tdmSHnnkk8z4/vOds0u3JL3YBNQHWHZbcAVyUeXwX8sp/LeCwwDXh/b2UETgGew1xrNgd4K4/KfD3w4yzbHgG8B7iBemDD/9/e2QdvUVVx/PMVCUkUBhRGc0aSbKQ/ykCcTC0UYqJMLHHSdJTUKXzJtLSG0TFzxpTM8g8rCEowoAlJI7VREIHeRHkRAV9Qx3GaihGnhCANwd/pj3MW1uV5fi+IPPvL85nZee7ee3ef7z37PHv33t09B+jRAs2HAcMifRDwXGirra3b0Vx3WwvoE+mewGNhw7nA2ZE/Bbgk0pcCUyJ9NvDrGmmeAYxvUL/lv4+Slm8Ac4D7Y70Wds4R1J4xDpgZ6ZnAGS3Ugpn9AfhXJbuZxnHAXeYsA/pJOmzfKN1FE83NqIXrKzPbYGarIr0FeAb3cFJbW7ejuRl1sbWZ2dZY7RmLAacC8yK/auviGMwDRklq9ML/O0Y7mpvR8t8HgKQjgM8C02Nd1MTO2UF1jOHumFbKXSsBDDKzDeAnAGBgy9Q1p5nGRi6o2jth7Wsuj+mOX5SmTmunOaY2PopfJXcLW1c0Q81tHdNOq4GNwEJ8NLfJzHY00LZTd5RvBgbsW8W7azazwtY3ha1/JKlXVXPQKlvfDnwLaIv1AdTEztlBdcyJZjYMGAtcJukTrRb0NumUG6kW8VNgCHAssAG4LfJrpVlSH+A3wJVm9u/2qjbIa4nuBpprb2sze9PMjsU9yRwPDG1ULT5robuqWe66bRJwDDAC6A98O6q3XLOk04CNZraynN2gakvsnB1UB5jZP+JzI3Av/kd5uRiKx+fG1ilsSjONtXUjZWYvxx+8DZjGrqml2miW1BM/0c82s3siu9a2bqS5O9i6wMw2AUvw+zT9JBUOBsraduqO8r50fgp5r1PS/OmYZjUz2wbcSb1sfSJwuqSX8IgTp+IjqlrYOTuodpB0oKSDijQwBliHu2K6IKpdAMxvjcJ2aabxd8D58QTRx4DNxfRUq6nMv38etzXUxPVVzLX/HHjGzH5YKqqtrZtp7ga2PlRSv0j3xiMdPAMsBsZHtaqti2MwHnjE4k7+vqKJ5mdLFy/C7+WUbd3S34eZTTKzI8xsMP7QwyNmdi51sfM7+QRGd1+Ao/Anmp4EngKujfwBwCLg+fjs32Kdv8KnabbjVzgXNdOID9F/jM/nrwWOq5HmX4amNfgf4bBS/WtD83pgbIs0n4RPZ6wBVsfymTrbuh3Ndbf1h4EnQt864PrIPwrvMF8A7gZ6Rf4Bsf5ClB9VI82PhK3XAbPY9aRfy38fFf0j2fUUXy3snK6OkiRJklqSU3xJkiRJLckOKkmSJKkl2UElSZIktSQ7qCRJkqSWZAeVJEmS1JLsoJJaIMkk3VZav1rSDXtp3zMkje+45tv+nrPkXsMXV/IHS/rSHu7zL52oM13Sh/Zk/3VF0taOayX/72QHldSFbcAXJB3SaiFlJPXoQvWLgEvN7JRK/mCgYQdVelu/IWb28Y6+1MwuNrOnOysySboL2UEldWEH8DPgqmpBdQRUXF1LGilpqaS5kp6TdIukc+UxedZKGlLazWhJf4x6p8X2PSTdKml5OPL8amm/iyXNwV+grOo5J/a/TtLkyLsefyl2iqRbK5vcApwsjwV0laQJku6WdB/uiLiPpEWSVsV+xzVp6xJJ8yQ9K2l2eCYg8o8r6ku6SR6TaJmkQZE/JNaXS7qx2QhF0nlhv9WSpoaNjpTHBTpE0n5hxzFR/7dyR8pPaZcz5ULH5Ch7WNLxofNFSadHnQmS5kt6UB576jtNNF1TOkZFjKUDJT0Q7Vwn6YuNtk26Oa18czmXXIoF2AocjMff6gtcDdwQZTMoxdMBtsbnSGATHvOoF/B34LtR9nXg9tL2D+IXZEfjnisOAL4CXBd1egEr8BhII4H/AO9voPNw4K/AocD+uJeAM6JsCQ28AVB6Qz/WJ4SGwuPE/sDBkT4Ef0tfDdq6GfeLth/wKHBS9XtxrxGfi/T3S+27Hzgn0hOL/VZ0DgXuA3rG+k+A8yN9MR5e4Rpgammbog29cU8JA0o6xkb6XmABHn7iI8Dqkh024J44iu2Pq7R7DH7homj3/XgssTOBaSUdfVv9G85l7y85gkpqg7mX7buAK7qw2XJzZ5zbcJcxCyJ/LT61VjDXzNrM7HngRdy79BjcF9pqPATFALwDA3jcPB5SlRHAEjN7xTzcwGz8hNlVFppZ4WRTwPckrQEexkMaDGqwzeNm9jdzB6+rK+0reAM/iQOsLNU5AXdRAx6YrhGjgOHA8rDJKNzlDWY2HQ94OBG/eCi4QtKTwDLciWhhvzfwiwLwY7HUzLaz+3FZaGb/NLPXgXvwUWiZMbE8AazCj9vRsZ/RMUo72cw2N2lT0o1pd/47SVrA7fiJ6M5S3g5iOjqmtd5TKttWSreV1tt46++76tPL8I7ha2b2ULlA0kh8BNWIvRWcrbz/c/ER2XAz2y73LH1Ag23KbX2Txv/f7WZmHdRphoCZZjZptwLpvfjoDaAPsCXsNBo4wcxek7SkpLusY+dxMbO2yn23RselqulmM5vaQNNw3K/gzZIWmNmNnWtm0l3IEVRSK2JUMRd/4KDgJfzKHjyiZ8892PVZcf9kCD4qWA88BFwiD0eBpA/Kvda3x2PAJ+N+TA/gHGBpB9tswUcfzeiLx+TZLukU4MhOtKerLMOnxcC9VjdiETBe0kAASf0lFVom46PF6/HwHIXuV6NzOgYPh9FVPhXf0xv39P3nSvlDwIXyeFZIep+kgZIOB14zs1nAD4Bhe/DdSc3JEVRSR24DLi+tTwPmS3ocP4k2G920x3q8IxkETDSz/0qajk83rYqR2SvsCm3dEDPbIGkSHo5AwO/NrKNwK2uAHTEVNgN4tVI+G7hP0gp86u7ZrjSsk1wJzJL0TeAB/H7WWzCzpyVdhz+4sR/uaf4yeSTeEXjwzjclnSnpy/hU4cSYmlyPd4Jd5U+4Z/UPAHPMbEVF0wJJQ4FH45mQrcB5Uf9WSW2h85I9+O6k5qQ38yR5FxBTdK+bmUk6G39gYlxH273DmibgD0Vc3lHd5N1JjqCS5N3BcOCOGCluAi5ssZ4k6ZAcQSVJkiS1JB+SSJIkSWpJdlBJkiRJLckOKkmSJKkl2UElSZIktSQ7qCRJkqSW/A9Fhej1MDtv7AAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "from sklearn.model_selection import learning_curve\n",
+ "\n",
+ "\n",
+ "pipe_lr = make_pipeline(StandardScaler(),\n",
+ " LogisticRegression(penalty='l2', random_state=1, solver='lbfgs'))\n",
+ "\n",
+ "train_sizes, train_scores, test_scores =\\\n",
+ " learning_curve(estimator=pipe_lr,\n",
+ " X=X_train,\n",
+ " y=y_train,\n",
+ " train_sizes=np.linspace(0.1, 1.0, 10),\n",
+ " cv=10,\n",
+ " n_jobs=1)\n",
+ "\n",
+ "train_mean = np.mean(train_scores, axis=1)\n",
+ "train_std = np.std(train_scores, axis=1)\n",
+ "test_mean = np.mean(test_scores, axis=1)\n",
+ "test_std = np.std(test_scores, axis=1)\n",
+ "\n",
+ "plt.plot(train_sizes, train_mean,\n",
+ " color='blue', marker='o',\n",
+ " markersize=5, label='training accuracy')\n",
+ "\n",
+ "plt.fill_between(train_sizes,\n",
+ " train_mean + train_std,\n",
+ " train_mean - train_std,\n",
+ " alpha=0.15, color='blue')\n",
+ "\n",
+ "plt.plot(train_sizes, test_mean,\n",
+ " color='green', linestyle='--',\n",
+ " marker='s', markersize=5,\n",
+ " label='validation accuracy')\n",
+ "\n",
+ "plt.fill_between(train_sizes,\n",
+ " test_mean + test_std,\n",
+ " test_mean - test_std,\n",
+ " alpha=0.15, color='green')\n",
+ "\n",
+ "plt.grid()\n",
+ "plt.xlabel('Number of training examples')\n",
+ "plt.ylabel('Accuracy')\n",
+ "plt.legend(loc='lower right')\n",
+ "plt.ylim([0.8, 1.03])\n",
+ "plt.tight_layout()\n",
+ "#plt.savefig('images/06_05.png', dpi=300)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "from sklearn.model_selection import validation_curve\n",
+ "\n",
+ "\n",
+ "param_range = [0.001, 0.01, 0.1, 1.0, 10.0, 100.0]\n",
+ "train_scores, test_scores = validation_curve(\n",
+ " estimator=pipe_lr, \n",
+ " X=X_train, \n",
+ " y=y_train, \n",
+ " param_name='logisticregression__C', \n",
+ " param_range=param_range,\n",
+ " cv=10)\n",
+ "\n",
+ "train_mean = np.mean(train_scores, axis=1)\n",
+ "train_std = np.std(train_scores, axis=1)\n",
+ "test_mean = np.mean(test_scores, axis=1)\n",
+ "test_std = np.std(test_scores, axis=1)\n",
+ "\n",
+ "plt.plot(param_range, train_mean, \n",
+ " color='blue', marker='o', \n",
+ " markersize=5, label='training accuracy')\n",
+ "\n",
+ "plt.fill_between(param_range, train_mean + train_std,\n",
+ " train_mean - train_std, alpha=0.15,\n",
+ " color='blue')\n",
+ "\n",
+ "plt.plot(param_range, test_mean, \n",
+ " color='green', linestyle='--', \n",
+ " marker='s', markersize=5, \n",
+ " label='validation accuracy')\n",
+ "\n",
+ "plt.fill_between(param_range, \n",
+ " test_mean + test_std,\n",
+ " test_mean - test_std, \n",
+ " alpha=0.15, color='green')\n",
+ "\n",
+ "plt.grid()\n",
+ "plt.xscale('log')\n",
+ "plt.legend(loc='lower right')\n",
+ "plt.xlabel('Parameter C')\n",
+ "plt.ylabel('Accuracy')\n",
+ "plt.ylim([0.8, 1.0])\n",
+ "plt.tight_layout()\n",
+ "# plt.savefig('images/06_06.png', dpi=300)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ "execution_count": 19,
+ "metadata": {
+ "image/png": {
+ "width": 500
+ }
+ },
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "Image(filename='images/06_07.png', width=500) "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CV accuracy: 0.974 +/- 0.015\n"
+ ]
+ }
+ ],
+ "source": [
+ "gs = GridSearchCV(estimator=pipe_svc,\n",
+ " param_grid=param_grid,\n",
+ " scoring='accuracy',\n",
+ " cv=2)\n",
+ "\n",
+ "scores = cross_val_score(gs, X_train, y_train, \n",
+ " scoring='accuracy', cv=5)\n",
+ "print('CV accuracy: %.3f +/- %.3f' % (np.mean(scores),\n",
+ " np.std(scores)))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CV accuracy: 0.934 +/- 0.016\n"
+ ]
+ }
+ ],
+ "source": [
+ "from sklearn.tree import DecisionTreeClassifier\n",
+ "\n",
+ "gs = GridSearchCV(estimator=DecisionTreeClassifier(random_state=0),\n",
+ " param_grid=[{'max_depth': [1, 2, 3, 4, 5, 6, 7, None]}],\n",
+ " scoring='accuracy',\n",
+ " cv=2)\n",
+ "\n",
+ "scores = cross_val_score(gs, X_train, y_train, \n",
+ " scoring='accuracy', cv=5)\n",
+ "print('CV accuracy: %.3f +/- %.3f' % (np.mean(scores), \n",
+ " np.std(scores)))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ "execution_count": 22,
+ "metadata": {
+ "image/png": {
+ "width": 300
+ }
+ },
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "Image(filename='images/06_08.png', width=300) "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[71 1]\n",
+ " [ 2 40]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "from sklearn.metrics import confusion_matrix\n",
+ "\n",
+ "pipe_svc.fit(X_train, y_train)\n",
+ "y_pred = pipe_svc.predict(X_test)\n",
+ "confmat = confusion_matrix(y_true=y_test, y_pred=y_pred)\n",
+ "print(confmat)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAKkAAACsCAYAAAAAGIycAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAMr0lEQVR4nO3df5RVZb3H8fcH5iZ6+akMCvmLFAXEgAGpENAbRFiW2MISNTNREu81r666C9NWWJneLE1CM7x4WWliWlkuAUnJGkBEJmQkMPwVhEAx3kEEf8CA3/vH3sMchjlnNsTe+8Hzfa01a52z957zfM/w4dn77P2c/cjMcC5kbfIuwLnWeEhd8DykLngeUhc8D6kLnofUBc9DCkgaI2m1pJclTc67nixJulfSJkl/zruWYso+pJLaAncCZwF9gfGS+uZbVaZmAmPyLqKUsg8pMAR42cxeNbMdwIPAOTnXlBkzqwbq866jFA8pfBBYV/D8tXiZC4SHFNTCMr9WHBAPadRzHlPw/GhgQ061uBZ4SGEp0EtST0kfAM4HHs25Jleg7ENqZjuB/wDmAS8AD5nZynyryo6kWcBi4GRJr0makHdNzcmH6rnQlX1P6sLnIXXB85C64HlIXfA8pDFJE/OuIU8hv38PaZNg/5EyEuz795C64AV1nrRT5y7W7ageubS95Y3NdOrcJZe2G3Vqf2hubdfV1VFZWZlb+8+vWPHmju3bO7W0riLrYkrpdlQP7pj+YN5l5OaTw07Nu4TcVHY9fFOxdb67d8HzkLrgeUhd8DykLngeUhc8D6kLnofUBc9D6oLnIXXB85C64HlIXfA8pC54HlIXPA+pC56H1AXPQ+qC5yF1wfOQuuB5SF3wPKQueB5SFzwPqQueh9QFz0PqguchdcHzkLrgBXWbnay89re/csuN/7X7+d83vMZFl17JEV2P5IGZP2Hd2le5/e4H6NX7lByrzMZlEy5l9uzH6NatG7XPhzm9aKo9aagTyx59bE+mzXiYaTMe5o7pD3JIu3YMHT6S43qeyPXfuY1+/QflXWJmLv7SJcye83jeZZSUWk9aMLHsJ4gm9Foq6VEzW5VWm/ujdtkSuvc4hrzu5pe3ESNGsGbNmrzLKCnNnvSgmFi2ev7jnDHyrLzLcCWkGdJEE8tKmiipRlLNljc2p1jO3hoaGljy9B8YduboTNt1+ybNkCaaWNbMppvZYDMbnPVNbGuWLOSEXn3ocvgRmbbr9k2aIQ1+Ytnq+XN9V38QSDOkQU8s++677/BczWKGjhi5e9nT1fO5eNwoXlhZy5TJ/843v3ZFjhVm48ILxjPs9I+xevVqjjv2aO6dMSPvkvaS6j3zJX0K+BHQFrjXzG4qtX2v3qeY3468PFV2Pfzl+vr6Xi2tS/VkvpnNAeak2YZ7//PLoi54HlIXPA+pC17RY1JJHUv9opm9eeDLcW5vpT44rSQ6+V54Ur7xuQHHpliXc7sVDamZHVNsnXNZSnRMKul8Sd+IHx8tqXzGsrnctRpSSdOAfwO+GC96G7g7zaKcK5TkZP5QM6uS9ByAmdXHlzmdy0SS3X2DpDbEI5gkHQG8l2pVzhVIEtI7gV8BlZJuBBYC/51qVc4VaHV3b2Y/k/QnYFS86DwzC/MbW+59KekAk7ZAA9Eu369SuUwl+XR/PTAL6EE0cPkBSdelXZhzjZL0pBcBg8zsbQBJNwF/Am5OszDnGiXZda9lzzBXAK+mU45zeys1wOR2omPQt4GVkubFz0cTfcJ3LhOldveNn+BXArMLlj+TXjnO7a3UAJPwvpHlylKrH5wknQDcBPQF2jUuN7OTUqzLud2SfHCaCfwv0TjSs4CHiG6Z41wmkoT0MDObB2Bmr5jZDUSjopzLRJLzpNslCXhF0hXAeqBbumU51yRJSK8B2gNfJTo27QRcmmZRzhVKMsBkSfxwK00Dn53LTKmT+Y/Qwl3wGpnZ51KpyLlmSvWk0zKrItax/aGMHtYv62aDUbthS94l5OatHbuKrit1Mn9+KtU4t498bKgLnofUBS9xSCUdkmYhzhWTZGT+EEkrgJfi5/0l/Tj1ypyLJelJpwJnA/8HYGa1+GVRl6EkIW1jZmubLSt+vsC5AyzJZdF1koYAFs9ydxXwYrplOdckSU86CbiW6FaP/wA+Gi9zLhNJrt1vIprexrlcJBmZfw8tz2Q3MZWKnGsmyTHpkwWP2wHnsuecoc6lKsnu/heFzyXdBzyRWkXONbM/l0V7Ascd6EKcKybJMelmmo5J2wD1wOQ0i3KuUMmQxt9t6k/0vSaA9yzNyUida0HJ3X0cyEfMbFf84wF1mUtyTPqspKrUK3GuiFLfcaows53AMOBySa8AbxFPNmZmHlyXiVLHpM8CVcDYjGpxrkWlQiqI7lqSUS3OtahUSCslXVtspZndlkI9zu2lVEjbEt25RCW2cS51pUK60cy+nVklzhVR6hSU96AuCKVCOjKzKpwroWhIzaw+y0KcK6bsbw6xbt06Ro38OKee0pf+p/Zj6tQ78i4pE7t27eKCMcO5+pIvALD+b2u4+DMjGTu8ismTvkzDjh05V9gktZBKulfSJklBz0NaUVHB92/9AStWrmLh04u5+667WLVqVd5lpW7WjJ9w/Ikn734+9eYpXHjZlfxmwTI6du7Mbx68L8fq9pRmTzoTGJPi6x8Q3bt3p6oqusLboUMHevfuw4b161v5rYPbPzauZ+Hvf8fY8dHtZs2MpYuqGfnpcwA4e9x4/jBvdqmXyFRqITWzaqKxpweNNWvWsHz5cwz5yEfyLiVVP5xyHVd/49u0aRP987+xuZ4OHTtRURGdkezWvQd1f9+YZ4l7yP2YVNJESTWSal6vq8utjm3btvH588bxw9tup2PHjrnVkbbqJx+nyxGV9PnwgKaFLY3AVDhnIJNOJZ4aM5sOTAcYNHhwLuNVGxoa+Py4cYy/4ALO/dz7+wbWtTVLqH5iLoue+h07tm9n29at/GDKdWx9cws7d+6koqKCTRs3UHnkUXmXulvuPWnezIzLL7uM3n16c801RYcqvG9cNflbzF26iscWr+B7d87gtNNHcNOP72Hw0OHMn/1bAB775SzOGP2pnCttUvYhXbRoET+//z6eeuopBlUNZFDVQObOmZN3WZn76nU38vN77uScYQN5Y3M9Y88PZw4PpfWNEEmzgDOBrkS35/lWa/OVDho82JY8uzSVeg4Gz294M+8ScvOx3se+vH3bll4trUvtmNTMxqf12q68lP3u3oXPQ+qC5yF1wfOQuuB5SF3wPKQueB5SFzwPqQueh9QFz0PqguchdcHzkLrgeUhd8DykLngeUhc8D6kLnofUBc9D6oLnIXXB85C64HlIXfA8pC54HlIXPA+pC56H1AXPQ+qCl9q9oPaHpDpgbU7NdwVez6ntEOT9/o8zs8qWVgQV0jxJqjGzwXnXkZeQ37/v7l3wPKQueB7SJtPzLiBnwb7/sg2ppF2Slkv6s6SHgfv/idc6U9Jj8ePPSppcYtvOkq7cjzamSPpa0uXNtpkpaVypbeK5Cxq3Pz6k+bfKNqTAO2Y2wMz6ATuAKwpXKrLPfx8ze9TMbimxSWdgn0Nazso5pIUWACfGPcgLku4ClgHHSBotabGkZZIeltQeQNIYSX+RtBDYPWWJpEskTYsfHynpEUm18c9Q4BbghLgXvzXe7uuSlkp6XtKNBa91vaTVkp4EmqavK0LS5fHr1Er6laTDClaPkrRA0ouSzo63byvp1oK2v/LP/iHTUPYhlVQBnAWsiBedDPzMzAYCbwE3AKPMrAqoAa6V1A64B/gMMBwoNp/MVOCPZtYfqAJWApOBV+Je/OuSRgO9gCHAAGCQpBGSBgHnAwOJ/hOcluDt/NrMTovbewGYULDueOAM4NPA3fF7mABsMbPT4te/XFLPBO1kKvd5nHJ0qKTl8eMFwAygB7DWzJ6Jl38U6AssUjT51geAxUBv4K9m9hKApPuBiS208XHgYgAz2wVskdSl2Taj45/n4uftiULbAXjEzN6O23g0wXvqJ+m7RIcU7YF5BeseMrP3gJckvRq/h9HAhwuOVzvFbb+YoK3MlHNI3zGzAYUL4iC+VbgIeKL5JBWSBgAH6iqIgJvN7KfN2vjP/WhjJjDWzGolXUI0+0uj5q9lcdtXmVlhmJF0/D62m6qy39234hngdEknAkg6TNJJwF+AnpJOiLcrNtPKfGBS/LttJXUEthL1ko3mAZcWHOt+UFI3oBo4V9KhkjoQHVq0pgOwUdK/ABc2W3eepDZxzR8CVsdtT4q3R9JJkv41QTuZKueetFVmVhf3SLMkHRIvvsHMXpQ0EZgt6XVgIdCvhZe4GpguaQKwC5hkZoslLYpP8cyNj0v7AIvjnnwbcJGZLZP0C2A50XiGBQlK/iawJN5+BXv+Z1gN/BE4ErjCzN6V9D9Ex6rLFDVeB4xN9tfJjl+7d8Hz3b0LnofUBc9D6oLnIXXB85C64HlIXfA8pC54/w9LulSO/Cy8jAAAAABJRU5ErkJggg==\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "fig, ax = plt.subplots(figsize=(2.5, 2.5))\n",
+ "ax.matshow(confmat, cmap=plt.cm.Blues, alpha=0.3)\n",
+ "for i in range(confmat.shape[0]):\n",
+ " for j in range(confmat.shape[1]):\n",
+ " ax.text(x=j, y=i, s=confmat[i, j], va='center', ha='center')\n",
+ "\n",
+ "plt.xlabel('Predicted label')\n",
+ "plt.ylabel('True label')\n",
+ "\n",
+ "plt.tight_layout()\n",
+ "#plt.savefig('images/06_09.png', dpi=300)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Additional Note"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Remember that we previously encoded the class labels so that *malignant* examples are the \"postive\" class (1), and *benign* examples are the \"negative\" class (0):"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([1, 0])"
+ ]
+ },
+ "execution_count": 25,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "le.transform(['M', 'B'])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[71 1]\n",
+ " [ 2 40]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "confmat = confusion_matrix(y_true=y_test, y_pred=y_pred)\n",
+ "print(confmat)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Next, we printed the confusion matrix like so:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[71 1]\n",
+ " [ 2 40]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "confmat = confusion_matrix(y_true=y_test, y_pred=y_pred)\n",
+ "print(confmat)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Note that the (true) class 0 examples that are correctly predicted as class 0 (true negatives) are now in the upper left corner of the matrix (index 0, 0). In order to change the ordering so that the true negatives are in the lower right corner (index 1,1) and the true positves are in the upper left, we can use the `labels` argument like shown below:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "[[40 2]\n",
+ " [ 1 71]]\n"
+ ]
+ }
+ ],
+ "source": [
+ "confmat = confusion_matrix(y_true=y_test, y_pred=y_pred, labels=[1, 0])\n",
+ "print(confmat)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We conclude:\n",
+ "\n",
+ "Assuming that class 1 (malignant) is the positive class in this example, our model correctly classified 71 of the examples that belong to class 0 (true negatives) and 40 examples that belong to class 1 (true positives), respectively. However, our model also incorrectly misclassified 1 example from class 0 as class 1 (false positive), and it predicted that 2 examples are benign although it is a malignant tumor (false negatives)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ "metadata": {
+ "needs_background": "light"
+ },
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "from sklearn.metrics import roc_curve, auc\n",
+ "from scipy import interp\n",
+ "\n",
+ "pipe_lr = make_pipeline(StandardScaler(),\n",
+ " PCA(n_components=2),\n",
+ " LogisticRegression(penalty='l2', \n",
+ " random_state=1,\n",
+ " solver='lbfgs',\n",
+ " C=100.0))\n",
+ "\n",
+ "X_train2 = X_train[:, [4, 14]]\n",
+ " \n",
+ "\n",
+ "cv = list(StratifiedKFold(n_splits=3, \n",
+ " random_state=1).split(X_train, y_train))\n",
+ "\n",
+ "fig = plt.figure(figsize=(7, 5))\n",
+ "\n",
+ "mean_tpr = 0.0\n",
+ "mean_fpr = np.linspace(0, 1, 100)\n",
+ "all_tpr = []\n",
+ "\n",
+ "for i, (train, test) in enumerate(cv):\n",
+ " probas = pipe_lr.fit(X_train2[train],\n",
+ " y_train[train]).predict_proba(X_train2[test])\n",
+ "\n",
+ " fpr, tpr, thresholds = roc_curve(y_train[test],\n",
+ " probas[:, 1],\n",
+ " pos_label=1)\n",
+ " mean_tpr += interp(mean_fpr, fpr, tpr)\n",
+ " mean_tpr[0] = 0.0\n",
+ " roc_auc = auc(fpr, tpr)\n",
+ " plt.plot(fpr,\n",
+ " tpr,\n",
+ " label='ROC fold %d (area = %0.2f)'\n",
+ " % (i+1, roc_auc))\n",
+ "\n",
+ "plt.plot([0, 1],\n",
+ " [0, 1],\n",
+ " linestyle='--',\n",
+ " color=(0.6, 0.6, 0.6),\n",
+ " label='random guessing')\n",
+ "\n",
+ "mean_tpr /= len(cv)\n",
+ "mean_tpr[-1] = 1.0\n",
+ "mean_auc = auc(mean_fpr, mean_tpr)\n",
+ "plt.plot(mean_fpr, mean_tpr, 'k--',\n",
+ " label='mean ROC (area = %0.2f)' % mean_auc, lw=2)\n",
+ "plt.plot([0, 0, 1],\n",
+ " [0, 1, 1],\n",
+ " linestyle=':',\n",
+ " color='black',\n",
+ " label='perfect performance')\n",
+ "\n",
+ "plt.xlim([-0.05, 1.05])\n",
+ "plt.ylim([-0.05, 1.05])\n",
+ "plt.xlabel('false positive rate')\n",
+ "plt.ylabel('true positive rate')\n",
+ "plt.legend(loc=\"lower right\")\n",
+ "\n",
+ "plt.tight_layout()\n",
+ "# plt.savefig('images/06_10.png', dpi=300)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "