Model Selection (Regularization and polynoms)/exercise5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Programming Exercise 5:\n",
    "# Regularized Linear Regression and Bias vs Variance\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In this exercise, you will implement regularized linear regression and use it to study models with different bias-variance properties. Before starting on the programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics.\n",
    "\n",
    "All the information you need for solving this assignment is in this notebook, and all the code you will be implementing will take place within this notebook. The assignment can be promptly submitted to the coursera grader directly from this notebook (code and instructions are included below).\n",
    "\n",
    "Before we begin with the exercises, we need to import all libraries required for this programming exercise. Throughout the course, we will be using [`numpy`](http://www.numpy.org/) for all arrays and matrix operations, [`matplotlib`](https://matplotlib.org/) for plotting, and [`scipy`](https://docs.scipy.org/doc/scipy/reference/) for scientific and numerical computation functions and tools. You can find instructions on how to install required libraries in the README file in the [github repository](https://github.com/dibgerge/ml-coursera-python-assignments)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# used for manipulating directory paths\n",
    "import os\n",
    "\n",
    "# Scientific and vector computation for python\n",
    "import numpy as np\n",
    "\n",
    "# Plotting library\n",
    "from matplotlib import pyplot\n",
    "\n",
    "# Optimization module in scipy\n",
    "from scipy import optimize\n",
    "\n",
    "# will be used to load MATLAB mat datafile format\n",
    "from scipy.io import loadmat\n",
    "\n",
    "# library written for this exercise providing additional functions for assignment submission, and others\n",
    "import utils\n",
    "\n",
    "# define the submission/grader object for this exercise\n",
    "grader = utils.Grader()\n",
    "\n",
    "# tells matplotlib to embed plots within the notebook\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submission and Grading\n",
    "\n",
    "\n",
    "After completing each part of the assignment, be sure to submit your solutions to the grader. The following is a breakdown of how each part of this exercise is scored.\n",
    "\n",
    "\n",
    "| Section | Part                                             | Submitted Function                | Points |\n",
    "| :-      |:-                                                |:-                                 | :-:    |\n",
    "| 1       | [Regularized Linear Regression Cost Function](#section1)      | [`linearRegCostFunction`](#linearRegCostFunction) |  25    |\n",
    "| 2       | [Regularized Linear Regression Gradient](#section2)           | [`linearRegCostFunction`](#linearRegCostFunction) |25      |\n",
    "| 3       | [Learning Curve](#section3)                                   | [`learningCurve`](#func2)         | 20     |\n",
    "| 4       | [Polynomial Feature Mapping](#section4)                       | [`polyFeatures`](#polyFeatures)          | 10     |\n",
    "| 5       | [Cross Validation Curve](#section5)                           | [`validationCurve`](#validationCurve)       | 20     |\n",
    "|         | Total Points                                     |                                   |100     |\n",
    "\n",
    "\n",
    "You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
    "\n",
    "<div class=\"alert alert-block alert-warning\">\n",
    "At the end of each section in this notebook, we have a cell which contains code for submitting the solutions thus far to the grader. Execute the cell to see your score up to the current section. For all your work to be submitted properly, you must execute those cells at least once.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section1\"></a>\n",
    "## 1 Regularized Linear Regression\n",
    "\n",
    "In the first half of the exercise, you will implement regularized linear regression to predict the amount of water flowing out of a dam using the change of water level in a reservoir. In the next half, you will go through some diagnostics of debugging learning algorithms and examine the effects of bias v.s.\n",
    "variance. \n",
    "\n",
    "### 1.1 Visualizing the dataset\n",
    "\n",
    "We will begin by visualizing the dataset containing historical records on the change in the water level, $x$, and the amount of water flowing out of the dam, $y$. This dataset is divided into three parts:\n",
    "\n",
    "- A **training** set that your model will learn on: `X`, `y`\n",
    "- A **cross validation** set for determining the regularization parameter: `Xval`, `yval`\n",
    "- A **test** set for evaluating performance. These are “unseen” examples which your model did not see during training: `Xtest`, `ytest`\n",
    "\n",
    "Run the next cell to plot the training data. In the following parts, you will implement linear regression and use that to fit a straight line to the data and plot learning curves. Following that, you will implement polynomial regression to find a better fit to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = loadmat(os.path.join('Data', 'ex5data1.mat'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(12,)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['y'][:, 0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(21, 1)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Xtest'].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = data['X'], data['y'][:, 0]\n",
    "Xtest, ytest = data['Xtest'], data['ytest'][:, 0]\n",
    "Xval, yval = data['Xval'], data['yval'][:, 0]\n",
    "\n",
    "\n",
    "m = y.size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['y'].size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxkdZnv8c9XaCHIptKWCGp3EHQQoZGEYcQNFGSUqBmXEbiKy7XHccPREQW9LlwcYVBAx6sjjmjjdIsIlhIWaUQQF6STBgLdIi4JKNiEoCCtCUozz/3j/EIXIVU5SdepJfV9v171ytnqnKdOp586+Z3feX6KCMzMrHM8qtkBmJlZYznxm5l1GCd+M7MO48RvZtZhnPjNzDrM1s0OII9ddtkllixZ0uwwzMzaytq1a++OiMXTl7dF4l+yZAlDQ0PNDsPMrK1Ium2m5W7qMTPrME78ZmYdxonfzKzDOPGbmXWYtri5a2bWaSYmJiiXy4yOjtLd3U1/fz9dXV112bcTv5lZixkcHKSvr4+xsbGHlpVKJQYGBujt7d3i/bupx8yshUxOTlYk/WXA8cB+jI2N0dfXx+Tk5BYfw4nfzKyFlMvliqS/BjgVGGQq+ZfL5S0+hhO/mVkLGRkZSVOHA4vS9KI0X7l+/pz4zcxaSHd3d5q6DHggTT8ArJ62fv6c+M3MWkh/fz+lUgkYBnrJ2vh7gWFKpRL9/f1bfAwnfjOzFtLV1cXAwEBF8j+NqaQ/MDBQly6d7s5pZtZient7GR0dpVwuMzIy4n78ZmadoKuri6OPPrqQfbupx8yswxSW+CVtK2mNpGFJ6yV9PC3/qqRRSTek17KiYjAzs0cqsqnnL8ChEfEnSYuAH0m6NK17f0ScX+CxzcysisISf0QE8Kc0uyi9oqjjmZlZPoW28UvaStINwF3A5RFxbVr1CUk3SjpD0jZV3rtc0pCkofHx8SLDNDNrqomJCVauXMnJJ5/MqlWr6lKPpxZlF+bFkrQzUAbeBfweuBN4NHAW8OuIOKnW+3t6esJj7prZQlRkJU5JayOiZ/ryhvTqiYh7gSuBIyJiQ2T+AnwFOLARMZiZtZpGVOKcSZG9ehanK30kdQGHAT+XtGtaJuCVwLqiYjAza2WNqMQ5kyJ79ewKrJC0FdkXzHkRcZGk70taDAi4AXhbgTGYmbWs2pU4h+tSiXMmRfbquRHYf4blhxZ1TDOzdvLwSpwnkyX9+lbinImf3DUza5JGVOKciRO/mVmTNKIS50xcpM3MrImKrsQ5Eyd+M7MmK7IS50xqJn5J2wJHAs8DngRMknW/vDgi1hcfnpmZ1VvVxJ+qaR4JXAVcS1Z2YVtgL+CU9KXwvtR7x8zM2kStK/41EfHRKutOl/QE4CkFxGRmZgWq2qsnIi4GkPSsKuvviggX0DEzazN5unN+Pg2o8nZJOxUekZmZFWrWxB8RzwOOAZ4MrJW0StJhhUdmZmaFyPUAV0T8Evgw8AHgBcBnJf1c0j8UGZyZmdXfrIlf0r6SzgBuBg4F+iLib9L0GQXHZ2ZmdZbnAa7/AP4LODEiHioOHRG/k/ThwiIzM7NCzJr4I+IFNdZ9rb7hmJlZ0ao29UgakNQnadEM67olnSTpzcWGZ2Zm9Vbriv+twHuBMyX9ARgne3J3CfBr4HMR8Z3CIzQzs7qqmvgj4k6y4tDHS1pCNqLWJPCLiJhoSHRmZlZ3uapzRsStwK2FRmJmZg3hgVjMzDpMYYlf0rap1MOwpPWp2ieSlkq6VtKvJH1D0qOLisHMzB6pyCv+vwCHRsR+wDLgCEkHAacCZ0TE04B7gLcUGIOZmU2T58ndIyVdL+kPku6TtFHSfbO9LzJ/SrOL0ivInvg9Py1fAbxynrGbmdk85LniPxM4Fnh8ROwYETtExI55di5pK0k3kA3icjlZN9B7I2JT2uR2YLcq710uaUjS0Pj4eJ7DmZlZDnkS/2+BdRERc915RDwYEcuA3YEDgWfM4b1nRURPRPQsXrx4roc2M7Mq8nTnPB64RNIPyNrtAYiI0/MeJCLulXQl8HfAzpK2Tlf9uwN3zDFmMzPbAnmu+D8BTJA9tbtDxasmSYsl7Zymu4DDyCp8Xgm8Om12LOCnf83MGijPFf+TImKfeex7V2CFpK3IvmDOi4iLJP0MOFfSycD1wJfnsW8zM5unPIn/EkmHR8Tquew4Im4E9p9h+QhZe7+ZmTVBnqaefwa+K2lyLt05zcysNeWpxz9re76ZmbWPXEXaJD0W2JPsBi8AEXF1UUGZmVlxZk38kv43cBxZ18sbgIOAa8iewDUzszaTp43/OKAXuC0iDiG7YXtvoVGZmVlh8iT++yPifgBJ20TEz4GnFxuWmZkVJU8b/+3pQaxvA5dLuge4rdiwzMysKHl69fSnyY+lsgs7Ad8tNCozMytM1cQv6XEzLL4p/dwe+EMhEZmZWaFqXfGvJaufL+ApZIOmCNgZ+A2wtPDozMys7qre3I2IpRHRDXwP6IuIXSLi8cCRwJzKN5iZWevI06vnoIi4ZGomIi4FnlNcSGZmVqQ8vXp+J+nDwH+n+WOA3xUXkpmZFSnPFf9RwGKgDHwrTR9VZFBmZlacPN05/0D29K6ZmS0Aea74zcxsAXHiNzPrME78ZmYdZtbEL2kvSVdIWpfm9029fMzMrA3lueL/EnAC8AA8NJbu62Z7k6QnS7pS0s8krZd0XFr+MUl3SLohvV66JR/AzMzmJk8//u0iYo2kymWbcrxvE/C+iLhO0g7AWkmXp3VnRMSn5hirmZnVQZ7Ef7ekPcjq9iDp1cCG2d4UERumtouIjZJuBnbbgljNzKwO8jT1vAP4IvAMSXcA7wH+eS4HkbSEbOSua9Oid0q6UdLZaTzfmd6zXNKQpKHx8fG5HM7MzGpQROTbUHoM8KiI2DinA0jbAz8APhER35JUAu4m+wvi/wK7RsSba+2jp6cnhoaG5nJYM7OOJ2ltRPRMX55nsPVtgFcBS4Ctp9r6I+KkHO9dBFwArIyIb6X3jVWs/xJwUb6PYGZm9ZCnjf87wB/J6vP/Je+OlX1DfBm4OSJOr1i+a2r/B+gH1uUP18zMtlSexL97RBwxj30fDLweuEnSDWnZicBRkpaRNfXcCvzTPPZtZmbzlCfx/0TSsyLiptk33SwifkQ2Ytd0l8ywzMzMGqTWmLs3kV2Vbw28SdIIWVOPgIiIfRsTopmZ1VOtK/4jGxaFmZk1TNXEHxG3AUj6WkS8vnKdpK+Rtd+bmTXdxMQE5XKZ0dFRuru76e/vp6urq9lhtaw8bfzPrJyRtBVwQDHhmJnNzeDgIH19fYyNPdRTnFKpxMDAAL29vU2MrHVVfXJX0gmSNgL7SrovvTYCd5F18TQza6rJycmKpL8MOB7Yj7GxMfr6+picnGxyhK2pauKPiE9GxA7AaRGxY3rtEBGPj4gTGhijmdmMyuVyRdJfA5wKDDKV/MvlclPja1Wz1upxkjezVjUyMpKmDgcWpelFab5yvVXyCFxm1ra6u7vT1GWkIUPSz9XT1lulWm38SxsZiJnZXPX391MqlYBhoJesjb8XGKZUKtHf39/U+FpVrSv+8wEkXdGgWMzM5qSrq4uBgYGK5H8aU0l/YGDAXTqrqNWd81GSTgT2kvTe6SsrC6+ZmTVLb28vo6OjlMtlRkZG3I8/h1qJ/3XAK9M2OzQmHDOzuevq6uLoo49udhhto9aTu7cAp0q6MSIubWBMZmZWoDy9en4i6fSpYRAlfVrSToVHZmZmhciT+M8GNgKvTa/7gK8UGZSZmRUnT62ePSLiVRXzH68YWMXMzNpMniv+SUnPnZqRdDDgAhhmZm0qzxX/24BzKtr17wGOLS4kMzMr0qyJPyKGgf0k7Zjm78uzY0lPBs4BSmQjeZ0VEZ+R9DjgG8ASsjF3XxsR98wrejMzm7PctXoi4r68ST/ZBLwvIvYGDgLeIWlv4IPAFRGxJ3BFmjczswYprEhbRGyIiOvS9EbgZmA34BXAirTZCrKHxMzMrEFmTfyStsmzbJZ9LAH2B64FShGxIa26k6wpyMzMGiTPFf81OZfNSNL2wAXAe6Y3FUVEkLX/z/S+5VMPjY2Pj+c9nJmZzaLqzV1JTyRrmumStD+gtGpHYLs8O5e0iCzpr4yIb6XFY5J2jYgNknYlG8rxESLiLOAsgJ6enhm/HMzMbO5q9ep5CfBGYHegshLnRuDE2XYsScCXgZunVfK8kKw76Cnpp8fvNTNroFpF2lYAKyS9KiIumMe+DwZeD9xU8aTviWQJ/zxJbwFuIysDYWZmDZLnAa59JD1z+sKIOKnWmyLiR2xuHpruRTmOa2ZmBciT+P9UMb0tcCRZ10wzM2tDeZ7c/XTlvKRPkY1sbGZmbWg+D3BtR3bD18zM2tCsV/ySbmJzX/utgMVAzfZ9MzNrXXna+I+smN4EjEXEpoLiMTOzgs3a1BMRtwE7A31AP7B30UGZmVlx8tTqOQ5YCTwhvVZKelfRgZmZWTHyNPW8BfjbiPgzgKRTyWr1/EeRgZmZWTHy9OoR8GDF/INUfzDLzMxaXJ4r/q8A10oqp/lXktXgMTOzNpTnAa7TJV0FTA24/qaIuL7QqMzMrDB5rvhJI2ldV3AsZmbWAIUNvWhmZq3Jid/MrMM48ZuZdZg8tXo28shxcf8IDAHvi4iRIgIzM7Ni5Lm5eyZwO7CKrP/+64A9yG72ng28sKjgzMys/vI09bw8Ir4YERsj4r40CPpLIuIbwGMLjs/MzOosT+KfkPRaSY9Kr9cC96d105uAzMysxeVJ/MeQDZp+FzCWpv+XpC7gndXeJOlsSXdJWlex7GOS7pB0Q3q9dAvjNzOzOcrz5O4IWUnmmfyoxlu/CnwOOGfa8jMi4lO5ojOztjYxMUG5XGZ0dJTu7m76+/vp6upqdlgdL0+vnsXAW4ElldtHxJtrvS8irpa0ZMvCM7N2NTg4SF9fH2NjYw8tK5VKDAwM0Nvb28TILE9Tz3eAnYDvARdXvObrnZJuTE1BVW8OS1ouaUjS0Pj4+BYczswabXJysiLpLwOOB/ZjbGyMvr4+JicnmxxhZ8uT+LeLiA9ExHkRccHUa57H+wJZV9BlwAbg09U2jIizIqInInoWL148z8OZWTOUy+WKpL8GOBUYZCr5l8vlmu+3YuVJ/BfV6yZsRIxFxIMR8T/Al4AD67FfM2stIyNTz3UeDixK04vSfOV6a4Y8if84suQ/Kek+SRsl3Tefg0natWK2H1hXbVsza1/d3d1p6jLggTT9ALB62nprhjy9enaYz44lfZ3sqd5dJN0OfBR4oaRlZP3/bwX+aT77NrPW1t/fT6lUYmxsGOglu9JfDQxTKpXo7+9vboAdrmril/SMiPi5pGfPtD7V6K8qIo6aYbFH7jLrAF1dXQwMDKQbvMPAMLC5V4+7dDZXrSv+9wLLmfkGbACHFhKRmS0Ivb29jI6OUi6XGRkZcT/+FqKI1q+60NPTE0NDQ80Ow8ysrUhaGxE905fneYDrR8APgB8CP46IjQXEZ2ZmDZKnV8/rgVuAVwE/SQ9VnVFsWGZmVpQ8vXpGJd0P/DW9DgH+pujAzMysGLNe8Uv6NfBtoETWK2efiDii6MDMzKwYeZp6Pgv8BjgKeDdwrKQ9Co3KzMwKM2vij4jPRMRrgBcDa4GPAb8oOC4zMytInl49nwaeC2wPXAN8hKyHj5mZtaE8g61fA/x7RIzNuqWZmbW8PL16zpf0cknPT4t+EBEDBcdlZmYFydOr55NkFTp/ll7vlvRvRQdmZmbFyNPU8zJgWaqhj6QVwPXAiUUGZmZmxcjTnRNg54rpnYoIxMzMGiPPFf8ngeslXQkIeD7wwUKjMjOzwuS5uft1SVeRjaYA8IGIuLPQqMzMrDC1BmKZPgDL7ennkyQ9abaBWMzMrDXVuuKfaQCWKR6IxcysTdVK/F+IiPMkdUfESMMiMjOzQtXq1TN1A/f8+exY0tmS7pK0rmLZ4yRdLumX6edj57NvMzObv1qJ//eSVgNLJV04/ZVj318Fppdv/iBwRUTsCVyBeweZmTVcraaelwHPBr5G7fb+GUXE1ZKWTFv8CuCFaXoFcBXwgbnu28zM5q9q4o+IvwI/lfSciBiv0/FKEbEhTd9JNrjLjCQtB5YDPOUpT6nT4c3MLE89/nol/en7DbLeQdXWnxURPRHRs3jx4iJCMDPrSHlLNtTLmKRdAdLPuxp8fDOzjlcz8UvaStK/1PF4FwLHpuljge/Ucd9mZpZDzcQfEQ+SjbU7Z5K+TjaIy9Ml3S7pLcApwGGSfkk2lOMp89m3mZnNX54ibT+W9DngG8CfpxbOVrIhIqp9Ybwof3hmZlZveRL/svTzpIplLtlgZtam8lTnPKQRgZiZWWPkGXqxJOnLki5N83un9nozM2tDeZp6vgp8BfhQmv8FWXv/lwuKyczqbGJignK5zOjoKN3d3fT399PV1dXssKxJ8iT+XVKVzhMAImKTpAcLjsvM6mRwcJC+vj7GxsYeWlYqlRgYGKC3t7fGO22hyvMA158lPZ70lK2kg4A/FhqVmdXF5ORkRdJfBhwP7MfY2Bh9fX1MTk42OUJrhjyJ/71kD17tIenHwDnAuwuNyszqolwuVyT9NcCpwCBTyb9cLjc1PmuOPE0964EXAE8nG2z9Fhpf6sHM5mFkZGoMpcOBRWl6UZofrlhvnSRPAr8mIjZFxPqIWBcRD5A9kWtmLa67uztNXQY8kKYfAFZPW2+dpNZg608EdgO6JO1PdrUPsCOwXQNiM7Mt1N/fT6lUYmxsGOglu9JfDQxTKpXo7+9vboDWFLWael4CvBHYHTi9YvlG4MQCYzKzOunq6mJgYCDd4B0GhoHNvXrcpbMz1RqIZQWwQtKrIuKCBsZkZnXU29vL6Ogo5XKZkZER9+O3XCUbLpD0MuCZwLYVy0+q/i4zayVdXV0cffTRzQ7DWkSekg3/Cfwj8C6ydv7XAE8tOC4zMytInl49z4mINwD3RMTHgb8D9io2LDMzK0qexD/1aN+EpCeR9QXbtbiQzMysSHke4LpI0s7AacB1ZKUbvlRoVGZmVpha/fjfA/wE+GREbAIukHQRsG1EuFaPmVmbqnXFvztwJvAMSTcBPyb7IvhJIwIzM7Ni1OrH/68Akh4N9ADPAd4EnCXp3ojYe74HlXQr2YNgDwKbIqJnvvsyM7O5ydPG30VWpmGn9PodcFMdjn1IRNxdh/2Ymdkc1GrjP4vsoa2NwLVkTTynR8Q9DYrNzMwKUKs751OAbYA7gTuA24F763TcAFZLWitp+UwbSFouaUjS0Pj4eJ0Oa2ZmiojqKyWRXfU/J732Af5AVqr5o/M+qLRbRNwh6QnA5cC7IuLqatv39PTE0NDQfA9nZtaRJK2d6R5qzQe4IrMOuAS4lKxnzx7AcVsSTETckX7eBZSBA7dkf2Zmll+tNv53s/lK/wE2d+U8my24uSvpMcCjImJjmj4caImCbxMTE5TLZUZHR13B0JrKv4tWpFq9epYA3wT+JSI21PGYJaCctSKxNbAqIr5bx/3Py+DgYMWg1JmpmuW9vb1NjMw6jX8XrWg12/hbRdFt/JOTkyxdurRiUOrDyYaqy0YpGh0d9dWWNYR/F62e5tXG3ynK5XLFf7Q1wKnAILAfY2NjlMvlpsZnncO/i9YITvzAyMhImjocWJSmF6X5yvVmxfLvojXCgk38ExMTrFy5kpNPPplVq1YxOTlZddvu7u40dRnZfWzSz9XT1psVy7+L1ggLso1/rjfHHt6uuh/Z1dVqmtmu6l4dnakVfxetfVVr4yciWv51wAEHRF4TExNRKpUCCFgWcHzAfgFEqVSKiYmJGd+3Zs2aivfx0PZr1qzJfex6aaVYrPH872/1AgzFDDl1wV3xr1q1imOOOYbNN8cWkf2p3AsMs3LlyqqDTk9OTlIulxkZGWnaVbZ7dRi0xu+itb9qV/x5qnO2ldo3x4Zr3hzr6uqq+qXQKI/s1bEIOBnoZWxsmHK53PQYG6kRTV6t2KzWCr+LtnAtuMT/8JtjJ7P5ir89bo5tyRfXQtOIB5n8sJR1ogXXq6e/v59SqQQMkzXvHM9UM0+pVKK/v7+p8c3GvToyk5OTFQl5Gdm/Y9aXva+vr2YvrVY6hlkrWnCJv6uri4GBgYrkfxpTSX9gYKDpf8LPpt2/uOqlEQ8y+WEp61QLrqkHoLe3l9HR0ba8OTb1xZVdiQ6TfQHQNl9c9dKIJi83q1mnWpCJH9r75li9v7ha8eblbBpxr6bd7weZzdeC685pD9euNy8b8SCTH5ayhc5F2jpQO9+8bMS9mna/H2Q2Xwu2qcfa/5mARtyraef7QWbz5cS/gC2Em5eNuFfTzveDzObDTT0LmJ8JMLOZOPEvYH4mwMxm0pTEL+kISbdI+pWkDzYjhk7gm5dmNpOGt/FL2gr4f8BhwO3AoKQLI+JnjY6lE/jmpZlN14ybuwcCv4qIEQBJ5wKvAJz4C+Kbl2ZWqRlNPbsBv62Yvz0texhJyyUNSRoaHx9vWHBmZgtdy97cjYizIqInInoWL17c7HDMzBaMZiT+O4AnV8zvnpaZmVkDNCPxDwJ7Sloq6dHA64ALmxCHmVlHakqRNkkvBc4EtgLOjohPzLL9OHBbI2Krk12Au5sdRIvyuanO56Y6n5vqap2bp0bEI9rK26I6Z7uRNDRTRTzzuanF56Y6n5vq5nNuWvbmrpmZFcOJ38yswzjxF+OsZgfQwnxuqvO5qc7npro5nxu38ZuZdRhf8ZuZdRgnfjOzDuPEXwBJ75MUknZJ85L02VSG+kZJz252jI0m6TRJP0+fvyxp54p1J6Rzc4uklzQzzmZxqfLNJD1Z0pWSfiZpvaTj0vLHSbpc0i/Tz8c2O9ZmkbSVpOslXZTml0q6Nv3+fCM9HFuVE3+dSXoy2diGv6lY/PfAnum1HPhCE0JrtsuBfSJiX+AXwAkAkvYme3r7mcARwOdT6e6OUVGq/O+BvYGj0nnpVJuA90XE3sBBwDvS+fggcEVE7AlckeY71XHAzRXzpwJnRMTTgHuAt9R6sxN//Z1BNtRV5V3zVwDnROanwM6Sdm1KdE0SEasjYlOa/SlZjSbIzs25EfGXiBgFfkVWuruTPFSqPCL+CkyVKu9IEbEhIq5L0xvJEtxuZOdkRdpsBfDK5kTYXJJ2B14G/FeaF3AocH7aZNZz48RfR5JeAdwREcPTVuUqRd1B3gxcmqZ9bnwOqpK0BNgfuBYoRcSGtOpOoNSksJrtTLKLy/9J848H7q24sJr196cZA7G0NUnfA544w6oPASeSNfN0pFrnJiK+k7b5ENmf8isbGZu1H0nbAxcA74mI+7IL20xEhKSO64su6UjgrohYK+mF892PE/8cRcSLZ1ou6VnAUmA4/YLuDlwn6UA6pBR1tXMzRdIbgSOBF8XmB0g64tzMwudgGkmLyJL+yoj4Vlo8JmnXiNiQmkrval6ETXMw8PJU6HJbYEfgM2TNx1unq/5Zf3/c1FMnEXFTRDwhIpZExBKyP7eeHRF3kpWdfkPq3XMQ8MeKP1k7gqQjyP48fXlETFSsuhB4naRtJC0luwG+phkxNpFLlVdIbdZfBm6OiNMrVl0IHJumjwW+0+jYmi0iToiI3VOOeR3w/Yg4BrgSeHXabNZz4yv+xrgEeCnZjcsJ4E3NDacpPgdsA1ye/iL6aUS8LSLWSzqPbMzlTcA7IuLBJsbZcBGxSdI7gcvYXKp8fZPDaqaDgdcDN0m6IS07ETgFOE/SW8jKtL+2SfG1og8A50o6Gbie7IuzKpdsMDPrMG7qMTPrME78ZmYdxonfzKzDOPGbmXUYJ34zsw7jxG91JemJks6V9GtJayVdImkvSS+cqiTYbJJOklTzYbM6HWdnSW+vw36uklTXgcYr95meL/m+pB1rbL9Y0nfrGYM1jxO/1U168KYMXBURe0TEAWRVOFuqpkpEfCQivteAQ+0MzCnxpyTc6P+XLwWGI+K+ahtExDiwQdLBjQvLiuLEb/V0CPBARPzn1IKIGI6IH6bZ7SWdn+ryr0xfFEj6iKRBSesknVWx/CpJp0paI+kXkp6Xlm8n6bxUr72c6pBPXb0eLukaSddJ+maq9/Iwkr4q6dVp+lZJH0/b3yTpGTNsf7GkfdP09ZI+kqZPkvRWSdtLuqJiH1OVNU8B9pB0g6TT0nvenz7rjZI+npYtUVaL/xxgHQ8v3zA9lkd8PmW1/L9Zsc1Df13lOR/AMaQnPSX1pti2lfQYZfXw90nbfTtta23Oid/qaR9gbY31+wPvIas53032hCbA5yKiNyL2AbrI6vlM2ToiDkzv+2ha9nbgnlSv/f8ABwAoG/jmw8CLI+LZwBDw3hxx3522/wLwrzOs/yHwPEk7kT1dPBX384CrgfuB/rSPQ4BPpy+vDwK/johlEfF+SYeTlaQ4EFgGHCDp+WlfewKfj4hnRsRtMwVZ4/N9D/hbSY9Jm/4j2VOcec/HwaR/t4gYJCuNcDLw78B/R8S6tN1Q+szW5lyywRppTUTcDpAexV8C/Ag4RNLxwHbA44D1wEB6z1SBrrVpe4DnkhWmIiLWSboxLT+I7Evlx+mPhkcD1+SIq/IY/zDD+h8C7wZGgYuBwyRtByyNiFuUFRT7t5TE/4esJO5MzVuHp9f1aX57soT/G+C2NFZDLTN+vlTy4btAn6TzyWq1Hw+8YKbtZ9jv41Ld+yknkdUPuj997il3AU+aJUZrA078Vk/r2VwoaiZ/qZh+ENha0rbA54GeiPitpI+RVR2c/p4Hmf33VcDlEXHUnKKe/RiDQA8wQjaS2C7AW9n8180xwGLggIh4QNKt0z5DZXyfjIgvPmxhVnP+zznirPX5zgXeCfwBGIqIjemvjjznY5OkR0VEZX337YFF6XNMxbYtMJkjTmtxbuqxevo+sI2k5VMLJO071TZfxVSCvDu1P9f64pjyY1KBLmVD8j0rLf8pcLCkp6V1j5G01xw/wyOkUbF+C7yG7Ir5h2RNQlenTXYiq5H+gKRDgKem5RuBHSp2dRnw5ql2dkm7SXrCHEKp9fl+ADyb7Avp3BzbV7qFrOltyhfJmtBWkg3pNxNmmiEAAAEQSURBVGUvsnsQ1uac+K1uUo39fuDFyrpzrgc+STZaUrX33At8iSyhXEZ2dT2bzwOLJf2MrC16PVmp63HgjcDXU/PPNcAjbtbO0w/Jkvtkmt49/YQsQfZIugl4A/BzgIj4PVkzyzpJp0XEamAVcE3a9nwe/sVQU63PlyqaXkQ2bu9Fs20/zcXACwEkvYHsBv0qspvTvZIOTdsdkra1NufqnNZ2lA1Ovigi7pe0B9nNzaenK3ObI2WDmpwTEYfNst3VwCsi4p7GRGZFcRu/taPtgCvTTVUBb3fSn780otWXJO1YrS+/pMXA6U76C4Ov+M3MOozb+M3MOowTv5lZh3HiNzPrME78ZmYdxonfzKzD/H9wNIPZ/xWwTAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pyplot.plot(X,y, 'bo', ms= 6, mec = 'k', mew = 2)\n",
    "pyplot.xlabel('Change in water level(x)')\n",
    "pyplot.ylabel('Water flowing out of the dam (y)');\n",
    "\n",
    "\n",
    "#seems quadratic function \n",
    "#also features are not scaled(but there is only one feature)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load from ex5data1.mat, where all variables will be store in a dictionary\n",
    "data = loadmat(os.path.join('Data', 'ex5data1.mat'))\n",
    "\n",
    "# Extract train, test, validation data from dictionary\n",
    "# and also convert y's form 2-D matrix (MATLAB format) to a numpy vector\n",
    "X, y = data['X'], data['y'][:, 0]\n",
    "Xtest, ytest = data['Xtest'], data['ytest'][:, 0]\n",
    "Xval, yval = data['Xval'], data['yval'][:, 0]\n",
    "\n",
    "# m = Number of examples\n",
    "m = y.size\n",
    "\n",
    "# Plot training data\n",
    "pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1)\n",
    "pyplot.xlabel('Change in water level (x)')\n",
    "pyplot.ylabel('Water flowing out of the dam (y)');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Regularized linear regression cost function\n",
    "\n",
    "Recall that regularized linear regression has the following cost function:\n",
    "\n",
    "$$ J(\\theta) = \\frac{1}{2m} \\left( \\sum_{i=1}^m \\left( h_\\theta\\left( x^{(i)} \\right) - y^{(i)} \\right)^2 \\right) + \\frac{\\lambda}{2m} \\left( \\sum_{j=1}^n \\theta_j^2 \\right)$$\n",
    "\n",
    "where $\\lambda$ is a regularization parameter which controls the degree of regularization (thus, help preventing overfitting). The regularization term puts a penalty on the overall cost J. As the magnitudes of the model parameters $\\theta_j$ increase, the penalty increases as well. Note that you should not regularize\n",
    "the $\\theta_0$ term.\n",
    "\n",
    "You should now complete the code in the function `linearRegCostFunction` in the next cell. Your task is to calculate the regularized linear regression cost function. If possible, try to vectorize your code and avoid writing loops.\n",
    "<a id=\"linearRegCostFunction\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linearRegCostFunction(X, y, theta, lambda_=0.0):\n",
    "    \"\"\"\n",
    "    Compute cost and gradient for regularized linear regression \n",
    "    with multiple variables. Computes the cost of using theta as\n",
    "    the parameter for linear regression to fit the data points in X and y. \n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array_like\n",
    "        The dataset. Matrix with shape (m x n + 1) where m is the \n",
    "        total number of examples, and n is the number of features \n",
    "        before adding the bias term.\n",
    "    \n",
    "    y : array_like\n",
    "        The functions values at each datapoint. A vector of\n",
    "        shape (m, ).\n",
    "    \n",
    "    theta : array_like\n",
    "        The parameters for linear regression. A vector of shape (n+1,).\n",
    "    \n",
    "    lambda_ : float, optional\n",
    "        The regularization parameter.\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    J : float\n",
    "        The computed cost function. \n",
    "    \n",
    "    grad : array_like\n",
    "        The value of the cost function gradient w.r.t theta. \n",
    "        A vector of shape (n+1, ).\n",
    "    \n",
    "    Instructions\n",
    "    ------------\n",
    "    Compute the cost and gradient of regularized linear regression for\n",
    "    a particular choice of theta.\n",
    "    You should set J to the cost and grad to the gradient.\n",
    "    \"\"\"\n",
    "    # Initialize some useful values\n",
    "    m = y.size # number of training examples\n",
    "\n",
    "    # You need to return the following variables correctly \n",
    "    J = 0\n",
    "    grad = np.zeros(theta.shape)\n",
    "\n",
    "    # ====================== YOUR CODE HERE ======================\n",
    "    temp = theta.copy()\n",
    "    temp[0] = 0\n",
    "    \n",
    "    h = X@theta\n",
    "    J = (1/(2*m))*np.sum(np.square((h-y))) + (lambda_/(2*m))*np.sum(np.square(temp))\n",
    "    grad = grad+ ((1/m)*(X.T@(h-y))+(lambda_/m)*temp)\n",
    "\n",
    "    # ============================================================\n",
    "    return J, grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you are finished, the next cell will run your cost function using `theta` initialized at `[1, 1]`. You should expect to see an output of 303.993."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost at theta = [1, 1]:\t   303.993192 \n",
      "This value should be about 303.993192)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "theta = np.array([1, 1])\n",
    "J, _ = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)\n",
    "\n",
    "print('Cost at theta = [1, 1]:\\t   %f ' % J)\n",
    "print('This value should be about 303.993192)\\n' % J)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After completing a part of the exercise, you can submit your solutions for grading by first adding the function you modified to the submission object, and then sending your function to Coursera for grading. \n",
    "\n",
    "The submission script will prompt you for your login e-mail and submission token. You can obtain a submission token from the web page for the assignment. You are allowed to submit your solutions multiple times, and we will take only the highest score into consideration.\n",
    "\n",
    "*Execute the following cell to grade your solution to the first part of this exercise.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance\n",
      "\n",
      "Login (email address): nehatrical@gmail.com\n",
      "Token: ggzrckSLpLK16VJ8\n",
      "                                  Part Name |     Score | Feedback\n",
      "                                  --------- |     ----- | --------\n",
      "Regularized Linear Regression Cost Function |  25 /  25 | Nice work!\n",
      "     Regularized Linear Regression Gradient |   0 /  25 | \n",
      "                             Learning Curve |   0 /  20 | \n",
      "                 Polynomial Feature Mapping |   0 /  10 | \n",
      "                           Validation Curve |   0 /  20 | \n",
      "                                  --------------------------------\n",
      "                                            |  25 / 100 |  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "grader[1] = linearRegCostFunction\n",
    "grader.grade()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section2\"></a>\n",
    "### 1.3 Regularized linear regression gradient\n",
    "\n",
    "Correspondingly, the partial derivative of the cost function for regularized linear regression is defined as:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "& \\frac{\\partial J(\\theta)}{\\partial \\theta_0} = \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left(x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} & \\qquad \\text{for } j = 0 \\\\\n",
    "& \\frac{\\partial J(\\theta)}{\\partial \\theta_j} = \\left( \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left( x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} \\right) + \\frac{\\lambda}{m} \\theta_j & \\qquad \\text{for } j \\ge 1\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "In the function [`linearRegCostFunction`](#linearRegCostFunction) above, add code to calculate the gradient, returning it in the variable `grad`. <font color='red'><b>Do not forget to re-execute the cell containing this function to update the function's definition.</b></font>\n",
    "\n",
    "\n",
    "When you are finished, use the next cell to  run your gradient function using theta initialized at `[1, 1]`. You should expect to see a gradient of `[-15.30, 598.250]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "J at theta = [1,1]:303.993192\n",
      "Gradient at theta = [1, 1]:  [-15.303016, 598.250744] \n",
      " (this value should be about [-15.303016, 598.250744])\n",
      "\n"
     ]
    }
   ],
   "source": [
    "theta = np.array([1, 1])\n",
    "J, grad = linearRegCostFunction(np.concatenate([np.ones((m, 1)), X], axis=1), y, theta, 1)\n",
    "\n",
    "print('J at theta = [1,1]:{:.6f}'.format(J))\n",
    "print('Gradient at theta = [1, 1]:  [{:.6f}, {:.6f}] '.format(*grad))\n",
    "print(' (this value should be about [-15.303016, 598.250744])\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*You should now submit your solutions.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance\n",
      "\n",
      "Use token from last successful submission (nehatrical@gmail.com)? (Y/n): Y\n",
      "                                  Part Name |     Score | Feedback\n",
      "                                  --------- |     ----- | --------\n",
      "Regularized Linear Regression Cost Function |  25 /  25 | Nice work!\n",
      "     Regularized Linear Regression Gradient |  25 /  25 | Nice work!\n",
      "                             Learning Curve |   0 /  20 | \n",
      "                 Polynomial Feature Mapping |   0 /  10 | \n",
      "                           Validation Curve |   0 /  20 | \n",
      "                                  --------------------------------\n",
      "                                            |  50 / 100 |  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "grader[2] = linearRegCostFunction\n",
    "grader.grade()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {},
   "outputs": [],
   "source": [
    "def trainLinearReg(linearRegCostFunction, X, y, lambda_=0.0, maxiter=200):\n",
    "    \"\"\"\n",
    "    Trains linear regression using scipy's optimize.minimize.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array_like\n",
    "        The dataset with shape (m x n+1). The bias term is assumed to be concatenated.\n",
    "\n",
    "    y : array_like\n",
    "        Function values at each datapoint. A vector of shape (m,).\n",
    "\n",
    "    lambda_ : float, optional\n",
    "        The regularization parameter.\n",
    "\n",
    "    maxiter : int, optional\n",
    "        Maximum number of iteration for the optimization algorithm.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    theta : array_like\n",
    "        The parameters for linear regression. This is a vector of shape (n+1,).\n",
    "    \"\"\"\n",
    "    # Initialize Theta\n",
    "    initial_theta = np.zeros(X.shape[1])\n",
    "\n",
    "    # Create \"short hand\" for the cost function to be minimized\n",
    "    costFunction = lambda t: linearRegCostFunction(X, y, t, lambda_)\n",
    "\n",
    "    # Now, costFunction is a function that takes in only one argument\n",
    "    options = {'maxiter': maxiter, 'disp':True}\n",
    "\n",
    "    # Minimize using scipy\n",
    "    res = optimize.minimize(costFunction, initial_theta, jac=True, method='TNC', options=options)\n",
    "    return res.x, res.fun"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fitting linear regression\n",
    "\n",
    "Once your cost function and gradient are working correctly, the next cell will run the code in `trainLinearReg` (found in the module `utils.py`) to compute the optimal values of $\\theta$. This training function uses `scipy`'s optimization module to minimize the cost function.\n",
    "\n",
    "In this part, we set regularization parameter $\\lambda$ to zero. Because our current implementation of linear regression is trying to fit a 2-dimensional $\\theta$, regularization will not be incredibly helpful for a $\\theta$ of such low dimension. In the later parts of the exercise, you will be using polynomial regression with regularization.\n",
    "\n",
    "Finally, the code in the next cell should also plot the best fit line, which should look like the figure below. \n",
    "\n",
    "![](Figures/linear_fit.png)\n",
    "\n",
    "The best fit line tells us that the model is not a good fit to the data because the data has a non-linear pattern. While visualizing the best fit as shown is one possible way to debug your learning algorithm, it is not always easy to visualize the data and model. In the next section, you will implement a function to generate learning curves that can help you debug your learning algorithm even if it is not easy to visualize the\n",
    "data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
    "theta, cost = trainLinearReg(linearRegCostFunction, X_aug, y, lambda_=0)\n",
    "\n",
    "#fitting the parameters to optimize training error using the gradient (here regularization parameter is zero)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([13.08790351,  0.36777923]), 22.373906495108915)"
      ]
     },
     "execution_count": 220,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta, cost\n",
    "\n",
    "#we obtzined theta0 = 13; theta 1 = 0.367 with traing error reduced to 22.37 from 300{theta = [1,1]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-5.0, 40.0)"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEKCAYAAAAb7IIBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3df5yVZZ3/8dcbNEcdFcXJ2FBQNE3AUEejzApN134X+SO3TVttqazNtGAjW0ltNWGL2i1NNt2wry2appk/UkPTMlFBUPC3qCgmAiIyAw4/xs/3j+ueZRjOOXPmx/kxc97Px+M85pzrvs+5P3MYPtd9X9d1X5ciAjMzqz0DKh2AmZlVhisAM7Ma5QrAzKxGuQIwM6tRrgDMzGqUKwAzsxpVtgpA0kBJ8yXdlL3eW9L9kp6RdLWkt5QrFjMzK+8VwJnA4+1eXwxMj4h9gdeA08sYi5lZzStLBSBpKPBR4BfZawFHAddmu8wEPlWOWMzMLNmmTMf5MTAJ2Cl7PRhYHRGbstdLgbfneqOkCcAEgB133PHQAw44oMShmpn1L/PmzVsZEQ0dy0teAUj6GLA8IuZJ+mBX3x8RM4AZAI2NjTF37txejtDMrH+TtCRXeTmuAI4APiHpI0AdsDPwE2CQpG2yq4ChwEtliMXMzDIl7wOIiMkRMTQihgOfBe6MiM8BdwHHZ7udCvyu1LGYmdlmlbwP4F+BsyU9Q+oTuLyCsZiZ1ZxydQIDEBF/Av6UPX8WOLycxzczs818J7CZWY1yBWBmVqNcAZiZ1ShXAGZmNcoVgJlZjXIFYGZWo1wBmJlVqebmZi6YMoXhDQ0MHDCA4Q0NXDBlCs3Nzb3y+a4AzMyqUHNzM0ePHcuiqVO5YeVK1kdww8qVLJw6laPHju2VSsAVgJlZFZo+bRrDFy9mVksLY0h37Y4Brm5pYdjixUyfNq3Hx3AFYGZWhS6/5BImt7SgDuUCJre0cMWll/b4GK4AzMyq0IuvvsqoPNtGZdt7yhWAmVkV2nPwYBbl2bYo295TrgDMzKrQ6WecwYV1dUSH8gAuqqvjtK98pcfHcAVgZlaFzpo4kSUjRnBSXR3zgY3AfOCkujqWjBjBWRMn9vgYrgDMzKpQfX09s+fMYfSkSYxvaGD7AQMY39DA6EmTmD1nDvX19T0+hiI6XmD0Lkl1wD3AdqSRTNdGxBRJvwQ+ALye7fqFiFhQ6LO8JrCZWddJmhcRjR3Ly7EgzHrgqIholrQt8BdJt2bbJkbEtWWIwczMOih5BRDpEqPtlrVts0dpLzvMzKxTZekDkDRQ0gJgOXBHRNyfbfp3SY9Imi5pu3LEYmZWjUo9708uZakAIqI1IsYAQ4HDJY0CJgMHAIcBu5EWid+KpAmS5kqau2LFinKEa2ZWVuWY9yeXso4CiojVwF3AcRHxciTrgf8hzwLxETEjIhojorGhoaGc4ZqZlUU55v3JpeQVgKQGSYOy59sDxwBPSBqSlQn4FOS96c3MrF8rx7w/uZRjFNAQYKakgaQK55qIuEnSnZIaSL/jAuDLZYjFzKzqlGPen1zKMQroEeDgHOVHlfrYZmZ9wZ6DB7No5UrG5NjWW/P+5FKwCUhSnaTjJf1E0m8kXSlpkqSRJYnGzKwGlWPen1zyVgCSzgPuBd4D3A9cBlwDbAJ+IOkOSQeVJCozsxpSjnl/cil0BfBARBwaEd+MiF9HxB8j4qaI+FFEfBz4HPCWkkRlZlZDyjHvTy6dzgUkaXRELCzJ0bvIcwGZmXVdvrmAihkGeomkBySdIWmXEsRmZmYV0GkFEBFHkpp79gTmSfq1pGNKHpmZmZVUUTeCRcTTwHdJ0zV8APhPSU9IGl/K4MzMrHQ6rQAkHSRpOvA4cBTw8Yh4Z/Z8eonjMzOzEinmRrD/An4BfCci3mgrjIi/SfpuySIzM7OS6rQCiIgPFNj2q94Nx8zMyqXQjWC/l/TxbBWvjtv2kXS+pNNKG56ZmZVKoSuAfwbOBn4saRWwAqgDhgOLgZ9GxO9KHqGZmZVE3gogIpYBk4BJkoaTZvV8A3gqItaVJTozMyuZomYDjYjngedLGomZmZVVWVcEMzOz6uEKwMysRpVjSci6bC6hhyU9mk0zjaS9Jd0v6RlJV0vyzKJmZmVUzJ3AH5M0X9IqSWskNUla04VjrAeOioh3kdY5Pk7SWOBiYHpE7Au8BpzenV/AzMy6p5grgB8DpwKDI2LniNgpInYu9gCRNGcvt80eQZpK4tqsfCZpYXgzMyuTYiqAF4FF0dnCAQVIGihpAbAcuIN0H8HqiNiU7bIUeHue906QNFfS3BUrVnQ3BDMz66CYYaCTgFsk3U1qzgEgIn5U7EEiohUYI2kQcD1wQBfeOwOYAWlBmGLfZ2ZmhRVTAfw70Ey6C7hHHbURsVrSXaR1hgdJ2ia7ChgKvNSTzzYzs64ppgL4u4gY1d0DSGoANmbJf3vgGFIH8F3A8cAsUh+Dp5UwMyujYvoAbpF0bA+OMQS4S9IjwIPAHRFxE2lxmbMlPQMMBi7vwTHMzKyLirkC+ArwLUnrgY2ASIN7ihoJFBGPAAfnKH8WOLwLsZqZWS8qZj2AncoRiJmZlVdRk8FJ2hXYj9QRDEBE3FOqoMzMrPQ6rQAkfRE4kzRSZwEwFriPdCOXmZn1UcV0Ap8JHAYsiYhxpPb81SWNyszMSq6YCqAlIloAJG0XEU8A+5c2LDMzK7Vi+gCWZnfw3gDcIek1YElpwzIzs1IrZhTQp7On38vu4t0F+ENJozIzs5LLWwFI2i1H8cLsZz2wqiQRmZlZWRTqA5gHzM1+rgCeAp7Ons8rfWhmZrXrzTfhN7+Bo4+GdetKc4y8FUBE7B0R+wB/BD4eEbtHxGDgY8DtpQnHzKy2bdoExx8PAwfCiSfCnXfCL39ZmmMVMwpobETc0vYiIm4F3luacMzMalNb4t92W7juus3lP/85nF6i9RKLGQX0N0nfBf5f9vpzwN9KE46ZWW1580249lo46aQtyw8+GO65B+rrS3fsYq4ATgYaSAu5/DZ7fnLpQjIz6/9aW6GpCY48csvkf8ghqfyhh0qb/KG4YaCrSHcDm5lZD23cCFdfDRddlJL/hz4EL74IkyfD5z9f+qTfXlGTwZmZWc9s3JjO9K+/fsuyBx6Ac86Bt/RovcXuKaYJqEck7SnpLkmPSXpU0plZ+fckvSRpQfb4SKljMTMrt40bYfz4lODbJ//LLoNFi2DQoMokfyjPFcAm4JsR8ZCknYB5ku7Itk2PiP8oQwxmZmXV2prG8Z/cocf00EPhT38qb1NPPsVMB/0O4FJgj4gYJekg4BMR8f1iDhARLwMvZ8+bJD0OvL0HMZuZVa3W1jR2v6lpy+RfTYm/TTFNQP8NTCYtB9m2xONnu3MwScNJ00nfnxV9TdIjkq7IFp3J9Z4JkuZKmrtixYruHNbMrOQ2boRJk2D0aDj2WNh11/Tzsstg/XqYO7e6kj8U1wS0Q0Q8IKl92aauHkhSPXAd8I2IWCPpUuACILKfPwRO6/i+iJgBzABobGyMrh7XzKyUNm5MN3DdeOPmsmHD4I034LbbKhdXMYq5AlgpaQQpUSPpeLImnWJJ2paU/K+KiN8CRMQrEdEaEW+SrjK8QLyZ9RmtrXDCCakDt33yP/VUeOop+EgfGNZSzBXAV0ln4AdIegl4DvjHYg+gdOlwOfB4RPyoXfmQrH8A4NPAoqKjNjOroFmz4Pzz4fHHN5cdfnhq+99xx8rF1VXF3Aj2LPAhSTsCAyKiqYvHOAL4PLBQ0oKs7DvAyZLGkK4snge+1MXPNTMrm9ZWuOYaqKuDX/0qJf8dd4SRI/te4m9TzCig7YDPAMOBbdr6AiLi/GIOEBF/AZRj0y05yszMqsrGjfCZz8Dvf59eDx+ergA+/Wk45ZTKjeHvDcU0Af0OeJ20BsD60oZjZlYdOiZ+SMn/nHPSfD3vfnfFQus1xVQAQyPiuJJHYmbWA83NzUyfNo3LL7mEF199lT0HD+b0M87grIkTqe/C+MvWVvjBD+C7392y/D3vSeP4+/IZf0fFjAL6q6TRJY/EzKybmpubOXrsWBZNncoNK1eyPoIbVq5k4dSpHD12LM3NzUV9zty5MGrUlsn/Pe+BtWvhr3/tX8kfClQAkhZKegR4H/CQpCezm7bays3MqsL0adMYvngxs1paGENq2hgDXN3SwrDFi5k+bVre97a2wq9/DY2NKcE//3xq6pk0aXPi32GH8vwe5aaI3PdWSRpW6I0RsaQkERXQ2NgYc+fOLfdhzazKDW9o4IaVKxmTY9t8YHxDA88tX75F+YYNaZK2m2/eXHb++XDMMWnahm23LWnIZSVpXkQ0dizP2wfQluAl/SoiPt/hw35FGtppZlZxL776KqPybBuVbW+zYUMawXNLh3GIl1+e5uPvT4m/M8X0AYxs/0LSQODQ0oRjZtZ1ew4enPdO0kXZ9tZWuOoq2G67LZP/EUekpp7TTqut5A+F+wAmS2oCDpK0Jns0ActJQ0PNzKrC6WecwYV1dXRs0A7gwu124LOnns1xx8E/tpvDoC3x/+Uv/beNvzN5K4CIuCgidgKmRcTO2WOniBgcEZPLGKOZWUFnTZzIkhEjOKmujvmkqYsfYFsO2+bL3BSLWPj4txg/HvbZB664Ik3UVsuJv03eTuBq5E5gM8tn830Al7Nk5eXA3//fthEj0tKLO+1Ue808kL8T2BWAmfUL69blno/nF79IUzbUYuJvk68CKNQHsHdpQzIz67n16+G447ZO/u9/f6oUTj+9tpN/IYVGAV0LIGl2mWIxMyvapk1pzd26uq0XXlmzBu6+G7bfvjKx9RWF5gIaIOk7wDsknd1xY/u5/c3MymXdOvjqV+Hee+Hpp2HoUFi6NJ3x/+EPTvpdUagC+CzwqWyfncoTjplZbrna+EeMgAsuSCtzbVPM1Ja2hUJ3Aj8JXCzpkYi4tbsHkLQncCWwB2lY7oyI+Imk3YCrSesMPA+cGBGvdfc4ZtY/rV+fmnk6OuWUdPeuE3/3FTsb6I8kzc0eP5S0SxeOsQn4ZkQcCIwFvirpQODbwOyI2A+Ynb02MwNSG/+HP5w7+a9ZAzNnOvn3VDEVwBVAE3Bi9lgD/E+xB4iIlyPioex5E/A48Hbgk8DMbLeZpOYmM6txmzbBlVfCUUelNv32mpogIo3nt54rpv4cERGfaff6vHZr+3aJpOHAwcD9wB7tFoVfRmoiyvWeCcAEgL322qs7hzWzPmDtWthrL1i1Kr3ecUcYPBhGj4Zbb819JWA9U8wVwBuS3tf2QtIRwBtdPZCkeuA64BsRsab9tkh3o+W8Iy0iZkREY0Q0NjQ0dPWwZlbl1q4FCerrNyf/ffaBn/0Mli2Du+5y8i+VYq4Avgxc2a7d/zXg1K4cRNK2pOR/VUT8Nit+RdKQiHhZ0hDSJHNmViNaWtK8+489tmX5zjvD44/3v9W3qlGnVwAR8XBEvAs4CDgoIg6OiKJXBJMk4HLg8Q73DtzI5orkVDzDqFlNePPNtNrW9ttvnfybm+H11538y6WYJiAAImJNx6abIh1BWjzmKEkLssdHgB8Ax0h6GvhQ9trM+qlNm9LInQMPhMsu21y+//4p8UfknsvHSqfkg6gi4i+A8mw+utTHN7PKampKzTrtvec9cOSRcN55bt+vpE4rAEnbRcT6zsrMzNp79VXYffety2fOhH/4B4/hrwbF/BPcBxxSRJmZGatXw6675t7W0pKWZLTqkLcCkPQ20g1b20s6mM3NODsDNb6Ojpl11Nyc/watVavyVwpWOYWuAP4e+AIwFGg/eqcJ+E4JYzKzPqSlBX7+czjrrK23vf761u3/Vj0KTQY3E5gp6TMRcV0ZYzKzPmDFCnjrW3Nvc+LvG4rpAxglaWTHwog4vwTxmFmVy5f4f/pT+OIX3cbflxRTATS3e14HfIw0oZuZ1ZDXXoPddsu9be1a2ME9g31OpxVARPyw/WtJ/wHclmd3M+tnco3jb/PaazBoUHnjsd5T9J3A7exA6hg2s36sqSnNx58r+bdNy+zk37cVcyPYQjbP1DkQaADc/m/WT73yCrztbbm3rVvnNXf7k2L6AD7W7vkm4JWI2FSieMysQlatSvPvd3TqqWnpxYEDyx+TlVYxfQBLJL0LODIrugcoejZQM6tua9bALnkWefUZf//WaR+ApDOBq4C3Zo+rJP1LqQMzs9LauDEtxJIr+b/+emrjd/Lv34ppAjodeHdErAWQdDFpLqD/KmVgZlYar7+ev/O2qSmtzGW1oZhRQAJa271uJf/0zmZWpZYtS2f8uZL/unXpjN/Jv7YUUwH8D3C/pO9J+h4wh7TCV1EkXSFpuaRF7cq+J+mlDgvEmFkJrFyZEv+QIVuW77orrF/vpp5aVsySkD8C/glYlT3+KSJ+3IVj/BI4Lkf59IgYkz1u6cLnmVkR1qxJib+hYettzc1p1E85ll5sbm7mgilTGN7QwMABAxje0MAFU6bQ3Nzc+ZutpIq6ESwiHoqI/8we87tygIi4h1RxmFkZbNgAJ5+cu3N3zZryLr3Y3NzM0WPHsmjqVG5YuZL1EdywciULp07l6LFjXQlUWHfuBO4tX5P0SNZElHemcEkTJM2VNHfFihXljM+sT2lr6tluO5g1a8ttbWvu5puvv1SmT5vG8MWLmdXSwhjSqJMxwNUtLQxbvJjp06aVNyDbQqUqgEuBEaS/hZeBH+bbMSJmRERjRDQ25LqWNatxf/tb7qaeL34R3nijsoutX37JJUxuadlq1IiAyS0tXHHppZUIyzIVWZUzIl5pey7pv4GbKhGHWV9WaD7+9evL077fmRdffZVRebaNyrZb5RRzI1iTpDUdHi9Kul7SPt05qKT24xE+DSzKt6+ZbWn16nTGnyv5t03SVg3JH2DPwYPz/udelG23yimmCejHwETS+sBDgW8BvwZmAVd09mZJ/0u6cWx/SUslnQ5MlbRQ0iPAOCDHYnJm1t7atXDBBbnX1l27tjrH8Z9+xhlcWFf3f7NJtgngoro6TvvKVyoRlmUU0fGfpsMO0sMR8a4OZQsiYkyubaXU2NgYc+fOLdfhzKrCSy/B0BwTsA8YkDp3q3kMf9sooGGLFzO5pYVRpDP/i+rqWDJiBLPnzKG+2mqtfkjSvIho7FhezBXAOkknShqQPU4EWrJthWsPM+u2559PTT0dk//PfpaGera2VnfyB6ivr2f2nDmMnjSJ8Q0NbD9gAOMbGhg9aZKTfxUo5gpgH+AnwHtICX8OqcnmJeDQiPhLqYNs4ysAqwX5zvgBWlq85q51Xb4rgGKmg34W+HiezWVL/mb9XaFRPatX55+y2ay7ilkRrAH4Z2B4+/0j4rTShWVWOwqtwNXcXLkx/Nb/FXMfwO+APwN/ZMtZQc2sB554At75ztzbnPitHIqpAHaIiH8teSRmNeKFF2DYsK3L6+rSOP5tKnJ7ptWiYkYB3eTpms167oUX0qieXMl/7do0bYOTv5VTMX9uZwLfkbQe2EiaxiMiYueSRmbWTzQ355+E7fXXYWf/T7IKKWYUUJnnDzTrH/I19UA6499hh/LGY9ZR3gpA0gER8YSkQ3Jtj4iHSheWWd/16KMwKs8MaOvWVf/NW1Y7Cl0BnA1MIPdUzQEcVZKIzPqoJUtg+PDc26pldk6z9vJWABExIfs5rnzhmPU9hRJ/U1P1TdBm1qaYG8H+AtxNuhfg3ohoKnlUZn3A66/DoEG5t3kcv/UFxQwD/TzwJPAZ4K/Z8ozTSxuWWfV68sk0nDNX8l+3rrIrcJl1RTGjgJ6T1AJsyB7jgDz3L5r1X/k6d/ffHx5+2JO0Wd9TzIpgi4EbgD2Ay4FREXFcsQfIFn1fLmlRu7LdJN0h6ensZ95F4c0q7bnn0hl/ruTf0pKmdHDyt76omCag/wReAE4Gvg6cKmlEF47xS6BjhfFtYHZE7AfMzl6bVZUlS1Li3yfHwqfNzampx4nf+rJOK4CI+ElEnAB8CJgHfA94qtgDRMQ9wKoOxZ8EZmbPZwKfKvbzzEptxYrUhp9rZM8bb7iN3/qPYpqAfijpfuB+4F3AucB+PTzuHhHxcvZ8Gal5Kd/xJ2Qdz3NXrFjRw8Oa5bdgwebF1tet23JbS0tK/HV1lYnNrBSKmQvoPmBqRLxSigAiIiTlXZYsImYAMyCtCFaKGKy2PfVU6sjtaNw4uO022Hbb8sdkVg7FjAK6VtInJL0/K7o7In7fw+O+ImlIRLwsaQiwvIefZ9ZlixfDvvvm3uY7d60WFNMEdBFpRtDHssfXJV3Yw+PeCJyaPT+VtOiMWVm88kpq6smV/NeuTU09Tv5WC4ppAvooMCYi3gSQNBOYD3ynmANI+l/gg8DukpYCU4AfANdIOh1YApzY9dDNuubFF2GvvXJv82LrVouKXX5iEJtH8nRpaeqIODnPpqO78jlm3TVvHjQ25t7mxG+1rJgK4CJgvqS7SIvBvB+P27c+4Omn4R3v2Lq8vh5Wr4aBA8sfk1k1KaYT+H8l/Qk4LCv614hYVtKozHogX+IHd+6atZe3E1jSIW0PYAiwNHv8Xb5FYswqafXq1LmbK/m3TdJWy8m/ubmZC6ZMYXhDAwMHDGB4QwMXTJlCc3NzpUOzCil0BZBrIZg2XhDGqsaTT8IBB+Te5jP+pLm5maPHjmX44sXc0NLCKGDRypVcOHUqR193HbPnzKHeCxfUnELDQC/NFoM5PSLGdXg4+VvFPfBAOuPPlfzXr/cZf3vTp01j+OLFzGppYQzpzG8McHVLC8MWL2b6tGkVjtAqQRG5b66V9FBEHNL2s8xx5dTY2Bhz586tdBhWYYXO+DduhG2KHdtWQ4Y3NHDDypWMybFtPjC+oYHnlvt+zP5K0ryI2GosXKH/Kq9Kuh3YW9KNHTdGxCd6M0CzzixYAAcfnHvbG294np5CXnz1VfKsU8+obLvVnkIVwEeBQ4BfUbg/wKykXngBhg3Lvc3j+Iuz5+DBLMpzBbAo2261J28fQERsiIg5wHsj4u6OjzLGaDXqvvtSG3+u5L9hg+fj74rTzziDC+vq6NjgG8BFdXWc9pWvVCIsq7Bi1gPwHMxWVm2J/73v3XpbW+L3DJ1dc9bEiSwZMYKT6uqYD2wktf2fVFfHkhEjOGvixApHaJVQzIpgZmXx+ONO/KVSX1/P7DlzGD1pEuMbGth+wADGNzQwetIkDwGtYXlHAQFIGgh8PSKmly+k/DwKqH967DEYOTL3NnfumvVcvlFABa8AIqKVtBawWa975pl0xp8r+bed8Tv5m5VOMSOm75X0U+BqYG1bYUQ8VLKorF/74x/hmGNyb/M4frPyKea/WtvIsfPblXkqCOuyhx+GMbnGIQKbNnl2TrNyK2Y20HGlOrik54EmoBXYlKuNyvq+J56Ad74z9zaf8ZtVTjFLQu4h6XJJt2avD8xW8uot4yJijJN//7N4cWrjz5X8W1pSG7+Tv1nlFDMM9JfAbcDfZa+fAr5RqoCs71u4MP+auxs3+gYus2pRTAWwe0RcA7wJEBGbSE02vSGA2yXNkzQh1w6SJkiaK2nuihW+J62a/eEPKfEfdNDW2zZt8hm/WbUp5r/jWkmDSckaSWOB13vp+O+LiJckvRW4Q9ITEXFP+x0iYgYwA9J9AL10XOtFCxfmTvrgzl2zalZMBXA2cCMwQtK9QANwQm8cPCJeyn4ul3Q9cDhwT+F3WbUolPg3bPBdu2bVrpgmoEeBDwDvBb4EjASe6OmBJe0oaae258CxpIkJK8rL5nVu6dL8TT2esqH3+G/RSq3gVBCweWGYzsq6fGBpH+D67OU2wK8j4t8LvafUU0G0XzZvctuyecCF2YRZtT5nyn335Z6nB9zU09v8t2i9qctTQUh6m6RDge0lHdxukfgPAjv0NKCIeDYi3pU9RnaW/MvBy+bldvPN+SdpaxvV4+Tfu/y3aOVQaEnIU4EvAI1A+9PuJuCXEfHbkkfXQamvALxs3pYKrcDlM/7S8t+i9aYuXwFExMzsLuAvdFgQ/hOVSP7d0dU2VC+bl7TNx58r+be18Tv5l5b/Fq0cilkQ5jpJH5U0SdK5bY9yBNcTbW2oi6ZO5YaVK1kfwQ0rV7Jw6lSOHjs2ZyWw5+DBeXuhK7FsXrk7AZ98svOmHnfulke1/S1a/1TMVBA/B04C/gUQaQhonhVaq0d32lCradm87lRg3XXrrSnxH3DA1tt8A1dlVNPfovVjEVHwATzS4Wc98OfO3leKx6GHHhrFGrb77jE/5a6tHg9BDG9o2Oo9TU1NcfjIkXFCXV08BLEh2/eEuro4fOTIaGpqKvr4PXX+uefGiXV18WaH2N/M4jn/3HN7fIzbb8/59cSOO0Zs2tQLv0QvaWpqivPPPTeG7b57DJBi2O67x/nnntur/x7lOEZX46mWv0Xr+4C5kSu/5yrcYge4P/s5hzQf0HbAM529rxSPrlQAA6TYmKcC2AAxcMCAnO9rSwTDGxpi4IABMbyhoSKJoDsVWLHmzcud+KG6En/E5kR4Yl1dzIfYCDG/lxNhOY7R3biq4W/R+r6eVAD/BgwCPgMsA14Gzu/sfaV4lPoKoJp0twIr5LHH8if+jRtL8Ev0gnJcCZXjGGaV1OUKgDTj5+HANu3KtgN2yfeeUj+6UgGcf+65cUIf/k/dmxXY0qURQ4bkTvytrSX8JXpBOSryvn6yYNaZfBVAoU7gocCPgeWS7pZ0IXAM0CcGAJ41cSJLRozgpLo65gMbSeOnT8rupDxr4sQKR1hYb3QC3nNP6twdOhRefnnLba2tKcMNKGYykAoqx3BID7m0WlXoPoBvRcR7gbcBk4FVwD8BiyQ9Vqb4uq2+vp7Zc+YwetIkxjc0sP2AAYxvaGD0pEl94jb6nlRgjz2WEv8HPrBl+bsPn8NegxsYoAHss0ffmFemHMMhPeTSalUx53/bAzsDu2SPvwH3lzKo3lJfX8+/nXcezy1fzqbWVp5bvpx/O++8qk/+0L0K7NFHU+IfOXLrzzvswNEMe2Qcv3u1tENKe1s5hkN6yKXVrFztQqnJiBnAvcAfgPOADwO75tu/HI+u9AHUkiVL8nfutrb27U7OcgyH9EsYVtEAAAy1SURBVJBL6+/oRh/AXqRO32XAS8BSYHUpKyPrmueeS2f8w3Lclte+jf/ySy5hcksL6rCPgMktLVxx6aXlCLdbytGU19ebC826q+B00JJEmv//vdljFKkv4L6ImFKWCNsp9WRwfcWdd8LRR+fe1tq6dcfuwAEDWB+Rc/WfjcD2AwawqbW3Vvk0s2rT5cngALKrh0XALcCtpCahEcCZJYnSClq0KJ3xd0z+J5wAb76Zf1SPOznNLJdC6wF8XdIsSS8AdwMfI60ENh7YrTcOLuk4SU9KekbSt3vjM/ujBx9MiX/06K23tbbCNdek7fm4k9PMcim0HsCPSGf8f42Il3Pu1JMDSwOBp0j3FiwFHgROjoi8Q0xrrQlo7dqU9J97buttuZp68mmbWG5Yh9WlLvLqUmY1oTvrAZwdEdeVIvlnDifNKfRsRGwAZgGfLNGx+pTFi9O0y4MGbZ38CzX15ONOTjPLpZL3gb4deLHd66VZWc1auDA15ey7b5qGedMmGDsWLrtsc+Iv1NRTSF++J8LMSqPqZ3mXNAGYALDXXntVOJrSWLw4Jf2OvvlNmDat+0nfzKyQSlYALwF7tns9NCvbQkTMIN2URmNjY/4xq31QvsQP6Yzfid/MSqmSTUAPAvtJ2lvSW4DPAjdWMJ6yaW6Giy/eOvmPGtXzph4zs2JV7AogIjZJ+hpwG2mG0Ssi4tFKxVMOCxbkXmj9sMPg/vud9M2svCraBxARt5BuMuvXciX+t74VrrwSjj3Wid/MKqPqO4H7smeegf3227p87Fj461+d+M2sslwBlMCyZTBkSO5t7tw1s2pR5etB9S3LlsFFF22d/N/1Lnfumln18RVAL5g/Hw45ZOtyd+6aWTXzFUAPPPVUSu4dk//tt6cz/gcecPI3s+rlK4BueOop2H//3Nvcxm9mfYUrgC5oaoKdd869zYnfzPoaNwEVYelS+NSnYPjwLcsbG925a2Z9l68ACpg3LyX59g46KLX5X3GFk76Z9W2uAHJYswZ22WXr8hkz4ItfdOI3s/7BFUA7r70Gl14KP/zh1tvcxm9m/Y37AEhn/F/4Auy2G5xzDqxaBWPGwLe+5TZ+M+u/avoKYNUq+PnP0xn/qlWby2fPhnHjnPTNrH+ryQrgpZfSJG1vvLG57H3vg69/HY4/3onfzGpDTVUAa9bAf/0XfPe7W5b7jN/MalFN9AEsWwZf+lIax98++Z9zTmrjP+ooJ38zqz0VqQAkfU/SS5IWZI+PlPJ4p5yShnC+9hoceSTceWdK/N//vhO/mdWuSjYBTY+I/yjHgcaPhzvugJtvhg9/2EnfzAxqpAnoy19OQzk/8hEnfzOzNpW8AviapFOAucA3I+K1XDtJmgBMyF42S3qyXAH2gt2BlZUOokr5u8nP301+/m7yK/TdDMtVqIgoSSSS/gi8Lcemc4A5pEADuAAYEhGnlSSQCpI0NyIaO9+z9vi7yc/fTX7+bvLrzndTsiuAiPhQMftJ+m/gplLFYWZmuVVqFFD7VXM/DSyqRBxmZrWsUn0AUyWNITUBPQ98qUJxlNqMSgdQxfzd5OfvJj9/N/l1+bspWR+AmZlVt5oYBmpmZltzBWBmVqNcAZSQpG9KCkm7Z68l6T8lPSPpEUmHVDrGcpM0TdIT2e9/vaRB7bZNzr6bJyX9fSXjrBRJx2W//zOSvl3peCpJ0p6S7pL0mKRHJZ2Zle8m6Q5JT2c/d610rJUiaaCk+ZJuyl7vLen+7O/naklvKfR+VwAlImlP4FjghXbFHwb2yx4TgEsrEFql3QGMioiDgKeAyQCSDgQ+C4wEjgMukTSwYlFWQPb7/oz0d3IgcHL2vdSqTaSbRA8ExgJfzb6PbwOzI2I/YHb2uladCTze7vXFpGl29gVeA04v9GZXAKUzHZhEGunU5pPAlZHMAQZ1GBLb70XE7RGxKXs5BxiaPf8kMCsi1kfEc8AzwOGViLGCDgeeiYhnI2IDMIv0vdSkiHg5Ih7KnjeREt3bSd/JzGy3mcCnKhNhZUkaCnwU+EX2WsBRwLXZLp1+N64ASkDSJ4GXIuLhDpveDrzY7vXSrKxWnQbcmj33d+PvIC9Jw4GDgfuBPSLi5WzTMmCPCoVVaT8mnWS+mb0eDKxud4LV6d9PTS0I05s6meriO6Tmn5pU6LuJiN9l+5xDusS/qpyxWd8jqR64DvhGRKxRuxkdIyIk1dxYdkkfA5ZHxDxJH+zu57gC6KZ8U11IGg3sDTyc/aEOBR6SdDjwErBnu92HZmX9SmfTgEj6AvAx4OjYfCNKTXw3nfB30IGkbUnJ/6qI+G1W/IqkIRHxctaEurxyEVbMEcAnsrVU6oCdgZ+QmpW3ya4COv37cRNQL4uIhRHx1ogYHhHDSZdhh0TEMuBG4JRsNNBY4PV2l7I1QdJxpMvWT0TEunabbgQ+K2k7SXuTOsofqESMFfQgsF82kuMtpE7xGyscU8VkbdqXA49HxI/abboRODV7firwu3LHVmkRMTkihmY55rPAnRHxOeAu4Phst06/G18BlNctwEdIHZzrgH+qbDgV8VNgO+CO7AppTkR8OSIelXQN8BipaeirEdFawTjLLiI2SfoacBswELgiIh6tcFiVdATweWChpAVZ2XeAHwDXSDodWAKcWKH4qtG/ArMkfR+YT6pA8/JUEGZmNcpNQGZmNcoVgJlZjXIFYGZWo1wBmJnVKFcAZmY1yhWA9SpJb5M0S9JiSfMk3SLpHZI+2DZjYaVJOl9SUWtW9/A4gySd0Quf8ydJvboQeqHPlHStpH0KvPctku6R5GHkfZwrAOs12Y071wN/iogREXEoabbPqpqrJSLOjYg/luFQg4AuVQDZTYIV+38paSQwMCKezbdPNlHdbOCksgVmJeEKwHrTOGBjRPy8rSAiHo6IP2cv67OzyyckXZVVGEg6V9KDkhZJmtGu/E+SLpb0gKSnJB2Zle8g6Zpsnvjrs/nPG7Ntx0q6T9JDkn6TzSOzBUm/lHR89vx5Sedl+y+UdECO/W+WdFD2fL6kc7Pn50v6Z0n1kma3+4y2GTx/AIyQtEDStOw9E7Pf9RFJ52Vlw5XWALgSWMSW00F0jGWr309pDYHftNvn/662ivk+Ovgc2d2jkoYpzbm/u6QBkv4sqW2Oqxuyfa0PcwVgvWkUMK/A9oOBb5Dmut+HdKcnwE8j4rCIGAVsT5onqM02EXF49r4pWdkZwGvZPPH/BhwKoLTwzneBD0XEIcBc4Owi4l6Z7X8p8K0c2/8MHClpF9Jdym1xHwncA7QAn84+Yxzww6wS+zawOCLGRMTELHnuR5r2eQxwqKT3Z5+1H3BJRIyMiCW5gizw+/0ReLekHbNdTyLdDdqd7+MIsn/DLI6Ls+/lm8BjEXF7tt8i4LBOPsuqnNvwrJweiIilANmt/cOBvwDjJE0CdgB2Ax4Ffp+9p20CsHnZ/gDvI018RUQskvRIVj6WVLncm11EvAW4r4i42h9jfI7tfwa+DjwH3AwcI2kHYO+IeFJpwrILs2T+JmkK3lzNXsdmj/nZ63pS4n8BWJKtEVFIzt8vm0LiD8DHJV1LmiN+EvCBXPt3cowhwIq2FxHxC0knAF8mVVpt5a2SNkjaKZur3/ogVwDWmx5l80RUuaxv97wV2EZSHXAJ0BgRL0r6Hml2w47vaaXzv1cBd0TEyV2KuvNjPAg0As+SVjTbHfhnNl/tfA5oAA6NiI2Snu/wO7SP76KIuGyLwjTX/doi4iz0+80CvgasAuZGRFN2FdLV7+ON9rFnFV3boj31QPtkvx3p6sf6KDcBWW+6E9hO0oS2AkkHtbXd59GWbFZm7dOFKpA295JNAKa0RODorHwOcISkfbNtO0p6Rxd/h61knZ4vAieQzqD/TGoquifbZRfS3OwbJY0DhmXlTcBO7T7qNuC0tnZ4SW+X9NYuhFLo97sbOIRUMc0qYv98Hgf2bff6YtKaDecC/91WKGkwqelsYxfityrjCsB6TTa3/6eBDykNA30UuIi0alO+96wmJZZFpAT5YBGHugRokPQY8H3SlcfrEbEC+ALwv1mz0H3AVp263fRnUpJ/I3s+NPsJKUE2SloInAI8ARARr5KaXxZJmpa1n/8auC/b91q2rCAKKvT7ZTOn3kRaT/imzvYv4GbggwCSPkBq5784Iq4CNkhqm8F2XLav9WGeDdT6HKXF07eNiBZJI0idoPtnZ+rWA5K2J80pf0Sh6bgl/Rb4dkQ8VbbgrNe5D8D6oh2Au7LOVwFnOPn3joh4Q9IUUkf2C7n2UVqs5gYn/77PVwBmZjXKfQBmZjXKFYCZWY1yBWBmVqNcAZiZ1ShXAGZmNer/A4x0hQogMArHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pyplot.plot(X, y,'ro', ms = 8, mec= 'k', mew = 1)\n",
    "pyplot.xlabel('Change in water level (x)')\n",
    "pyplot.ylabel('Water flowing out of the dam (y)')\n",
    "pyplot.plot(X, np.dot(X_aug, theta), 'b--', lw=2)\n",
    "pyplot.ylim(-5,40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxU5dn/8c8VFBBRaQBXRKAVyYILRNGyuOFu1ao/q7XWShqw1rY0PlVQa+tW8PEBtU+tDQm2arXVx2q1YotoUagiuwJJRFHBfQNUXFjn+v1xTjBklkzCbMl8369XXpm57zNzrhmGa07uc5/rNndHRETyR0G2AxARkcxS4hcRyTNK/CIieUaJX0Qkzyjxi4jkmR2yHUAyevTo4X369Ml2GCIibcrChQs/cveeTdvbROLv06cPCxYsyHYYIiJtipmtitWuoR4RkTyjxC8ikmfaxFCPiEg+cXfq6upYs2YNhYWFFBcXY2Ype34d8YuI5Ah3p6amhoElJZSWljJixAhKS0sZWFJCTU0NqSqxo8QvIpID3J0xY8ZQUVFB5+XLqQJmAFVA5+XLqaioYMyYMSlJ/kr8IiI5YOrUqVRXVzMemB+JMBoYCYwO748DqqurufPOO7d7X9YWqnOWlZW5pnOKSHvl7gwsKaHz8uXMj0SINZrvQFlBARsHDGDJsmVJjfmb2UJ3L2variN+EZEsq6uro7a+ntFxkj6AAaMjEZbV1VFfX79d+1PiFxHJsjVr1gDQr5ntGvpXr169XftT4hcRybLCwkIAXmtmu4b+7t27b9f+lPhFRLKsuLiYkqIiphQUEO+sqwNTCgooLS6mqKhou/anxC8ikmVmxtjKShZGIlwFUcnfgSuBRZEIYysrt/tiLl25KyKSA8rLy5k3bx4TqquZXlDA6EiEfgTDO1MKClgUiVBRUcGoUaO2e1864hcRyQFmRlVVFTU1NWw44AAuBo4HLgY2HHAANTU1VFVVpaR0g474RUSyrHFtnsMPP5wly5bx0ksvsXr1arp3705RUZFq9YiItAfxavMcWFrKc889x7Bhw1JeoA2U+EVEsiKTtXmaUuIXEcmCTNbmaUq1ekREMixdtXmaUq0eEZEckenaPE0p8YuIZFima/M0pcQvIpJhma7N05QSv4hIhmW6Nk9TSvwiIhmW6do8TenKXRGRLMhkbZ6mdMQvIpIFmazNE7VvzeMXEckud6e+vj7ltXnizePXUI+ISJaZGcXFxRnbX8LEb2adgVOB4cDewJfAMmCau9emPzwREUm1uInfzK4lSPpPA3OBD4DOQH9gYvilcJm7L8lAnCIikiKJjvjnufuv4vRNNrPdgd5piElERNIo7qwed58GYGYD4/R/4O464yoi0sYkM53z92Y2z8wuMbPd0h6RiIikVbOJ392HA+cD+wILzew+Mzsu7ZGJiEhaJHUBl7u/AlwNXAEcCfzWzF4yszPTGZyIiKRes4nfzA40s1uAeuAY4FvuXhTeviXN8YmISIolcwHX/wI1wJXu/mVDo7u/Y2ZXpy0yERFJi2YTv7sfmaDvntSGIyIi6RZ3qMfM/mFm3zKzHWP09TOz68wsbtk4M+sczgZ60cxqwwvCMLO+ZjbXzFaY2f1m1jE1L0VERJKRaIy/gqBUw0tmNt/MHjezf5vZa0AVsNDdEy3/vgE4xt0PAg4GTjSzw4GbgFvc/RvAWqA8Ja9ERESSEneox93fAy4HLjezPsBeBLV6Xnb3L5p7Yg/Kfn4W3t0x/HGCk8LfDdvvAn4N3NGq6EVEpMWSqs7p7iuBlS19cjPrACwEvgHcDrwKfOzum8NN3gL2ifPY0cBogN69VRlCRCRV0roQi7tvcfeDgV7AYcCAFjx2iruXuXtZz5490xajiEi+ycgKXO7+MTATOALoZmYNf2n0At7ORAwiIhJIW+I3s55m1i28vRNwHMFFYDOBs8PNLgQeSVcMIiISLZkrd081s8VmtsbMPjWzdWb2aRLPvRcw08yWAPOBGe7+GEHZh0ozWwF0B6ZuzwsQEZGWSebk7q3AmcBSb8ECveECLYfEaH+NYLxfRESyIJmhnjeBZS1J+iIikruSOeK/HHjczJ4huCgLAHefnLaoREQkbZJJ/DcSXIjVGVB5BRGRNi6ZxL+3u5emPRIREcmIZMb4Hzez49MeiYiIZEQyif9HwL/M7MsWTucUEZEclEw9/l0yEYiIiGRGUkXazOxrwP4EJ3gBcPdZ6QpKRETSJ5krd38IzAKmA9eGv3+d3rBERPLTZxs288dnX+fLjVvSto9kjvh/BhwKPO/uR5vZAOA3aYtIRCQPfbp+E2ff8Rwvvx8sY+IOo4b1Tcu+kkn86919vZlhZp3c/SUzOyAt0YiI5JlPvtjE6bf/h5Wrt13fqndhl7TtM5nE/1ZYZfPvwAwzWwusSltEIiJ54MuNWzh3yhxefOuTbdpPO2hvJp9zEDt0SF/V/GRm9Xw7vPlrM5sJ7Ab8K20RiYjkgRfe/HibpH/24F7cdNaBdCiwtO87buI3s8IYzUvD312BNWmJSESkHXr/0/VU3L2A4fv34BcnDODwfoWcP6Q3O3Yo4JpTiynIQMJvkOiIfyHB4ugG9AbWhre7AW8A6TnrICLSjrzz8ZccM+lp1m+KALDkrU+48Ig+7L5rZ2789sCsxBQ38bt7XwAzqwYedvfHw/snAWdkJjwRkbbpzTVfMOLmmTQtaH/FiQPouUun7AQVSubk7uHuXtFwx93/aWb/ncaYRETatGv/Ucsfn125TdtPjvkGlcf1xyxzQzrxJJP43zGzq4E/h/fPB95JX0giIm3Pp+s3sWvnHQG2ufjq5yP787OR+2crrJiSSfznAb8CHiYY858VtomI5L2X31/H8bcEFWweGHMEh/Ut5NJjvsGpB+7NsP17ZDm62JKZzrmG4OpdEREJ1b3zKSf/dvY2bbNf+ZDD+hbS62td6PW19F2Atb2SKtImIiKBJW99zGm/ezaq/frTS7jgiD6ZD6gVlPhFRJL07IqPOL9m7jZtN501kO8c2jtLEbWOEr+ItHnuTl1dHWvWrKGwsJDi4uKUzZ5Z8cFnrP1iI4f2KeSQ3t0wCwqoTT7nIM4c1Csl+8i0ZhO/mfUH7gD2cPdSMzsQOM3db0h7dCIiCbg7U6dO5dbJk6mtr9/aXlJUxNjKSsrLy1v9BfDcio/4bnh0333njjx/5bF06bgDL/7q+K2zd9qqZKoAVQPjgU0A7r4EODedQYmINMfdGTNmDBUVFXRevpwqYAZQBXRevpyKigrGjBmDN72CqhnPvPwhfcZN25r0ATZsjvD5hs0AbT7pQ3JDPV3cfV6Tb83NaYpHRCQpU6dOpbq6mvHAjZEIjTNURSTClcDE6mqGDBlCeXl5s8+3+I21fPv3z0Xv58Iyji3aI2Vx54Jkjvg/MrOvE8zhx8zOBt5Na1QiIgm4O7dOnszgggJuBJoO5hjBalGDCgq4dfLkpI76r/77sm3u3zXqMFZOPKXdJX1ILvH/mOCvpwFm9jYwFvhRWqMSEUmgrq6O2vp6Rjc50m/MgNGRCMvq6qhvNP7f4B8vvsN3quaw+rMNQHCFbb8eO3PfD4ewcuIpHNm/Z/peQJYlcwHXa8BIM9sZKHD3dekPS0QkvjVrgqrw/ZrZrqF/9erVW9seXvwWP7//xa33q2e/zriTBjCyeA9GFre/o/tYkpnV0wk4C+gD7NAw1u/u16U1MhGROAoLg+VCXmtmu4b+7t2788D8N7n8b0uitvn+EfulNrg2IJmTu48AnxDU59+Q3nBERJpXXFxMSVERU5YvpyLOcI8DUwoKKCobzsl3vx7V/+ilQzmwV7e0x5qLkkn8vdz9xLRHIiKSJDNjbGUlFRUVXAVRJ3gduKLDjizasonfXHgRVW981Tftp8Mo2Xu3zAacY5JJ/M+Z2UB3X9r8piIimVFeXs68efOYUF3N9IICRkci9APuO+wsZh59EV+8PIeKHh8w7kffZ8ALb3Ngr258vWfXbIedEyzeNCczW0rwxbkDsD/BcNkGgi9Wd/cDMxVkWVmZL1iwIFO7E5E2wt258847uWXSJN7tdSS7DDp1m/4FVx1Lj106Zym67DOzhe5e1rQ90RH/qQn6RERywtpew/jstD3ZpUn70/91VF4n/UQSrbm7CsDM7nH3Cxr3mdk9wAUxHygikmbuzisffMZlD7zI0rc/2aZv9uVHs29h7tbCzwXJjPGXNL5jZh2AwekJR0QkPnfnxmn1zF+1lmVvf8KpB+7F0rc/odMOBcz8r6PYu9tO2Q6xTYib+M1sPHAlsJOZfdrQDGwEpmQgNhERACIR55sT/817n67f2la4c0eOK96DG789kK6dVGG+JRIN9UwAJpjZBHcf39InNrN9gbuBPQin1Lr7bWZWCNxPcEHYSuAcd1/bithFpJ2LRJzBN8xg7Rebtmkv3mtXHvzREXTpqITfGsmUbGhx0g9tBi5z90Vmtguw0MxmAD8AnnL3iWY2DhgHXNHKfYhIOxSJOOMfWsr9C96M6ntu3DEa0tlOafu6dPd3Cat4uvs6M6sH9gFOB44KN7sLeBolfhEBtkScfy57lxff/Dgq6c+98lj22FWzdFIh0Rh/X3ePvs65FcysD3AIMJdgJa+Gss7vEQwFiUgeW79pCyP+eyYfrAuqwhy8bzcG9e7GFxu38MDFR7SLxU9ySaIj/geBwWb2lLsf29odmFlX4G/AWHf/tPGCLu7uZhbzCjIzGw2MBujdu20tZCwiyfli42aKr5m+TVu3Ljvy/8p6cd6hvSkoSM26ubKtRIm/wMyuBPqbWWXTTnef3NyTm9mOBEn/Xnd/KGx+38z2cvd3zWwv4INYj3X3KYSzh8rKylq2dpqI5LT1m7Yw+PoZfL5xS1Tfc+OO0UnbNEv07p4LnBFu0/SiuGZZcGg/Fahv8iXxKHAhMDH8/UhLn1tE2iZ357dPreCWJ1+O6lt27QmalpkhiaZzLgduMrMl7v7PVjz3UIKre5ea2Qth25UECf8BMysHVgHntOK5RaQNWb9pCw8ufIuHF79Nxw5fLfzXo2snZl1+lI7wMyzZ6pyTgRHh/WeA69z9kwSPwd3/Q/RSmA1afc5ARNqO1z78jGMmPbNN26ihfTlmwO5c+M0+dNwhmdVfJdWSSfx3Asv46sj8AuCPwJnpCkpE2rZVqz/nyJufjmq//buDOKl0T520zbJkEv/X3f2sRvevbTR0IyKy1YoPPmPk5Gdi9r36m5PpoISfE5JJ/F+a2bBw6AYzGwp8md6wRKQt+XDdBg698cmYfStuPIkdOmhIJ5ckk/gvBu42s4a1ytYSzMYRkTz36oefceyk6CP8AoNXbtQRfq5KplbPi8BBZrZreP/TZh4iIu3cf175iO9NnRvVfmT/nvzpokNpfKGm5J6k51Ap4YvIcys+4rs10Qn/u0N6c+MZpUr4bYQmz4pIs2a/8iEXTJ0Xs+/1CScr4bcxzSZ+M+vk7huaaxOR9ifeLJ3OOxbw0vUnZSEiSYVkjvjnAIOSaBORduKB+W9y+d+WRLX36NqJBVePzEJEkkqJyjLvSVA/fyczO4SvrsLdFdBKxiLt0P8teJNfPBid8K8+pYgfDu+XhYgkHRId8Z9AsFpWL6BxkbV1BDV3RKSdmFH3PhV3L4hqP++w3kw4c2AWIpJ0SlSk7S7gLjM7y93/lsGYRCRDHl/6Lpfcuyiq/WtddmTxNcdnISLJhGTG+EvNrKRpo7tfl4Z4RCQD5r62mu9MeT6qfUT/ntw96rAsRCSZlEzi/6zR7c7AqUB9esIRkXT6yV8W848X34lqP2XgXtx+vuZr5Itkrtyd1Pi+mf0PMD3O5iKSg679Ry1/fHZlVPsfvjeYE0v3TMs+3Z26ujrWrFlDYWEhxcXFmu+fI1pzAVcXghO+IpLjrnp4KffOfSOq/XuH9+aGM9Jz0tbdmTp1KrdOnkxt/VeDAyVFRYytrKS8vFxfAFmWzAVcS4GGNW87AD0Bje+L5LBpS97lx/dFn7StGN6Xq04pTtt+3Z0xY8ZQXV3N4IICqoB+wGvAlOXLqaioYN68eVRVVSn5Z1EyR/ynNrq9GXjf3TenKR4R2Q63z1zBzdOXR7XfcEYp3zt8v7Tvf+rUqVRXVzMeuDES2WYJvopIJFh7tbqaIUOGUF5envZ4JDZz9+Y3MjsIGB7eneXu0Vd4pFFZWZkvWBA9x1hEAmU3PMlHn0VXUbn1OwdzxiH7ZCQGd2dgSQmdly9nfpOkv3UboKyggI0DBrBk2TId9aeZmS1097Km7ckM9fwMqAAeCpvuNbMp7v6/KY5RRFro279/lsVvfBzV/vAl3+SQ3l/LaCx1dXXU1tdTRfzFtg0YHYlwcV0d9fX1FBenb9hJ4ktmqKccGOLunwOY2U0EtXqU+EWy5IRbZrH8/XVR7WNH7s/Ykf2zEBGsWbMGCMb0E2noX716dVrjkfiSSfwGbGl0fwvxv9BFJI2m/ud1rn+sLqp98jkHceag7E62KywsBIITuYk09Hfv3j2t8Uh8yST+PwJzzezh8P4ZwNT0hSQiTZ3y29nUvhO9FlL198s4rniPLEQUrbi4mJKiomD2ToIx/ikFBZQOGEBRUVGmQ5RQMhdwTTazp4FhYdNF7r44rVGJCAB9xk2L2f7Ez0fQf49dMhxNYmbG2MpKKioquAq4kW2HBpyguuOiSISaykqd2M2ipC7gcvdFQPSkYBFJOXen+JrpfLlpS1Tf9LEjOGDP3Er4jZWXlzNv3jwmVFczvaCA0ZHIV/P4CwpYFIlQUVHBqFGjsh1qXtPSiyI5wt3pO/7xmH25NKSTiJlRVVXFkCFDuGXSJC5ufOXuAQdQc9lljBo1Skf7WZbUPP5s0zx+ac8iEafflbET/kOXfJNBGZ6WmSruTn19PatXr6Z79+4UFRUp4WdYq+fxi0h6bNoSYf+r/hmz748/OJSjB+ye4YhSy8w0Tz9HJXMB1zq+qtXT4BNgAXCZuzc3e0tEGkmU8HN9DF/ah2SO+G8F3gLuIzhJfy7wdYKTvXcCR6UrOJH25MuNWyi65l8x+6b9dBgle++W4YgkXyWT+E9z94Ma3Z9iZi+4+xVmprV3RZqxftMWBvwydsL/92VH0q9n1wxHJPkumcT/hZmdAzwY3j8bWB/ezv0zwyJZ8un6TRz46ydi9j037hj27rZThiMSCSST+M8HbgN+T5Donwe+Z2Y7AZemMTaRNmn1ZxsYfMOTMfue+cVR7Nd95wxHJLKtZK7cfQ34Vpzu/6Q2HJG2a+3nGznk+hkx+xZePZLuXTtlOCKR2JKZ1dOToCxzn8bbu7suvRMB3lj9BSNunhmzb874Y9hrNw3pSG5JZqjnEWA28CTbVukUyWvvfbKewyc8FbNv2bUn0LWTLpOR3JTMJ7OLu1+R9khE2oi6dz7l5N/Ojtn30vUn0nnHDhmOSKRlkkn8j5nZye4e+5pykTyx8qPPOep/no7ZV3/diezUUQlf2oZkEv/PgCvNbAOwieAiLnf3XdMamUiOeGvtFwy7KfYY/ss3nETHHQoyHJHI9klmVk+rrh83szuBU4EP3L00bCsE7ic4UbwSOMfd17bm+UXSbcHKNZz9hzkx+5TwpS2Lm/jNbIC7v2Rmg2L1hzX6E/kT8Dvg7kZt44Cn3H2imY0L7+v8geSUF978mDNufzZm36u/OZkOBaowKW1boiP+SmA0MClGnwPHJHpid59lZn2aNJ/OV7V97gKeRolfcsQ9z6/il39fFrPvtd+cTIESvrQTcRO/u48Ofx+dwv3t4e7vhrffA+KuLGFmowm+eOjdu3cKQxDZ1rMrPuL8mrkx+16fcLJqyEu7k8wFXP8BniGYy/+su69LxY7d3c0sbq0fd58CTIFgIZZU7FOksXvmrOSXj9TG7FPCl/YsmVk9FwDDgbOAm8PZPbPd/eet2N/7ZraXu79rZnsBH7TiOUS2y7Ql7/Lj+2Kfolo58ZQMRyOSecnM6nndzNYDG8Ofo4GiVu7vUeBCYGL4+5FWPo9Ii/1l3huMf2hpVPshvbvx8CVDsxCRSHYkM9TzKvARwUIsU4GfuHskicf9heBEbg8zewv4FUHCf8DMyoFVwDmtD10kOZfcu5DHl74Xs09H+JKPkhnq+S0wDDgPOAR4xsxmufuriR7k7ufF6Tq2ZSGKtM7dc1ZyTYwx/IH77MY/fjIs8wGJ5IhkhnpuA24zs67ARcCvgV6Ark+XnDTmngVMr30/Zl++HuG7O3V1daxZs4bCwkKKi4t18jqPJTPUM4ngiL8rMAe4hmCGj0hOue3JV7jlyZej2o8+oCd/vOiwLESUfe7O1KlTuXXyZGrr67e2lxQVMbaykvLycn0B5KFkhnrmAP/t7rEPoUSybPTdC3iiLvrjWbrPrjz2k+FZiCg3uDtjxoyhurqawQUFVAH9gNeAKcuXU1FRwbx586iqqlLyzzPJDPU8aGanmdmIsOkZd/9HmuMSadaP71vEtCXvRrWPLNqDmgvLshBRbpk6dSrV1dWMB26MRGic2isiEa4EJlZXM2TIEMrLy7MUpWSDuSe+NsrMJgCHAfeGTecB8939yjTHtlVZWZkvWLAgU7uTHHfVw0u5d+4bUe3nHrovE886MAsR5R53Z2BJCZ2XL2d+k6S/dRugrKCAjQMGsGTZMh31t0NmttDdo46CkhnqOQU4uGEKp5ndBSwGMpb4RQAGXT+DNZ9vjGrftfMOLPn1CVmIKHfV1dVRW19PFcRM+oTtoyMRLq6ro76+nuLi4gxGKNmU7Npw3YA14e3d0hSLSExH3TyTlau/iGofO3J/xo7sn4WIct+aNcF/137NbNfQv3r16rTGI7klmcQ/AVhsZjMJDhJGEJRTFkmrPuOmxWw/vngPpnxfY/iJFBYWAsGJ3EQa+rt3757WeCS3NDvGDxDW1Tk0vDvP3WNfBpkmGuPPL33HTyPWx/LX3yrmB0P7Zj6gNkhj/AKtGOOPsQDLW+Hvvc1s7yQWYhFpkX7jpxGJkfAvOHw/rj+jNPMBtWFmxtjKSioqKrgKuJFtx/qd4CTdokiEmspKJf08k2ioJ9YCLA2aXYhFJFnxhnSU8LdPeXk58+bNY0J1NdMLChgdiXw1j7+ggEWRCBUVFYwaNSrboUqGJUr8d7j7A2bWz92bGyoUabF4Cf/6M0q54PD9MhxN+2NmVFVVMWTIEG6ZNImLG1+5e8AB1Fx2GaNGjdLRfh6KO8ZvZovcfVDD7wzHtQ2N8bcf7k7f8Y/H7Lv6lCJ+OLy5eSjSGu5OfX09q1evpnv37hQVFSnh54HWzONfbWZPAH3N7NGmne5+WioDlPYtUcKvumAwJ5TsmeGI8ouZaZ6+bJUo8Z8CDALuIfF4v0hckYjT78rYCV+zdESyI9Fi6xuB583sm+7+YQZjknYgUcL/c/kQhu3fI8MRiUiDZIq0KelL0jZujtD/6n/G7HvsJ8Mo3UcXfotkW7IlG0QS2rB5Cwdc/a+YfY/8eCgH7dstwxGJSDwJE7+ZdQB+6u63ZCierNIqRS23ftMWBvwydsL/58+GU7TXrhmOqH3QZ1HSqSBRp7tvISjD3K65OzU1NQwsKaG0tJQRI0ZQWlrKwJISampqSKasRb75bMNm+oybFjPp/6XicFZOPEVJvxX0WZRMSGao51kz+x1wP/B5Q2N7KdmgVYpa5pMvN3HQtU/E7Jv1i6Pp3b1LhiNqP/RZlExJZiGWmTGa3d0zVrIhnRdw1dTUUFFREaxSROx6JhPD7fJ5laI313zB8P+O9VGAmf91FH177JzhiNoffRYl1eJdwJVUdc5sS1fiVwXD5n3w6XoO+81TMfsW/fI4CnfumOGI2id9FiUdWr0Cl5ntAfwG2NvdTzKzYuAId5+ahjhTJpmTY1qlKL6VH33OUf/zdMy+BVePpEfXTpkNqJ3TZ1EyKZkx/j8BfwSuCu+/TDDen5OJ392ZOnUqt06eTG3jolRFRYytrKS8vHzrF0Aur1KUrVkdb639gmE3xR7SmXflsey+a+e0x5CPcvmzKO1PMom/R1ilczyAu282sy1pjqtVWnpyLBdXKWrJF1cqrfhgHSMnz4rZV3vtCezcSZd8pFMufhal/Urm5O7TwFnAjLBa5+HATe5+ZAbiA5If42/pybFcG1dt+sXVtH76wrB+eipndfz7pfcZ9afY7239dSeyU8cOKdlPa2XiL59cmDOfa59FaR/ijfHj7gl/CAq1PQt8Ev5+GTioucel8mfw4MHenEgk4iVFRT64oMAj4B7jJwI+qKDAS4uLPRKJuLt7dXW1Az4+7G+6/bjg/5vX1NQ0G8P2ymQsi99Y6/td8VjMn42bt6Tg1WyfSCTi1dXVXlJU5ISvG/CSoiKvrq7e+u+X6/toiVz6LEr7ACzwWHk9VuM2G0AngiGhEqAU2BHo1NzjUvmTTOJftmyZA14VJ+k3/Pwh/M9TW1vr7sF//oqKCif8UvgD+BPhdoMKChzwioqKtCeB1n5xtdS/69/P6YTvvu2/yeCCAq8CnxH+2w5O0b9JJvaxPTFl87Mo7cf2JP5FybSl8yeZxD9r1iwn/M+bKPE/ESb+WbNmbX1sJBLxmpqamEd+NTU1GfmP1tovrmQtWrUmbsLfvCW3Ekkmjnxz9eg6Fz6L0n7ES/yJVuDaE9gH+DPwXb4aMt8V+IO7D0g4uJRCyYzx19bWUlpaShUwOsF2VcDF4fZNp8O5Z2+VotmzZzNixAhmACMTbDcDOB6YNWsWw4cPb/Z5n6x7nx/eHfu9e33CyTk3Tuye/rHuTOxje2XzsyjtR2vm8Z8A/ADoBUxu1L6O4DxpTikuLqakqCiYvZPgP/OUggJKBwygqKgoqj+bqxSlelbHzJc+4KI/zY/Zt/Ojv+DnlZUtjDAzMjGfvS3MmdeKWZJOiRZiuQu4y8zOcve/ZTCmVjEzxlZWUlFRwVXEn9WzKBKhprIy546eUvHFBbBw1RrOumNOzL6am07dOkMoV+u+ZGI+u+bMS75LZiGWv5nZKQQndzs3ar8unYG1Rnl5OfPmzWNCdTXTY0yHXBROhxw1alS2Q42yvV9c9819gysfXhrzuV+/6dRtnqsiEgmmtlZXM2DYWmMAAA5kSURBVGTIkJyq+5KJ+eyaMy/5Lpl5/H8AugBHAzXA2cA8d89YtmhJrR5358477+SWSZOiLoD6+WWXMWrUqJw6wm3M/at5/IMSfHE1Pkr/++K3GXv/C1HPtcPbL9Ltvl/m7Bh2PBrjF0md7ZnHv6TJ767A7OYel8qfZGb1NBWJRLy2ttZnzZrltbW1bWY2RLKzOm7+10txZ+mke4ZQuuXzrB6RVCLOrJ5krsP/Mvz9hZntDawG9mr9d1BmtNWTY2ZGeXk5o0aNijmr41/L3uXiP0cvhXBgr9149NJhQDBDCNruGHYmhuza8rCgyPZKJvE/ZmbdgJuBRQRHQtVpjUqivrieXfER59fMjdquR9dOLLh62wmgbX0M28yoqqpiyJAh3DJpEhc3HrI74ABqUjBkl4l9iOSqRPP4xwLPEVystTls6wR0dvdPMhdiehdiyXW3zHiZ2556Jap99106Me+q2DP+vR2NYXsG5rNnYh8i2dCaefy9gFuBAWa2lKBOz3Phz/YGcyJwG9ABqHH3idv7nO3N9Nr3GHPPwqj2yuP689Nj90/42LY+tbWxTAzZtdVhQZHWSmZWT0egDPgmcET487G7t+p/ipl1ICj0dhzwFjAfOM/d6+I9Jp+O+K95ZBl3z1kV1X5Y30IeGHNE0s/jrZghJCLtS6tX4AJ2IijTsFv48w4Qe8J4cg4DVrj7a2FgfwVOB+Im/nxwx9OvctO/Xopqv2hoH371rZIWP5/GsEUknkRj/FMILtpaB8wFngeed/e127VDs7OBE939h+H9C4Ah7n5pk+1GE5bd6d279+BVq6KPgtuDW598mVufjB7Dn3DmQM47rHdK9qExbJH81Joj/t4EJZlfAd4mGJb5OD3hRXP3KcAUCIZ6MrXfTHl48Vv8/P4Xo9onn3MQZw7qldJ9aQxbRBpLVKvnRAsOC0sIxvcvA0rNbA0wx91/1cp9vg3s2+h+r7AtL/zvU68wacbLUe1/Lh/CsP17ZCEiEck3Ccf4wyu/lpnZxwQrcH0CnEowTt/axD8f2N/M+hIk/HMJyj63a6ff/iwvvhn9B9NDl3yTQb2/loWIRCRfxU38ZvZTgiP9bwKb+Goq551sx8ldDxZrvxSYTjCd8053r23t8+W6sX9dzN9feCeq/bGfDKN0n92yEJGI5LtER/x9gP8Dfu7u76Zyp+7+OPB4Kp8z14y5ZwHTa9+Par/t3IM5/eB9shCRiEgg0Rh/bq7UkeNun7mCm6cvj2q/f/ThDOmXW6URRCQ/JTOPX5rh7pxTNYf5K6Nnuj7y46EctG+3LEQlIhKbEv92cHeOu2UWKz74LKpv9uVHs29hlyxEJSKSmBJ/K7g7Z/9hDgtXRR/hK+GLSK5T4m+BSMQ5v2Yuc16Lrl//6KVDObCXhnREJPcp8SchEnHGPbSEBxa8FdX3zC+OYr/uO2chKhGR1lHiT2Dj5gj9r/5nzL7/XHE0vb6mIR0RaXuU+GOIl/B7F3bh0UuH0q1LxyxEJSKSGkr8jWzcHGHMPQuYufzDqL55Vx3L7rt0zkJUIiKppcQf+vvitxl7/wtR7QuvHkn3rp2yEJGISHrkdeLfvCXC+s0RunbagWlLv6pK8fWeO/PQj4ayW5cdsxidiEh65GXi/3LjFr43dS4LV61l1NC+XPOtYiqP68+xA3bn9IP3YaeOHbIdoohI2uRV4v98w2bOnfI8S9/+ZGvb86+tZkvEKdprV4r22jWL0YmIZEZeJP516zdx9h1zWP7+um3ajx2wO1O+X0aHAi1DKCL5o90n/oWr1nDWHXO2aTupdE9+e94h7NihIEtRiYhkT7tP/Pt135mdduzAl5u2cNpBezP5nIPYQQlfRPJYu0/8Pbp24pFLh/KNnl0p0JCOiEj7T/wA/ffYJdshiIjkDI15iIjkGSV+EZE8o8QvIpJnlPhFRPKMEr+ISJ5R4hcRyTNK/CIieUaJX0Qkz5i7ZzuGZpnZh8CqbMfRAj2Aj7IdRI7SexOf3pv49N7El+i92c/dezZtbBOJv60xswXuXpbtOHKR3pv49N7Ep/cmvta8NxrqERHJM0r8IiJ5Rok/PaZkO4AcpvcmPr038em9ia/F743G+EVE8oyO+EVE8owSv4hInlHiTzEzu8zM3Mx6hPfNzH5rZivMbImZDcp2jJlmZjeb2Uvh63/YzLo16hsfvjfLzeyEbMaZTWZ2YvgerDCzcdmOJ5vMbF8zm2lmdWZWa2Y/C9sLzWyGmb0S/v5atmPNFjPrYGaLzeyx8H5fM5sbfn7uN7OOiR6vxJ9CZrYvcDzwRqPmk4D9w5/RwB1ZCC3bZgCl7n4g8DIwHsDMioFzgRLgROD3ZtYha1FmSfiabyf4rBQD54XvTb7aDFzm7sXA4cCPw/djHPCUu+8PPBXez1c/A+ob3b8JuMXdvwGsBcoTPViJP7VuAS4HGp8xPx242wPPA93MbK+sRJcl7v6Eu28O7z4P9Apvnw781d03uPvrwArgsGzEmGWHASvc/TV33wj8leC9yUvu/q67LwpvryNIcPsQvCd3hZvdBZyRnQizy8x6AacANeF9A44BHgw3afa9UeJPETM7HXjb3V9s0rUP8Gaj+2+FbflqFPDP8Lbem4DehzjMrA9wCDAX2MPd3w273gP2yFJY2XYrwQFmJLzfHfi40cFVs5+fvFhsPVXM7ElgzxhdVwFXEgzz5KVE7427PxJucxXBn/H3ZjI2aZvMrCvwN2Csu38aHNgG3N3NLO/mopvZqcAH7r7QzI5q7fMo8beAu4+M1W5mA4G+wIvhh7MXsMjMDgPeBvZttHmvsK1diffeNDCzHwCnAsf6VxeP5MV7kwS9D02Y2Y4ESf9ed38obH7fzPZy93fD4dIPshdh1gwFTjOzk4HOwK7AbQRDyDuER/3Nfn401JMC7r7U3Xd39z7u3ofgT61B7v4e8Cjw/XB2z+HAJ43+XM0LZnYiwZ+mp7n7F426HgXONbNOZtaX4AT4vGzEmGXzgf3DmRkdCU54P5rlmLImHLOeCtS7++RGXY8CF4a3LwQeyXRs2ebu4929V5hnzgX+7e7nAzOBs8PNmn1vdMSffo8DJxOcuPwCuCi74WTF74BOwIzwL6Ln3f1id681sweAOoIhoB+7+5YsxpkV7r7ZzC4FpgMdgDvdvTbLYWXTUOACYKmZvRC2XQlMBB4ws3KCMu3nZCm+XHQF8FczuwFYTPDFGZdKNoiI5BkN9YiI5BklfhGRPKPELyKSZ5T4RUTyjBK/iEieUeKXlDCzPc3sr2b2qpktNLPHzay/mR3VUEEw28zsOjNLeKFZivbTzcwuScHzPG1mKV1gPNFzmtmDZtYvwWM7mtksM9M08DZOiV+2W3jBzcPA0+7+dXcfTFCBM6dqqbj7Ne7+ZAZ21Q1oUeIPL/DL2v9HMysBOrj7a/G2CQvIPQV8J2OBSVoo8UsqHA1scvc/NDS4+4vuPju82zU8mnzJzO4Nvygws2vMbL6ZLTOzKY3anzazm8xsnpm9bGbDw/YuZvZAWKf94bD+eFnYd7yZzTGzRWb2f2Gdl22Y2Z/M7Ozw9kozuzbcfqmZDYix/TQzOzC8vdjMrglvX2dmFWbW1cyeavQcDRU1JwJfN7MXzOzm8DG/CF/rEjO7NmzrY0EN/ruBZWxbtqFpLFGvz4Ia/v/XaJutf10l8340cT7h1Z5mtp8FNe97mFmBmc02s4Y6VH8Pt5U2TIlfUqEUWJig/xBgLEGt+X4EV2YC/M7dD3X3UmAnglo+DXZw98PCx/0qbLsEWBvWaf8lMBjAgkVvrgZGuvsgYAFQmUTcH4Xb3wH8V4z+2cBwM9uN4MrihriHA7OA9cC3w+c4GpgUfnmNA15194Pd/Rdh0tyfoPzywcBgMxsRPtf+wO/dvcTdV8UKMsHrexIYYmY7h5t+h+Dqzda8H0MJ/w3DOG4K35fLgDp3fyLcbhlwaDPPJTlOY3WSCfPc/S2A8BL8PsB/gKPN7HKgC1AI1AL/CB/TUJhrYbg9wDCCglS4+zIzWxK2H07wpfJs+EdDR2BOEnE13seZMfpnAz8FXgemAceZWRegr7svt6CQ2G/CJB4hKIUba3jr+PBncXi/K0HCfwNYFa7TkEjM1xeWevgX8C0ze5CgRvvlwJGxtm9mH3sBHzbccfcaM/t/wMUEX1YN7VvMbKOZ7RLWypc2SIlfUqGWrwpExbKh0e0twA5m1hn4PVDm7m+a2a8Jqg02fcwWmv+cGjDD3c9rUdTN72M+UAa8RrCKWA+ggq/+ujkf6AkMdvdNZrayyWtoHN8Ed6/apjGoNf95EnEmen1/BS4F1gAL3H1d+FdHS9+PLxvHHn7BNSyY0xVonOQ7Efy1I22UhnokFf4NdDKz0Q0NZnZgw9h8HA1J5qNw/DnRF0eDZwkLc1mwFN/AsP15YKiZfSPs29nM+rfwNUQJT2a+Cfw/giPm2QRDQrPCTXYjqI2+ycyOBvYL29cBuzR6qunAqIZxdjPbx8x2b0EoiV7fM8Aggi+kvyaxfTz1wDca3b+JYN2Ea4DqhkYz604wRLapBfFLjlHil+0W1tf/NjDSgumctcAEglWS4j3mY4KEsowgMc5PYle/B3qaWR1wA8FfGp+4+4fAD4C/hMM/c4Cok7WtNJsguX8Z3u4V/oYgMZaZ2VLg+8BLAO6+mmCYZZmZ3RyOj98HzAm3fZBtvxgSSvT6wmqmjxGs1/tYc9snMA04CsDMjiQYx7/J3e8FNppZQ1XZo8NtpQ1TdU5pMyxYlHxHd19vZl8nOLl5QHhkLtvBzHYiqOk+NFFpbDN7CBjn7i9nLDhJOY3xS1vSBZgZnlQ14BIl/dRw9y/N7FcEJ6jfiLWNBYvE/F1Jv+3TEb+ISJ7RGL+ISJ5R4hcRyTNK/CIieUaJX0Qkzyjxi4jkmf8P7outqCE9f4sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# add a columns of ones for the y-intercept\n",
    "X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
    "theta, cost = trainLinearReg(linearRegCostFunction, X_aug, y, lambda_=0)\n",
    "\n",
    "#  Plot fit over the data\n",
    "pyplot.plot(X, y, 'ro', ms=10, mec='k', mew=1.5)\n",
    "pyplot.xlabel('Change in water level (x)')\n",
    "pyplot.ylabel('Water flowing out of the dam (y)')\n",
    "pyplot.plot(X, np.dot(X_aug, theta), '--', lw=2);\n",
    "\n",
    "#we can see here that fitting a linear function is (one feature; theta1) is high bias here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section3\"></a>\n",
    "## 2 Bias-variance\n",
    "\n",
    "An important concept in machine learning is the bias-variance tradeoff. Models with high bias are not complex enough for the data and tend to underfit, while models with high variance overfit to the training data.\n",
    "\n",
    "In this part of the exercise, you will plot training and test errors on a learning curve to diagnose bias-variance problems.\n",
    "\n",
    "### 2.1 Learning Curves\n",
    "\n",
    "You will now implement code to generate the learning curves that will be useful in debugging learning algorithms. Recall that a learning curve plots training and cross validation error as a function of training set size. Your job is to fill in the function `learningCurve` in the next cell, so that it returns a vector of errors for the training set and cross validation set.\n",
    "\n",
    "To plot the learning curve, we need a training and cross validation set error for different training set sizes. To obtain different training set sizes, you should use different subsets of the original training set `X`. Specifically, for a training set size of $i$, you should use the first $i$ examples (i.e., `X[:i, :]`\n",
    "and `y[:i]`).\n",
    "\n",
    "You can use the `trainLinearReg` function (by calling `utils.trainLinearReg(...)`) to find the $\\theta$ parameters. Note that the `lambda_` is passed as a parameter to the `learningCurve` function.\n",
    "After learning the $\\theta$ parameters, you should compute the error on the training and cross validation sets. Recall that the training error for a dataset is defined as\n",
    "\n",
    "$$ J_{\\text{train}} = \\frac{1}{2m} \\left[ \\sum_{i=1}^m \\left(h_\\theta \\left( x^{(i)} \\right) - y^{(i)} \\right)^2 \\right] $$\n",
    "\n",
    "In particular, note that the training error does not include the regularization term. One way to compute the training error is to use your existing cost function and set $\\lambda$ to 0 only when using it to compute the training error and cross validation error. When you are computing the training set error, make sure you compute it on the training subset (i.e., `X[:n,:]` and `y[:n]`) instead of the entire training set. However, for the cross validation error, you should compute it over the entire cross validation set. You should store\n",
    "the computed errors in the vectors error train and error val.\n",
    "\n",
    "<a id=\"func2\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [],
   "source": [
    "def learningCurve(X, y, Xval, yval, lambda_=0):\n",
    "    \"\"\"\n",
    "    Generates the train and cross validation set errors needed to plot a learning curve\n",
    "    returns the train and cross validation set errors for a learning curve. \n",
    "    \n",
    "    In this function, you will compute the train and test errors for\n",
    "    dataset sizes from 1 up to m. In practice, when working with larger\n",
    "    datasets, you might want to do this in larger intervals.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array_like\n",
    "        The training dataset. Matrix with shape (m x n + 1) where m is the \n",
    "        total number of examples, and n is the number of features \n",
    "        before adding the bias term.\n",
    "    \n",
    "    y : array_like\n",
    "        The functions values at each training datapoint. A vector of\n",
    "        shape (m, ).\n",
    "    \n",
    "    Xval : array_like\n",
    "        The validation dataset. Matrix with shape (m_val x n + 1) where m is the \n",
    "        total number of examples, and n is the number of features \n",
    "        before adding the bias term.\n",
    "    \n",
    "    yval : array_like\n",
    "        The functions values at each validation datapoint. A vector of\n",
    "        shape (m_val, ).\n",
    "    \n",
    "    lambda_ : float, optional\n",
    "        The regularization parameter.\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    error_train : array_like\n",
    "        A vector of shape m. error_train[i] contains the training error for\n",
    "        i examples.\n",
    "    error_val : array_like\n",
    "        A vecotr of shape m. error_val[i] contains the validation error for\n",
    "        i training examples.\n",
    "    \n",
    "    Instructions\n",
    "    ------------\n",
    "    Fill in this function to return training errors in error_train and the\n",
    "    cross validation errors in error_val. i.e., error_train[i] and \n",
    "    error_val[i] should give you the errors obtained after training on i examples.\n",
    "    \n",
    "    Notes\n",
    "    -----\n",
    "    - You should evaluate the training error on the first i training\n",
    "      examples (i.e., X[:i, :] and y[:i]).\n",
    "    \n",
    "      For the cross-validation error, you should instead evaluate on\n",
    "      the _entire_ cross validation set (Xval and yval).\n",
    "    \n",
    "    - If you are using your cost function (linearRegCostFunction) to compute\n",
    "      the training and cross validation error, you should call the function with\n",
    "      the lambda argument set to 0. Do note that you will still need to use\n",
    "      lambda when running the training to obtain the theta parameters.\n",
    "    \n",
    "    Hint\n",
    "    ----\n",
    "    You can loop over the examples with the following:\n",
    "     \n",
    "           for i in range(1, m+1):\n",
    "               # Compute train/cross validation errors using training examples \n",
    "               # X[:i, :] and y[:i], storing the result in \n",
    "               # error_train[i-1] and error_val[i-1]\n",
    "               ....  \n",
    "    \"\"\"\n",
    "    # Number of training examples\n",
    "    m = y.size\n",
    "\n",
    "    # You need to return these values correctly\n",
    "    error_train = np.zeros(m)\n",
    "    error_val   = np.zeros(m)\n",
    "\n",
    "    # ====================== YOUR CODE HERE ======================\n",
    "    for i in range(1, m+1):\n",
    "        theta,_ = trainLinearReg(linearRegCostFunction, X[:i, :], y[:i], lambda_)\n",
    "        error_train[i-1],_ = linearRegCostFunction(X[:i, :], y[:i],theta, lambda_=0)\n",
    "        error_val[i-1], _ = linearRegCostFunction(Xval, yval,theta, lambda_=0)\n",
    "        \n",
    "\n",
    "        \n",
    "    # =============================================================\n",
    "    return error_train, error_val"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you are finished implementing the function `learningCurve`, executing the next cell prints the learning curves and produce a plot similar to the figure below. \n",
    "\n",
    "![](Figures/learning_curve.png)\n",
    "\n",
    "In the learning curve figure, you can observe that both the train error and cross validation error are high when the number of training examples is increased. This reflects a high bias problem in the model - the linear regression model is too simple and is unable to fit our dataset well. In the next section, you will implement polynomial regression to fit a better model for this dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Training Examples\tTrain Error\tCross Validation Error\n",
      "  \t1\t\t0.000000\t205.121096\n",
      "  \t2\t\t0.000000\t110.302641\n",
      "  \t3\t\t3.286595\t45.010231\n",
      "  \t4\t\t2.842678\t48.368911\n",
      "  \t5\t\t13.154049\t35.865165\n",
      "  \t6\t\t19.443963\t33.829962\n",
      "  \t7\t\t20.098522\t31.970986\n",
      "  \t8\t\t18.172859\t30.862446\n",
      "  \t9\t\t22.609405\t31.135998\n",
      "  \t10\t\t23.261462\t28.936207\n",
      "  \t11\t\t24.317250\t29.551432\n",
      "  \t12\t\t22.373906\t29.433818\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhcZdn48e+dvWmSbkmbNCldoJSWrjQWihZKAVmKLYoIVVZRBEFxxaL4Un0VUVnUnwsgYIuy9UXZcSnQSiuytKX7TumS7gtN0iX7/fvjOTOdTCfNOnNmJvfnus41Zz/3mUnmnvM85zyPqCrGGGMMQIrfARhjjIkflhSMMcYEWVIwxhgTZEnBGGNMkCUFY4wxQZYUjDHGBFlSMK0mIhNEZK3fccSSiNwsIrtE5KCI9OqA/W0SkfO88e+LyCPtj7LzEZETvM8k1e9YkoXYcwqJRUQ2AV9S1df8jqWzEJF0oAI4Q1WXdtA+N2Gfo4lDdqVgjpEMv7o6+Bz6AFnAyjbEISISl/9nzb1HHR27iKR11L5M9MTlH6tpPRFJEZHpIvKBiOwTkdki0jNk+f+JyE4RKReRN0Xk1JBlM0XkDyLyqogcAs7xije+IyLLvG2eEZEsb/2JIlIWsn2T63rLbxeRHSKyXUS+JCIqIic1cR49ReRP3roficjz3vzrRGRB2LrB/UQ4h+9455sasv6nRWRZS96vkG1OBgJFZQdE5A1v/pki8p53vu+JyJkh28wTkZ+KyH+Aw8CgZj67GSLyF298gHde14rIFhHZKyI/CFm3Qz/nCLEcE7uInCIic0Rkv4isFZHPhazfS0ReEpEK7334Sejn5J3LLSKyHljvzbtERJaIyAEReUtERoas/z0R2SYild6xzvXmjxORhd5xdonI/WHvV5o33VdEXvRi3SAiXw57n2eLyOPe/leKSOnxPptOSVVtSKAB2AScF2H+bcDbQAmQCTwEPBWy/ItArrfsV8CSkGUzgXLg47gfClnecd4F+gI9gdXATd76E4GysJiaWvdCYCdwKpAN/AVQ4KQmzu8V4BmgB5AOnO3Nvw5YELZucD9NnMMHwPkh6/8fML0l71fYcQZ4x0rzpnsCHwFXA2nANG+6l7d8HrDFO+c0IP14nyMwA/hL2LH+CHQBRgHVwNBofM4R4gqPvRuwFbjemx4D7AWGees/7Q3ZwDBv3QVhn9Ec7z3r4m2/GzgdSAWu9d6LTGCIt33fkPfiRG/8v8DV3ngOrigv0mfzJvB77/MfDewBJoW8z1XAxd6xfwa87ff/dLwNvgdgQys/sKaTwmrg3JDpIqA28M8Stm537x+pmzc9E3g8wnGuCpn+BfCgNz6RY5NCU+s+BvwsZNlJNJEUvJgbgB4Rll1H80kh/Bx+AjzmjecCh4D+bXi/wr94rgbeDVvnv8B13vg84Mct/RyJnBRKQtZ9F7gyGp9zhG0axQ5cAcwPW+ch4C7cF2stMCTsPQ9PCpNCpv8A/G/Y/tYCZ3t/G7uB8whLpLgv+x8B+U19NkA/oB7IDVn+M2BmyPv8WsiyYcCRaP/PJtpgxUfJoz/wnHdJfgD35VEP9BGRVBG5xytyqMB9IQHkh2y/NcI+d4aMH8b9QmtKU+v2Ddt3pOME9AP2q+pHx1nneML3/STwGRHJBD4DLFbVzd6yJt+vFhynL7A5bN5moPg4sbRWU+9nND7ncKHr9AdODxzPO+YXgEKgAPdl3NznG76/b4ftrx/u6mAD8A3cl/duEXlaRPp6290AnAys8YqpLolwnL64v5/KkHnhn0v4+5olVtfRiCWF5LEVuEhVu4cMWaq6Dfg8MBX3C6wb7tcVgIRsH63b0HbgijoC+h1n3a1ATxHpHmHZIVwRBQAiUhhhnUbnoKqrcF8KF+HegyfDjtXU+9Wc7bgvt1AnAKHbRuv9jMXnHLrOVuDfYcfLUdWbcUUzdTT/+Ybv76dh+8tW1acAVPVJVf0E7v1V4Ofe/PWqOg3o7c17VkS6hh1nO+7vJzdkXvjnYpphSSExpYtIVsiQBjwI/FRE+gOISIGITPXWz8WVS+/DfbHeHcNYZwPXi8hQEckGftjUiqq6A/g78HsR6SEi6SJylrd4KXCqiIwWV4k9o4XHfxJXDn8Wrk4h4HjvV3NeBU4Wkc+LSJqIXIErini5hdu3R6w/55dx53q193mki8jHRGSoqtYDfwNmiEi2iJwCXNPM/v4I3CQip4vTVUQmi0iuiAwRkUnelV0VcARXnIiIXCUiBaraABzw9tUQumNV3Qq8BfzM+78YibvC+EsHvA+dhiWFxPQq7h8mMMwAfg28CPxLRCpxlZGne+s/jvvFvA1Y5S2LCVX9O/AbYC6wIeTY1U1scjWunHoNrnz5G95+1gE/Bl7D3cWyoIntwz2FK69+Q1X3hsw/3vvV3DntAy4Bvo37Ar4duCRs/9ES08/ZK4r5JHAl7pf4Ttwv9UxvlVtxVyU7gT/j3u+mPltUdSHwZeC3uMr5Dbj6Irx93oOryN6Juyq4w1t2IbBSRA7i3oMrVfVIhENMw10hbQeeA+5SexakVezhNRNTIjIUWAFkqmqd3/GYjiUiPwcKVfVav2MxbWNXCibqxD0fkCkiPXC/Ml+yhJAcvGcYRnpFQeNwxTXP+R2XaTtLCiYWvoIrCvoAd6fMzf6GYzpQLq5e4RDu+ZL7gBd8jci0ixUfGWOMCbIrBWOMMUEJ/dBGfn6+DhgwwO8wImuog53LQVKgaJTf0RhjTNCiRYv2qmpBpGUJnRQGDBjAwoUL/Q6jafefChVlcOtTkD/Y72iMMQYAEQl/Ij/Iio+iqXCEe925zN84jDGmhSwpRFMwKSz3Nw5jjGkhSwrRZEnBGJNgErpOIe5ZUjAJqLa2lrKyMqqqqvwOxbRTVlYWJSUlpKent3gbSwrR1L0/ZObBwV1QuQtyW9IqszH+KisrIzc3lwEDBiAizW9g4pKqsm/fPsrKyhg4cGCLt7Pio2hKSYE+w934LrtaMImhqqqKXr16WUJIcCJCr169Wn3FZ0kh2qwIySQgSwjJoS2foyWFaLOkYIxJIJYUos2SgjGtsm/fPkaPHs3o0aMpLCykuLg4OF1TU3PcbRcuXMjXv/71GEWanKyiOdoKToGUNNi7HmoOQUZ4D4LGmFC9evViyZIlAMyYMYOcnBy+853vBJfX1dWRlhb5q6u0tJTS0tKYxJmsonalICKPichuEVkRYdm3RURFJN+bFhH5jYhsEJFlInJatOKKufQsyB8CKOxe7Xc0xiSk6667jptuuonTTz+d22+/nXfffZfx48czZswYzjzzTNauXQvAvHnzuOSSSwCXUL74xS8yceJEBg0axG9+8xs/TyFhRPNKYSauy73HQ2eKSD9c935bQmZfBAz2htOBP9DCrhETQuEI2L3SNXdRYr9iTOIYMP2VqOx30z2TW71NWVkZb731FqmpqVRUVDB//nzS0tJ47bXX+P73v89f//rXY7ZZs2YNc+fOpbKykiFDhnDzzTe36p79zihqSUFV3xSRAREWPYDr0za0I46pwOPqOnd4W0S6i0iR15F74iscAcuetnoFY9rh8ssvJzU1FYDy8nKuvfZa1q9fj4hQW1sbcZvJkyeTmZlJZmYmvXv3ZteuXZSUlMQy7IQT0zoFEZkKbFPVpWG3ShUDW0Omy7x5xyQFEbkRuBHghBNOiF6wHckqm02Cassv+mjp2vVofdwPf/hDzjnnHJ577jk2bdrExIkTI26TmZkZHE9NTaWuznqBbU7M7j4SkWzg+8D/tGc/qvqwqpaqamlBQcTmwONPICnsWgkN9f7GYkwSKC8vp7i4GICZM2f6G0ySieUtqScCA4GlIrIJKAEWi0ghsA3oF7JuiTcvOWT3hLwSqD0M+zf6HY0xCe/222/njjvuYMyYMfbrv4NFtY9mr07hZVUdHmHZJqBUVfeKyGTgVuBiXAXzb1R1XHP7Ly0t1bjuZCfUk1fCur/DZx+D4Zf5HY0xTVq9ejVDhw71OwzTQSJ9niKySFUj3vUSzVtSnwL+CwwRkTIRueE4q78KbAQ2AH8EvhqtuHxj9QrGmAQQzbuPpjWzfEDIuAK3RCuWuGBJwRiTAKyZi1ixpGCMSQCWFGIlvG8FY4yJQ5YUYsX6VjDGJABLCrFkRUjGmDhnSSGWLCkY0yI7d+7kyiuv5MQTT2Ts2LFcfPHFrFu3LqrHnDVrFtOmNb4/Zu/evRQUFFBdXR1xm5kzZ3LrrbcC8OCDD/L4448fs86mTZsYPvyYu/KPWefJJ58MTvvZBLg1nR1Lhd4fhiUFY5qkqnz605/m2muv5emnnwZg6dKl7Nq1i5NPPjm43vGa0G6LT3/603z729/m8OHDZGdnA/Dss8/yqU99qlFzGU256aab2nzsQFL4/Oc/D/jbBLhdKcRSwVCQ1KN9KxhjjjF37lzS09MbfcmOGjWKCRMmMG/ePCZMmMCUKVMYNmwYVVVVXH/99YwYMYIxY8Ywd+5cAFauXMm4ceMYPXo0I0eOZP369Rw6dIjJkyczatQohg8fzjPPPNPouHl5eZx99tm89NJLwXlPP/0006ZN46WXXuL0009nzJgxnHfeeezadezNIjNmzODee+8FYNGiRYwaNYpRo0bxu9/9LrjOpk2bmDBhAqeddhqnnXYab731FgDTp09n/vz5jB49mgceeKBRE+D79+/n0ksvZeTIkZxxxhksW7YseLxoNA1uVwqxlJ4FBUNg9yrXt4I1o23i3YxuUdpveZOLVqxYwdixY5tcvnjxYlasWMHAgQO57777EBGWL1/OmjVr+OQnP8m6det48MEHue222/jCF75ATU0N9fX1vPrqq/Tt25dXXnHNgZeXHxvDtGnTeOKJJ7jiiivYvn0769atY9KkSVRUVPD2228jIjzyyCP84he/4L777msyxuuvv57f/va3nHXWWXz3u98Nzu/duzdz5swhKyuL9evXM23aNBYuXMg999zDvffey8svvwy4fiEC7rrrLsaMGcPzzz/PG2+8wTXXXBPshCgaTYPblUKsBesVlvkbhzEJaty4cQwcOBCABQsWcNVVVwFwyimn0L9/f9atW8f48eO5++67+fnPf87mzZvp0qULI0aMYM6cOXzve99j/vz5dOt2bMKbPHky//nPf6ioqGD27NlcdtllpKamUlZWxgUXXMCIESP45S9/ycqVK5uM78CBAxw4cICzzjoLgKuvvjq4rLa2li9/+cuMGDGCyy+/nFWrVjV7vgsWLAjuY9KkSezbt4+KiopgvJmZmeTn5webBm8vu1KItcIRsOwZq1cwieE4v+ij5dRTT+XZZ59tcnloE9pN+fznP8/pp5/OK6+8wsUXX8xDDz3EpEmTWLx4Ma+++ip33nkn5557Lv/zP40bbe7SpQsXXnghzz33HE8//TT3338/AF/72tf41re+xZQpU5g3bx4zZsxo07k98MAD9OnTh6VLl9LQ0EBWVlab9hMQjabB7Uoh1uwOJGOOa9KkSVRXV/Pwww8H5y1btoz58+cfs+6ECRN44oknAFi3bh1btmxhyJAhbNy4kUGDBvH1r3+dqVOnsmzZMrZv3052djZXXXUV3/3ud1m8eHHE40+bNo3777+fXbt2MX78eKBxU92zZs06bvzdu3ene/fuLFiwACAYX2A/RUVFpKSk8Oc//5n6eteUfm5uLpWVlRH3F3qO8+bNIz8/n7y8vOPG0B6WFGKtj/WtYMzxiAjPPfccr732GieeeCKnnnoqd9xxB4WFhces+9WvfpWGhgZGjBjBFVdcwcyZM8nMzGT27NkMHz6c0aNHs2LFCq655hqWL18erHz+0Y9+xJ133hnx+Oeffz7bt2/niiuuINAZ2IwZM7j88ssZO3Ys+fn5zZ7Dn/70J2655RZGjx5NaEvUX/3qV5k1axajRo1izZo1wauekSNHkpqayqhRo3jggQca7WvGjBksWrSIkSNHMn369GaTUntFtensaEuoprND3T8MKrbBrQshf7Df0RjTiDWdnVzipulscxxW2WyMiVOWFPxg9QrGmDhlScEPwaSwwt84jGlCIhcrm6Pa8jlaUvCDXSmYOJaVlcW+ffssMSQ4VWXfvn2tvu3VnlPwQ/cBkJELB3fCwd2Q09vviIwJKikpoaysjD179vgdimmnrKwsSkpKWrWNJQU/pKS4xvG2/NddLZx0rt8RGROUnp4efGLYdD5WfOQXK0IyxsShqCUFEXlMRHaLyIqQeb8UkTUiskxEnhOR7iHL7hCRDSKyVkQuiFZcccOSgjEmDkXzSmEmcGHYvDnAcFUdCawD7gAQkWHAlcCp3ja/F5HUKMbmP0sKxpg4FLWkoKpvAvvD5v1LVQMtNr0NBGpApgJPq2q1qn4IbADGRSu2uBDoW2Hfeqg57Hc0xhgD+Fun8EXg7954MbA1ZFmZN+8YInKjiCwUkYUJfXdEoG8FbXB9KxhjTBzwJSmIyA+AOuCJ5tYNp6oPq2qpqpYWFBR0fHCxZM1dGGPiTMyTgohcB1wCfEGPPh2zDegXslqJNy+5Wb2CMSbOxDQpiMiFwO3AFFUNLUh/EbhSRDJFZCAwGHg3lrH5wpKCMSbORO3hNRF5CpgI5ItIGXAX7m6jTGCO107526p6k6quFJHZwCpcsdItqpr8nQ2E962Qktw3XBlj4l/UkoKqTosw+9HjrP9T4KfRiicude0FecWub4X9H0L+SX5HZIzp5OyJZr9ZZbMxJo5YUvCb1SsYY+KIJQW/WVIwxsQRSwp+6zPcvVpSMMbEAUsKfusxEDJyjvatYIwxPrKk4LeUFLtaMMbEDUsK8cDqFYwxccKSQjwIJIUdS/2NwxjT6VlSiAclH3Ovm/8D1lm6McZHlhTiQe+hkFMIB3fB7lV+R2OM6cQsKcQDEThxkhv/4A1/YzHGdGqWFOLFiee4V0sKxhgfWVKIF4MmutfNb0FtlZ+RGGM6MUsK8SKnt2tKu64KtvzX72iMMZ2UJYV4YkVIxhifWVKIJ4HK5o1z/Y3DGNNpWVKIJyeMh7Qs92SztYNkjPGBJYV4kp4F/c904xvn+RqKMaZzsqQQb4LPK1gRkjEm9iwpxJtBIZXN1uSFMSbGopYUROQxEdktIitC5vUUkTkist577eHNFxH5jYhsEJFlInJatOKKe31Oha69Xf8Ku1f7HY0xppOJ5pXCTODCsHnTgddVdTDwujcNcBEw2BtuBP4QxbjiW2iTF3YXkjEmxqKWFFT1TWB/2OypwCxvfBZwacj8x9V5G+guIkXRii3u2fMKxhifxLpOoY+q7vDGdwJ9vPFiYGvIemXevGOIyI0islBEFu7Zsyd6kfpp0ET3uuk/1uSFMSamfKtoVlUFWl2TqqoPq2qpqpYWFBREIbI4kFvouuisOwJb3/E7GmNMJxLrpLArUCzkvQae0NoG9AtZr8Sb13kNmuherQjJGBNDsU4KLwLXeuPXAi+EzL/GuwvpDKA8pJipc7L+FYwxPkiL1o5F5ClgIpAvImXAXcA9wGwRuQHYDHzOW/1V4GJgA3AYuD5acSWM/mdCaibsXAaH9kLXfL8jMsZ0AlFLCqo6rYlF50ZYV4FbohVLQkrvAv3Hu+YuNs6DEZ/1OyJjTCdgTzTHMytCMsbEmCWFeBbaDpI1eWGMiQFLCvGs96nQtQAqt8OetX5HY4zpBCwpxLOUlMYN5BljTJRZUoh31g6SMSaGLCnEu0ET3eumBVBX7WckxphOwJJCvMsrgt7DoPawNXlhjIk6SwqJwHpjM8bEiCWFRGCVzcaYGLGkkAj6nwmpGbBjKRza53c0xpgkZkkhEWRkwwnjAYUP5/kdjTEmiVlSSBTWG5sxJgYsKSSKYGXzPGvywhgTNZYUEkWfEZCdDxVlsHe939EYY5KUJYVEkZJivbEZY6LOkkIisSYvjDFRZkkhkQQqmz+cD3U1/sZijElKlhQSSV5fKDgFag9B2bt+R2OMSUKWFBKNNXlhjIkiSwqJxrroNMZEUbNJQURSROTMjjyoiHxTRFaKyAoReUpEskRkoIi8IyIbROQZEcnoyGMmjUCTF9vfh8P7/Y7GGJNkmk0KqtoA/K6jDigixcDXgVJVHQ6kAlcCPwceUNWTgI+AGzrqmEkloyv0Ox3X5MW//Y7GGJNkWlp89LqIXCYi0kHHTQO6iEgakA3sACYBz3rLZwGXdtCxko8VIRljoqSlSeErwP8BNSJSISKVIlLRlgOq6jbgXmALLhmUA4uAA6pa561WBhRH2l5EbhSRhSKycM+ePW0JIfEF20Gaa01eGGM6VIuSgqrmqmqKqqarap43ndeWA4pID2AqMBDoC3QFLmzp9qr6sKqWqmppQUFBW0JIfIWjoEtPKN8K+z7wOxpjTBJp8d1HIjJFRO71hkvacczzgA9VdY+q1gJ/Az4OdPeKkwBKgG3tOEZyS0mxVlONMVHRoqQgIvcAtwGrvOE2EflZG4+5BThDRLK9OopzvX3OBT7rrXMt8EIb9985WG9sxpgoSGt+FQAuBkZ7dyIhIrOA94E7WntAVX1HRJ4FFgN13n4eBl4BnhaRn3jzHm3tvjuVwJXCpvlQXwup6f7GY4xJCi1NCgDdgcCN8d3ac1BVvQu4K2z2RmBce/bbqXQrgfwhsHctlL3nnl8wxph2ammdwt3A+yIy07tKWAT8NHphmRaxegVjTAdr0RPNQANwBq5S+K/AeFV9JsqxmeZYO0jGmA7WbPGRqjaIyO2qOht4MQYxmZbq/3FISYfti12TF9k9/Y7IGJPgWlp89JqIfEdE+olIz8AQ1chM8zJzXJMX2gAfvul3NMaYJNDSpHAFcAvwJq4+YRGwMFpBmVYI1CtYb2zGmA7Q0jqF6ao6MGwYFIP4THMC9Qob3rAmL4wx7dbSVlK/G4NYTFsUjYIuPaB8C+zf6Hc0xpgEZ3UKiS4lFQZNdON2a6oxpp2sTiEZ2K2pxpgO0qInmlV1YLQDMe0wyJq8MMZ0jONeKYjI7SHjl4ctuztaQZlW6t4Peg2G6grYtsjvaIwxCay54qMrQ8bDG79rcR8IJgasyQtjTAdoLilIE+ORpo2frF7BGNMBmksK2sR4pGnjpwGfgJQ02LYQjhzwOxpjTIJqLimMCvTJDIz0xgPTI2IQn2mpzFwoGWdNXhhj2uW4SUFVU0P6ZE7zxgPTdotLvAkUIVmTF8aYNmpxH80mAQTrFayy2RjTNpYUkknf0ZDVHT7aZE1eGGPaxJJCMklJhUFnu3G7C8kY0waWFJKNFSEZY9rBl6QgIt1F5FkRWSMiq0VkvNfI3hwRWe+99vAjtoQXaPLiwzehvs7fWIwxCcevK4VfA/9Q1VOAUcBqYDrwuqoOBl73pk1r9egPPU90TV5sX+x3NMaYBBPzpCAi3YCzgEcBVLVGVQ8AU4FZ3mqzgEtjHVvSsCIkY0wb+XGlMBDYA/xJRN4XkUdEpCvQR1V3eOvsBPpE2lhEbhSRhSKycM+ePTEKOcFYO0jGmDbyIymkAacBf1DVMcAhwoqKVFVpohkNVX1YVUtVtbSgoCDqwSakARNAUqFsIVSV+x2NMSaB+JEUyoAyVX3Hm34WlyR2iUgRgPe624fYkkNWHvQbB1oPH873OxpjTAKJeVJQ1Z3AVhEZ4s06F1gFvAhc6827Fngh1rEllUFWhGSMab0W9bwWBV8DnhCRDGAjcD0uQc0WkRuAzcDnfIotOZw4Cebdbe0gGWNaxZekoKpLgNIIi86NdSxJq+8YyOrmmrvY/yH0tB5VjTHNsyeak1VqGgw8y43b1YIxpoUsKSQz643NGNNKlhSSWSApfPhva/LCGNMilhSSWY8B0GOge1Zh+/t+R2OMSQCWFJKd9cZmjGkFSwrJztpBMsa0giWFZDfQa/Ji67tQVeF3NMaYOGdJIdlldYOSUtfkxaYFfkdjjIlzlhQ6AytCMsa0kCWFziDQDlIsKptrq9ydTjuWRf9YxpgO51fbRyaWisdCZh7s2wAfbXa9s3WEw/th1wqXAHYud8PetdDgPRMx8kq4+BeuCMsYkxAsKXQGgSYv1rzsrhbGXte67VWhfKv70g9NAOVbjl1XUiD/ZDiwFZY9DZvfgs88BP3P7JBTMcZElyWFzuLEc1xS+KCZpFBfC3vXhSQALwlUHTh23bQu0OdUKBwBRSOhcCT0HgYZ2bB3Pfz1S7BjCcycDJ/4Jpw9HdIyonaKxpj2s6TQWQQfYpsHDfWQkgrVlbBrZeMv/92rob762O2ze7kv/cIRUDTKvfY6ye0nkvzB8KXXYN49sOB+mH8fbHgdPvNHKDg5aqdpjGkfSwqdRc9BrtmLjzbBE5e71/0bidjraY+B7ku/cKR3BTACcotApHXHTE2Hc38IJ50Hz93orhoeOgsu+AmU3tD6/Rljok5cd8iJqbS0VBcuXOh3GInj5W/BwkePTqekQ++hjb/8+5wanYrhqnL4+/dg6VNuevAnYervIKd3xx/LGHNcIrJIVSP1aWNJoVM5uBsWzYJuxS4B5A+JfRn/ir/By990dRTZ+TD1tzDkotjGYEwnZ0nBxJfybfD8za5Jb4Cx18MFP4WMrv7GZUwncbykYA+vmdjrVgxXPw8X3A2pGbDoT/DgBNi2yO/IjOn0LCkYf6SkwPhb4Mtz3W2s+z+AR86Hf//SOgQyxke+JQURSRWR90XkZW96oIi8IyIbROQZEbEb2juDwuEuMZxxi2u0b+5PYObFsP9DvyMzplPy80rhNmB1yPTPgQdU9STgI+AGX6IysZeeBRfe7YqUcotg6zvw4Cfg/Sfc09TGmJjxJSmISAkwGXjEmxZgEvCst8os4FI/YjM+OvEcuPktGDYVag7CC1+F2de4NpaMMTHh15XCr4DbgQZvuhdwQFUDhcllQHGkDUXkRhFZKCIL9+zZE/1ITWxl94TLZ8GlD0JGLqx+Ef5wpjX7bUyMxDwpiMglwG5VbdOtJqr6sKqWqmppQUFBB0dn4oIIjJ4GNy+AfmdA5Q7486fhH3e4prmNMVHjx5XCx4EpIrIJeBpXbPRroLuIBJrdKAG2+RCbiSc9BsB1r8CkOyElDd7+PTw80bXRZIyJipgnBVW9Q1VLVHUAcCXwhnE1zb0AABXgSURBVKp+AZgLfNZb7VrghVjHZuJQahqc9V244V+uAb49q+GPk+Ct/wcNDc1vb4xplXh6TuF7wLdEZAOujuHRZtY3nUnxWPjKm1D6RaivgX/dCY9PgV2rrEjJmA5kzVyYxLP27/DCrXB479F5XQsgrxi6lbghr9g9Od2tnxvPLWy6mW9jOpnjNXNhTWebxDPkIvjqf+GfP4At/4WK7XBojxt2LIm8jaRCXt+QZFECeSWNx7N7WnPeptOzpGASU05vuOyPbryhHg7ugvIyN1RsO3b80B7XpWj5VtjaxD7TurgkEemKI6/YPViX1c0Sh0lqlhRM4ksJXAX0hX7jIq9TWwWV271ksQ0qvNdg4tgG1eWwb4MbmpKe7YqicvtCXpFLFHl9G8/LKbRuR03CsqRgOof0LNf7XM9BTa9TVXE0QZRvbTxeuQMqdkDtIddj3f6Nxz9edr6XNPoefc0t9BKIl0i69LCrDhN3LCkYE5CV54beQyMvV3X9WlfucPUYwdedjecd3OUqwQ/vPf4zFamZjRNFbpFLFFndmh4yuloiMVFlScGYlhI5mjgKhjS9Xn2dq8Oo3O6uLholj5B51RVwYLMbWhxD6vGTRlb31ieVhgbXQm1DXcjQ3HQL1snIhe793B1gGdlte89NzFlSMKajpaa5IqO8oiZa8PJUH2ycKA7udH1ZH2+oPQxH9ruhLSTV1YuEJgGNwUOAXQug+wkuQXQ/ofHQrR9k5kQ/BtMilhSM8UtmDmSeBPkntXybuhp3hVFV7vq5bjKBVDSRVA5BTeWx+01JCxlSjz8tqc2vk5Lijndg69G7vw7tabp3vexekRNGIGlk5bXtPe4IDQ1QV+WusCTFDQTGJemK8ywpGJNI0jIgLR+65rdt+/paqD3S+Is88OUWLQ317orowBZXaX9gsxs/sPXovMP73NDUcyZZ3ZtOGCmp7u6yuiPHvtZVu/Otq2ritTrydrVVbp26KvcEfXMCyaJRwkgJSSRynGWh2wlc9ij0+1gHfgCtY0nBmM4kNd0NsZSS6j0kWAyMP3Z5QwMc2u0lighD+VZ3VbTzAOxcFtvYA1Iz3Re2NoQMCngtQgTmdYSWJKEosqRgjPFXSor3nEdh5OdMVF3RU6RkUV7m1knLdA8fpmdFePWG4y7rEuHV22daZtNXUqpHk0N4wggmCg2bF2lZyHa5hVF6o1vGkoIxJr6JuCfYc3pDScTmevzTqE4hOdrWiqdWUo0xxvjMkoIxxpggSwrGGGOCrE7BGNPpqSpVtQ1UVNVSWVVLRVUdlVV1bvxIHUdq68nPyaCoWxeKumXROy+TzLTkqEMIZ0nBGJPwquvqqTjivsQrvS/0wBe8G6+j4kht8Iu+sqqOyuraRtvUNbSuw7H8nAwKu2VRmOcSRWG3rJDXLhTmZdElI/EShyUFY0xUqCrVdQ1U1dZTVdvAkdp6b7yeI7X1VDea13h55HkNwW1Dpyur66ipa/8zAhlpKeRlpZGXlU5uVhq5wdc0stJT2Xewhh3lR9hZXsWuymr2Hqxh78EaVmyraHKf3bPTKcwLJIvGySMwLyczvr6G4ysaY0xCUlW27D/M2xv38fbG/byzcR87KqqIVW+/6alCblY6eWFf5oHxvLDX3Kx08ro0Xrc1xUH1Dcreg9XsKK9iZ/kR77Xq6GuFSx4HDtdy4HAta3ZGaFrEk5uZ5q44vETxxU8M5JRC/5r1sKRgjGk1VWXzvkAScIlgZ0XVMetlpKXQJT2VrPQUstJT6ZKeSmZ6KllpKXTJSCUrLdW9pqeQGRhPS6VLhls/Ky2VrIyQ9dOPLs9Mc9PuCz0FiWEbRKkpQp+8LPrkZUG/7hHXaWhQ9h+uCUkWYcmjoood5UeorK6jcvdB1u8+CMDlpf1idh6RxDwpiEg/4HGgD+4Z8YdV9dci0hN4BhgAbAI+p6ofxTo+Y8yxWpIEemSnc8agXsHhpN45pKYkV2NxrZGSIuTnZJKfk8nw4m4R11FVyo/UNkoWJxX422KsH1cKdcC3VXWxiOQCi0RkDnAd8Lqq3iMi04HpwPd8iM+YDlffoI3K0wPl4aFl5/UNDeTnZAZ/gWak+XfHeEuSQM+uGZw+sGcwCQzunUNKJ04CbSEidM/OoHt2BkOLfGwJNkTMk4Kq7gB2eOOVIrIa1+r8VGCit9osYB6WFIzPKqtqeXPdXvYdquZITeQK0aa+5EMrVGvqW18Rmp+TSVE3lyAa3d2SlxUsg87O6Jh/YVVlU6MksI9dFdWN1rEk0Dn4WqcgIgOAMcA7QB8vYQDsxBUvRdrmRuBGgBNOOCH6QZpOp6augXlrd/PCku28tnoX1R1wZwtAVnqgfD2kbD1kXooIew9Ws7O8it2VVew9WM3eg9Us31be5D7zstLc7Y9ewmicQNxtkXld0o4pb29pEjhj0NEkcFKBJYHOwLekICI5wF+Bb6hqRegfraqqiES8b0FVHwYeBigtLY3RvQ0m2TU0KO9u2s8LS7bx6vKdlB+pDS4bN6AnJxfmhFSKBoaURhWlWSFf+FlhX/itrQitq29gb8gtkDsrGt/dEpiuqKqjoqqStbuavrulS3pqo4RR26C8+6ElAROZL0lBRNJxCeEJVf2bN3uXiBSp6g4RKQJ2+xGb6TxUldU7KnlhyTZeXLqdHeVHy8xPKczl0jHFfGpUX4q7d4l5bGmpKcEioqaoKvsP1TSZMAIJ5VBNPRv3HmLj3kONtg9PAoN758T0Dh4Tn/y4+0iAR4HVqnp/yKIXgWuBe7zXF2Idm+kctu4/zItLt/P8+9uCtwECFHfvwtTRfZk6upghhbk+RtgyIkKvnEx65WRyat/Id7eAqxcJvQ2yrl4pHdDDkoCJyI8rhY8DVwPLRSTQ9973cclgtojcAGwGPudDbCZJ7T9UwyvLtvP8ku0s2nz0Tuce2elMHlnE1NHFjD2hR1IWl7gHtNIZ3Cf+E53xnx93Hy0AmvrPOzeWsZjkdrimjjmrdvH8+9uYv35vsG2bLumpnD+sD5eO6cuEwQWkp1pjwcYE2BPNJqnU1jewYP1enl+yjX+t3MWR2nrAPYE6cUgBl44u5vxhfegaZ+3NGBMv7D/DJDxVZdHmj3hhyXZeWb6D/YeOdnx+2gnduXRMMZNHFNErJ9PHKI1JDJYUTML6cO8hnl20lReWbKfsoyPB+Sf1zuHS0X2ZMqqYE3pl+xihMYnHkoJJOAcO1/DAnHX85Z0t1Hv1BIV5WUwZ3Zepo/syrCjP7qoxpo0sKZiEUVffwFPvbuG+Oes4cLiWFIHLTivhs2NLGDewZ6dufM2YjmJJwSSEtz7Yy49fWhVsl378oF7cNWWYr+3OG5OMLCmYuLZ1/2F++spq/rFyJwAlPbpw5+ShXHBqoRURGRMFlhRMXDpcU8cf5n3AQ29upKaugS7pqdxyzol8acIgstITr99bYxKFJQUTV1SVF5du52evrgm233/p6L5Mv2jocdsBMsZ0DEsKJm4sLytnxksrg81QjCjuxowpwxjbv6fPkRnTeVhSML7bU1nNvf9cy+xFW1GF/JwMbr/gFD47tiQp2yIyJp5ZUjC+qalrYNZbm/jN6+uprK4jPVW4/uMDuXXSSeRlpfsdnjGdkiUF44u5a3bzvy+vCrbxP+mU3tw5eSiDfO603JjOzpKCiakP9hzkf19exby1ewAYlN+VH35qGOcM6e1zZMYYsKRgYqSiqpbfvLaemW9toq5Byc1M47bzBnPN+AFkpFnT1cbEC0sKJqrqG5RnF23ll/9cy96DNYjAlR/rx3cuGEK+tVpqTNyxpGCiZuGm/cx4aSUrtlUAUNq/BzOmnMrw4qa7jjTG+MuSQhI7cLiGVTsqWL2jktU7Kli1vYLN+w6RIkJ6WgoZqSmkpwnpqW48IzAvNSW4PCNNIswLXU8ibvvaql28uHQ7AEXdsph+0SlMGdXXmqYwJs5ZUkgCDQ3K5v2HWb2jIvjlv3pHBdvLq5reqDr6cWWmpfCVswZx08QTyc6wPzVjEoH9pyaYwzV1rNlZ2SgBrN1ZyaGa+mPWzUpPYUhhHsOKchlWlMfQojxO6p2DiFBb30BNXQO19W6ormugtl6D82pClh+dpxHmheynTt10fQM9szO48axB9OtpndwYk0jiLimIyIXAr4FU4BFVvcfnkHyhquyqqGbVjnJW76h0xUDbK/hw3yFUj12/T14mQ4vygl/+Q4vyGJjf1foYMMa0SlwlBRFJBX4HnA+UAe+JyIuqusqPeOoblFXbK2JyrLqGBjbuOeSuAHa6K4CPDtces15ainBSn5ywBJBr/Q8bYzpEXCUFYBywQVU3AojI08BUwJekUFVbz6d+u8CPQwPQrUs6Q4tyGVbUjaFFuQwtymNwnxwy06zpaGNMdMRbUigGtoZMlwGnh64gIjcCN3qT1SKyIkaxxUI+sDd0xjKfAukAx5xLgkum80mmc4HkOp9YnUv/phbEW1Jolqo+DDwMICILVbXU55A6TDKdTzKdCyTX+STTuUBynU88nEu8tS+wDegXMl3izTPGGBMD8ZYU3gMGi8hAEckArgRe9DkmY4zpNOKq+EhV60TkVuCfuFtSH1PVlcfZ5OHYRBYzyXQ+yXQukFznk0znAsl1Pr6fi2ikm96NMcZ0SvFWfGSMMcZHlhSMMcYEJWxSEJELRWStiGwQkel+x9NWItJPROaKyCoRWSkit/kdU0cQkVQReV9EXvY7lvYQke4i8qyIrBGR1SIy3u+Y2kNEvun9na0QkadEJMvvmFpDRB4Tkd2hzyeJSE8RmSMi673XHn7G2FJNnMsvvb+1ZSLynIh0j3VcCZkUQprDuAgYBkwTkWH+RtVmdcC3VXUYcAZwSwKfS6jbgNV+B9EBfg38Q1VPAUaRwOckIsXA14FSVR2Ou5njSn+jarWZwIVh86YDr6vqYOB1bzoRzOTYc5kDDFfVkcA64I5YB5WQSYGQ5jBUtQYINIeRcFR1h6ou9sYrcV86xf5G1T4iUgJMBh7xO5b2EJFuwFnAowCqWqOqB/yNqt3SgC4ikgZkA9t9jqdVVPVNYH/Y7KnALG98FnBpTINqo0jnoqr/UtU6b/Jt3LNaMZWoSSFScxgJ/UUKICIDgDHAO/5G0m6/Am4HGvwOpJ0GAnuAP3lFYY+ISFe/g2orVd0G3AtsAXYA5ar6L3+j6hB9VHWHN74T6ONnMB3oi8DfY33QRE0KSUdEcoC/At9Q1dg0zRoFInIJsFtVF/kdSwdIA04D/qCqY4BDJE7RxDG8svapuGTXF+gqIlf5G1XHUnePfcLfZy8iP8AVLT8R62MnalJIquYwRCQdlxCeUNW/+R1PO30cmCIim3DFepNE5C/+htRmZUCZqgau3J7FJYlEdR7woaruUdVa4G/AmT7H1BF2iUgRgPe62+d42kVErgMuAb6gPjxIlqhJIWmawxDXafGjwGpVvd/veNpLVe9Q1RJVHYD7XN5Q1YT8NaqqO4GtIjLEm3UuPjXj3kG2AGeISLb3d3cuCVxxHuJF4Fpv/FrgBR9jaRevk7HbgSmqetiPGBIyKXgVMYHmMFYDs5tpDiOefRy4GveLeok3XOx3UCboa8ATIrIMGA3c7XM8beZd8TwLLAaW4/7/fW9WoTVE5Cngv8AQESkTkRuAe4DzRWQ97mooIXprbOJcfgvkAnO874IHYx6XNXNhjDEmICGvFIwxxkSHJQVjjDFBlhSMMcYEWVIwxhgTZEnBGGNMkCUF02YioiJyX8j0d0RkRgfte6aIfLYj9tXMcS73Wj+dGzZ/gIh8vo37fKsF6zySJA0fAsH3a0Xza5p4Z0nBtEc18BkRyfc7kFBeY28tdQPwZVU9J2z+ACBiUmhu/6ra7FPCqvolVU3kB+FMkrKkYNqjDvfw0zfDF4T/0heRg97rRBH5t4i8ICIbReQeEfmCiLwrIstF5MSQ3ZwnIgtFZJ3XplKgn4Zfish7XpvzXwnZ73wReZEITx2LyDRv/ytE5OfevP8BPgE8KiK/DNvkHmCC9wDRN0XkOhF5UUTeAF4XkRwReV1EFnv7nRpyrNBznSdH+2N4wnuSGG9+aWB9EfmpiCwVkbdFpI83/0RvermI/CSw3wjndpX3/i0RkYe89+hj3vuTJSJdxfWhMLypuL1f+mu8z22dF+t5IvIfcf0UjPPWmyEifxaR/3rzvxwhnqY+oyIRedOLc4WITIh0PsZnqmqDDW0agINAHrAJ6AZ8B5jhLZsJfDZ0Xe91InAAKAIycW1W/chbdhvwq5Dt/4H74TIY1w5RFnAjcKe3TiawENfA20Rcg3UDI8TZF9fEQwGukbs3gEu9ZfNw/QuEbzMReDlk+jovhp7edBqQ543nAxs4+jBo6LmW49rmSsE9vfqJ8OPiGnD7lDf+i5DzexmY5o3fFNhvWJxDgZeAdG/698A13vhPcK2i/g6443hx466M6oARXqyLgMe8ZVOB571tZgBLgS7e9lu993cAsMJbp6nP6NvAD7z5qUCu33/DNhw7tOYy25hjqGqFiDyO67zlSAs3e0+9po5F5AMg0HzzciC0GGe2qjYA60VkI3AK8ElgZMhVSDdc0qgB3lXVDyMc72PAPFXd4x3zCVw/Cc+3MN6AOaoaaP9egLtF5CxcE+HFuCabd4Zt866qlnnHXYL78lwQtk4NLgGA+zI+3xsfz9G+AZ7EfcGHOxcYC7znXYR04WiDcD/GtRNWhft8jhc3uMbylnuxrsR1XKMistyLO+AFVT0CHPHqYsYBS0KWN/UZvQc8Jq4ByOdVNXQbEycsKZiO8Ctcezp/CplXh1c8KSIpQEbIsuqQ8YaQ6QYa/02Gt8GiuC+1r6nqP0MXiMhE3JVCNIXu/wu4K4+xqlorrlXYSF1bhp5rPZH/52rV+/l8nHWaIsAsVY3UQ1cvIAdI92I71Ezc7flcwmM65jMC8JLRZGCmiNyvqo8f//RMrFmdgmk379fzbFylbcAm3C9YgCm4L6bWulxEUrx6hkHAWlwjiDd7vzYRkZOl+Y5v3gXOFpF8cV25TgP+3cw2lbiGyZrSDddvRK2InAP0b8H5tNbbwGXeeFPdZr4OfFZEekOwv+JALA8BP8S1yf/zDox7qldX0QtXRPZe2PKIn5EX1y5V/SOuV75EboY8admVguko9+Farg34I/CCiCzF1Q205Vf8FtwXeh5wk6pWicgjuKKMxV6l7R6a6X5RVXeIyHRgLu5X7Cuq2lzzysuAei/+mcBHYcufAF7yilYWAmtac2It9A3gL+I6XPkHrn6iEVVdJSJ3Av/yrshqcf18n427AnnSS4RvicikDop7Ge69zAf+V1W3i+s1MKCpz2gi8F0RqcXVR13ThmObKLNWUo2JUyKSDRzxyvWvxFU6+9oXubjnUA6qaqT6DZME7ErBmPg1Fvit92v7AK7PXmOiyq4UjDHGBFlFszHGmCBLCsYYY4IsKRhjjAmypGCMMSbIkoIxxpig/w+YcaecdtRClAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_aug = np.concatenate([np.ones((m, 1)), X], axis=1)\n",
    "Xval_aug = np.concatenate([np.ones((yval.size, 1)), Xval], axis=1)\n",
    "error_train, error_val = learningCurve(X_aug, y, Xval_aug, yval, lambda_=0)\n",
    "\n",
    "pyplot.plot(np.arange(1, m+1), error_train, np.arange(1, m+1), error_val, lw=2)\n",
    "pyplot.title('Learning curve for linear regression')\n",
    "pyplot.legend(['Train', 'Cross Validation'])\n",
    "pyplot.xlabel('Number of training examples')\n",
    "pyplot.ylabel('Error')\n",
    "pyplot.axis([0, 13, 0, 150])\n",
    "\n",
    "print('# Training Examples\\tTrain Error\\tCross Validation Error')\n",
    "for i in range(m):\n",
    "    print('  \\t%d\\t\\t%f\\t%f' % (i+1, error_train[i], error_val[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*You should now submit your solutions.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance\n",
      "\n",
      "Use token from last successful submission (nehatrical@gmail.com)? (Y/n): Y\n",
      "                                  Part Name |     Score | Feedback\n",
      "                                  --------- |     ----- | --------\n",
      "Regularized Linear Regression Cost Function |  25 /  25 | Nice work!\n",
      "     Regularized Linear Regression Gradient |  25 /  25 | Nice work!\n",
      "                             Learning Curve |  20 /  20 | Nice work!\n",
      "                 Polynomial Feature Mapping |   0 /  10 | \n",
      "                           Validation Curve |   0 /  20 | \n",
      "                                  --------------------------------\n",
      "                                            |  70 / 100 |  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "grader[3] = learningCurve\n",
    "grader.grade()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section4\"></a>\n",
    "\n",
    "## 3 Polynomial regression\n",
    "\n",
    "The problem with our linear model was that it was too simple for the data\n",
    "and resulted in underfitting (high bias). In this part of the exercise, you will address this problem by adding more features. For polynomial regression, our hypothesis has the form:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "h_\\theta(x)  &= \\theta_0 + \\theta_1 \\times (\\text{waterLevel}) + \\theta_2 \\times (\\text{waterLevel})^2 + \\cdots + \\theta_p \\times (\\text{waterLevel})^p \\\\\n",
    "& = \\theta_0 + \\theta_1 x_1 + \\theta_2 x_2 + \\cdots + \\theta_p x_p\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Notice that by defining $x_1 = (\\text{waterLevel})$, $x_2 = (\\text{waterLevel})^2$ , $\\cdots$, $x_p =\n",
    "(\\text{waterLevel})^p$, we obtain a linear regression model where the features are the various powers of the original value (waterLevel).\n",
    "\n",
    "Now, you will add more features using the higher powers of the existing feature $x$ in the dataset. Your task in this part is to complete the code in the function `polyFeatures` in the next cell. The function should map the original training set $X$ of size $m \\times 1$ into its higher powers. Specifically, when a training set $X$ of size $m \\times 1$ is passed into the function, the function should return a $m \\times p$ matrix `X_poly`, where column 1 holds the original values of X, column 2 holds the values of $X^2$, column 3 holds the values of $X^3$, and so on. Note that you don’t have to account for the zero-eth power in this function.\n",
    "\n",
    "<a id=\"polyFeatures\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "for i in range(10):\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {},
   "outputs": [],
   "source": [
    "def polyFeatures(X, p):\n",
    "    \"\"\"\n",
    "    Maps X (1D vector) into the p-th power.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array_like\n",
    "        A data vector of size m, where m is the number of examples.\n",
    "    \n",
    "    p : int\n",
    "        The polynomial power to map the features. \n",
    "    \n",
    "    Returns \n",
    "    -------\n",
    "    X_poly : array_like\n",
    "        A matrix of shape (m x p) where p is the polynomial \n",
    "        power and m is the number of examples. That is:\n",
    "    \n",
    "        X_poly[i, :] = [X[i], X[i]**2, X[i]**3 ...  X[i]**p]\n",
    "    \n",
    "    Instructions\n",
    "    ------------\n",
    "    Given a vector X, return a matrix X_poly where the p-th column of\n",
    "    X contains the values of X to the p-th power.\n",
    "    \"\"\"\n",
    "    # You need to return the following variables correctly.\n",
    "    X_poly = np.zeros((X.shape[0], p))\n",
    "\n",
    "\n",
    "    # ====================== YOUR CODE HERE ======================\n",
    "    for i in range(0,X.shape[0]):\n",
    "#         print(\"\\n\")\n",
    "#         print(\"i:\", i)\n",
    "        for j in range(1, p+1):\n",
    "#             print(\"p:\", j)\n",
    "            X_poly[i,j-1]= X[i]**(j)\n",
    "\n",
    "    # ============================================================\n",
    "    return X_poly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you have a function that will map features to a higher dimension. The next cell will apply it to the training set, the test set, and the cross validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(12, 1)"
      ]
     },
     "execution_count": 224,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = 8\n",
    "\n",
    "# Map X onto Polynomial Features and Normalize\n",
    "X_poly = polyFeatures(X, p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Normalized Training Example 1: \n",
      " [ 1.         -0.36214078 -0.75508669  0.18222588 -0.70618991  0.30661792\n",
      " -0.59087767  0.3445158  -0.50848117]\n",
      "Normalized Test Example 1:\n",
      " [ 1.         -0.94220394  0.33149914 -0.50128942 -0.03307354 -0.11950565\n",
      " -0.24013304  0.11556833 -0.33419137]\n",
      "Normalized Val Example 1:\n",
      " [ 1.         -0.38916543 -0.72149541  0.16876064 -0.69803716  0.30354037\n",
      " -0.58940525  0.34396872 -0.50824813]\n"
     ]
    }
   ],
   "source": [
    "p = 8\n",
    "\n",
    "# Map X onto Polynomial Features and Normalize\n",
    "X_poly = polyFeatures(X, p)\n",
    "X_poly, mu, sigma = utils.featureNormalize(X_poly)\n",
    "X_poly = np.concatenate([np.ones((m, 1)), X_poly], axis=1)\n",
    "\n",
    "# Map X_poly_test and normalize (using mu and sigma)\n",
    "X_poly_test = polyFeatures(Xtest, p)\n",
    "X_poly_test -= mu\n",
    "X_poly_test /= sigma\n",
    "X_poly_test = np.concatenate([np.ones((ytest.size, 1)), X_poly_test], axis=1)\n",
    "\n",
    "# Map X_poly_val and normalize (using mu and sigma)\n",
    "X_poly_val = polyFeatures(Xval, p)\n",
    "X_poly_val -= mu\n",
    "X_poly_val /= sigma\n",
    "X_poly_val = np.concatenate([np.ones((yval.size, 1)), X_poly_val], axis=1)\n",
    "\n",
    "print('Normalized Training Example 1: \\n', X_poly[0, :])\n",
    "\n",
    "\n",
    "print('Normalized Test Example 1:\\n', X_poly_test[0, :])\n",
    "\n",
    "\n",
    "print('Normalized Val Example 1:\\n', X_poly_val[0, :])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*You should now submit your solutions.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance\n",
      "\n",
      "Use token from last successful submission (nehatrical@gmail.com)? (Y/n): Y\n",
      "                                  Part Name |     Score | Feedback\n",
      "                                  --------- |     ----- | --------\n",
      "Regularized Linear Regression Cost Function |  25 /  25 | Nice work!\n",
      "     Regularized Linear Regression Gradient |  25 /  25 | Nice work!\n",
      "                             Learning Curve |  20 /  20 | Nice work!\n",
      "                 Polynomial Feature Mapping |  10 /  10 | Nice work!\n",
      "                           Validation Curve |   0 /  20 | \n",
      "                                  --------------------------------\n",
      "                                            |  80 / 100 |  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "grader[4] = polyFeatures\n",
    "grader.grade()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Learning Polynomial Regression\n",
    "\n",
    "After you have completed the function `polyFeatures`, we will proceed to train polynomial regression using your linear regression cost function.\n",
    "\n",
    "Keep in mind that even though we have polynomial terms in our feature vector, we are still solving a linear regression optimization problem. The polynomial terms have simply turned into features that we can use for linear regression. We are using the same cost function and gradient that you wrote for the earlier part of this exercise.\n",
    "\n",
    "For this part of the exercise, you will be using a polynomial of degree 8. It turns out that if we run the training directly on the projected data, will not work well as the features would be badly scaled (e.g., an example with $x = 40$ will now have a feature $x_8 = 40^8 = 6.5 \\times 10^{12}$). Therefore, you will\n",
    "need to use feature normalization.\n",
    "\n",
    "Before learning the parameters $\\theta$ for the polynomial regression, we first call `featureNormalize` and normalize the features of the training set, storing the mu, sigma parameters separately. We have already implemented this function for you (in `utils.py` module) and it is the same function from the first exercise.\n",
    "\n",
    "After learning the parameters $\\theta$, you should see two plots generated for polynomial regression with $\\lambda = 0$, which should be similar to the ones here:\n",
    "\n",
    "<table>\n",
    "    <tr>\n",
    "        <td><img src=\"Figures/polynomial_regression.png\"></td>\n",
    "        <td><img src=\"Figures/polynomial_learning_curve.png\"></td>\n",
    "    </tr>\n",
    "</table>\n",
    "\n",
    "You should see that the polynomial fit is able to follow the datapoints very well, thus, obtaining a low training error. The figure on the right shows that the training error essentially stays zero for all numbers of training samples. However, the polynomial fit is very complex and even drops off at the extremes. This is an indicator that the polynomial regression model is overfitting the training data and will not generalize well.\n",
    "\n",
    "To better understand the problems with the unregularized ($\\lambda = 0$) model, you can see that the learning curve  shows the same effect where the training error is low, but the cross validation error is high. There is a gap between the training and cross validation errors, indicating a high variance problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial Regression (lambda = 0.000000)\n",
      "\n",
      "# Training Examples\tTrain Error\tCross Validation Error\n",
      "  \t1\t\t0.000000\t160.721900\n",
      "  \t2\t\t0.000000\t160.121511\n",
      "  \t3\t\t0.000000\t59.071635\n",
      "  \t4\t\t0.000000\t77.998004\n",
      "  \t5\t\t0.000000\t6.449033\n",
      "  \t6\t\t0.000000\t10.829905\n",
      "  \t7\t\t0.000000\t27.917628\n",
      "  \t8\t\t0.001442\t18.841672\n",
      "  \t9\t\t0.000185\t31.270978\n",
      "  \t10\t\t0.014306\t76.118843\n",
      "  \t11\t\t0.032764\t38.041801\n",
      "  \t12\t\t0.030051\t39.119110\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd3hUddbA8e8JgYQSCAm9FykJiAoBe+/dtS2667qSDbjNghV01/Lq2lndppDEFVddxb6LbbGCDQRESAFFeocESChJSOa8f9ybMIRkMklm5mbC+TzPPJm59UzJPff+2hVVxRhjjPEX43UAxhhjmh5LDsYYYw5iycEYY8xBLDkYY4w5iCUHY4wxB7HkYIwx5iCWHCJARFaJyBlex+FPRN4TkWuDXLbJxR8qIrJLRAY05f2JyL9F5BL3+S9F5PMwxfapiPyqHss329+FF0QkTkSWikhnr2MBSw714v4z7HX/wTeLyHMi0s7ruBpCVc9V1emN3Y77GZS5n0mhiMwSkaGhiDESVLWdqq4I9Xar/VYqHz389+d+dg/UsZ0RwBHA26GOMRqJyOnuAXSPiHwiIn0DLNvPXWaPu84Z1ebfLCKbRKRIRJ4VkTgv11XVUuBZ4M7GfUqhYcmh/i5U1XbASCANuNvjeJqCR93PpCewHsgO9Q5EJDbU24yAC91kUPnY0IBtTABeVOutioh0At4A/gAkAfOBVwKs8m/gWyAZuAt4rfKsXETOxjkInw70BQYA9zWBdV8CrvVPNp5RVXsE+QBWAWf4vX4MmOk+vwjIBXYAnwIp1dcDugF7gGS/eSOBrUBL4JfA58DjwHZgJXCu37I9gP8AhcByIMNv3r3Aq8ALQDGwBBgMTAK2AGuBs/yW/xT4lft8IPAxUABsA14EEmt739U+k+eAB/xenwfsrhbz6+57XAnc4DevNTDdfa/5wO3Aumr7vQNYDJQCscAxwJfu5/wdcIrf8r8EVrjvfyXwM3f6YcBnwE73/b3it44Ch7nPOwDPu7Guxkn8MX7brvW7qeu3Un1/wHhgH1AG7AL+W8t2VgAnVHuPn/u9fsr9bouABcCJjfxNPATMc7f3NpDkN/8a93MpwDnoVb1HYAzwlfu9bAT+BrQK8f/feOBLv9dtgb3A0BqWHez+ZhL8ps0BrnefvwT8yW/e6cAmL9f1m/YDcHKoj1/1fdiVQwOJSG+cA+G3IjIY52zhJqAz8C7wXxFp5b+Oqm7C+Qe80m/yNcDLqrrPfX00sAzoBDwKZIuIuPNeBtbhHHAvB/4kIqf5betC4F9AR5wzlw9wrg57AvcDU2t7OzgHhR5ACtAb58BSLyLSFrgKJ3EhIjHAf3EO4j1x/hFucs+eAO4B+uGcPZ0J/LyGzV4FnA8kAl2Bd4AHcM4cbwVeF5HO7r7/gnPATgCOAxa52/g/4H84n0sv4K+1vIW/4iSIAcDJwC+A6/zmB/pu6kVVp+Ek4UfVuaq4sPoy7nvq7+6zNt8AR+J8Hi8Br4pIvN/8+v4mfgGMA7oD5TifKSKSCjyN83vtgXNW3MtvvQrgZpzP5lic7/o3tQUtIjsCPGorVhmG81sCQFV3Az+602tadoWqFvtN+85v2QO25T7vKiLJHq5bKR+nKNFTlhzq7y0R2YFzFvkZ8Cfgp8A7qjrLPcg/jnNWfFwN60/HPQiKSAucg9+//OavVtVMVa1wl+2O8+PpDRwP3KGqJaq6CMjC+WeuNEdVP1DVcpwzxs7Aw25MLwP9RCSxekCqutyNvVRVtwJTcA6OwbrV/UyKgRNwDiAAo4HOqnq/qpapU9aeCYx151+Jcxa1XVXX4R6IqvmLqq5V1b3u5/auqr6rqj5VnYVTtHCeu6wPGC4irVV1o6rmutP34VzC93A/u4MqdN3vYiwwSVWLVXUV8ITfe4FavpsAn8tbfge8twIsV5vK76q4tgVU9QVVLVDVclV9AogDhvgtUt/fxL9UNcc98P4BuNL9bC7HuUqerU7Z+B9wPu/KOBao6tduHKtwkk6tvyFVTQzweLiW1drhXP352wkkNGDZ6vMrnyd4uG6lYvZ/956x5FB/l7g/4L6q+hv3oNUD53IbAFX14Vyy96xh/beBVBHpj3O2vFNV5/nN3+S3nT3u03buPgqrnZGsrraPzX7P9wLb3ANZ5evKbR1ARLqKyMsisl5EinCKITrV8v5r8riqJuJcBexl/8GpL9DD/6wQmMz+A2oPnM+pkv/zmqb1Ba6otr0TgO7uweynwPXARhF5x69i/Hacq6N5IpIrIuNq2E8nnKK91X7Tqn++tX03tbnE74B3SYDlarPD/VvTwQ8AEblVRPJFZKf7eXTgwO+uvr8J/897Nc5n0olq35X7eRf4xTFYRGZWVrTinDTV5zcUjF1A+2rT2lNz8qxr2erzK58Xe7hupQT2f/eeseQQGhtwDlwAuEUNvXEqZw+gqiXADJyz4Gs48Kqhrn0kiYj/gaJPTftogD/hlIUfrqrt3djqXVyiqmuAG4GnRKQ1zsFkZbWzwgRVrTzT38iBRRO9a9qs3/O1OGe2/ttrW3mm6Z4hn4lzRr8U5yoFVd2kqhmq2gOngvcfInJYtf1sY/8VRqVQfb61CVjJ7FdsMrim+SJyIk7iuxLo6CbonTTgu/Pj/x30wflMtuF8V1XzRKQNTtFSpadxPvNB7m9ocqA4qrXiqv6YXMtqufgVt7jFbgPd6TUtO6Da/8sRfssesC33+WZVLfBw3UopHFj05AlLDqExAzjfbWbXErgFp1Lqy1qWfx6nYvEigkwOqrrW3d5DIhIvThPHdJyz/MZKwDmj2SkiPYHbGroht6hnA07l4TygWETuEJHWItJCRIaLyGh38RnAJBHp6O73d3Vs/gXgQhE5291WvIicIiK93Kufi90DRqn7fnwAInKFiFQmoe04B2Wf/4bds+kZwIMikiBOE8mJhObzrc1mnPqNQN6l9uKZBJx6ga1ArIj8kYPPWuvr5yKS6h787wdecz+b14ALROQEty7tfg48fiTgVGLvcq/Yfh1oJ3pgK67qjz/VstqbOMWGl7n1Kn8EFqvq0hq2/z1OndM97u/kJ8AInMYR4PwPprvvNRGn8cFzXq4L4P4fJAFfB/r8IsGSQwio6jKcs+2/4pxlXYjTjLGsluW/wDk4LVTV1TUtU4urcIpuNuD8o9yjqh82IvRK9+G0mtqJU+H7RiO39xjOGW0scAFOhelKnM8mC6foA5wDzDp33oc4B6DS2jbqJsiLcc5Kt+JcSdyG8zuOwTmYb8BpzXUy+w9Qo4G5IrILp7XXjVpz34bfA7txWgh9jlPB+2x933w9ZOMUMQaqk5gG/KyWiu8PgPeB73GKgEqouWiuPv6Fc7DaBMQDNwC49Te/xflMNuIk2XV+690KXI1TPJJJ4CamDeLWh10GPOju/2j2118hIs+IyDN+q4zFaW6+HXgYuNzdBqr6Pk6jgk+ANTif3z1NYN2rgeluvY6nRPWQbz7tCRH5GHhJVbO8jqWpEJFfA2NVtT6V4c2eiLwEzFDVhlRqmyghTt+G74CTVHWL5/FYcog8t1hlFtC7WgXzIUVEuuMUq3wFDMK5avmbqj7paWDGGDztdSoiq3AuQyuAclVNE5EknEvSfjidbK5U1e1exRhqIjIduASnaOOQTQyuVjhNHvvjtM54GfiHpxEZYwCPrxzc5JCmqtv8pj2K02TzYbczTEdVvcOrGI0x5lDUFCukL8bpYIT7tyHtw40xxjSC11cOK9nftHCqqk4TkR1ue+3K/gLbK19XW3c8TnNJ2rZtO2ro0KgZCNQYYwLaWlzKpqIS4mNbMKhr+AZ+XrBgwTZVrXGIcK9HujxBVdeLSBdglogc0F5ZVVVEasxe6oxNMw0gLS1N58+fH/5ojTEmzPaUlXPMnz5CSsp5If1oThgU6o7m+4lIrU3pPU0Oqrre/btFRN7EGdlxs4h0V9WNbmsWz5t0GWNMpLRpFcur1x/H+zmbwpoY6uJZnYOItK3sYu72aj0LyMHppFR5h7JrsZucGGMOMUO6JXDjGYM8jcHLCumuwOci8h3OMAvvuL0HHwbOFJEfcO6BUNsIjcYY02zsq/DxUf5mmkrfM8+KldzhCw4as9wdgOr0yEdkjDHeyZqzkkfeX8rPj+nDA5cc7nU4TbIpqzHGHFLWFOzhqY++B+Cs1G4eR+Ow5GCMMR6q8Cm3vvodJft8XHRED04aXGPL0oiz5GCMMR569vOVzFtVSOeEOO69qKY7nnrDkoMxxnjk+83FPPY/5xbhj1x2OEltW9WxRuRYcjDGGI8889mPlJX7GDu6N6cNDXQ78sjzuoe0McYcsv70k8PpmdiaCScP9DqUg1hyMMYYj8S3bMEtZw3xOowaWbGSMcZE0LJNxdz8yiKKSvZ5HUpAduVgjDERsrW4lHHPfcP6HXvpkhDHpPNSvA6pVnblYIwxEVCyr4KM5+ezfsdejuydyM1nDvY6pIAsORhjTJhV+JRbZnzHorU76JnYmsxfpBHfsoXXYQVkycEYY8JIVbnrzSW8s2Qj7eJiefaXo+mcEOd1WHWy5GCMMWH0+sL1vPzNWuJiY8i6No0h3RK8DikoViFtjDFhdMmRPViwupCzh3XjmAHJXocTNEsOxhgTYhU+ZU9ZOQnxLYltEcNDl47wOqR6s2IlY4wJobJyHze+/C3XZM9jd2m51+E0mF05GGNMiBTsKuXXLyxk3qpC2sXFsmLrbg7v1cHrsBrEkoMxxoTA0k1FpD/n9GPo2j6O7GtHM7xndCYGsORgjDGN9ta367nrzSXsLqvgiN6JTLtmFF3bx3sdVqNYcjDGmEb4cvk2bnplEeC0THr4shFNvoNbMCw5GGNMIxw7MJlLR/ZkTL8kfjq6NyLidUghYcnBGGPqYW9ZBU9+9D2XHtWLId0SEBGmXHmk12GFnOdNWUWkhYh8KyIz3df9RWSuiCwXkVdEpOncN88Yc0j78sdtnPPUbKZ+toJJbyxGVb0OKWw8Tw7AjUC+3+tHgD+r6mHAdiDdk6iMMca1aWcJE19ZxNWZc1ldsIeh3RK458JhzaYIqSYBi5VEJB64ADgR6AHsBXKAd1Q1t7E7F5FewPnAg8BEcT7p04Cr3UWmA/cCTzd2X8YYU18l+yrI/nwlf/9kOXvKKmjVIobfnnoYvz5lIK1im8K5dfjUmhxE5D6cxPApMBfYAsQDg4GH3cRxi6oubsT+nwRuBypHokoGdqhqZbfCdUDPWuIbD4wH6NOnTyNCMMaYmu3cu68qMZwzrBuTz0uhT3Ibr8OKiEBXDvNU9Z5a5k0RkS5Ag4/KInIBsEVVF4jIKfVdX1WnAdMA0tLSmm/BnzEmon7YXEz/Tm2JbRFD1/bx3HfRMHomtua4wzp5HVpE1ZocVPUdABE5XFWX1DB/C87VREMdD1wkIufhXJG0B54CEkUk1r166AWsb8Q+jDEmKIW7y5gyaxkvzV3DfRcP55pj+gJwRVpvjyPzRjCFZv8QkXki8hsRCVlfcFWdpKq9VLUfMBb4WFV/BnwCXO4udi3wdqj2aYwx1e2r8PHs5ys55bFPeOHrNYgIW4tKvA7Lc3X2c1DVE0VkEDAOWCAi84B/quqsMMV0B/CyiDwAfAtkh2k/xphmQlXJy8ujsLCQpKQkUlNTg2pJ9OmyLfzfzDx+3LobgBMHdeIPF6QyuGt03JAnnILqBKeqP4jI3cB84C/AUW7Losmq+kZjg1DVT3EqvlHVFcCYxm7TGNP8qSrZ2dk8OWUKufn7W8QPS0nhpokTSU9PrzVJfJS/mfTp8wHol9yGu89P5fSULs26eWp91JkcRGQEcB1Ok9NZwIWqulBEegBfAY1ODsYYU1+qyoQJE8jMzGRUTAxTgQHACmDasmVkZGQwb948pk6dWnXAV9Wq56cM6cKYfkmckdqFa4/rR1xs9I+HFFKqGvABfAZcA7SuYd41da0ficeoUaPUGHNoyczMVEAngfpA1e/hA70TFNCsrCxVVZ3z/VY9/y+zddPOvVXb8Pl8XoXfJADztZbjqmgz6P6dlpam8+fP9zoMY0yEqCqHDxtG/LJlfOPzUVNBkAJpMTGUjBjJmXdM461FGwD41Qn9ufuC1IjG21SJyAJVTatpXq2tlUTkvyJyoYi0rGHeABG5X0TGhTJQY4wJRl5eHrn5+YyvJTEACHBq/1EUHX8jby3aQFxsDLefM4Q7zh0ayVCjVqA6hwxgIvCkiBQCW3H6I/QDfgT+pqrWzNQYE3GFhYWAU8dQk12tWvPAab/itSPOpgUwNKkFU9NPpG9y24jFGO0CdYLbhDO0xe0i0g/ojjO20vequici0RljTA2SkpIAp/K5JsuTezPj8DNoUV7G1tnP8/g/H7DEUE9W52CMiTqqyuGpqU6dg2qNRUsvHnE2j25Yiq9zWxbn5FgT1Ro0qM7BGGOaInX7NmzfsYMFqtwFFLVqze8uup1P+490lgFyvvuAxVtXc9PEiZYYGsDuBGeMiRrq17dhpAgDgMc69uDly/6IL7kXX3cfTPq08WQJLPT5yMjIYNw4azfTEHblYIyJGtnZ2WRmZjIJmK/Kw31GMPCaJ/Al96Jsy0q+m/FHfqM+NnftSlZW1gEd4Ez91JkcROQC9zaehSJSJCLFIlIUieCMMaaSqvLklCmMionhQeDfR5zNtVfeT0nrBM744Wv++8JtfLR9AykiJCYmMm7cOEsMjRBMsdKTwKXAEm0OtdfGmKhU2bdhKvDk8Vfz1AnODSMnzH2d2z+bTgv1AXCjKtfn55Ofn09qqnV2a6hgksNaIMcSgzHGS/59G9quWUJW6R7u+SiTK5ccOEB0Zd+HgoKCyAbYzASTHG4H3hWRz4DSyomqOiVsURljTDUdO3YEnL4N49cu4fNn0ulYUnzQcpV9H5KTkyMXXDMUTIX0g8AenN7RCX4PY4yJiN2l5Tw6dzdDTv4J02JiUKgxMSgwLSaG4amppKSkRDzO5iSYK4ceqjo87JEYY0wNdpeWc90/v2HeqkIST7iWBXP+y134eBAO6PymwGScJqxZ1reh0YK5cnhXRM4KeyTGGFPNnrJyrnvOSQzd2sfz5o2nk5F+HQ/hjLg6FecmM1Pd1w+D9W0IkWCSw6+B90VkrzVlNcZEyp6ycsY99w3zVhbStX0cL48/hv6d2zF16lSysrIoHTKE64GzgOuB0iFDrG9DCNnYSsaYJqdkXwXjnvuGL38soEtCHK9MOJb+nQ4cOE9Vyc/Pp6CggOTkZFJSUiwp1FOgsZWCGj5DRDoCg3AqpQFQ1dmhCc8YYw60dFMxC9dsp3NCHP8ef8xBiQFARKwfQxgFcw/pXwE3Ar2ARcAxOPeOPi28oRljDlVH9k7kxV8dTdu4WAZ2bud1OIekYOocbgRGA6tV9VTgKGBHWKMyxhyS1hbuv1XMqL5JDO3W3sNoDm3BJIcSVS0BEJE4VV0KDGnsjkUkXkTmich3IpIrIve50/uLyFwRWS4ir4hIq8buyxjT9D33xUpOn/IZ7+ds9DoUQ3DJYZ2IJAJvAbNE5G1gdQj2XQqcpqpHAEcC54jIMcAjwJ9V9TBgO5Aegn0ZY5qwD3I3cd/MPMrKfRSXlHsdjiGIOgdV/Yn79F4R+QToALzf2B27YzXtcl+2dB+KU5dxtTt9OnAv8HRj92eMaZpy1u/kppcXoQq3njWYK9J6ex2SIUByEJGkGiYvcf+2Awobu3MRaQEsAA4D/g78COxQ1cpTh3VAz1rWHQ+MB+jTp09jQzHGeGDTzhLSp3/D3n0VXDayF7899TCvQzKuQFcOC3DO5AXog1PEI0AisAbo39idq2oFcKRbbPUmMLQe604DpoHTz6GxsRhjImtPWTm/ev4bNheVMqZfEn+6dLj1U2hCaq1zUNX+qjoA+BC4UFU7qWoycAHwv1AGoao7gE+AY4FEEalMWr2A9aHclzGmaVi2qZiVW3fTN7kNz1wzirjYFl6HZPwEUyF9jKq+W/lCVd8DjmvsjkWks3vFgIi0Bs4E8nGSxOXuYtcCbzd2X8aYpueoPh157dfH8ewvR5PU1holNjXB9JDeICJ3Ay+4r38GbAjBvrsD0916hxhghqrOFJE84GUReQD4FsgOwb6MMU3EnrJy2rRyDj0p3a0fQ1MVTHK4CrgHp05AgdnutEZR1cU4HeqqT18BjGns9o0xTc+PW3dx5TNfcdvZQxg7xhqSNGXBNGUtxOklbYwxDbartJzr/7WAgt1lzP5hKz8d3dsqoJuwYOocjDGmUVSV21/7jh+27GJQl3Y8evkRlhiaOEsOxpiwe+azFby7ZBMJcbFMvWYU7eKCGhDaeMiSgzEmrL5eUcBjHywFYMpPj2SAjbIaFepMDiIyWEQ+EpEc9/UIt/WSMcYEVOFTJr+5BJ/Cb08dyJmpXb0OyQQpmCuHTGASsA+qWhmNDWdQxpjmoUWMkPmLNK4a04ebzxjsdTimHoIp+GujqvOqVR7ZsInGmKAM7NyOhy493OswTD0Fc+WwTUQG4vRxQEQuB2zAdWNMrfI3FvHS3DU0h3vUH6qCuXL4Lc4Ad0NFZD2wEvh5WKMyxkStvWUV/P7f37J8yy4U5WdH9/U6JNMAwXSCWwGcISJtgRhVLQ5/WMaYaHX/zFyWb9nFYV3acelRvbwOxzRQnclBROKAy4B+QGxl3YOq3h/WyIwxUeedxRv597y1tIqN4a9XHUXrVjbSarQKpljpbWAnzv0dSsMbjjEmGqgqeXl5FBYWkpSURGpqKut37OXONxYDcPf5KTaoXpQLJjn0UtVzwh6JMabJU1Wys7N5csoUcvPzq6anpg6j4xX/R3FJK85I6co1x1g9Q7QLprXSlyJi7dCMOcSpKhMmTCAjI4P4ZcuYCswCpgJxazexYs164nx7efSyw23cpGYg0D2kl+A0X40FrhORFTjFSgKoqo6ITIjGmKYgOzubzMxMJgEP+nz4H/4zigu44/mJ/DmxK28O2Ul6erpXYZoQkdraIYtIwOtCVV0dlogaIC0tTefPn+91GMY0W6rK4cOGEb9sGd/4JYaSFi2Jq9jnnDECaTExlA0dyuKcHLt6iAIiskBV02qaF+ge0qvdBPBA5XP/aeEK1hjT9OTl5ZGbn894v8SgwO8uvpMbLryNolZtEGC8z0dOXh75fvURJjoFUyE9zP+Fe1vPUeEJxxjTFBUWFgIwwG/a8yMv4MNBR9O+ZBdF8W1pX7anan5BQUHEYzShVeuVg4hMEpFiYISIFLmPYmALTvNWY8whIikpCYAV7uu8zv158NRxADz8/l/pVbT1gPnJyckRjtCEWqBipYdUNQF4TFXbu48EVU1W1UkRjNEY47HU1FSGpaQwLSaG3S3j+P3Ft1MW24qrFr3Hecu+AJxipmkxMQxPTSUlJcXbgE2j1dmU1RKBMUZEuGniRBb4fFx0+nh+TO7NYdvW8MePsgAnMUwGFvp83DRxolVGNwN2rz5jTFDS09N559vVfJtwDJSXcfLbj/B5eSkrcK4YFvp8ZGRkMG7cOK9DNSEQqM6hfzh3LCK9ReQTEckTkVwRudGdniQis0TkB/dvx3DGYYwJjojw0p/vYVSHPcTlzeSP21ZzFnA9UDpkCFlZWUydOtWuGpqJQP0cFqjqKBH5SFVPD/mORboD3VV1oYgk4IzddAnwS6BQVR8WkTuBjqp6R6BtWT8HYyJLVcnPz6egoIDk5GRSUlIsKUShQP0cAhUrxYjIZGCwiEysPlNVpzQmKFXdiHvTIFUtFpF8oCdwMXCKu9h04FMgYHIwxoTXe0s2cuLgzrSLcw4ZIkJqaqrHUZlwClQhPRaowEkgCTU8QkZE+gFHAXOBrm7iANgE2B3JjfHQlz9u4zcvLeSiv35Oyb4Kr8MxEVLrlYOqLgMeEZHFqvpeuAIQkXbA68BNqlrkf2mqqioiNZZ7ich4YDxAnz59whWeMYe0wt1l3PzKIlThgiN6EN/S7s9wqAh2VNYpIjLffTwhIh1CsXMRaYmTGF5U1TfcyZvd+ojKeoktNa2rqtNUNU1V0zp37hyKcIwxflSV2179js1FpYzu15EbTjvM65BMBAWTHJ4FioEr3UcR8M/G7licS4RsIL9a/cV/gGvd59divbGN8cT0L1fx0dIttI+P5cmxRxHbIpjDhWkugunnMFBVL/N7fZ+ILArBvo8HrgGW+G1vMvAwMENE0oHVOAnJGBNBuRt28qd3lwLwyGUj6JnY2uOITKQFkxz2isgJqvo5gIgcD+xt7I7d7dXW9i3kTWeNMcH7KH8LZRU+rj66D+ce3t3rcIwHgkkO1wPP+9UzbGd/sY8xphm64fRBpHZvzwmDOnkdivFInclBVb8DjhCR9u7rorBHZYzxhKpWdWY7I9VakR/Kgq5hUtUiSwzGNF8rtu7igr9+Ts76nV6HYpoAa35gjKG0vILfvfQtuRuKePaLlV6HY5qAOpODiMQFM80YE70eencpeRuL6JPUhvsuGlb3CqbZC+bK4asgpxljmjhVJTc3lzlz5pCbm4uq8r/cTTz35SpathD+dvVRJMS39DpM0wTUWiEtIt1wBsJrLSJHsb/ZaXugTQRiM8aEiKqSnZ3Nk1OmkJufXzU9ZeSx+M6eBMRwxzlDGdEr0bsgTZMSqLXS2TjDZ/cC/HswF+N0VjPGRAFVZcKECWRmZjIqJoapwABgucTw+LDLKffF0KlsE+OOP9frUE0TEmjgvenAdBG5TFVfj2BMxpgQys7OJjMzk0nAgz5fVRHAGeqj5zdvcfdxP+Xbl+7kn8Ocu70ZAwFu9lO1gMg9OLeIPYCq3h+uoOrLbvZjTM1UlcOHDSN+2TK+8UsM/solhqMFyoYOZXFOjt205xAS6GY/wVRI7wJ2u48K4FygX8iiM8aETV5eHrn5+Yz3Swzfd+pDTteBVcvEqo/xPh85eXnk+9VHmENbMD2kn/B/LSKPAx+ELSJjTMgUFhYCTh0DQHGr1lx/yWTWdejCc6/ew3Frlhwwv6CgIPJBmiapIZ3g2uBUUhtjmrikpCQAVuCUDd9x7o2sSO7FgML1HLXh+6rlVrh/k5OTIx6jaZrqvHIQkSXsr3NoAXQGmkx9gzGmdqmpqQxLSWHasrgkAGcAACAASURBVGXsG3MZ7w49gYTS3Tz91kO0Li8FnH/uaTExDB86lJSUFG8DNk1GMKOyXuD3vBzYrKrlYYrHGBNCIsJNEydywyOZPHbSNYj6mDLzCfpv3wA4iWEysNDnI2viRKuMNlWCqXNYLSJHACe6k2YDi8MalTEmZE684Eq6/9CZCokhfva/WLF8HrNwipKmxcSw0OcjIyODcePGeR2qaUKCGVvpRuBFoIv7eFFEfh/uwIwxobGpqJT4uDhS2+0lZsdirgfOwrlRS+mQIWRlZTF16lS7ajAHCKafw2LgWFXd7b5uC3ylqiMiEF9QrJ+DMYEt37KLnomtiW8ZQ35+PgUFBSQnJ5OSkmJJ4RAWqJ9DMHUOgtO/oVIFtd/e0xjTBKgqy7fsYlDXBAAO69Kual5qaqpXYZkoEkxT1n8Cc0XkXhG5F/gayA5rVMaYRnn6sx8596k5vLFwndehmCgVTIX0FBH5FDjBnXSdqn4b1qiMMQ329qL1PPr+MkSgdcsWXodjolQwxUqo6kJgYZhjMcY00tcrCrjtVacx4V3npXDu4d09jshEK09vEyoiz4rIFhHJ8ZuWJCKzROQH929HL2M0Jlp8v7mY8c/Pp6zCx3XH9+NXJw6oeyVjauH1PaSfA86pNu1O4CNVHQR85L42xgSwumA3P8+aS1FJOWcP68rd51uls2kcT5ODqs4GCqtNvhiY7j6fDlwS0aCMiULb9+yjZF8Fxw5I5qmxR9EixhoUmsYJZmylYg6+n8NOYD5wi6quOHitRumqqhvd55uAriHevjHNzpG9E3nt18e5fRmsEto0XjAV0k8C64CXcPo3jAUG4lRQPwucEq7gVFVFpMZeeiIyHhgP0KdPn3CFYEyTtX13GYvW7eDUIV0AGOz2aTAmFIIpVrpIVaeqarGqFqnqNOBsVX0FCEdl8WYR6Q7g/t1S00KqOk1V01Q1rXPnzmEIwxjvqSq5ubnMmTOH3NxcKkc02LarlKsyvyb9uW94Z/HGOrZiTP0Fc+WwR0SuBF5zX18OlLjPA4+90TD/Aa4FHnb/vh2GfRjTpKkq2dnZPDllCrl+d2cblpJC+g238s6egSzfsouBnduS1s8a9JnQCyY5/Ax4CvgHTjL4Gvi5iLQGfteYnYvIv3GKpTqJyDrgHpykMENE0oHVwJWN2Ycx0UZVmTBhApmZmYyKiWEqzp3aVgBPbyjg8YUVtEzaxZCu7XjhV8fQOSHO44hNcxRMD+kVwIW1zP68MTtX1atqmXV6Y7ZrTDTLzs4mMzOTScCD1e79/NwV99KyfRfKNv/IBQPa0TnhZC9DNc1YMKOydgYygH74JRNVbTKDv9uorKa5UFUOHzaM+GXL+MYvMVRIDGem/50Vyb0ZuT6fla/fT3n/3izOybFRVU2DBRqVNZgK6beBDsCHwDt+D2NMiOXl5ZGbn894v8QA0EJ9PDnzCS7J/YSXXr6L6/cWk5OXR75ffYQxoRRMnUMbVb0j7JEYYygsdPqEDgB8CF/2HcEJq78DYMSm5Tw584mq+QAFBQUeRGkOBcFcOcwUkfPCHokxhqSkJADyW7Vm/KV38/OxD/LukOMPWq6y52lycnIEozOHkmCuHG4EJotIKbAPpyOcqmr7sEZmzCEoNTWVoWNO4ckRP6UiuTftS3bRrnTPAcsozr2fhw8dSkpKijeBmmYvmNZK1u3SmAhQVV74ejX7TruFChU6bF3F2288QL8dm/YvA0wGFvp8ZE2caJXRJmxqTQ4iMlRVl4rIyJrmu/d4MMaEwM49+7jl1UV8mL8FEHqWruarf93KZRVljGd/P4dpMTEs9PnIyMhg3Lgm02DQNEOBrhwm4oxd9EQN8xQ4LSwRGXMIahkrLN1UTPv4WB66dATnHX4ezw6P489PPMH1/j2khwwh65ZbGDdunF01mLCqs59DNDiU+jmoKnl5eRQWFpKUlERqaqodJKJE9e8uoVs/ktrF0S7OOUdbuGY7XdvH0zOx9QHr5OfnU1BQQHJyMikpKfZ9m5AJ1M8hmCG7Pwc+A+YAX6hqcYjjM0EINNbOTRMnkp6ebgeNJqr6dyet2tDh2CvoMPoSRieVMePOKxARRvY5eIwkESE11W7cYyIvmKas1wDLgMuAL0Vkvoj8ObxhGX+VY+1kZGQQv2wZU4FZwFQgftkyMjIymDBhAs3hKrC58f/u4pb/SMZR55M6IZMOx1wBLVryyRdzGW/fnWmCgmmttFJESoAy93EqYO3nwqC2IqPaxtoByPD5mAw8nJnJ0UcfTXp6ukfRm5pkZ2eT9c/nuXTUhaw/+jL+l9AJgLR1udz1cTYzNn7Pw8Ax9t2ZJiaYsZV+BLbh3OxnDrBIVX0RiC1o0V7nEKjI6Mabb+apKVNotWodU7oPZk1idwradKDrrgKu/u4DZ31gVGxL9g0e1Kixdqw+I7Qqx0mK3VHOjl84F9tDtq7ips9f4pzvv3Q6DAFpMTGUDR1q4ySZiAtU5xBMcrgROAHoDSzFqX+Yrao/hjrQhorm5FB9eObxPl9Vs8WpMS3IHziahJEX0Lb3cHwt9l/ojdj4Pf95fiLgDMqWesNLFBdt5cqTj+DSY4cypn9S0LeLjPb6jFAktVAlxrJyH58v38rXKwq5uK+P4cOHMxXYeuo4Rq/N5Yzl84ipdhuUqcD1QG5urtUvmIhqVIW0qj4FPCUi7YDrgHuBXoDdqDYEAhUZbTx1HAVpFzsvfBWMWpfH4G1r6Ly7kG7F+8fUWdehC2WtWtOqS3/eyi/irfx5xLeM4aRBnTl/RHfOSOlK27iav+pA9w6Y5tZnzJs3j6lTpza5BBGKpBaKbZSV+5i3spCZizfwXs4mdu7dB0CvE9oCzuc5/pNna13fxkkyTZKqBnzg9HOYC+QCWTh3ZxtQ13qRfIwaNUqjkc/n02EpKToqJkZ9oFrtsTypp46ZkKUJoy7SJ+PaHjTf//HX2DiN6z1cb/vXHD3vqdna946ZVY9v12w/YJ/+MjMzFdBJcFAMPtA7nZIPzcrKivTHE5DP59OMjAwFdFRMjE4FnQU61X0NaEZGxkHvN5Tb2FJUounPzdPUP7x3wOd91pTP9G8f/6BfzP9OcbcX6Lt7xv2Mc3Nzw/VxGVMjYL7WduyvbUbVAs5tQbvWtZyXj2hNDjk5OQcdPOb3HHrAgWOfxGhP0KNqOHj7H8RHxsTo8NTUqgPZxh179Z+fr9BfvzD/gIPbz7O+1mufnav/nrtatxTtDZicatt2dT6fT3NycnT27Nmak5MT8IAcKqFIasFuY8rTz+onSzfrUx9+rw++k1e1fll5hQ774/va946ZeuaUT/WJD5bqsk1FVfPrSv7Bfr7GhEujkoOzPhcBj7uPC4NZJ5KPaE0Os2fPVtyzVQXNHnWR9r1jpj5xws8OOIDc7B6kGnt2v313qQ6c9E7VGW7/O2dq16sf1vS0i3VlYvdaD2C1ndn6fD7NzMzUYSkpirsMoMNSUjQzMzNsB7tQHHT9t1Ea00L3tmh5QIK+9dwb9eJrHtd+N8044Kpg8F3vaum+iqrtzPl+q27YsafWWKP1yswcGgIlh2A6wT0EjAFedCfdICLHqurkBpVjmSqVwzOvAD7vewT/d/qvAOi1c/MByw12/z4EfFCt0ro+Y+0ktmnF15NP58O8zXyQu4k5P2wlvvdwPuw9nA9Pz+CZNx7knB++AqAkthVx5WUINZeJq3pXV1F5Q5ypQPUtK7CnZTxF8W25PDaOye4NcVJTU/n7J8vZWlxK4e4y1m4ppGDMBJJP7ciQdh2598NpXLtwJgBrOnTj1RFnVm2zYs9ORvTrwtGDenBUn0TUr0L5hEGdAsaanp7OvHnzeCgzs1HfnTGRFsyQ3ecDR6rbfFVEpgPf4gwOaRohNTWVYSkpPL1uKzEX3oZKDDd88RJXLvmwahkFMmNiGDZkCDdNnMiTU6Y0aqydTu3iGDumD2PH9OGbb5dwytW/5aTBx7K57xGMXpdbtdyd5/yeOf2OYvjm5ezbtJw2W1exnXYUl+wjIb5lWPteqCpFJeUU7d3Hzr379v8tcf4m7dkGOMloxuFn8tKRZ1MU146d8e0oim/LvhYtAUgq2gJPj6tKav/8YhXbdpVW7Seu22HsAkR97IxvVzV95Ial/N8Hf2dQwVo2bFvDZXuLeGn2bE48sf4tiUSEqVOncvTRR9s4SSaqBJMcABKBQvd5hzDFcsgREW6aOJFJ76ygbdtEjl+1iBu/eLlq/gHDM99yC+np6aSnp4dsrJ20I4fTX7axdebjzPf5DuguvzqxOwVtE/lsQBoMSKMzcMM7G+CdDVwxqhfvTZnCqJgYfte2Iw+M/gntSvcQV15GXEUZceX7OMJXTmpMLH/+yz+qDn7v52wib2MRu0vL2VNWzu7SCvaUlbOrtJyU7u2558JhAGzbVcboBz+sMWaASSd1Bpyz74o27VnUY+gB8+P2ldKhZBcxu3cA+2+I85tTBqJAUtuWFG/bRMY1Y3lgTxE37S6kpa+iav2+OzZxzaL3AKeZqf82GkJESE9PZ9y4cTZOkokawSSHh4BvReQTnKv4k4A7wxrVIaTvcRfQdvkCfKV7yH3/L2Rp4GKHUI61U5mcMjIyuBt4kP3FNG+8cCtrO3RlUrfDeLfrYYw+40KkXWdWFexGSourinXWJHYne/Qlte5jfdb1VcU67+Vs5O1FG2pcbl/F/qKaDq1b0i4ulg6tW9K+dUvaxzvPK18fN6IHw1JSmLZsGW/lzWbMulw6lOyifcku2pfsJr5iX1XnsuGpqVU3xBl3Qv+qfaj25NGkVryytYBbfTX36Qz1TXVsnCQTTYLp5/BvEfkUGO1OukNVNwVYxdTDXz9eDsA5Pcv5skcy1+/cUjUvEsUOAcvEi7eycOdmMk5KZerdlyIi+HzKJ7Pn8BhOsU6vnZuZ/Ek2u1u2pjS2JaWxrSiJbUV5TCyb1Me/S/dUFeucM6wb/Tu1pW2rWNrEtXD+tmpB27hYktu1qoqpVWwMOfedHTDuyqT2j+KtPFi89YBirWBuiOOfGO/iwMQY7DaMadZqq6kGRgZ61LZeqB7AOTgD/i0H7gy0bLS2VlJV3bm3TLPmrNB95RXq8/k0NzdXZ8+erbm5uRFr2ujz+TQrK6vGVkdZWVkHxVFTE9xIt9/376MwMiZGnwH9n7vPkQ3o59DQbRgTzWhIU1bgkwCPj2tbLxQPnN7XP+KcnLYCvgNSa1s+mpNDUxJscmoq7ffrm9TCtQ1jolWg5FDr2EoicqWqzhCRAaq6orFXKPUhIscC96rq2e7rSQCq+lBNy0fj2Eort+2me4f4oMc/amqysrLIyMhwWitRc5HMw+5y4R5tVLXxN8QJxTaMiTYNHVvpTmAG8BpOUVIk9QTW+r1eBxztv4CIjMe5jSl9+vSJXGQhoKpM+Nd8CnaV8WLG0Qzt1t7rkOqtKbXfD0VFr1UWG3OgQMmhQET+B/QXkf9Un6mqF4UvrLqp6jRgGjhXDl7GUl9f/VjA95t30SUhjgGd2tW9QhNk7feNad4CJYfzca4Y/oUz+F4krccZIrxSL3das/Dcl6sA+PkxfWkVG8zN+Joma79vTPNVa3JQ1TLgaxE5TlW3RjAmgG+AQSLSHycpjAWujnAMYVGwq5SPl26hRYwwdkzvuleIAlYkY0zzE0w/h0gnBlS1XER+B3yA03LpWVXNrWO1qDBz8UbKfcqpQzrTJSHe63CMMaZGwQ6fEXGq+i7wrtdxhNobC9cBcOnIXh5HYowxtQtY4C0iLUTk5kgF09w5g8eVkxAXy5mpXb0OxxhjahXwykFVK0TkKuDPEYqnWevQuiUf33IyW3eVRm3/BmPMoSGYYqUvRORvwCvA7sqJqrowbFE1YyJidQ3GmCYvmORwpPv3fr9pCpwW+nCar6KSfRSXlNMzsbXXoRhjTJ2Caa10aiQCae7++90G7nozh3HH9+ePF1qzT2NM01ZnDywR6Soi2SLynvs6VUTCO1hOMzQrz7n159DuCR5HYowxdQume+5zOP0NerivvwduCldAzVFpeQVfr3DuaXDqkC4eR2OMMXULJjl0UtUZgA+cDmpAReBVjL+Fq3dQss/H0G4JdE6I8zocY4ypUzDJYbeIJONUQiMixwA7wxpVM/PF8m0AHH9YJ48jMcaY4ATTWmki8B9goIh8AXQGrghrVM3M525yOMGSgzEmSgSTHHKBk4EhOPd0WUZwVxwG2FNWTt7GImJjhDH9k7wOxxhjghJMcvhKVUfiJAkARGQhkb8BUFRq0yqWhX84k6Ubi2gb12SHsjLGmAPUerQSkW44d2RrLSJHsf9OkO2BNhGIrdloFxdLWj+7ajDGRI9Ap7JnA7/EudHOFL/pxTi3CDbGGNNMBbrZz3RguohcpqqvRzCmZqNkXwWnP/EZI3p14G9Xj6RFjN0dzRgTHYIZPuN1ETkfGAbE+02/v/a1DEDuhp2s37GXhPhYSwzGmKgSzPAZzwA/BX6PU+9wBdA3zHE1C/NXbQdgZN+OHkdijDH1E0yT1ONU9RfAdlW9DzgWGBzesJqHBaud5DCqjyUHY0x0CSY57HX/7hGRHsA+oHv4QmoeVJWFa3YAMMquHIwxUSaYhvczRSQReAxYiDOMRmZYo2oGNhWVsG1XKYltWtI32Vr+GmOiS6B+DjcBXwIPuYPtvS4iM4F4VbWxleqwZJ3zEQ3v0QERq4w2xkSXQFcOvYAngaEisgT4AidZfBmJwKLdkG4JTD5vKF3b2y1BjTHRp9Y6B1W9VVWPA7oBk4BC4DogR0TyGrNTEblCRHJFxCciadXmTRKR5SKyTETObsx+vNQ3uS3jTxrIxUf29DoUY4ypt2DqHFrjDJnRwX1sAJY0cr85wKXAVP+JIpIKjMXpU9ED+FBEBquq3T/CGGMiKFCdwzScg3QxMBenOGmKqm5v7E5VNd/dR/VZFwMvq2opsFJElgNjgK8au89IKthVyktz1zCqb0eOs2G6jTFRKFBT1j5AHLAJWA+sA3aEOZ6ewFq/1+vcaQcRkfEiMl9E5m/dujXMYdXP4nU7eWLW9/zl4x+8DsUYYxok0NhK54hzaj8MOA64BRguIoU4w3jfE2jDIvIhTn1FdXep6tuNiLkyvmnANIC0tDRt7PZCKWf9/pZKxhgTjQLWOaiq4lRA78C5NehO4AKcop6AyUFVz2hAPOuB3n6ve7nTosrSzcUApHRv73EkxhjTMLUWK4nIDSLysoisAT7DSQpLcSqSw3Vzgv8AY0UkTkT6A4OAeWHaV9h8v8lJDoO7JngciTHGNEygK4d+wKvAzaq6MZQ7FZGfAH/FuR/1OyKySFXPVtVcEZkB5AHlwG+jraVSWbmPldt2IwKHdWnndTjGGNMggeocJoZrp6r6JvBmLfMeBB4M177DbeW23ZT7lH7JbWjdqoXX4RhjTIMEM/CeqYede/fRO6k1Q7pZkZIxJnrZHe9DbEz/JObcfhoVvibVgMoYY+rFrhzCxO78ZoyJZpYcQmxXabnXIRhjTKNZcgihkn0VjLj3A45/+GMrVjLGRDVLDiG0fMsufArxLWOsWMkYE9UsOYTQim27AevfYIyJfpYcQmiVmxz6dWrrcSTGGNM4lhxCaKWbHAZYcjDGRDlLDiFUWazUv5MVKxljopslhxBRVVZu3QVAf7tyMMZEOeshHUJPXXUUq7ftplO7Vl6HYowxjWLJIUREhFOHdIEhXkdijDGNZ8VKxhhjDmJXDiHy3pKN/Lh1F2emdrMRWY0xUc+SQ4j8d/EG3l2yiV4d21hyMMZEPStWCpEVWyubsVpLJWNM9LPkEAKqyprCPQD0S7bkYIyJfpYcQqBwdxl7yipoHx9LhzYtvQ7HGGMazZJDCKzdvheAXh3beByJMcaEhiWHEFjrFin1TmrtcSTGGBMalhxCQHEqogd0tjGVjDHNgydNWUXkMeBCoAz4EbhOVXe48yYB6UAFcIOqfuBFjPVx0RE9uOiIHl6HYYwxIePVlcMsYLiqjgC+ByYBiEgqMBYYBpwD/ENEWngUozHGHLI8SQ6q+j9VLXdffg30cp9fDLysqqWquhJYDozxIsb62FNWXvdCxhgTRZpCD+lxwCvu8544yaLSOnfaQURkPDDefblLRJaFKb5OwLYwbTsSoj1+iP73EO3xQ/S/h2iPH8LzHvrWNiNsyUFEPgS61TDrLlV9213mLqAceLG+21fVacC0RgUZBBGZr6pp4d5PuER7/BD97yHa44fofw/RHj9E/j2ELTmo6hmB5ovIL4ELgNNVVd3J64Hefov1cqcZY4yJIE/qHETkHOB24CJV3eM36z/AWBGJE5H+wCBgnhcxGmPMocyrOoe/AXHALBEB+FpVr1fVXBGZAeThFDf9VlUrPIqxUtiLrsIs2uOH6H8P0R4/RP97iPb4IcLvQfaX6BhjjDEO6yFtjDHmIJYcjDHGHMSSQy1E5PcislREckXkUb/pk0RkuYgsE5GzvYwxGCJyi4ioiHRyX4uI/MV9D4tFZKTXMdZERB5zP//FIvKmiCT6zYua70BEznHjXC4id3odT11EpLeIfCIiee5v/0Z3epKIzBKRH9y/Hb2ONRARaSEi34rITPd1fxGZ634Pr4hIK69jDEREEkXkNfd/IF9Ejo30d2DJoQYicipOb+0jVHUY8Lg7PaqG9xCR3sBZwBq/yefitAIbhNOJ8GkPQgtG1A+x4sb1d5zPPBW4yo2/KSsHblHVVOAY4LduzHcCH6nqIOAj93VTdiOQ7/f6EeDPqnoYsB1n/Lam7CngfVUdChyB814i+h1YcqjZr4GHVbUUQFW3uNOjbXiPP+M0GfZvdXAx8Lw6vgYSRaS7J9EF0EyGWBkDLFfVFapaBryME3+TpaobVXWh+7wY56DUEyfu6e5i04FLvImwbiLSCzgfyHJfC3Aa8Jq7SFOPvwNwEpANoKpl7sCkEf0OLDnUbDBwonsZ+pmIjHan9wTW+i1X6/AeXhORi4H1qvpdtVlR8x78jAPec59HU/zRFOtBRKQfcBQwF+iqqhvdWZuArh6FFYwncU6KfO7rZGCH38lGU/8e+gNbgX+6RWNZItKWCH8HTWFsJU8EGt4D53NJwrmsHg3MEJEBEQwvKHW8h8k4RUpNVriHWDENJyLtgNeBm1S1yO2PBICqqog0yTbwInIBsEVVF4jIKV7H00CxwEjg96o6V0SeoloRUiS+g0M2OQQa3kNEfg284Q7rMU9EfDiDXjWp4T1qew8icjjO2cd37j91L2ChiIyhCb2HQ2CIlWiKtYqItMRJDC+q6hvu5M0i0l1VN7rFkFtq34KnjgcuEpHzgHigPU75faKIxLpXD039e1gHrFPVue7r13CSQ0S/AytWqtlbwKkAIjIYaIUzGmJUDO+hqktUtYuq9lPVfjg/tpGqugnnPfzCbbV0DLDT71K1yWgmQ6x8AwxyW8q0wqlI/4/HMQXkls9nA/mqOsVv1n+Aa93n1wJvRzq2YKjqJFXt5f7uxwIfq+rPgE+Ay93Fmmz8AO7/6VoRGeJOOh1n1IiIfgeH7JVDHZ4FnhWRHJy71V3rnrk2xeE96utd4Dycitw9wHXehlOraBpipUaqWi4ivwM+AFoAz6pqrsdh1eV44BpgiYgscqdNBh7GKV5NB1YDV3oUX0PdAbwsIg8A3+JW9jZhvwdedE8qVuD8n8YQwe/Ahs8wxhhzECtWMsYYcxBLDsYYYw5iycEYY8xBLDkYY4w5iCUHY4wxB7HkYCJGRLqJyMsi8qOILBCRd0VksIicUjl6ptdE5H4RCdg5L0T7SRSR34RgO5+KSEhvOh9om+5IobWOFiAirURktohYM/koZ8nBRITbuepN4FNVHaiqo3BGWm1SY/So6h9V9cMI7CoRqFdycDsuevY/KyLDgBaquqK2ZdwBBj8CfhqxwExYWHIwkXIqsE9Vn6mcoKrfqeoc92U7v/HrX3STCSLyRxH5RkRyRGSa3/RPReQREZknIt+LyInu9DYiMkOc+xG86Q6emObOO0tEvhKRhSLyqjt+0AFE5DkRudx9vkpE7nOXXyIiQ2tY/h0RGeE+/1ZE/ug+v19EMkSknYh85LeNylFZHwYGisgiEXnMXec2970uFpH73Gn9xLkfxPNADgcOx1E9loPenzj3k3jVb5mqq7RgPo9qfobbK1dE+opzX4FOIhIjInNEpHIsr7fcZU0Us+RgImU4sCDA/KOAm3DuezAAp6cuwN9UdbSqDgda44y1VClWVce4693jTvsNsN29H8EfgFEA4tzs6G7gDFUdCcwHJgYR9zZ3+aeBW2uYPwdnBN8OOD22K+M+EZgNlAA/cbdxKvCEm+DuBH5U1SNV9Tb3wDoIZ5jvI4FRInKSu61BwD9UdZiqrq4pyADv70PgaHFG9QTnjP7lBn4ex+N+h24cj7ifyy1Anqr+z10uB2fAShPFrFzQNBXzVHUdgDtsQz/gc+BUEbkdaIMzUm4u8F93ncpB4Ra4ywOcgDPQGqqaIyKL3enH4CSeL9yLj1bAV0HE5b+PS2uYPwe4AVgJvAOcKSJtgP6qukycQez+5B7ofThDRddUlHaW+/jWfd0OJymsAVa7994IpMb35w7h8T5woYi8hnOfg9uBk2tavo59dMcZShoAVc0SkSuA63ESWuX0ChEpE5EE954QJgpZcjCRksv+gc9qUur3vAKIFZF44B9AmqquFZF7cUbarL5OBXX/lgWYpapX1SvquvfxDZCGM/7NLJzRezPYf5X0M6AzMEpV94nIqmrvwT++h1T1/9u7e9cmoyiO498fIsU3HEQXBQcLTjq5dVDBf8DBVUTQQfwDnHQRpLgL4uzLIOJQBxdFi7Sgm/UFRx19QSkSscNxOOeBlMc8bWwJpvw+S5Kbm+Q+Ge7JvSfcc3NZY9ZU+LmKcXZd3z3gIvANeBURi7V6Gfb76PWPvYJgU4RpTHxlVAAAAYlJREFUO9AfCCbIVZONKW8r2ag8ASYknW8aJB1ucgUDNBPRl9oP7woujRfUgWTK8paHqn0emJI0Wc9tU564uyaVgP0EnCJ/ec+S20/Pq8tOsr7AkrL87P5qXwR29L3VY+Bss+8vaa+kPUMMpev6npH1Ac6RgWKl/oO8Ayb7Hk+TdTYuA7eaRkm7yO24pSHGb/8ZBwcbiTrV9iRwQvlX1jfANbKi1aDXfCcnnQVy8ny5io+6AeyW9Ba4Sq5YfkTEZ+AMcLe2muaAVoL5H82SAaBX9/fVLeTkeUTSa+A08B4gIr6SWzoLkq7Xfv0dYK763md58OjUdX11au0MWct6ZqX+HR4BxwAkHSXzCtMRcRv4Lak54fd49bUx5lNZbUORtAnYHBG/JB0gE7IH6xe+rYGkLWRdhKmuY9IlPQAuRcSHkQ3O1p1zDrbRbAWeViJYwAUHhvURET1JV8ik+se/9VHWH3jowDD+vHIwM7MW5xzMzKzFwcHMzFocHMzMrMXBwczMWhwczMys5Q+qHVtmH7g2hAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3hUZfbA8e9JISGU0Ca0IE1IkC4oooKIqCxBwcIqdteVVXcta3dXV7a57Nq3/FZZdUHFggULiopIEHQtgEhvhhZpoXcI5P398d4JQ5jUKXfmzvk8T54pt507M7nnvve+RYwxKKWUUqFIcjsApZRS8U+TiVJKqZBpMlFKKRUyTSZKKaVCpslEKaVUyDSZKKWUClncJxMRWS0ig9yOI5CITBGRa6s4b8zFHy4iskdE2rkdRySISD8RWeZ2HG4QEZ+ILBWR2s7rfBH5eQS200ZEjIikVHH+ASJSGO44EpmIXCAir1dl3phJJs5Bdb9zANokIuNEpK7bcdWEMeYnxpjxoa7H+QwOOZ/JNhGZKiK54YgxGowxdY0xBeFebywkYGPMTGNMTqTWLyLni8jnIrJbRIpEZIaIXBip7VXT/cA4Y8x+twNxm1h/FZGtzt9fRUQqmP8KEVkjIntF5B0RaRQwrZGITHKmrRGRK9xe1hjzPtBZRLpV9lnETDJxXGCMqQucDPQGHnQ5nljwN+czaQn8CDwf7g1U9cwvkYhIsovbvhR4A3gRyAaaAr8DLqjBukREwvZ/LiJpwLXAy+FaZ5wbBQwHugPdsN/RL4LNKCKdgWeBq7Hf6T7g/wJm+RdwyJl2JfBvZxnXlnW86uxnxYwxMfEHrAYGBbx+FJjsPL8QWATsAPKBTmWXA5o5H1LjgGknA0VAKnAdMAt4DNgOrAJ+EjBvC+A9YBuwErgxYNpo7D/3y8BuYAHQEXgA2AysA84LmD8f+LnzvD3wGbAV2AJMABqUt99lPpNxwJ8CXg8B9paJ+S1nH1cBtwVMqw2Md/Z1CXAvUFhmu/cB84GDQApwGvCl8zl/DwwImP86oMDZ/1XAlc77JwIzgJ3O/r0esIwBTnSeZ2IPjkXAGuyJQlLAusv9bir7rQS8n4Q9a/7B+bwnAo0Cpr8BbHRi/RzoXOaz/jfwIbAX+5taDdztfEY7gdeBdGf+AUE+z6DzOtPvBTYA64GfB342ZfZBgLXAPRXs/2jg5YDXbZz1pQT8/v4MfAHsd77n2WXW8WvgPed5mvPZrwU2Ac8AtcvZdn9gZZn38qne7/0e53Paiz05agpMwf62PgUaltmvUc7ntgG4u8xvfJzzm1nsrDfwO/H/FnY70y+KwHHrS2BUwOsbgK/KmfcR4JWA1+2xB/F6QB3neceA6S8BY9xc1nl9BrCqss8i1komAIhIK+yB8zsR6YjNjHcAPuw/+/siUitwGWPMRuyP+qcBb18NvGaMKXZe9wGWAU2AvwHPBxRJXwMKsQfoS4FHRGRgwLouwH7IDYHvgI+xB6+WwB+wmT/o7gB/cdbbCWiFPRhUi4jUAUZiEx3O2eb72IN+S+Ac4A4ROd9Z5GHsP2M74FzgqiCrHQnkAQ2w/9AfAH8CGmEPjG8518frAH/HHuDrAacD85x1/BH4BPu5ZAP/KGcX/oFNKO2As4BrgOsDplf03VTVrdizxLOwn/d27FmX3xSgA5AFzMUe6AJdgT0I18MmN7C/p8FAW+yZ53UVbD/ovCIyGLgTm6BOxCai8uRgfyNvVjBPVVyNPQjXwyaHHBHpEDD9CuAV5/kY7MlRDye+ltiSUDBdsd9Tearye78E+5vsiP2/mgL8Bvv/nQTcVmb+s7Hf23nAfQGXOB/GHhjbA+djS0yBfgD6YX93vwdeFpHmQYO2l4F2VPB3Qjn72xn7P+j3vfNepfMaY37AOZA7f4eNMcvLWZdby4I9GW0jIvXL2a/SFcfEH/aMZQ/2rHgNthhWG3gImBgwXxL2cs+AgOUGOc8vA75wnidjz0JPdV5fR8AZFZCBPetphv3BHwHqBUz/C/a6MNh/hqkB0y5wYk12Xtdz1tWg7JlakP0cDnxXZr8rKpkccD6TEuwZezdnWh9gbZn5HwD+6zwvAM4PmPZzjj+T/lnA6/uAl8qs72PsP2gdJ4ZLKHPGii1tjAWyg8RvsAenZOyP96SAab8A8iv7bir4rQQrmSwBzgl43RwoxjljLzNvA2cbmQGf9YtBtnNVwOu/Ac84zwcE+TzLm/cF4C8B006k/JLJGc609LLTAuYZTeUlkz+UWeZl4HfO8w7Ys/UM7MF/L9A+YN6+lHMmCvwWe4IW+F4+1fu9Xxnw+i3g3wGvbwXeKbNfuWU+1+cDfuODA6aNCvxOgsQyDxhW3vSa/GGPG4HxdXBiliDzTgNuKvPej85vqR+wscy0Gzn6P+LKss7rVGefTqjos4i1kslwY0wDY0xrY8wtxt7ga4FNLgAYY0qwl5VaBln+XeAkEWmLPfPZaYz5JmD6xoD17HOe1nW2sc0Ysztg3jVltrEp4Pl+YIsx5kjAa/+6jiEiTUXkNRH5UUR2Yf+pm5Sz/8E8ZoxpgP3H2o89cwVoDbQIPHvCnt01daa3wH5OfoHPg73XGhhRZn1nAs2NMXuxifomYIOIfBBQEeBe7AHpGxFZJCI/C7KdJtgf5JqA98p+vuV9N9XRGpgUEP8S7D97UxFJFpExIvKD8z2sDojNL9hntDHg+b5KYipv3qp8F35bncegZ9DVUHYbr2BLomBLJe84n7MPm1TmBHxuHznvB7Mde/IUVBV/72X/l8q+LvsZB+7LGuznCcd/roG/L0TkGhGZF7BfXYLEEqo9QOAZe31gj3GOwpXM659/dyXT3FwWjn7fO6hArCWTYNZjDxKAvaGILUn8WHZGY8wB7HXyq7DF/JeqsY1GIhL4T3JCsG3UwCPYrN7VGFPfia26l28wxqwFbgeeFlslcx327LFBwF89Y8wQZ5EN2MtOfq2CrTbg+TpsySRwfXWMMWOc7X9sjDkXe5BbCvzHeX+jMeZGY0wLbGnj/0TkxDLb2YItIbQOeC9cn2+gddhLcYH7kG6M+RF7AB2GvdSUiU3OcOx3EewAEA5V+S78lmH345IK5tmLTQB+zYLMU3ZfpgI+EemBTSr+S1xbsAfwzgGfWaaxlT6CmY+9NFKesPzeywj8vE7A/r+C/VzLTgNARFpjf6O/wt5HbQAsLC8WEblSbK3J8v7Ku8y1CHvz3a+7816l84qtNp8GLHf+Uspcigxcl1vLgr1cudoYs6uc/QLiI5lMBPJE5BwRSQXuwt4w/rKc+V/EXja5kComE2PMOmd9fxGRdKca3A2Ep8ZKPWz23ykiLbE3CWvEGDMV+480CvgG2C0i94lIbefMu4uInOLMPhF4QEQaOtv9VSWrfxm4QGyV1GTncxggItnO2eYw597JQWd/SgBEZISI+A+U27EHkpIycR9x4vmziNRz/tHvJLTPN9WJ0f+Xgr038Gdn/f72EMOc+es5sW/FHogfCWHb1TURuF5EOolIBvbSbVDOGe2dwEMicr2I1BeRJBE5U0TGOrPNA/qLyAkikom9vFkhY+8bvoGt2NIIm1z8Jf3/AE+KSBaAiLQMuPdW1jdAA+c3FUzYfu8BHhKRDLE1jK7HVm6AY3/j2dhLZH51sL/FIgARuR5bMgnKGDPB2Krs5f2tLWfRF4E7nc+sBfb4NK6ceSdg/8f6Of9LfwDeNsbsdkr/bwN/EJE6InIG9uTnJZeXBXsPckp5n51fzCcTY8wy7NnNP7BnURdgqxAfKmf+L7AHs7nGmDXB5inHSOzZ6npgEvCwMebTEEL3+z22VtlO7A3ut0Nc36PYS0spwFDsTdNV2M/mOexZN9gfTKEz7VPsDd2D5a3USajDsJfKirBnx/dgfyNJ2APcemxtt7OAm51FTwG+FpE92Npwt5vgbUtuxZ5RF2Bvbr+CvZdQUx9iz6j9f6OBp50YPhGR3cBX2HtLYP/p12BLQ4udaVFhjJmCrcAwHVuBwr/toN+HMeZN7GXFn2E/803YihHvOtOnYg+o84E5wOQqhvIKtmT2hjHmcMD79/njci5NfcrRy6llYzuEPVgGq9AB4f+9g60tuBJ77f8xY8wnAdtag/2Nf0LAAdAYsxh4HPgf9vPriq3dFm7PYivCLMCWfD4goDKOU6rp58S0CHupeAK2Fmg94JaAdd2CvU+8GVvp6GZnGdeWdYyk/ApGpST4pb34JiKfYavCPed2LLFCRG4GLjfGnOV2LIlORDphDzxpZQ7qcUFEfMBMoKfRhoueJiIXAFcbY35a6bxeSybOZZ6pQKsyN9QTitgqkO2wZ2YdsGdM/zTGPOVqYAlKRC7ClqYysO1/Sowxw92NSqnwidhlLhF5QUQ2i8jCgPcaie0SZIXz2NB5X0Tk7yKyUkTmi8jJNdzmeGwR/Y5ETiSOWtii6W5sI7J3ObbVq4quX2AvI/yArWF2c8WzKxVfIlYyEZH+2BtxLxpjujjv/Q1bBXeMiNyPbel6n4gMwV5TH4K9xv20MaZPeetWSikVWyJWMjHGfI69WRtoGLaIj/M4POD9F431Fba2SKj17JVSSkVJtDv4a2qM2eA838jRBnYtObbxUaHz3gbKEJFROJ2O1alTp1dubtx0ohs5m5dAajo0bOt2JEqpODBnzpwtxpjyGqbWiGu9xRpjjIhU+xqbMWYstvsOevfubWbPnh322OLOSxfD/m0wKt/tSJRScUBEqtNsokqi3c5kk//ylfO42Xn/R45tyZpN+FtHe1dmNuzUMYGUUu6JdjJ5j6M9e16L0wjLef8ap1bXadg+tY67xKXKkdkK9hZBsVb5V0q5I5JVg1/FtnHIEZFCEbkB29X1uSKyAtsSd4wz+4fYltErsV073BJklao8mU5vJrvWVzyfUkpFSMTumRhjRpYz6Zwg8xrgl5GKxfP8yWTnOmjc3t1YVNwrLi6msLCQAwcOuB2KClF6ejrZ2dmkpqZGfFs6XKsXlCYTvW+iQldYWEi9evVo06YNUu3xyVSsMMawdetWCgsLads28jU9Y76jR1UF9VsAoslEhcWBAwdo3LixJpI4JyI0btw4aiVMTSZekJIGdZvay1xKhYEmEm+I5veoycQrtHqwUspFmky8QpOJ8oitW7fSo0cPevToQbNmzWjZsmXp60OHgg5jVGr27NncdtttUYpUBdIb8F6RmQ3LPwJjQC9RqDjWuHFj5s2bB8Do0aOpW7cud999d+n0w4cPk5IS/NDVu3dvevfuHZU41bG0ZOIVma3g8AHYt9XtSJQKu+uuu46bbrqJPn36cO+99/LNN9/Qt29fevbsyemnn86yZcsAyM/PZ+jQoYBNRD/72c8YMGAA7dq14+9//7ubu+B5WjLxisC2JnWauBuL8ozfv7+Ixet3hXWdJ7Woz8MXdK72coWFhXz55ZckJyeza9cuZs6cSUpKCp9++im/+c1veOutt45bZunSpUyfPp3du3eTk5PDzTffHJU2F4lIk4lXBLY1adHT3ViUioARI0aQnJwMwM6dO7n22mtZsWIFIkJxcXHQZfLy8khLSyMtLY2srCw2bdpEdnZ2NMNOGJpMvCLT6SdTb8KrMKpJCSJS6tSpU/r8oYce4uyzz2bSpEmsXr2aAQMGBF0mLS2t9HlycjKHDx+OdJgJS++ZeEVGI0iprclEJYSdO3fSsmVLAMaNG+duMArQZOIdIk71YG24qLzv3nvv5YEHHqBnz55a2ogRERsDPhp0cKwyXhwOB3fBjZ+5HYmKY0uWLKFTp05uh6HCJNj3KSJzjDFhrUOtJRMv0YaLSimXaDLxksxWsGcTHD7odiRKqQSjycRLSgfJ0hGPlVLRpcnES3RcE6WUSzSZeIkmE6WUSzSZeEl9W+9ek4lSKto0mXhJajrUydK2Jirubdy4kcsvv5z27dvTq1cvhgwZwvLlyyO6zfHjxzNy5Mhj3tuyZQs+n4+DB4NXahk3bhy/+tWvAHjmmWd48cUXj5tn9erVdOnSpcJtr169mldeeaX0dTx2pa/dqXiNVg9Wcc4Yw0UXXcS1117La6+9BsD333/Ppk2b6NixY+l8FXVFXxMXXXQRd911F/v27SMjIwOAN998kwsuuOCYblnKc9NNN9V42/5kcsUVVwDx2ZW+lky8RpOJinPTp08nNTX1mINz9+7d6devH/n5+fTr148LL7yQk046iQMHDnD99dfTtWtXevbsyfTp0wFYtGgRp556Kj169KBbt26sWLGCvXv3kpeXR/fu3enSpQuvv/76MdutX78+Z511Fu+//37pe6+99hojR47k/fffp0+fPvTs2ZNBgwaxadOm4+IePXo0jz32GABz5syhe/fudO/enX/961+l86xevZp+/fpx8sknc/LJJ/Pll18CcP/99zNz5kx69OjBk08+eUxX+tu2bWP48OF069aN0047jfnz55duL5a62NeSiddktoKVn+ogWSo8ptwPGxeEd53NusJPxpQ7eeHChfTq1avc6XPnzmXhwoW0bduWxx9/HBFhwYIFLF26lPPOO4/ly5fzzDPPcPvtt3PllVdy6NAhjhw5wocffkiLFi344IMPANu/V1kjR45kwoQJXHbZZaxfv57ly5czcOBAdu3axVdffYWI8Nxzz/G3v/2Nxx9/vNwYr7/+ev75z3/Sv39/7rnnntL3s7KymDp1Kunp6axYsYKRI0cye/ZsxowZw2OPPcbkyZMBOy6L38MPP0zPnj155513+Oyzz7jmmmtKBw+LpS72tWTiNZnZULwP9m93OxKlIuLUU0+lbdu2AMyaNYurrroKgNzcXFq3bs3y5cvp27cvjzzyCH/9619Zs2YNtWvXpmvXrkydOpX77ruPmTNnkpmZedy68/Ly+OKLL9i1axcTJ07kkksuITk5mcLCQs4//3y6du3Ko48+yqJFi8qNb8eOHezYsYP+/fsDcPXVV5dOKy4u5sYbb6Rr166MGDGCxYsXV7q/s2bNKl3HwIED2bp1K7t27SqNNy0tjSZNmpR2se8WLZl4TeAgWRmN3I1Fxb8KShCR0rlzZ958881ypwd2RV+eK664gj59+vDBBx8wZMgQnn32WQYOHMjcuXP58MMPefDBBznnnHP43e9+d8xytWvXZvDgwUyaNInXXnuNJ554AoBbb72VO++8kwsvvJD8/HxGjx5do3178sknadq0Kd9//z0lJSWkp6fXaD1+sdTFvpZMvEbbmqg4N3DgQA4ePMjYsWNL35s/fz4zZ848bt5+/foxYcIEAJYvX87atWvJycmhoKCAdu3acdtttzFs2DDmz5/P+vXrycjI4KqrruKee+5h7ty5Qbc/cuRInnjiCTZt2kTfvn2BY7u8Hz9+fIXxN2jQgAYNGjBr1iyA0vj862nevDlJSUm89NJLHDlyBIB69eqxe/fuoOsL3Mf8/HyaNGlC/fr1K4zBDZpMvEYHyVJxTkSYNGkSn376Ke3bt6dz58488MADNGvW7Lh5b7nlFkpKSujatSuXXXYZ48aNIy0tjYkTJ9KlSxd69OjBwoULueaaa1iwYEHpTfnf//73PPjgg0G3f+6557J+/Xouu+wyxLnvOHr0aEaMGEGvXr1o0qTyYbH/+9//8stf/pIePXoQ2DP7Lbfcwvjx4+nevTtLly4tLWV169aN5ORkunfvzpNPPnnMukaPHs2cOXPo1q0b999/f6XJzC3aBb3XGAN/agp9RsF5f3I7GhWHtAt6b9Eu6FXNlA6SpSUTpVT0aDLxIk0mSqko02TiRZmtNJmokMTz5W91VDS/R00mXpSZDbs3wuFDbkei4lB6ejpbt27VhBLnjDFs3bo15OrHVaXtTLwoMxswsHs9NGzjdjQqzmRnZ1NYWEhRUZHboagQpaenk52dHZVtaTLxosC2JppMVDWlpqaWtjBXqqr0MpcXaVsTpVSUuZJMROTXIrJIRBaKyKsiki4ibUXkaxFZKSKvi0gtN2LzhEz/IFk6rolSKjqinkxEpCVwG9DbGNMFSAYuB/4KPGmMORHYDtwQ7dg8I7U2ZDTRkolSKmrcusyVAtQWkRQgA9gADAT8vbuNB4a7FJs3aFsTpVQURT2ZGGN+BB4D1mKTyE5gDrDDGOPv8rIQaBlseREZJSKzRWS21japgCYTpVQUuXGZqyEwDGgLtADqAIOrurwxZqwxprcxprfP54tQlB7gb7iobQWUUlHgxmWuQcAqY0yRMaYYeBs4A2jgXPYCyAZ+dCE278jMhkN74MAOtyNRSiUAN5LJWuA0EckQ27/zOcBiYDpwqTPPtcC7LsTmHW6Oa7LuG1g2JfrbVUq5xo17Jl9jb7TPBRY4MYwF7gPuFJGVQGPg+WjH5ilutTXZtw1evRzevz2621VKucqVFvDGmIeBh8u8XQCc6kI43uRWyWTq72DfVvt83zYdOlipBKEt4L2qjg+Sa0W34eLqL+C7l6B5D/t6y/LobVsp5SpNJl6VlAT1W0avZHL4IEy+AxqcABc9Y9/bvCQ621aJyRhY8CYc2ut2JApNJt4WzbYmXzxtSyJ5T0CTHEjNgKJl0dm2SkyF38JbN8A3Y92ORKHJxNuiNUjWlpXw+WPQ+SLocK4tFflyoGhp5LetEldBvn1c+qGrYShLk4mXZWbD7g1wpDhy2zAGPvg1pKTD4DFH3/flajJRkVUwwz4Wfgu7N7kbi9Jk4mmZ2WBKbEKJlPmvw6rPYdDDUK/Z0fd9OXa7+7XRpIqAQ3th3ddw4iDAwHJt1+Q2TSZeFunqwXu3wse/gexToNf1x07zdbKPet9ERcKa/0FJMZx2sx0AbukHbkeU8DSZeFmkGy5O/R0c2AkXPG3vkwTy5dhHvdSlImFVvq36fsLpkJNn758c3O12VAlNk4mXRXKQrFUzYd7L0PdX0LTz8dMbtIaU2loyUZFRkA+t+kCtDMjNgyOHYOU0t6NKaJpMvKxWHajdKPwlk8MHYfKvbcI4677g8yQlga8jFGlbExVme7fCxgXQ7iz7ulUfyGisl7pcpsnE6yLR1mTWU7B1hW1TUiuj/Pl8uVoyUeG3yqnF1XaAfUxOgY6DYcXHka25qCqkycTrwt3WZMsKmPkYdLkEOgyqeF5fLuz6EQ7sCt/2lSrIh7T60KLn0fdy8+z9uzVfuBZWotNk4nXhLJkYYy9vpdSG8/9S+fy+XPuopRMVTqtmQJszbYnEr93Z9nepl7pco8nE6zKz4eAue9YWqu9fhdUz4dzRUK9p5fNrjS4VbttX2792A459v1YGtB9oW8Pr6KKu0GTideFqa7J3K3z8W3uz8+TrqrZMwza2ZbwmExUu/lbv7QYcPy03D3YVwobvoxmRcmgy8bpwtTWZ+pAt4Qx96vg2JeVJSoYmHTSZqPApyIe6zaBJx+OndRwMkqSXulyiycTrSksmIbQ1WfU5zJsAp98GTU+q3rK+TnrPRIVHSYm9X9JuAIgcP71OYzihLyzTjh/doMnE6+o2haTUmpdMig/Ym+4N20D/e6q/vC/HJjJtnaxCtXmRHcXT374kmNw82LQQtq2KXlwK0GTifUlJUL9FzZPJrCdh60rIe7ziNiXlKa3RpaMuqhD5u5xvW0EyyRliH7V0EnWaTBJBTduaFC2HWU9Al0ud3llrIMvf4aO2hFchKphh75X4uwkKplFbyOqsY5y4QJNJIqhJWxN/m5LU2jC4Cm1KytOwDSSn6U14FZrDh2yDxIpKJX65ebD2S1sDUUWNJpNEkJkNu9bDkcNVX2beBFgzC879A9TNqvm2k5Lt2aTehFehKPwWivcFrxJcVu4QO47Pio8jHZUKoMkkEWRmgzkCezZWbf69W+CTB6HVadDzmtC378uBzVoyUSFYNcNW+21zZuXzNu8B9VtqFeEo02SSCKrb1uSTB+HgHrigGm1KKuLLhZ1r7TqVqomCfNsXV+0Glc8rYi91rZwGh/ZFPDRlaTJJBNVpBV8ww3abcsbtR2+ehyrLqdG1RWt0qRo4sAsKZ1ftEpdfzhA4vP9oDTAVcZpMEkFVB8kqbVPSFvrfHb7tl1YP1ktdqgbWfGkv01bl5rtfmzMhLVMvdUVRSuWzqLiXVg/SG1ReMpn5OGz7Aa6eZGtxhUvDtnaIVU0mqiYK8m0fb636VH2Z5FToeB4snwIlR2xFEBVRWjJJFJW1NSlaZhsodv2p7X01nJJToHEHvQmvambVDDjhNEhNr95yuXm2xfy6ryMTlzqGJpNEUVFbk5ISeP8OO8zv+Y9EZvu+HC2ZqOrbvQk2L67e/RK/EwfZErFe6ooKTSaJIjO7/Hsm8ybYRl7n/RHq+iKz/axOsGMtHNobmfUrb1r1uX1sN6D6y6bVs/dZlk7WMU6iQJNJosjMtgNklR1Cd0+RrQp8wunQ46rIbd+XAxit0aWqpyDf3u9r1q1my+fm2cG0Nmt3PpGmySRR+KsH7/rx2Pc/+a0tLQx9MjxtSsqjQ/iq6jLGJpO2/Wt+Az1nCCB6qSsKNJkkimANF3+YDvNfhzPvONoWJFIatbNd4et9E1VV2wrsyIkVdTlfmXpNIbs3LNNkEmmaTBJF2UGyivfDB3fag3y/uyK//eRUaHyi1uhSVVcw3T62Ozu09eTmwfrvQh9tVFXIlWQiIg1E5E0RWSoiS0Skr4g0EpGpIrLCeWzoRmyeVa8ZSPLRf6iZj9szv6FPhrdNSUW0RpeqjoIZtkTdqF1o68kdah+XTQk9JlUut0omTwMfGWNyge7AEuB+YJoxpgMwzXmtwiUp2XZ+t7PQlg5mPQXdLq9ZLZmayupkb4Zqf0mqMiVHbE2utmcFH6K3Opp0sO2c9L5JREU9mYhIJtAfeB7AGHPIGLMDGAaMd2YbDwyPdmyel5ltq+dOvgPS6sL5f47u9v01urauiO52VfzZ8D0c2BG+k53cPFg9E/bvCM/61HHcKJm0BYqA/4rIdyLynIjUAZoaYzY482wEmgZbWERGichsEZldVFQUpZA9IjMb1n4Fa/8H5/4R6jSJ7vZ9/lEXtUaXqsSqGfaxbf/wrC83D0oOw4qp4VmfOo4bySQFOBn4tzGmJ7CXMpe0jDEGCNrKyBgz1hjT2xjT2+eLUAM7r44PI3gAAB3ISURBVMrMBgy0PgN6RrBNSXkatYOkFK3zrypXkA9ZJ9naWOHQsjfUydJaXRHkRjIpBAqNMf4Oc97EJpdNItIcwHnc7EJs3ta8G9SqC0OfCv06dE2k1IJG7bVkoipWfMCWoNsNCN86k5LsCIwrpsLhg+FbryoV9WRijNkIrBORHOetc4DFwHvAtc571wLvRjs2z+t8EdxbAL6O7sWQlas1ulTF1n0Nhw9Ur8v5qsjJg0N7YNXM8K5XAe51QX8rMEFEagEFwPXYxDZRRG4A1gA/dSk2b0tJc3f7vlxY8r49+6xuL7AqMRTk22rsbc4I73rb9rcl86WTocOg8K5buZNMjDHzgN5BJp0T7VhUlPlywJTYGl3NurodjYpFq2bYVutp9cK73tR025Pwsg8h74nIdh+UgPTTVNHlr9GlLeFVMPu329bq7QZEZv25ebBnE6yfG5n1JzBNJiq6Gre3lzD0vokKZvUsW3JtNyAy6+9wrq1RuHRyZNafwDSZqOhKSbMJRZOJCqZgBqRm2Kq8kVC7oR0fXlvDh50mExV92keXKk9Bvm0HlVIrctvIybPj6mzRnhjCSZOJij5fru1kUuv7q0A7f7QVM0Lpcr4qcofYRy2dhJUmExV9vlynRtdKtyNRscTfhUq7AZHdTmY2NO9ua3WpsKk0mYhIkoicHo1gVILwj7qo3aqoQAX5kNEEsjpHflu5Q2HdN7B7U+S3lSAqTSbGmBLgX1GIRSWKxieCJGm3KuooY+zN97b9o9P+IzcPMLBcxzgJl6p+a9NE5BIRNzp0Up6Tmm47fSzSkolyFC2DPRujN75O1knQoDUs1Utd4VLVZPIL4A3gkIjsEpHdIrIrgnEpr/PlaslEHVV6vyTCN9/9ROylroJ8OLg7Otv0uColE2NMPWNMkjEm1RhT33ldP9LBKQ/z5cLWH+DwIbcjUbGgIB8atrF/0ZKbB0cOwspp0dumh1X54qSIXCgijzl/QyMZlEoAvlwwR7RGl4Ijh23L93YDorvdVn2gdiOt1RUmVUomIjIGuB3bVfxi4HYR+UskA1Me53NGINDGi2r9d3BwV/i7nK9Mcgrk/ASWfwRHiqO7bQ+qaslkCHCuMeYFY8wLwGAgL3JhKc9r0kFrdCmrIN8+RjuZAOQMgQM7Yc2X0d+2x1SnDl6DgOeZ4Q5EJZjU2vb6uNboUqtm2OEI6jSO/rbbD4SU2toaPgyqmkweAb4TkXEiMh6YA/w5cmGphODrpCWTRHdorx1Zsd0Ad7ZfK8MmlKUf2LYuqsaq1AIeKAFOA94G3gL6GmNej3Bsyut8OfYGvNboSlxr/wdHDrmXTMD21bWrEDbOdy8GD6hqC/h7jTEbjDHvOX8boxCb8jpfLpQctp0+qsRUMAOSUuGEvu7F0HGwvX+nl7pCUtXLXJ+KyN0i0kpEGvn/IhqZ8r4sp48urdGVuArybRXdWnXci6FOE5vMNJmEpKrJ5DLgl8Dn2Pslc4DZkQpKJYjGHQDRZJKo9m6FjQui1+q9IjlDYNNC2L7a7UjiVlXvmdxvjGlb5q9dFOJTXlYrAxq21mSSqFZ/Dhh375f4lY5xog0Ya6qq90zuiUIsKhH5OsFmTSYJqSAfatWDFie7HYnteDSrs17qCoHeM1Hu8tfo0hbIiadghh2PPTnF7Uis3CGw9kvYt83tSOKS3jNR7srqBCXFsG2V25GoaNq+Bravio1LXH65eXYE0OUfuR1JXKpqr8Fl75foPRMVHqV9dGlL+IQS7S7nq6J5D6jfUi911VCFyURE7g14PqLMtEciFZRKIE062kdtCZ9YCvKhbtOjQzjHAhFbq2vlNDi0z+1o4k5lJZPLA54/UGba4DDHohJRrTp2xDut0ZU4Skrs/ZJ2A+wBPJbk5sHh/Uc7n1RVVlkykXKeB3utVM34crVGVyLZvBj2bXGnl+DKtDkT0jJhmV7qqq7Kkokp53mw10rVjC8Htq6wgyQp7/Of9cfS/RK/5FToeB4smwIlR9yOJq5Ulky6+8d8B7o5z/2vu0YhPpUIsjrZzv62a42uhLBqBjQ+ETKz3Y4kuNw82LfV9masqqzCZGKMSQ4Y8z3Fee5/nRqtIJXH6aiLiePwIVj9RWxVCS7rxEGQXEtrdVVTdQbHUioymmgySRg/zobivbGdTNLq2fs5OsZJtWgyUe5LqwuZJ+hN+ERQMMN2997mTLcjqVhunr3sqic4VabJRMUGX462NUkEBfm2cWDthm5HUrGcn9jHpZPdjSOOaDJRsSErF7Ys1xo0XnZwt73M1W6A25FUrl4zyD5F75tUg2vJRESSReQ7EZnsvG4rIl+LyEoReV1EarkVm3KBLxeOHNTxJLxszZd2ZM1YrBIcTM4QWP8d7PzR7Ujigpslk9uBwA6Z/go8aYw5EdgO3OBKVModPh110fMK8iElHVqd5nYkVZM71D4u0zFOqsKVZCIi2UAe8JzzWoCBwJvOLOOB4W7Eplzirx68WTt89KyCGXaI3tR0tyOpGl9HOxqoXuqqErdKJk8B9wIlzuvGwA5jjL8JdCHQMtiCIjJKRGaLyOyioqLIR6qiI60e1M/Wm/BetWczbF4UH/dLAuUOgdUzYf+OyG/LGCjeH/ntREjUR6URkaHAZmPMHBEZUN3ljTFjgbEAvXv31krgXpKVq5e5vGrV5/ax3QA3o6i+3KHwxdOw8lPoemn1lzcGDu2xyXRvEezZFPB8s/N889Hn9VvAbXPDvx9R4MYQZ2cAF4rIECAdqA88DTQQkRSndJIN6F2vROPLhdWzbI2upGS3o1HhVDAd0jOheXe3I6melr2hTpa91OVPJtVJEHuLoDhYd/YCdZrYddf12ct/dbOgwQlR3b1winoyMcY8gNOdvVMyudsYc6WIvAFcCrwGXAu8G+3YlMt8OXD4AOxYY8fkVt5gjL1f0rZ//J0kJCXZNifzJ8Jzg2qWIOpmHX2vblP7PKNx7AxXHCaxtDf3Aa+JyJ+A74DnXY5HRZuvk30sWqbJxEu2FcDOdXDG7W5HUjO9roONC+zYOwmWIKrD1T03xuQD+c7zAuBUN+NRLvM5oy5uXnK0BbKKf6Vdzp/tahg11vJkGDXd7ShinraAV7EjPRPqtdAaXV6zaoatqde4vduRqAjSZKJiS1YuFGlbE88oOWJrcrU7K/aG6FVhpclExRZfLhQtt+OEq/i3cT7s3x5/VYJVtWkyUbHFlwuH98POtW5HosKhYIZ9bNvf3ThUxGkyUbHF30eXjm3iDQX5tpZevWZuR6IiTJOJii06hK93FB+AtV/pJa4EoclExZbaDaBec63R5QWF39hLlvHS5bwKiSYTFXt8OVqjywsK8kGSofUZbkeiokCTiYo9WqPLGwpmQMtekF7f7UhUFGgyUbHHlwvFe20XHCo+7d8B6+fq/ZIEoslExZ7SURf1vkncWj0LTIkmkwSiyUTFHq3RFf9WzYDUDMg+xe1IVJRoMlGxJ6OR7YlVk0l82rcNln4IJ/SFlFpuR6OiRJOJik2+HE0m8ejALnj5YjvmR7+73I5GRZEmExWbfJ3sPROjIzPHjUP74JXL7NgfPx0PbbRKcCLRZKJiky/HDo26s9DtSFRVHD4Ir18J676Ci8fqeDQJSJOJik1ZAaMuqth2pBjeuB5++Awu/Ad0ucTtiJQLNJmo2FRaPVhbwse0kiPwzs2w7AP4yaPQ8yq3I1Iu0WSiYlNGI6jj05vwscwYmHwHLHgDBo2GPqPcjki5SJOJil2+3MS5zLVlBXz8W9i7xe1IqsYY+OgBmPsi9L8Hzvy12xEpl2kyUbHLn0y8XqPr8EGYeC3875/wTD9Y+7XbEVVu+p/h63/DabfA2b91OxoVAzSZqNjly4GDu2DXercjiazP/gSbF8G5f7SN/MYNgf/9K3aT6Mwn4PNH4eRr4PxHdGx3BWgyUbGstEaXh2/Cr/4CvvwH9LoezrgNRs2ADufDx7+BidfYRoCx5OuxMO330HUEDH1KE4kqpclExS6vd/h4YBdMugkatoHz/mTfq90ALp9gSylLP4CxA2DjQjejPOq7l2HKPZA7FIb/G5KS3Y5IxRBNJip21WkCGU28W6ProwdgV6Ft5JdW9+j7IraUct1kOLQXnjsHvpvgXpwAC9+C926F9gPh0hcgOdXdeFTM0WSiYpsvFzZ7MJksmQzzXoYz74RWpwafp/XpcNNM2/Puu7fYg3nx/ujGCbBsCrw9ClqdBpdNgJS06MegYp4mExXbfDneq9G1ZzO8fxs06wZn3VfxvHWz4Jp3od/dthru8+fCtoLoxAnww3Rb06xZN7jidaiVEb1tq7iiyUTFtqxOcHAn7N7odiThYQy8dxsc3AMX/6dqXbQnJcM5D8EVE2HHOnh2gC3ZRNrar+C1K6DxiXDVWzr8rqqQJhMV20oHyvJIja65L8LyKbbFeFZu9ZbteD784nNo3M52qvjJg7ZfrEhY/x1MGAH1W8A179geCZSqgCYTFdu8VKNrW4G96d62P/S5qWbraNgafvYxnPJzW6V4/IWwa0N449y0GF66CNIb2EtsdbPCu37lSZpMVGyr44PajWBznJdMSo7ApJshKcWpVhvCv15KGuQ9Dhc/BxvmwbP9oGBGeOLc+gO8NByS0+DadyEzOzzrVZ6nyUTFNhFv9NH1xdN2rI8hj4bvAN1tBNw4HWo3tAng88egpKTm69ux1pZ0Sg7bEkmjduGJUyUETSYq9mXl2rYm8Vqja8N8mP4InDQcuv00vOvOyrUJpfNF8Nkf4dXL7Bjs1bV7I7w4DA7uhqsnVf9+jkp4mkxU7PPlwoEdsGeT25FUX/EB20YjozEMfTIy3Y+k1YVLnochj9mqvM+eBT/Oqfrye7fCi8Nh9ya46k1o3j38MSrPi3oyEZFWIjJdRBaLyCIRud15v5GITBWRFc5jw2jHpmJUaY2uOGy8+NkfbU20Yf+KbI0oETj1RntzHgMvDIZv/lN5ae7ATnj5Iti+Cq54rfwGlEpVwo2SyWHgLmPMScBpwC9F5CTgfmCaMaYDMM15rRT44nQI31Uzbe+/vW+ADoOis83sXrb6cNuz4MO74e0bbZuWYA7ttdV/Ny2Gn75ka5kpVUNRTybGmA3GmLnO893AEqAlMAwY78w2Hhge7dhUjKqbZaupxlONrgM77XC2jdrBeX+M7rYzGtkGjgMftH1q/Wfg8Ym4+AC8OhIKv4VLnoOO50U3RuU5rt4zEZE2QE/ga6CpMcZfYX4j0LScZUaJyGwRmV1UVBSVOJXL4rFG15T77DgsF4+FWnWiv/2kJDsC4tWTYN9WGHs2LHjTTjtSDG9cB6tmwLD/g8563qZC51oyEZG6wFvAHcaYYwZtMMYYIOjFXmPMWGNMb2NMb5/PF4VIVUzIyrX3HuKhRtfid+H7V6HfXZDd291Y2g2wnUU26wpv3QAf3GUrBCyfYtuq9BjpbnzKM1xJJiKSik0kE4wxbztvbxKR5s705sBmN2JTMcqXC/u3w94YL43u3gjv3wHNe8BZ97odjVW/he3Ovu+v4NvnYNHbdryUU37udmTKQ1KivUEREeB5YIkx5omASe8B1wJjnMd3ox2bimGl3aosjd3uPYxxuonfZztxjKUxP5JT4fw/2xvze4ug55VuR6Q8JurJBDgDuBpYICLznPd+g00iE0XkBmANEObWXSqu+ZPJ5qWxW+tozjhY8Qn85G/g6+h2NMHpjXYVIVFPJsaYWUB5LbfOiWYsKo7UawZpmbHb1mTrD3bc9nYD4JQb3Y5GqajTFvAqPog4N+FjsEbXkcN2LPfkVFs7KpROHJWKU/qrV/HDlxOb45p88SQUfgN5T0BmS7ejUcoVmkxU/PDl2jYTe7e4HclR6+dB/hjofDF0vdTtaJRyjSYTFT9Kb8LHSOmkeL9ts1HHZ9tsKJXANJmo+BFYPTgWTPsDbFkW+U4clYoDmkxU/KjfAtLqx8ZN+IJ8+Or/4NRRcKJWQlRKk4mKHyLOTXiXSyb7d8A7t0DjDjDo9+7GolSM0GSi4kssJJMp99puUy5+FmpluBuLUjFCk4mKL75OtjuQvVvd2f6iSTD/ddvvVste7sSgVAzSZKLii5s34XdtgMm/hhYn2x6BlVKlNJmo+JLlUjIxBt79pR1U6uKxsdWJo1IxwI2OHpWqufotoVbd6CeT2c/DD9NgyGPQpEN0t61UHNCSiYovbtTo2rISPnkI2g/UMUCUKocmExV/fJ2i19Zky0p4+0ZIrmUbJ0p5HV4rldj0MpeKP74cmPcy7NsWmZbn+7fDwrft0LuF34Ikw4j/2kaTSqmgNJmo+FNao2sZtO4bnnUeKYaV0+D7V2DZFDhyCLJOssPbdh0B9ZuHZztKeZQmExV/Amt0hZpMNsy3JZAFb9j2KxmNoffPoPtIaN5dL2spVUWaTFT8qZ8NqXVqfhN+9yZYMBG+fw02LYSkVMgZDN2vgA7narVfpWpAk4mKP0lJ1a/RVXwAln1gE8jKaWCO2BbsQx6DLpdor79KhUiTiYpPvlz44bOK5zEG1n1tL2MtnAQHd9p2Kmfcbi9j+TpGJ1alEoAmExWffDn2Zvn+7VC74bHTtq+x/Wd9/ypsK4DUDOh0IXS/HNr2h6Rkd2JWysM0maj4lNXJPhYthxP6wMHdsPhdmPcqrJllp7XpB/3uhpMuhLR67sWqVALQZKLiky/HPs5/Hb59Dpa8D4f3Q6P2cPaD0P0yaHCCuzEqlUA0maj4lHmCrdE1+3lIz7SXsHpcAdmnaHVepVygyUTFp6QkuOQ5OHIQOv4EUtPdjkiphKbJRMWv3CFuR6CUcmhHj0oppUKmyUQppVTINJkopZQKmSYTpZRSIdNkopRSKmSaTJRSSoVMk4lSSqmQaTJRSikVMk0mSimlQhZTyUREBovIMhFZKSL3ux2PUkqpqomZ7lREJBn4F3AuUAh8KyLvGWMWuxuZUspLjDEBz53HINMD34sWAVKSY+ocv8piJpkApwIrjTEFACLyGjAMcDWZnDHmM3bsO+RmCCpBuHHwipaA4zeG4w/m9n3KeVG1ZYIlARNnH2p2w9rMum+g22HUSCwlk5bAuoDXhUCfsjOJyChglPPyoIgsjEJs0dIE2OJ2EGHipX0Bb+2Pl/YFPLQ/a6CJ3B+VfWkd7hXGUjKpEmPMWGAsgIjMNsb0djmksPHS/nhpX8Bb++OlfQFv7U8870ssXZz7EWgV8DrbeU8ppVSMi6Vk8i3QQUTaikgt4HLgPZdjUkopVQUxc5nLGHNYRH4FfAwkAy8YYxZVstjYyEcWVV7aHy/tC3hrf7y0L+Ct/YnbfRETb9UdlFJKxZxYusyllFIqTmkyUUopFbK4TSZe6XpFRFqJyHQRWSwii0TkdrdjCpWIJIvIdyIy2e1YQiUiDUTkTRFZKiJLRKSv2zGFQkR+7fzOForIqyKS7nZMVSUiL4jI5sC2ZSLSSESmisgK57GhmzFWRzn786jzW5svIpNEpIGbMVZHXCaTgK5XfgKcBIwUkZPcjarGDgN3GWNOAk4DfhnH++J3O7DE7SDC5GngI2NMLtCdON4vEWkJ3Ab0NsZ0wVZ0udzdqKplHDC4zHv3A9OMMR2Aac7reDGO4/dnKtDFGNMNWA48EO2gaioukwkBXa8YYw4B/q5X4o4xZoMxZq7zfDf2YNXS3ahqTkSygTzgObdjCZWIZAL9gecBjDGHjDE73I0qZClAbRFJATKA9S7HU2XGmM+BbWXeHgaMd56PB4ZHNagQBNsfY8wnxpjDzsuvsO3t4kK8JpNgXa/E7QHYT0TaAD2Br92NJCRPAfcCJW4HEgZtgSLgv85lu+dEpI7bQdWUMeZH4DFgLbAB2GmM+cTdqELW1BizwXm+EWjqZjBh9jNgittBVFW8JhPPEZG6wFvAHcaYXW7HUxMiMhTYbIyZ43YsYZICnAz82xjTE9hLfF1GOYZzP2EYNkm2AOqIyFXuRhU+xrZz8ERbBxH5LfYS+AS3Y6mqeE0mnup6RURSsYlkgjHmbbfjCcEZwIUishp76XGgiLzsbkghKQQKjTH+kuKb2OQSrwYBq4wxRcaYYuBt4HSXYwrVJhFpDuA8bnY5npCJyHXAUOBKE0cNAeM1mXim6xUREew1+SXGmCfcjicUxpgHjDHZxpg22O/kM2NM3J75GmM2AutEJMd56xxcHhIhRGuB00Qkw/ndnUMcVyhwvAdc6zy/FnjXxVhCJiKDsZeJLzTG7HM7nuqIy2Ti3KDyd72yBJhYha5XYtUZwNXYs/h5zt8Qt4NSpW4FJojIfKAH8IjL8dSYU8J6E5gLLMD+/8dN9x0i8irwPyBHRApF5AZgDHCuiKzAlrzGuBljdZSzP/8E6gFTnWPBM64GWQ3anYpSSqmQxWXJRCmlVGzRZKKUUipkmkyUUkqFTJOJUkqpkGkyUUopFTJNJqpGRMSIyOMBr+8WkdFhWvc4Ebk0HOuqZDsjnJ6Ap5d5v42IXFHDdX5ZhXme80BnnqWcz2th5XMqL9NkomrqIHCxiDRxO5BATgeGVXUDcKMx5uwy77cBgiaTytZvjKm0Rbkx5ufGmHhu/KjUcTSZqJo6jG3w9uuyE8qWLERkj/M4QERmiMi7IlIgImNE5EoR+UZEFohI+4DVDBKR2SKy3Onzyz9OyqMi8q0z3sMvAtY7U0TeI0gLdREZ6ax/oYj81Xnvd8CZwPMi8miZRcYA/ZxGY78WketE5D0R+QyYJiJ1RWSaiMx11jssYFuB+5ovR8dCmeC0Osd5v7d/fhH5s4h8LyJfiUhT5/32zusFIvIn/3qD7NtVzuc3T0SedT6jU5zPJ11E6ogdv6RLeXE7JYulzve23Il1kIh8IXackFOd+UaLyEsi8j/n/RuDxFPed9RcRD534lwoIv2C7Y+KY8YY/dO/av8Be4D6wGogE7gbGO1MGwdcGjiv8zgA2AE0B9Kw/an93pl2O/BUwPIfYU92OmD7yEoHRgEPOvOkAbOxnRYOwHbC2DZInC2w3Yj4sB03fgYMd6blY8f2KLvMAGBywOvrnBgaOa9TgPrO8ybASo42AA7c153YfuOSsC2dzyy7XWzHhBc4z/8WsH+TgZHO85v86y0TZyfgfSDVef1/wDXO8z9hewj+F/BARXFjS2KHga5OrHOAF5xpw4B3nGVGA98DtZ3l1zmfbxtgoTNPed/RXcBvnfeTgXpu/4b1L7x/1bkkoNQxjDG7RORF7IBL+6u42LfG6TJcRH4A/F2gLwACLzdNNMaUACtEpADIBc4DugWUejKxyeYQ8I0xZlWQ7Z0C5BtjipxtTsCOUfJOFeP1m2qM8Y89IcAjItIf29V+S2zX5xvLLPONMabQ2e487EF3Vpl5DmETB9iD+LnO874cHZvjFWxiKOscoBfwrVPoqc3Rjg7/gO3D7gD2+6kobrAdQC5wYl2EHXDKiMgCJ26/d40x+4H9zr2mU4F5AdPL+46+BV4Q26npO8aYwGWUB2gyUaF6CtvX038D3juMcwlVRJKAWgHTDgY8Lwl4XcKxv8ey/fwY7MHwVmPMx4ETRGQAtmQSSYHrvxJb0ulljCkW20tysOFvA/f1CMH/34qNMaaSecojwHhjTLDR+BoDdYFUJ7a9lcQdyvdSNqbjviMAJ4nlAeNE5AljzIsV756KJ3rPRIXEOVufiL2Z7bcae8YMcCH2gFZdI0QkybmP0g5Yhu3Y82bn7BYR6SiVD1b1DXCWiDQRO9zzSGBGJcvsxna2V55M7LgtxSJyNtC6CvtTXV8BlzjPyxtadxpwqYhkQel46P5YngUewo6H8dcwxj3MuRfTGHsp79sy04N+R05cm4wx/8GOwhnPXfmrILRkosLhcWwvzn7/Ad4Vke+x9z5qUmpYi00E9YGbjDEHROQ57CWXuc7N7CIqGabVGLNBRO4HpmPPmj8wxlTWTfl84IgT/zhge5npE4D3nUtAs4Gl1dmxKroDeFnsIEkfYe+/HMMYs1hEHgQ+cUqAxcAvReQsbInnFSeBfikiA8MU93zsZ9kE+KMxZr3YEUL9yvuOBgD3iEgx9n7bNTXYtoph2muwUjFIRDKA/c59i8uxN+OHVbZchGMaja0IEOz+jUpwWjJRKjb1Av7pnN3vwI4HrlTM0pKJUkqpkOkNeKWUUiHTZKKUUipkmkyUUkqFTJOJUkqpkGkyUUopFbL/B/WpuNvFhJEoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "lambda_ = 0\n",
    "theta = utils.trainLinearReg(linearRegCostFunction, X_poly, y,\n",
    "                             lambda_=lambda_, maxiter=55)\n",
    "\n",
    "# Plot training data and fit\n",
    "pyplot.plot(X, y, 'ro', ms=10, mew=1.5, mec='k')\n",
    "\n",
    "utils.plotFit(polyFeatures, np.min(X), np.max(X), mu, sigma, theta, p)\n",
    "\n",
    "pyplot.xlabel('Change in water level (x)')\n",
    "pyplot.ylabel('Water flowing out of the dam (y)')\n",
    "pyplot.title('Polynomial Regression Fit (lambda = %f)' % lambda_)\n",
    "pyplot.ylim([-20, 50])\n",
    "\n",
    "pyplot.figure()\n",
    "error_train, error_val = learningCurve(X_poly, y, X_poly_val, yval, lambda_)\n",
    "pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_val)\n",
    "\n",
    "pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)\n",
    "pyplot.xlabel('Number of training examples')\n",
    "pyplot.ylabel('Error')\n",
    "pyplot.axis([0, 13, 0, 100])\n",
    "pyplot.legend(['Train', 'Cross Validation'])\n",
    "\n",
    "print('Polynomial Regression (lambda = %f)\\n' % lambda_)\n",
    "print('# Training Examples\\tTrain Error\\tCross Validation Error')\n",
    "for i in range(m):\n",
    "    print('  \\t%d\\t\\t%f\\t%f' % (i+1, error_train[i], error_val[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to combat the overfitting (high-variance) problem is to add regularization to the model. In the next section, you will get to  try different $\\lambda$ parameters to see how regularization can lead to a better model.\n",
    "\n",
    "### 3.2 Optional (ungraded) exercise: Adjusting the regularization parameter\n",
    "\n",
    "In this section, you will get to observe how the regularization parameter affects the bias-variance of regularized polynomial regression. You should now modify the the lambda parameter and try $\\lambda = 1, 100$. For each of these values, the script should generate a polynomial fit to the data and also a learning curve.\n",
    "\n",
    "For $\\lambda = 1$, the generated plots should look like the the figure below. You should see a polynomial fit that follows the data trend well (left) and a learning curve (right) showing that both the cross validation and training error converge to a relatively low value. This shows the $\\lambda = 1$ regularized polynomial regression model does not have the high-bias or high-variance problems. In effect, it achieves a good trade-off between bias and variance.\n",
    "\n",
    "<table>\n",
    "    <tr>\n",
    "        <td><img src=\"Figures/polynomial_regression_reg_1.png\"></td>\n",
    "        <td><img src=\"Figures/polynomial_learning_curve_reg_1.png\"></td>\n",
    "    </tr>\n",
    "</table>\n",
    "\n",
    "For $\\lambda = 100$, you should see a polynomial fit (figure below) that does not follow the data well. In this case, there is too much regularization and the model is unable to fit the training data.\n",
    "\n",
    "![](Figures/polynomial_regression_reg_100.png)\n",
    "\n",
    "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"section5\"></a>\n",
    "### 3.3 Selecting $\\lambda$ using a cross validation set\n",
    "\n",
    "From the previous parts of the exercise, you observed that the value of $\\lambda$ can significantly affect the results of regularized polynomial regression on the training and cross validation set. In particular, a model without regularization ($\\lambda = 0$) fits the training set well, but does not generalize. Conversely, a model with too much regularization ($\\lambda = 100$) does not fit the training set and testing set well. A good choice of $\\lambda$ (e.g., $\\lambda = 1$) can provide a good fit to the data.\n",
    "\n",
    "In this section, you will implement an automated method to select the $\\lambda$ parameter. Concretely, you will use a cross validation set to evaluate how good each $\\lambda$ value is. After selecting the best $\\lambda$ value using the cross validation set, we can then evaluate the model on the test set to estimate\n",
    "how well the model will perform on actual unseen data. \n",
    "\n",
    "Your task is to complete the code in the function `validationCurve`. Specifically, you should should use the `utils.trainLinearReg` function to train the model using different values of $\\lambda$ and compute the training error and cross validation error. You should try $\\lambda$ in the following range: {0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10}.\n",
    "<a id=\"validationCurve\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [],
   "source": [
    "def validationCurve(X, y, Xval, yval):\n",
    "    \"\"\"\n",
    "    Generate the train and validation errors needed to plot a validation\n",
    "    curve that we can use to select lambda_.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array_like\n",
    "        The training dataset. Matrix with shape (m x n) where m is the \n",
    "        total number of training examples, and n is the number of features \n",
    "        including any polynomial features.\n",
    "    \n",
    "    y : array_like\n",
    "        The functions values at each training datapoint. A vector of\n",
    "        shape (m, ).\n",
    "    \n",
    "    Xval : array_like\n",
    "        The validation dataset. Matrix with shape (m_val x n) where m is the \n",
    "        total number of validation examples, and n is the number of features \n",
    "        including any polynomial features.\n",
    "    \n",
    "    yval : array_like\n",
    "        The functions values at each validation datapoint. A vector of\n",
    "        shape (m_val, ).\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    lambda_vec : list\n",
    "        The values of the regularization parameters which were used in \n",
    "        cross validation.\n",
    "    \n",
    "    error_train : list\n",
    "        The training error computed at each value for the regularization\n",
    "        parameter.\n",
    "    \n",
    "    error_val : list\n",
    "        The validation error computed at each value for the regularization\n",
    "        parameter.\n",
    "    \n",
    "    Instructions\n",
    "    ------------\n",
    "    Fill in this function to return training errors in `error_train` and\n",
    "    the validation errors in `error_val`. The vector `lambda_vec` contains\n",
    "    the different lambda parameters to use for each calculation of the\n",
    "    errors, i.e, `error_train[i]`, and `error_val[i]` should give you the\n",
    "    errors obtained after training with `lambda_ = lambda_vec[i]`.\n",
    "\n",
    "    Note\n",
    "    ----\n",
    "    You can loop over lambda_vec with the following:\n",
    "    \n",
    "          for i in range(len(lambda_vec))\n",
    "              lambda = lambda_vec[i]\n",
    "              # Compute train / val errors when training linear \n",
    "              # regression with regularization parameter lambda_\n",
    "              # You should store the result in error_train[i]\n",
    "              # and error_val[i]\n",
    "              ....\n",
    "    \"\"\"\n",
    "    # Selected values of lambda (you should not change this)\n",
    "    lambda_vec = [0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]\n",
    "\n",
    "    # You need to return these variables correctly.\n",
    "    error_train = np.zeros(len(lambda_vec))\n",
    "    error_val = np.zeros(len(lambda_vec))\n",
    "\n",
    "    # ====================== YOUR CODE HERE ======================\n",
    "    X = np.concatenate([np.ones((y.shape[0], 1)), X], axis=1)\n",
    "    Xval = np.concatenate([np.ones((yval.shape[0], 1)), Xval], axis=1)\n",
    "    \n",
    "    for i in range(1, len(lambda_vec)+1):\n",
    "        lambda_ = lambda_vec[i-1]\n",
    "        \n",
    "        theta,_ = trainLinearReg(linearRegCostFunction, X, y, lambda_)\n",
    "        error_train[i-1],_ = linearRegCostFunction(X, y,theta, lambda_=0)\n",
    "        error_val[i-1], _ = linearRegCostFunction(Xval, yval,theta, lambda_=0)\n",
    "\n",
    "\n",
    "\n",
    "    # ============================================================\n",
    "    return lambda_vec, error_train, error_val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10,)"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lambda_vec = [0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]\n",
    "np.zeros(len(lambda_vec)).shape\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After you have completed the code, the next cell will run your function and plot a cross validation curve of error v.s. $\\lambda$ that allows you select which $\\lambda$ parameter to use. You should see a plot similar to the figure below. \n",
    "\n",
    "![](Figures/cross_validation.png)\n",
    "\n",
    "In this figure, we can see that the best value of $\\lambda$ is around 3. Due to randomness\n",
    "in the training and validation splits of the dataset, the cross validation error can sometimes be lower than the training error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lambda\t\tTrain Error\tValidation Error\n",
      " 0.000000\t0.028901\t55.043068\n",
      " 0.001000\t0.112439\t9.828218\n",
      " 0.003000\t0.170841\t16.305636\n",
      " 0.010000\t0.221527\t16.944736\n",
      " 0.030000\t0.281857\t12.828376\n",
      " 0.100000\t0.459315\t7.586964\n",
      " 0.300000\t0.921768\t4.636832\n",
      " 1.000000\t2.076187\t4.260628\n",
      " 3.000000\t4.901509\t3.822929\n",
      " 10.000000\t16.092218\t9.945512\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deZxU1Z338c+vF7ob6EWgabobFTSK7KA9LnE0ikaNKyZxITEak5c+RhPNZIIBH0fJqsZt4it5kjAmkcyQKA9BXJ/JoMJE4ySRRdkFNai90CzaG/Te5/nj3m6qu6rohVq6637fr1e/qupW1b2nGvjW5dxzfsecc4iISHCkJbsBIiKSWAp+EZGAUfCLiASMgl9EJGAU/CIiAZOR7Ab0xZgxY9yECROS3QwRkSFl3bp1+5xzhT23D4ngnzBhAmvXrk12M0REhhQzez/SdnX1iIgEjIJfRCRgFPwiIgEzJPr4RST2WltbKS8vp6mpKdlNkSOUnZ3N+PHjyczM7NPrFfwiAVVeXk5ubi4TJkzAzJLdHBkg5xz79++nvLyciRMn9uk9qdvVs3EZPDoNFhV4txuXJbtFIoNKU1MTo0ePVugPcWbG6NGj+/U/t9Q849+4DJ67HVobvce1H3qPAWZcnbx2iQwyCv3U0N8/x9Q843/5e4dCv1Nro7ddRCTgUjP4a8v7t11EEm7//v3MmjWLWbNmMW7cOEpLS7set7S0HPa9a9eu5fbbb09QS1NPanb15I/3uncibReRAVm5oYIH//g2lTWNlBTkMP/CScydXTrg/Y0ePZo333wTgEWLFjFy5Ei+/e1vdz3f1tZGRkbkiCorK6OsrGzAxw661DzjP+8eyMzpvi0zx9suIv22ckMFC1dsoqKmEQdU1DSycMUmVm6oiOlxvvzlL3PLLbdw2mmnceedd/K3v/2NM844g9mzZ/PJT36St99+G4A1a9Zw6aWXAt6Xxle+8hXOOeccjjvuOB577LGYtikVpeYZf+cF3BU3Aw7ySuH8RbqwKxLFhAUv9Ps9ja3tfPOpN/nmU29Gfc2u+y/p937Ly8t5/fXXSU9Pp66ujldffZWMjAxeeukl7rrrLv7whz+EvWf79u2sXr2a+vp6Jk2axNe+9rU+j2kPotQMfvBC/rk7oPUgfP0NGDYi2S0SkT646qqrSE9PB6C2tpYbbriBnTt3Yma0trZGfM8ll1xCVlYWWVlZjB07lurqasaPV9duNKkb/CLSZ72dmZ95/ytU1DSGbS8tyOHPC+bEtC0jRhw6SfuXf/kXzj33XJ5++ml27drFOeecE/E9WVlZXffT09Npa2uLaZtSTWr28YtITM2/cBI5mendtuVkpjP/wklxPW5tbS2lpd4F5CeeeCKuxwoSBb+I9Gru7FLu++x0SgtyMLwz/fs+O/2IRvX0xZ133snChQuZPXu2zuJjyJxzyW5Dr8rKytyAFmL5YbHXx39Xpfr4RXrYtm0bkydPTnYzJEYi/Xma2TrnXNi4V53xi4gEjIJfRCRgFPwiIgGj4BcRCRgFv4hIwCj4RUQCJq7Bb2a7zGyTmb1pZmv9baPMbJWZ7fRvj4pnG0Rk8Nq9ezfXXnstxx9/PKeccgoXX3wxO3bsiOsxlyxZwrx587pt27dvH4WFhTQ3N0d8zxNPPMHXv/51AH7xi1/w29/+Nuw1u3btYtq0aYc99q5du/jd737X9ThZ5aUTccZ/rnNuVshY0gXAy865E4CX/cciMtjFeDlT5xxXXnkl55xzDu+++y7r1q3jvvvuo7q6utvrYj1x68orr2TVqlUcPHiwa9vy5cu57LLLupV+iOaWW27h+uuvH9CxewZ/WVlZUqqJJqOr5wpgiX9/CTA3CW0Qkf7oXM609kPAHVrO9AjCf/Xq1WRmZnLLLbd0bZs5cyZnnXUWa9as4ayzzuLyyy9nypQpNDU1ceONNzJ9+nRmz57N6tWrAdiyZQunnnoqs2bNYsaMGezcuZMDBw5wySWXMHPmTKZNm8ZTTz3V7bh5eXl86lOf4rnnnuva9uSTTzJv3jyee+45TjvtNGbPns35558f9iUEXhnohx56CIB169Yxc+ZMZs6cyc9+9rOu1+zatYuzzjqLk08+mZNPPpnXX38dgAULFvDqq68ya9YsHn300W7lpT/66CPmzp3LjBkzOP3009m4cWPX8WJddjreRdoc8F9m5oBfOucWA0XOuSr/+d1AUaQ3mtnNwM0AxxxzTJybKRJwi/L7/57WRlhxk/cTdb+1UZ/avHkzp5xyStTn169fz+bNm5k4cSIPP/wwZsamTZvYvn07F1xwATt27OAXv/gFd9xxB1/84hdpaWmhvb2dF198kZKSEl54wSs1XVsb3oZ58+axdOlSrrnmGiorK9mxYwdz5syhrq6Ov/zlL5gZjz/+OD/+8Y95+OGHo7bxxhtv5Kc//Slnn3028+fP79o+duxYVq1aRXZ2Njt37mTevHmsXbuW+++/n4ceeojnn38e8NYV6HTvvfcye/ZsVq5cySuvvML111/ftVBNrMtOx/uM/x+dcycDnwFuM7OzQ590Xr2IiDUjnHOLnXNlzrmywsLCI2vFEChLISLdnXrqqUycOBGA1157jeuuuw6Ak046iWOPPZYdO3Zwxhln8KMf/YgHHniA999/n5ycHKZPn86qVav4zne+w6uvvkp+fviX2iWXXMKf//xn6urqWLZsGZ/73OdIT0+nvLycCy+8kOnTp/Pggw+yZcuWqO2rqamhpqaGs8/2Yu1LX/pS13Otra3cdNNNTJ8+nauuuoqtW7f2+nlfe+21rn3MmTOH/fv3U1dX19XerKwsxowZ01V2+kjE9YzfOVfh3+4xs6eBU4FqMyt2zlWZWTGwJ34t6N/K8yKBdZgzc8Dr04+4nOnR8E+bB3TIqVOnsnz58qjPh5ZnjuYLX/gCp512Gi+88AIXX3wxv/zlL5kzZw7r16/nxRdf5O677+a8887jnnu6r76Xk5PDRRddxNNPP82TTz7JI488AsA3vvENvvWtb3H55ZezZs0aFi1aNKDP9uijj1JUVMRbb71FR0cH2dnZA9pPp1iXnY7bGb+ZjTCz3M77wAXAZuBZ4Ab/ZTcAz8SrDSISI3FYznTOnDk0NzezePHirm0bN27k1VdfDXvtWWedxdKlSwHYsWMHH3zwAZMmTeK9997juOOO4/bbb+eKK65g48aNVFZWMnz4cK677jrmz5/P+vXrIx5/3rx5PPLII1RXV3PGGWcA3ctAL1myJOL7OhUUFFBQUMBrr70G0NW+zv0UFxeTlpbGv//7v9Pe3g5Abm4u9fX1EfcX+hnXrFnDmDFjyMvLO2wbBiqeXT1FwGtm9hbwN+AF59x/AvcDnzazncD5/mMRGcxmXA2XPead4WPe7WWPHdFypmbG008/zUsvvcTxxx/P1KlTWbhwIePGjQt77a233kpHRwfTp0/nmmuu4YknniArK4tly5Yxbdo0Zs2axebNm7n++uvZtGlT1wXf7373u9x9990Rj//pT3+ayspKrrnmGsy83oFFixZx1VVXccoppzBmzJheP8NvfvMbbrvtNmbNmkVopeNbb72VJUuWMHPmTLZv3971v5cZM2aQnp7OzJkzefTRR7vta9GiRaxbt44ZM2awYMGCXr94jkSKl2UugdYDsLACskbGvmEiQ5jKMqcWlWUWEZGoFPwiIgGj4BcJsKHQ1Su96++fo4JfJKCys7PZv3+/wn+Ic86xf//+fg0ZjffMXREZpMaPH095eTl79+5NdlPkCGVnZzN+/Pg+v17BLxJQmZmZXTNjJVjU1SMiEjAKfhGRgFHwi4gEjIJfRCRgFPwiIgGj4BcRCRgFv4hIwCj4RUQCRsEvIhIwCn4RkYAJSPCrCJWISKfUDn7TYusiIj2ldvCLiEgYBb+ISMAo+EVEAkbBLyISMAp+EZGAUfCLiASMgl9EJGAU/CIiAaPgFxEJmLgHv5mlm9kGM3vefzzRzP5qZu+Y2VNmNizebRARkUMSccZ/B7At5PEDwKPOuU8AHwNfTUAbRETEF9fgN7PxwCXA4/5jA+YAy/2XLAHmxrMNIiLSXbzP+P8VuBPo8B+PBmqcc23+43KgNNIbzexmM1trZmv37t0b52aKiARH3ILfzC4F9jjn1g3k/c65xc65MudcWWFhYYxbJyISXBlx3PeZwOVmdjGQDeQBPwEKzCzDP+sfD1TEsQ0iItJD3M74nXMLnXPjnXMTgGuBV5xzXwRWA5/3X3YD8Ey82iAiIuGSMY7/O8C3zOwdvD7/XyWhDSIigRXPrp4uzrk1wBr//nvAqYk4roiIhAvGzF2nNXdFRDqlePBrzV0RkZ5SPPhFRKQnBb+ISMAo+EVEAkbBLyISMAp+EZGAUfCLiASMgl9EJGAU/CIiAaPgFxEJGAW/iEjAKPhFRAJGwS8iEjAKfhGRgFHwi4gEjIJfRCRgFPwiIgGj4BcRCRgFv4hIwAQk+LXmrohIp9QOftOauyIiPaV28IuISBgFv4hIwCj4RUQCRsEvIhIwCn4RkYCJW/CbWbaZ/c3M3jKzLWb2XX/7RDP7q5m9Y2ZPmdmweLVBRETCxfOMvxmY45ybCcwCLjKz04EHgEedc58APga+Gsc2iIhID70Gv5mlmdkn+7tj52nwH2b6Pw6YAyz3ty8B5vZ33yIiMnC9Br9zrgP42UB2bmbpZvYmsAdYBbwL1Djn2vyXlAOlUd57s5mtNbO1e/fuHcjhRUQkgr529bxsZp8z699UWOdcu3NuFjAeOBU4qR/vXeycK3POlRUWFvbnsCIichh9Df7/BfxfoMXM6sys3szq+noQ51wNsBo4Aygwswz/qfFARX8aLCIiR6ZPwe+cy3XOpTnnMp1zef7jvMO9x8wKzazAv58DfBrYhvcF8Hn/ZTcAzwy8+SIi0l8Zvb/EY2aXA2f7D9c4557v5S3FwBIzS8f7glnmnHvezLYCT5rZD4ANwK8G0G4RERmgPgW/md0P/AOw1N90h5md6ZxbGO09zrmNwOwI29/D6+8XEZEk6OsZ/8XALH+ED2a2BO9sPWrwi4jI4NSfCVwFIffzY90QERFJjL6e8f8I2GBmqwHD6+tfELdWiYhI3PQa/GaWBnQAp+P18wN8xzm3O54NExGR+Og1+J1zHWZ2p3NuGfBsAtokIiJx1Nc+/pfM7NtmdrSZjer8iWvLYslpsXURkU597eO/xr+9LWSbA46LbXNiTYuti4j01Nc+/gXOuacS0B4REYmzvlbnnJ+AtoiISAIEo49fRES6pHgfv4iI9NSn4HfOTYx3Q0REJDEO29VjZneG3L+qx3M/ilejREQkfnrr47825H7PgmwXxbgtIiKSAL0Fv0W5H+mxiIgMAb0Fv4tyP9JjEREZAnq7uDvTX1vXgJyQdXYNyI5ry0REJC4OG/zOufRENURERBKjPwuxiIhIClDwi4gEjIJfRCRgFPwiIgGj4BcRCRgFv4hIwCj4RUQCJiDBr0nGIiKdUjv4VU1IRCRM3ILfX61rtZltNbMtZnaHv32Uma0ys53+7VHxaoOIiISL5xl/G/DPzrkpwOnAbWY2BVgAvOycOwF42X8sIiIJErfgd85VOefW+/frgW1AKXAFsMR/2RJgbrzaICIi4RLSx29mE4DZwF+BIudclf/UbqAoyntuNrO1ZrZ27969iWimiEggxD34zWwk8Afgm865utDnnHOOKENunHOLnXNlzrmywsLCeDdTRCQw4hr8ZpaJF/pLnXMr/M3VZlbsP18M7IlnG0REpLt4juox4FfANufcIyFPPQvc4N+/AXgmXm0QEZFwva3AdSTOBL4EbDKzN/1tdwH3A8vM7KvA+8DVcWyDiIj0ELfgd869RvQpVOfF67giInJ4qT1zV0REwij4RUQCJnWDf+MyaPJHj/78TO+xiIikaPBvXAbP3U7XFIG6Cu+xwl9EJEWD/+XvQWtj922tjd52EZGAi+dwzuSp/bB/20VEBpGVGyp48I9vU1nTSElBDvMvnMTc2aUx239qBr+lg2uPvF1EZBBbuaGChSs20djqZVhFTSMLV2wCiFn4p2ZXT6TQP9x2EZFBYF9DM99/fmtX6HdqbG3nwT++HbPjpOYZf/7Rkbt18o9OfFtERHro6HB88NFBtlbVsaWylq2VdWytqqO6rjnqeyprGqM+11+pGfzn3eON4gm9wJuZ420XEUmg5rZ2dlY3dAv4bVX1NDS3hb12xLB0WtsdLe0dYc+VFOTErE2pGfwz/PI/K27ybtOHwWWPHdouIhIHtQdb2VJ1KOC3Vtbxzp4G2jrCq88X5WUxpTiPKSV5TC3JZ0pxHseMGs6zb1V26+MHyMlMZ/6Fk2LWztQMfvBCvjP4x05W6ItIzDjnqKhpZGtlHVtCQr4iQneMGRxfOIIpJflMLcnrCvsxI7Mi7rvzAq5G9RypnmP6RUT6qLW9g3f3NrCl4lDAb62qo7axNey1WRlpnFSc1y3gTxqXy/Bh/YvaubNLYxr0PSn4RUR8Dc1tbOsM98o6tlTVsmN3Q8Q+91EjhnUL+KkleUwYPYKM9ME/WDIgwX8w2S0QkUHEOcee+uaus/fOC6+79kfOimNHD2dK55l8SR5TivMpysvCW29q6AlI8OuMXySo2jscf993wAt3/2x+W1Ud+xpawl47LD2NE8eN9M7ii/OYUpLP5OJccrMzk9Dy+FHwi0jKaGxpZ/vuQ33xWyrr2L67jqbW8K6avOyMrrP3zjP54wtHMixj8HfVHKlgBD8OHp3mjePX6B6RlLC/oblbwG+tquO9vQ1EGDlJaUEOk7t11eQx/qicIdtVc6RSN/h7lmCu/dAv1YzCX2QI6ehwfPjxwbChk7vrmsJem55mTBo7slvATynJo2D4sCS0fPBK3eCPVIK5szSzgl9kUOqc5Rp60fVws1wnFx8K+Kkl+ZxQNJLsTBVj7E3qBn9tef+2i0hC1R5s9c7eQ0bVRJvlOjY3q1vATynJ49hRw0lLC2ZXzZFK3eDPHx+lUNv4xLdFJMCcc1TWNvldNYfKGZR/HHmW63GFI7pKGHSGfWFu5FmuMjCpG/zn3XOoZEMnFWoTiavW9g7e23ugW8Bvraqj5mD0Wa6hE6AGMstV+i91f8OhtXoAckbBZx5Q/75IjDQ0t7G9s6vGL2fwdnU9LW3hQyePGp7J1JL8bhddJ44ZGrNcU1HqBn9PZ9yq0BcZAOcce+ub2RJSymBrVR279h/ARRg6ecyo4T1KGQztWa6pKDjBX1uR7BaIDHqds1xDi5FtrayNOMs1M904sSi3W8CfVJxLXorNck1FwQn+OgW/SKjGlnberq4/dNG1qo7tVfVhy/4B5GZnhNWO/8TYYMxyTUVxC34z+zVwKbDHOTfN3zYKeAqYAOwCrnbOfRyvNnSjM34JsI8OtHQL+K2VdbwbZZZrSX621w/vB/zUkmDPck2Kjcu8OUe15d5IxBhXHYjnGf8TwE+B34ZsWwC87Jy738wW+I+/E5ej95y5+9F7cTmMyGDinOPDjxq7BfyWXma5Hhofn8fk4jyOGqFZrkm1cVn3pWPjUHUgbsHvnPuTmU3osfkK4Bz//hJgDfEI/s5fXKi2Rlj/Wzj5+pgfTiQZWto62FFdf6g/3q86WR9hluvwzlmuIfVqTizK1SzXwaC9FRqqoa4K6qvgxfnhhSVjXHUg0X38Rc65Kv/+bqAo2gvN7GbgZoBjjjmmf0d5+XuRK3K+8kMFvwxJtY2tbKvya9X4F13f2VNPa3t4X01hblaP2vHeAiGa5ZpgzsHB/V6Yd4Z6/W6or/Ru6/zbA3uBCH1uPcWw6kDSLu4655yZRf20zrnFwGKAsrKyPvxWQkT7BTVU92s3IonmnKOqtikk4GvZUnn4Wa6hM1ynlOQxNjc7CS0PmOYGP8hDQz0k3OuqoGE3tIePhgpjaTCiCPKKIbcY/v7f0HIg/HUxrDqQ6OCvNrNi51yVmRUDe+JylGjlGnIK4nI4kYFoa+/g3b0H2FpV263yZNRZruNyu110PWlcLiOygjMwLyHaWrzArt8dOdTr/GBvqe/b/rILvDDvDPXcYsgd133biLGQHvLn2LOPH2JedSDRf2ueBW4A7vdvn4nLUc67J/wXB3DsP8blcCK9OdDc5i0QEhLw23dHn+UaOmxySkkex2mW65Hp6PC7XXp0s/QM9YP7+ra/jOxDQd4z1PNKvNuR42DY8P63tbMffyiO6jGz3+NdyB1jZuXAvXiBv8zMvgq8D8RnKm3PX1xOATR+DDn5cTmcSKg99U3d+uK9tVyjz3IN7aqZWprHuLxsDZ3sj6a6yH3nnY87fzrC/ycVxtJhZFH3AM8dB7kl3bdlF3h9bfEy4+q4VhqI56ieeVGeOi9ex+wm9Be344/wu6s1ll9iqr3DsWv/gZDa8V7I72toDnttZrpxwtjcrmJkU4rzOKk4j/wczXKNqq25x1l5pDP13dDS0Lf95RwVEuCh3S4hoT6iENJSf6RTMDoIq7d4t++t1hKMMiBNre28vbu+W+347bvrOdgSYZZrVgaTQwJ+SkkeJ4zN1SzXTh0d3kiWsAuiPUL94P6+7S8jxw/ynqHeo089Uxe9O6V+8G9cBv/940OPtQSj9OLjAy3dAn5rVR3v7j1Ae4RprsX52WEFyQI7y9U5aK6LckG06lCoN1RDR/hcgzCWfii0e14QDe1fz8qLb7dLCkr94H/5e97krVBaglE4NMu156iaqtrwWa5pBicWjewW8JOL8xgVlFmurU3dw7vnKJfO/vTWg33b3/DR0Ue5dP6MGBOIbpdkSP3g1xKMgjfLdeee+m4Bv60y8izXnMx0Jhfndi3xN6U4j0njUnSWa0e71+0SdkG0R6g39rGkVuaI6KNcQrdlaEWtZEr94NcSjIFT19TarW781so6dkaZ5TpmZFa3Ga5TS/I4dvQI0of6LFfnoKkm+iiXzm0N1eDCr1OEScsIOTuPMMql83FWrrpdhoDUD/6IY/oN5tydtCZJbHTOcj00qsYrTPbhR1FmuY4ZweSS7vVqhuQs19bG6BdEQ/vTe3ZxRjN8TPRRLp2Ph4+GNF2cThWpH/w9x/RbmneG0xY+5E4Gr7b2Dt7bdyCstPDHEWa5DvNnuYZedD1pXN7gn+Xa3gYH9kS+IBq6rammb/sblht9lEvoJKOMgFynkC6D/F9CjISO6d+0HP7wVXjl+zD1SsjOS27bJIw3y7WerSEBv313Pc0RZrkWDM8MG1Uz6Ga5Ouf1kUe7INq57cAecOGfMUxaZkiARxjl0vlcVm78P5sMScEI/lDTPgd/Wwwf/hX+9CBc8P1ktyjQ9tQ3deuL31pZx9+jzHI9elSOF/DFhxbtLs5P8izXloNRxqL3mEXa3pf/YZpXt+VwQxdziyFnlLpd5IgEL/jN4KL74N/mwF9+Dqd8GUYfn+xWDWkrN1Tw4B/fprKmkZKCHOZfOIm5s0u7vaajc5Zrj9LCe+vDAzEjzThhXG63vvjJiZ7l2t4KDXt6CfUqaK7t2/6y8g4zysUP9ZFFkK6ZvBJ/wQt+gNJTYOYX4K3fwc/PhLamuBRCCoKVGypYuGJT1zqtFTWNLFixkQ8+OkBhbnZXwG+rqjvsLNfQejUnFI0kKyNOQyedg4MfHWaCUedolz30qUZ6+rDDj3Lp6nYZGZ/PIzIAwQx+gJLZXvC3xW95s1TX0eG47/9tC1ucu6m1g0dW7Qx7fXF+dkhfvNdlM/6onNgtENLcEGWCUY/SAH2pkY75xboijUcPCfmcozR8UYac4Ab/64+Fb2tt9JY962iD7Hz/p+DQ/b6MUY7zIsnJ0NHhKP+4kZ176tlR3cDO6np27mngnT0NYaEf6opZJV0BP7k4l9EjBzhpp701pMpizwlGIYHeXNe3/WXnR78g2hnqI4u610gXSSHB/ZsdbeZuUw2s/Frk5ywt5AshwhdDTTm8/cKh8q+1H8IzX/cWep/0GUjP8obOpQ8Lvz8ILtYNJODTDC6117gzYxklto9KN4Yft13NurxP85NrZ/d2QGj86DCzRv2fA/voW7dL1uFHuXTeDhvR/1+OSAoJbvBHm9E7bCScdAk01Xo/jTWH7rce8Ibl9XX6OnijOdbc5/0cjqV709jT/S+DjCzvQl/UL4vQ13Xej/K60NekD6MjbRj7muCD2lZ21bSzq6aFdz9qZef+VupbjRYyaCWDFjJpIQMwxuZmcWJRLp8YO5ITi3I5sWgkJ4zNZcdLv2bausfJMa/7ZLzt44HMx9k2aRzsOzr6KJfOgO9TjfQ0b7x5pAuioWfp6nYR6RNzkcbNDTJlZWVu7dq1sd1ptOXNLnssetdMe6u36ENTjf/T48vhpXujH69omte33Nbs3ba3eMu8tbf0cahf8ri0TCzaF8z+Hd7vZaCyCyJcEO3Rnz6iUN0uIgNgZuucc2U9twf3X9NAljdLz4QRo72fSN54PEpdoKPha3+Ovl/nvPDs+kII+XI43JeFf7+jrZna+gb21TbwcV0DNQ0HqG84wIGDB0nraGEYbQyzVjJp8+7TxoiMdvIyOxiZ4Rie1kZWWjtZtJHWEXqcZuhowzpaoWUA4X7UxPChiz27XjJz+r9fETkiwQ1+iP3yZpHqAvVlkWQz7yy6l6nzHR2OippGdlTXs2NvAzv31LOz+vB98GNzszjB75Y5sSjXvz+SguF9nKbf0RH9S6i9Bf7js/7Qxx7yj4Y73uzbMUQkoYId/LEWo0WSuwV8dYICPpq0NEjLjr560QU/HNiXnYgkjYI/1vrxv4hBFfADFaMvOxFJHAV/jEUqX3D5zJKugN+5p8G77SXgC3OzukbOnFDkjaRJasAfTqy7zEQkroI7qicOVm6oYMGKjTS1HqqwaAYZZrRGWK8VhljAi8iQolE9MeacY099M5sratlcUcfmylpe2VZNz0WenINW5xTwIjJoKPj7wDlvRuvmilo2V3pBv6Wyln0Nfan5Aga88b/Pj28jRUT6KNDBH6k//rKZJfx93wG2VNaypbLOP6Ovpa4pfFHuvOwMppbkM600j2ml+fzwhW3siVBmuKRAY9VFZPAIRPBHCniAhSs20uj3x1fUNPKtZW8yf/lbEQ8k4YsAAAauSURBVBflHj1iGNNK/ZAvyWdaqVdZMnQREOfoVqIYICczvet4IiKDQcoG/90rN7H0Lx+ElfaqqGnkzuVvgRktPZby63DQ0e4oyc9mamk+00q8lZ6mleZTlJfV60pPnYuP9LYoiYhIMiUl+M3sIuAnQDrwuHPu/lju/+6Vm/iPv3wQ9fmWdke0ao8GvL7wvAEfe+7sUgW9iAxqCa8FbGbpwM+AzwBTgHlmNiWWx1h6mNDvjfrjRSTVJaMI/KnAO86595xzLcCTwBWxPEBfZiYU5GSSk9l9eT/1x4tIECQj+EuB0BKW5f62bszsZjNba2Zr9+7dG9MG5GSms+jyqdz32emUFuRgQGlBDvd9drq6aUQk5Q3ai7vOucXAYvBm7sZqv0cNz+Tey6Z2BbyCXkSCJhln/BXA0SGPx/vbYua604+JuP3M40ex4Z4LFPYiEmjJCP43gBPMbKKZDQOuBZ6N5QF+MHc6151+DOn+8Mt0M647/RiW3nRGLA8jIjIkJbyrxznXZmZfB/6IN5zz1865LbE+zg/mTucHc6fHerciIkNeUvr4nXMvAi8m49giIkGXjK4eERFJIgW/iEjAKPhFRAJGwS8iEjBDYulFM9sLvD/At48B9sWwOUOBPnMw6DOnviP9vMc65wp7bhwSwX8kzGxtpDUnU5k+czDoM6e+eH1edfWIiASMgl9EJGCCEPyLk92AJNBnDgZ95tQXl8+b8n38IiLSXRDO+EVEJISCX0QkYFI6+M3sIjN728zeMbMFyW5PPJnZ0Wa22sy2mtkWM7sj2W1KFDNLN7MNZvZ8stuSCGZWYGbLzWy7mW0zs5SvN25m/+T/vd5sZr83s+xktynWzOzXZrbHzDaHbBtlZqvMbKd/e1QsjpWywZ+IRd0HmTbgn51zU4DTgdtS/POGugPYluxGJNBPgP90zp0EzCTFP7uZlQK3A2XOuWl45dyvTW6r4uIJ4KIe2xYALzvnTgBe9h8fsZQNfhKwqPtg4pyrcs6t9+/X44VByi81ZmbjgUuAx5PdlkQws3zgbOBXAM65FudcTXJblRAZQI6ZZQDDgcoktyfmnHN/Aj7qsfkKYIl/fwkwNxbHSuXg79Oi7qnIzCYAs4G/JrclCfGvwJ1AR7IbkiATgb3Ab/zurcfNbESyGxVPzrkK4CHgA6AKqHXO/VdyW5UwRc65Kv/+bqAoFjtN5eAPJDMbCfwB+KZzri7Z7YknM7sU2OOcW5fstiRQBnAy8HPn3GzgADH67/9g5fdrX4H3pVcCjDCz65LbqsRz3tj7mIy/T+Xgj/ui7oONmWXihf5S59yKZLcnAc4ELjezXXhdeXPM7D+S26S4KwfKnXOd/5tbjvdFkMrOB/7unNvrnGsFVgCfTHKbEqXazIoB/Ns9sdhpKgd/3Bd1H0zMzPD6fbc55x5JdnsSwTm30Dk33jk3Ae/P9xXnXEqfCTrndgMfmtkkf9N5wNYkNikRPgBON7Ph/t/z80jxC9ohngVu8O/fADwTi50mZc3dREjUou6DyJnAl4BNZvamv+0uf31jSS3fAJb6JzTvATcmuT1x5Zz7q5ktB9bjjV7bQAqWbjCz3wPnAGPMrBy4F7gfWGZmX8UrTX91TI6lkg0iIsGSyl09IiISgYJfRCRgFPwiIgGj4BcRCRgFv4hIwCj4JbDMrCFG+1lkZt/uw+ueMLPPx+KYIkdCwS8iEjAKfgk8MxtpZi+b2Xoz22RmV/jbJ/g1758wsx1mttTMzjezP/v10U8N2c1MM/sff/tN/vvNzH7qrwnxEjA25Jj3mNkbfn35xf6MVJGEUPCLQBNwpXPuZOBc4OGQIP4E8DBwkv/zBeAfgW8Dd4XsYwYwBzgDuMfMSoArgUl460FcT/f6Mj91zv2DX18+B7g0Tp9NJEzKlmwQ6QcDfmRmZ+OVdy7lUPnbvzvnNgGY2Ra8RTGcmW0CJoTs4xnnXCPQaGar8daDOBv4vXOuHag0s1dCXn+umd2JV1t+FLAFeC5un1AkhIJfBL4IFAKnOOda/WqfnUv7NYe8riPkcQfd//30rH0StRaKv2zg/8FbUepDM1sUcjyRuFNXjwjk49X1bzWzc4FjB7CPK8ws28xG4xXaegP4E3CNvyZwMV43EhwK+X3++gka6SMJpTN+EVgKPOd336wFtg9gHxuB1cAY4PvOuUozexqv338rXmnh/wFwztWY2b8Bm/FWVXrjyD+CSN+pOqeISMCoq0dEJGAU/CIiAaPgFxEJGAW/iEjAKPhFRAJGwS8iEjAKfhGRgPn/dGxudUunrzQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "lambda_vec, error_train, error_val = validationCurve(X_poly, y, X_poly_val, yval)\n",
    "\n",
    "pyplot.plot(lambda_vec, error_train, '-o', lambda_vec, error_val, '-o', lw=2)\n",
    "pyplot.legend(['Train', 'Cross Validation'])\n",
    "pyplot.xlabel('lambda')\n",
    "pyplot.ylabel('Error')\n",
    "\n",
    "print('lambda\\t\\tTrain Error\\tValidation Error')\n",
    "for i in range(len(lambda_vec)):\n",
    "    print(' %f\\t%f\\t%f' % (lambda_vec[i], error_train[i], error_val[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*You should now submit your solutions.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Submitting Solutions | Programming Exercise regularized-linear-regression-and-bias-variance\n",
      "\n",
      "Use token from last successful submission (nehatrical@gmail.com)? (Y/n): Y\n",
      "                                  Part Name |     Score | Feedback\n",
      "                                  --------- |     ----- | --------\n",
      "Regularized Linear Regression Cost Function |  25 /  25 | Nice work!\n",
      "     Regularized Linear Regression Gradient |  25 /  25 | Nice work!\n",
      "                             Learning Curve |  20 /  20 | Nice work!\n",
      "                 Polynomial Feature Mapping |  10 /  10 | Nice work!\n",
      "                           Validation Curve |  20 /  20 | Nice work!\n",
      "                                  --------------------------------\n",
      "                                            | 100 / 100 |  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "grader[5] = validationCurve\n",
    "grader.grade()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4  Optional (ungraded) exercise: Computing test set error\n",
    "\n",
    "In the previous part of the exercise, you implemented code to compute the cross validation error for various values of the regularization parameter $\\lambda$. However, to get a better indication of the model’s performance in the real world, it is important to evaluate the “final” model on a test set that was not used in any part of training (that is, it was neither used to select the $\\lambda$ parameters, nor to learn the model parameters $\\theta$). For this optional (ungraded) exercise, you should compute the test error using the best value of $\\lambda$ you found. In our cross validation, we obtained a test error of 3.8599 for $\\lambda = 3$.\n",
    "\n",
    "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "for lambda: 0\n",
      "theta [  5.60876297   5.60876297   9.50243647  18.91963388  30.55295805\n",
      " -28.04449915 -78.68088346   6.15757986  64.11142117  22.26158208]\n",
      "for lambda: 0.001\n",
      "theta [ 11.2769312   -0.05930445  11.36936796  15.72473807  11.97003612\n",
      " -17.21130387 -20.35233803  12.10772092  10.3361476   -3.57847056]\n",
      "for lambda: 0.003\n",
      "theta [ 11.24314493  -0.02553791  11.76272942  13.81482676   8.76914847\n",
      "  -9.15968966 -10.69038771   3.77140471   1.38969715  -3.50455321]\n",
      "for lambda: 0.01\n",
      "theta [ 1.12220551e+01 -4.46329259e-03  1.22180406e+01  1.23918175e+01\n",
      "  5.75975893e+00 -4.62022317e+00 -4.95529145e+00 -7.56886996e-01\n",
      " -2.06751845e+00 -2.39285361e+00]\n",
      "for lambda: 0.03\n",
      "theta [ 1.12174098e+01  1.83793625e-04  1.24496936e+01  1.10209909e+01\n",
      "  3.61840790e+00 -2.27814898e+00 -2.35493814e+00 -1.75497100e+00\n",
      " -1.98047046e+00 -1.62668939e+00]\n",
      "for lambda: 0.1\n",
      "theta [ 1.12177864e+01 -2.08174660e-04  1.20332289e+01  9.02462865e+00\n",
      "  2.46364001e+00 -3.10710871e-02 -7.51961600e-01 -1.11332779e+00\n",
      " -7.18641708e-01 -1.25019353e+00]\n",
      "for lambda: 0.3\n",
      "theta [ 1.12174872e+01  1.05083057e-04  1.06843469e+01  7.06170665e+00\n",
      "  2.72159047e+00  1.48307934e+00  5.34485092e-01  2.83812109e-02\n",
      "  3.72043345e-01 -5.75668500e-01]\n",
      "for lambda: 1\n",
      "theta [1.12175505e+01 4.05193684e-05 8.58518772e+00 5.21268402e+00\n",
      " 3.59110702e+00 2.28293968e+00 1.76030269e+00 1.06813586e+00\n",
      " 1.10763224e+00 3.94006115e-01]\n",
      "for lambda: 3\n",
      "theta [ 1.12175858e+01 -1.34975369e-06  6.65674491e+00  3.89422201e+00\n",
      "  3.75900250e+00  2.25303961e+00  2.20642560e+00  1.33196648e+00\n",
      "  1.35011961e+00  7.62008780e-01]\n",
      "for lambda: 10\n",
      "theta [ 1.12175888e+01 -9.15740171e-07  4.36656650e+00  2.39947073e+00\n",
      "  2.90177132e+00  1.50155092e+00  1.86591961e+00  8.98138608e-01\n",
      "  1.19375122e+00  5.09233108e-01]\n",
      "lambda\t\tTrain Error\tValidation Err\tTest Error\n",
      " 0.000000\t0.028901\t55.043068\t17.102619\n",
      " 0.001000\t0.112439\t9.828218\t15.130962\n",
      " 0.003000\t0.170841\t16.305636\t16.965853\n",
      " 0.010000\t0.221527\t16.944736\t15.914861\n",
      " 0.030000\t0.281857\t12.828376\t13.022547\n",
      " 0.100000\t0.459315\t7.586964\t9.162786\n",
      " 0.300000\t0.921768\t4.636832\t5.024850\n",
      " 1.000000\t2.076187\t4.260628\t2.725978\n",
      " 3.000000\t4.901509\t3.822929\t3.859979\n",
      " 10.000000\t16.092218\t9.945512\t17.005350\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3zU1Z3/8ddnJslM7oEkhCTIHUEgN2W9dbWKVt3iBdt6t9puH/Xn9iK7XS/YtTbdbSu9+rOP9vezru1Kt+wqPxStxt0WFbYq9QJIEu4oBAhJyAUyk/vczu+P7yRkciEhzGTCzOf5eOQxM9/5Xs43gfecOd/zPUeMMSillIoftmgXQCml1PjS4FdKqTijwa+UUnFGg18ppeKMBr9SSsWZhGgXYDRycnLMzJkzo10MpZQ6q2zdurXZGJM7cPlZEfwzZ85ky5Yt0S6GUkqdVUTk0FDLtalHKaXijAa/UkrFGQ1+pZSKM2dFG79SKvy8Xi+1tbV0d3dHuyjqDDmdTqZNm0ZiYuKo1tfgVypO1dbWkp6ezsyZMxGRaBdHjZExhpaWFmpra5k1a9aotondpp6qtfDkYijPsh6r1ka7REpNKN3d3WRnZ2von+VEhOzs7NP65habNf6qtfDqA+Dtsl67jlivAYpvjV65lJpgNPRjw+n+HWOzxv/mP58M/V7eLmu5UkrFudgMflft6S1XSo27lpYWSktLKS0tZerUqRQWFva99ng8p9x2y5YtPPDAA+NU0tgTm009mdOs5p2hliulxuTlj47ykz/upa61i4KsZB66dj7LywrHvL/s7Gy2b98OQHl5OWlpaTz44IN97/t8PhISho6oJUuWsGTJkjEfO97FZo3/qschMTl0WWKytVwpddpe/ugoj75UzdHWLgxwtLWLR1+q5uWPjob1OF/60pe4//77ueiii3j44Yf54IMPuOSSSygrK+PSSy9l7969AGzatInrr78esD40/vZv/5YrrriC2bNn84tf/CKsZYpFsVnj772A+9J9gIGMQri6XC/sKjWMmSsrTnubLq+fv39hO3//wvZh16lZtey091tbW8vmzZux2+243W7efvttEhISeOONN/j2t7/Niy++OGibPXv2sHHjRtra2pg/fz5/93d/N+o+7fEoNoMfrJB/dQV4O+EbH0JSarRLpJQahVtuuQW73Q6Ay+Xi3nvvZf/+/YgIXq93yG2WLVuGw+HA4XAwZcoUjh07xrRp2rQ7nNgNfqXUqI1UM//Uqrc42to1aHlhVjLvrlwa1rKkpp6spH3nO9/hyiuvZP369dTU1HDFFVcMuY3D4eh7brfb8fl8YS1TrInNNn6lVFg9dO18khPtIcuSE+08dO38iB7X5XJRWGhdQH7uuecieqx4osGvlBrR8rJCnvhcEYVZyQhWTf+JzxWdUa+e0Xj44Yd59NFHKSsr01p8GIkxJtplGNGSJUvMmCZi+UG+1cb/7Tpt41dqgN27d3PeeedFuxgqTIb6e4rIVmPMoH6vWuNXSqk4o8GvlFJxRoNfKaXijAa/UkrFGQ1+pZSKMxr8SikVZyIa/CJSIyLVIrJdRLYEl00WkQ0isj/4OCmSZVBKTVwNDQ3cfvvtzJkzhwsuuIDPfvaz7Nu3L6LHXL16NXfccUfIsubmZnJzc+np6Rlym+eee45vfOMbADz99NP87ne/G7ROTU0NixcvPuWxa2pq+I//+I++19EaXno8avxXGmNK+/UlXQm8aYyZB7wZfK2UmujCPJ2pMYabb76ZK664gk8++YStW7fyxBNPcOzYsZD1wn3j1s0338yGDRvo7OzsW7Zu3TpuuOGGkKEfhnP//fdzzz33jOnYA4N/yZIlURlNNBpNPTcBq4PPVwPLo1AGpdTp6J3O1HUEMCenMz2D8N+4cSOJiYncf//9fctKSkq47LLL2LRpE5dddhk33ngjCxcupLu7my9/+csUFRVRVlbGxo0bAdi5cycXXnghpaWlFBcXs3//fjo6Oli2bBklJSUsXryYF154IeS4GRkZfPrTn+bVV1/tW/b8889zxx138Oqrr3LRRRdRVlbG1VdfPehDCKxhoH/6058CsHXrVkpKSigpKeFXv/pV3zo1NTVcdtllnH/++Zx//vls3rwZgJUrV/L2229TWlrKk08+GTK89PHjx1m+fDnFxcVcfPHFVFVV9R0v3MNOR3qQNgP8SUQM8GtjzDNAnjGmPvh+A5A31IYich9wH8D06dMjXEyl4lx55ulv4+2Cl75q/Qy7X9ewb+3YsYMLLrhg2Pe3bdvGjh07mDVrFj/72c8QEaqrq9mzZw/XXHMN+/bt4+mnn2bFihXcddddeDwe/H4/r7/+OgUFBVRUWENNu1yDy3DHHXewZs0abrvtNurq6ti3bx9Lly7F7Xbz3nvvISI8++yz/PjHP+ZnP/vZsGX88pe/zC9/+Usuv/xyHnroob7lU6ZMYcOGDTidTvbv388dd9zBli1bWLVqFT/96U957bXXAGtegV7f/e53KSsr4+WXX+att97innvu6ZuoJtzDTke6xv/Xxpjzgb8Bvi4il/d/01jjRQw5ZoQx5hljzBJjzJLc3NwzK8VZMCyFUirUhRdeyKxZswB45513uPvuuwFYsGABM2bMYN++fVxyySX88Ic/5Ec/+hGHDh0iOTmZoqIiNmzYwCOPPMLbb79NZubgD7Vly5bx7rvv4na7Wbt2LZ///Oex2+3U1tZy7bXXUlRUxE9+8hN27tw5bPlaW1tpbW3l8sutWPviF7/Y957X6+WrX/0qRUVF3HLLLezatWvE833nnXf69rF06VJaWlpwu9195XU4HOTk5PQNO30mIlrjN8YcDT42ish64ELgmIjkG2PqRSQfaIxcCU5v5nml4tYpauaA1aY/5HSm58A/7BjTIRctWsS6deuGfb//8MzDufPOO7nooouoqKjgs5/9LL/+9a9ZunQp27Zt4/XXX+exxx7jqquu4vHHQ2ffS05O5rrrrmP9+vU8//zz/PznPwfgm9/8Jt/61re48cYb2bRpE+Xl5WM6tyeffJK8vDwqKysJBAI4nc4x7adXuIedjliNX0RSRSS99zlwDbAD+ANwb3C1e4FXIlUGpVSYRGA606VLl9LT08MzzzzTt6yqqoq333570LqXXXYZa9asAWDfvn0cPnyY+fPnc+DAAWbPns0DDzzATTfdRFVVFXV1daSkpHD33Xfz0EMPsW3btiGPf8cdd/Dzn/+cY8eOcckllwChw0CvXr16yO16ZWVlkZWVxTvvvAPQV77e/eTn52Oz2fj3f/93/H4/AOnp6bS1tQ25v/7nuGnTJnJycsjIyDhlGcYqkk09ecA7IlIJfABUGGP+G1gFfEZE9gNXB18rpSay4lvhhl9YNXzEerzhF2c0namIsH79et544w3mzJnDokWLePTRR5k6deqgdb/2ta8RCAQoKiritttu47nnnsPhcLB27VoWL15MaWkpO3bs4J577qG6urrvgu/3vvc9HnvssSGP/5nPfIa6ujpuu+02RKzWgfLycm655RYuuOACcnJyRjyHf/u3f+PrX/86paWl9B/p+Gtf+xqrV6+mpKSEPXv29H17KS4uxm63U1JSwpNPPhmyr/LycrZu3UpxcTErV64c8YPnTMT4sMwF4O2AR4+CIy38BVPqLKbDMscWHZZZKaXUsDT4lVIqzmjwK6VUnNHgV0qpOKPBr5RScUaDXyml4kykx+pRSqkhtbS0cNVVVwHW8Mx2u53e4Vk++OADkpKSTrn9pk2bSEpK4tJLL414WWONBr9SalQqDlTw1LanaOhoYGrqVFacv4Jls5eNeX/Z2dl9g5CVl5eTlpbGgw8+OOrtN23aRFpamgb/GGhTj1JqRBUHKijfXE59Rz0GQ31HPeWby6k4UBHW42zdupVPf/rTXHDBBVx77bXU11sD+f7iF79g4cKFFBcXc/vtt1NTU8PTTz/Nk08+SWlp6ZDDPKjhaY1fKUXR6qLT3qbb383Kt1ey8u3h51Kqvrd61PszxvDNb36TV155hdzcXF544QX+6Z/+id/+9resWrWKgwcP4nA4aG1tJSsri/vvv/+0vyUoiwa/UmpC6OnpYceOHXzmM58BwO/3k5+fD1hj3Nx1110sX76c5ct17qYzpcGvlBqxZn7Numuo76gftDw/NZ8/feFPYSmDMYZFixbxl7/8ZdB7FRUV/PnPf+bVV1/lBz/4AdXVo/8moQbTNn6l1IhWnL8Cpz10THmn3cmK81eE7RgOh4Ompqa+4Pd6vezcuZNAIMCRI0e48sor+dGPfoTL5aK9vf2UQxyrU9PgV0qNaNnsZZRfWk5+aj6CkJ+aT/ml5WfUq2cgm83GunXreOSRRygpKaG0tJTNmzfj9/u5++67++bbfeCBB8jKyuKGG25g/fr1enF3DHRYZqXilA7LHFt0WGallFLDipPgn/jfapRSarzEdvCLTrau1KmcDU29amSn+3eM7eBXSg3L6XTS0tKi4X+WM8bQ0tKC0+kceeUg7cevVJyaNm0atbW1NDU1Rbso6gw5nU6mTZs26vU1+JWKU4mJicyaNSvaxVBRoE09SikVZzT4lVIqzmjwK6VUnNHgV0qpOKPBr5RScUaDXyml4kzEg19E7CLykYi8Fnw9S0TeF5GPReQFETn1jMpKKaXCajxq/CuA3f1e/wh40hgzFzgBfGUcyqCUUiooosEvItOAZcCzwdcCLAXWBVdZDeg8akopNY4iXeP/38DDQCD4OhtoNcb4gq9rgcKhNhSR+0Rki4hs0VvKlVIqfCIW/CJyPdBojNk6lu2NMc8YY5YYY5bk5uaGuXRKKRW/IjlWz6eAG0Xks4ATyACeArJEJCFY658GHI1gGZRSSg0QsRq/MeZRY8w0Y8xM4HbgLWPMXcBG4AvB1e4FXolUGZRSSg0WjX78jwDfEpGPsdr8fxOFMiilVNwal2GZjTGbgE3B5weAC8fjuEoppQaLjzt3dYYhpZTqE+PBr3PuKqXUQDEe/EoppQbS4FdKqTijwa+UUnFGg18ppeKMBr9SSsUZDX6llIozGvxKKRVnNPiVUirOaPArpVSc0eBXSqk4o8GvlFJxRoNfKaXijAa/UkrFGQ1+pZSKMxr8SikVZzT4lVIqzmjwK6VUnNHgV0qpOBMnwa9z7iqlVK/YDn7ROXeVUmqg2A5+pZRSg2jwK6VUnNHgV0qpOKPBr5RScUaDXyml4kzEgl9EnCLygYhUishOEflecPksEXlfRD4WkRdEJClSZVBKKTVYJGv8PcBSY0wJUApcJyIXAz8CnjTGzAVOAF+JYBmUUkoNMGLwi4hNRC493R0bS3vwZWLwxwBLgXXB5auB5ae7b6WUUmM3YvAbYwLAr8aycxGxi8h2oBHYAHwCtBpjfMFVaoHCYba9T0S2iMiWpqamsRxeKaXUEEbb1POmiHxe5PRuhTXG+I0xpcA04EJgwWls+4wxZokxZklubu7pHFYppdQpjDb4/xfw/wCPiLhFpE1E3KM9iDGmFdgIXAJkiUhC8K1pwNHTKbBSSqkzM6rgN8akG2NsxphEY0xG8HXGqbYRkVwRyQo+TwY+A+zG+gD4QnC1e4FXxl58pZRSpyth5FUsInIjcHnw5SZjzGsjbJIPrBYRO9YHzFpjzGsisgt4XkS+D3wE/GYM5VZKKTVGowp+EVkF/BWwJrhohYh8yhjz6HDbGGOqgLIhlh/Aau9XSikVBaOt8X8WKA328EFEVmPV1ocNfqWUUhPT6dzAldXveWa4C6KUUmp8jLbG/0PgIxHZCAhWW//KiJVKKaVUxIwY/CJiAwLAxVjt/ACPGGMaIlkwpZRSkTFi8BtjAiLysDFmLfCHcSiTUkqpCBptG/8bIvKgiJwjIpN7fyJasnAyOtm6Ukr1Gm0b/23Bx6/3W2aA2eEtTrjpZOtKKTXQaNv4VxpjXhiH8iillIqw0Y7O+dA4lEUppdQ4iI82fqWUUn1ivI1fKaXUQKMKfmPMrEgXRCml1Pg4ZVOPiDzc7/ktA977YaQKpZRSKnJGauO/vd/zgQOyXRfmsiillBoHIwW/DPN8qNdKKaXOAiMFvxnm+VCvlVJKnQVGurhbEpxbV4DkfvPsCuCMaMmUUkpFxCmD3xhjH6+CKKWUGh+nMxGLUkqpGKDBr5RScUaDXyml4owGv1JKxRkNfqWUijMa/EopFWc0+JVSKs7ESfDrTcZKKdUrtoNfRxNSSqlBIhb8wdm6NorILhHZKSIrgssni8gGEdkffJwUqTIopZQaLJI1fh/wj8aYhcDFwNdFZCGwEnjTGDMPeDP4Wiml1DiJWPAbY+qNMduCz9uA3UAhcBOwOrjaamB5pMqglFJqsHFp4xeRmUAZ8D6QZ4ypD77VAOQNs819IrJFRLY0NTWNRzGVUiouRDz4RSQNeBH4e2OMu/97xhjDMF1ujDHPGGOWGGOW5ObmRrqYSikVNyIa/CKSiBX6a4wxLwUXHxOR/OD7+UBjJMuglFIqVCR79QjwG2C3Mebn/d76A3Bv8Pm9wCuRKoNSSqnBRpqB60x8CvgiUC0i24PLvg2sAtaKyFeAQ8CtESyDUkqpASIW/MaYdxj+FqqrInVcpZRSpxbbd+4qpZQaRINfKaXiTMwG//df+xIlUzMomnkOJS9cxvdf+1K0i6SUUqNScaCCa9ZdQ/HqYq5Zdw0VByrCuv+YDP7vv/YlXmjeQkAERAiI8ELzFg1/pdSEV3GggvLN5dR31GMw1HfUU765PKzhH5PB//+at4AMuK4sYi1XSqkJqMvXxZaGLfzgvR/Q7e8Oea/b381T254K27Ei2Z0zagKnuVwppcaTMYba9loqmyqpbKyksqmSfSf24Tf+Ybdp6GgI2/FjMvhtDB3yMfn1Rik14XV6O9nZstMK+qZKqpqqON59PGQdm9hYMHkBB1oP4Ql0DdpHRmL4hq6JyeC/sKuL95KTQ5t7jOHCrsG/TKWUCidjDEfajoSE/FC1+UmOSZTkllAypYTinGIW5yyms8fO1b9+EjN5LWLzntxnIJGexmvDVsaYDP5DjuQh2/gPOZKjUyClVMzq9Hayo3kHVc1VVDZWUtU8uDZvFzvnTT6P4txiSnJLKMouxniz2d3Qxs46F/93i5td9Zs55u4Bikjo8eHI/SOS2IrxZtHTdC0d7kVhK3NMBn+DfegbhodbrpRSo2GM4XDb4b6afG/bfMCENi5Pdk7uC/mFk4tI8s/gk2MedtW5+d12N7vr99De4xu0/9QkO16/weMuw+cuC3mvMCt8FdeYDP5MRxatPa1DLldKqdHq9HZS3VzdF/RVTVWc6DkRsk5vbb4kt4RzMxeTFJhFY0squxvaWFfl5uPGFnyB5kH7zstwsDA/g4UFGSwqyGRhfgbTJ6fwh8o6Hn2pmi7vyaah5EQ7D107P2znFZPBbw3zP/rlSilljOGQ+1BI2/z+1v1D1uZLckuYmbaQ5MBs2l357DvWw+s73fxraxdwJGR9EZiTm8rCgkwWFWT0hX1OmmPIciwvKwTgJ3/cS11rFwVZyTx07fy+5eEQk8Hv9rhPa7lSKv50eDus2nywO2VVcxWuHlfIOgmSwPzJC5iRdh4pgTl0t5/DoWMO/mdPG3/o8gJe4HDf+o4EGwvyM0ICfsHUdFKSTi9ql5cVhjXoB4rJ4J+aOpX6jvohlyul4o8xhhp3TV9tvrKpko9PfIwZMAHgZGc2M1LPI525eDrO4WhjNtv3evjA31vr7wj+wOTUpJCAX1SQwczsVBLsE7/jeEwG/4rzV1C+uTzk7jen3cmK81dEsVRKqfHS7mnva5vvbbYZ+I3fLnYKU+aSIXPxdp7DscY8DjU5ORQymryVITOyU1jYW5MvyGBhfiZ5GQ5kYO/Bs0RMBv+y2csA+PHGhzieYCcpYCi/rLxvuVIqdgRMgBpXaG3+k9ZPBtXm0xMmk2Wfi79zBk3NeRw/kUerSQxZJ8lu49ypaVYtPj+DhQWZnJefTrozdL2zXUwGP1jhf/iV+/k/k7PwCH3jXGj4K3V2a/O0Ud1UTWWzFfLVTdWDavM2Esi0zcB0z6Dl+FS6286hzZdFXb/afIYzoa/23luTn5ObRlLCxG+qOVMxG/wVByr4TVaG9ULEGuHune8AGv5KnS0CJsBB18GQfvND1eaTmIT0zMDtKsDfOR1/dyGufrX5wqxkzgtpqslg2qTks7ap5kzFbPA/9d4T9NhCP7m7jZen3ntCg1+pCcrtcVu1+d5+881VtHnaQtYRY8fmnUZ32zT8XTPwd03H+DIBwW4T5uamsWjByYBfWJBBVkpSdE5ogorZ4G/wtA4etqF3uVIq6gImwIHWAyEXYD9xfTJoPePLxNc5HX+X9RPoLgCTSGqSnaL8kwG/qCCTeXlpOBPtUTibs0vMBv9Un5/6xMGnN9U3/LCnSqnIcfW4Tva0aaykqqmaDl97yDomYCfQXdgX8v6uGRhfJlPSHSEBv7AggxmTU7DZ4rOp5kzFbPCv6LFTbg/Q3a+5xxkIsKJHawNKRZo/4OeA60BfyG89tp0j7TWD1gt4M0+GfOcMjKeAWdmZfUMY9IZ9bvrQd7mqsYnZ4F922ePwxxX8S85kOmw20v1+/qm1g2VX/yTaRVMq5rh6XFQ1VfFR43ber/uIvSd20hPoDFnHBBKCtflz8HfNwO6ZyYIp57Cw8OQNUGO5y1Wdvtj9DRffyrKXvkqHzca/5EzmSk/ACv3iW6NdMqXOav6An49bP+bD+u28e2QLu45Xc9x7dNB6AW+W1cOmazqpZjYLc86jaEZ2Xy1+Vs7ZcZdrLIrd4A+a5bUmM6jJytfQV2oMTnSd4O0jW/nz4a3saK6ioWcffkLnhDWBBPzdhQS6ZjDJPpeFk4somzWjb+TJs/ku11gUN8F/0NOKMUb/8Sl1Ch6fj/+pqWbToQ/Z0VzF0a499MixQesFPFmY7hnkJJ7L/KzFXDRtMcWFOSzITycjxu5yjUUxH/zZ/gBpgQBtNh/Hu4+TnZwd7SIpNSF0efxsOXKEt2q2UNm4ndquPXRKDWLrObmSWLV58ZxDdsI8zs1azCWFF3DR9JnMnRIfd7nGoogFv4j8FrgeaDTGLA4umwy8AMwEaoBbjTEnhttHWMoBzPR62eFwUOOu0eBXcel4h4fqoyd4u6aKj5oqOdyxm045gC2p3wQhduv/i/gmMzlhHnMzFnNRQRlXzSllVnaGflseT1Vr4c1/BlctZE6Dqx4Pa1N1JGv8zwG/BH7Xb9lK4E1jzCoRWRl8/UhEjl61tu/pLI+PHQ4HB10HuSDvgogcTqmJwBjDkeNd7Kxzsa32CFuPVVLTvosu2wHsybWIzWOt6AAbICaRTNts5mQs4sL8Mq6ZeyFzswuieg5xr2otvPoAeLus164j1msIW/hHLPiNMX8WkZkDFt8EXBF8vhrYRCSCv/cXFzSz9wLv/go49wthP5xS0eDxBdh3rI1d9W52HD3B9obdHGzfiTexBnvyYWxJLVYVPv3kf/QUmcKs9IUsmVrKVbMvZHHuAhJt2iYfVX4vtB8Ddz201cPrD50M/V7eLusbwEQP/mHkGWN6Z0hpAPKGW1FE7gPuA5g+ffrpHeXNfw75xbUEu4ytbvqQP627hhXnr9DxetRZxdXlZXe9m511bnbVualuOMqh9l0YxyHsyYes2rzTC07ojXE7DqanzadsSgmXT19CyZRicpJzonoeccUY6Gyxwrw31NsaoK3OenQHHzuagFFMC+uqDVvRonZx1xhjRGTYszXGPAM8A7BkyZLTmyy33y+oIjWFF9PTrBe9o3RuLgd0lE418RhjqHd19wX8rnoXO+pOUN91AHvykWDIH8aWeZykzNBtcxwFlEwp5qKC8ynJLWHepHlam4+UnvZgkPcP9X7h7q6H9gbwe0bel9ggNQ8y8iE9Hw7+D3g6Bq+XOS1sxR/v4D8mIvnGmHoRyQcaI3KUzGlWuxjw1KSswaN0+rt5attTGvwqqnz+AJ80dbCr3sWuumBtvt6Nq+e4Fe7Jh7EnH8Y+pZZUmzdk2ySbk8U5iymbUkJJbgnFucXacSEcfB4rsNsahg51dzDYB4wYOixnlhXmvaGeng/pU0OXpU4Be78oHtjGD5CYbF3gDZPxDv4/APcCq4KPr0TkKFc93veLa0gYemyeho6GiBxaqaF09PjY0+AOCfg9DW14fF5szvpgTf4I9vzDpCUdH7T9OenTKQ0GfG9tPsEW872xwycQCDa7DGhmGRjqnc0j7wsgwXkyyAeGekaB9Zg2FZJSTr+sve34Z2OvHhH5T6wLuTkiUgt8Fyvw14rIV4BDQGRupe33ixt2lE6deF1FSGNb/6Ya67GmpQNjQOxt2FOs5hp74WHSk4+ChNbmkxOSKcop6qvJF+cWM9k5OUpncxbodg/ddt77uvcn4B15X2KHtLzQAE+fCukFocucWUMO+x42xbdGdKSBSPbquWOYt66K1DFDBH9xKw5UUP7ud+ju90d32B068bo6Y/6Aoaaloy/ge8O+ub33BihfsDZ/mOSCwySl1uK3twzaz4yMGZTknmyymZs1V2vzAL6eAbXyoWrqDeBpH3lfAMmT+gV4/2aXfqGemgu22B/BN+b/dS2bvQz2vM5TdW9Rn2AHEZakFGr7vjot3V4/exvaggFvtcnvaWij03NyfgdJcGNPPkx6/hGSM2rpsR0mwMkKhx9ISUihKKeI4txiSqeUUpRTxCTnpCicURQFAlZPlkEXRAeEeufgD8khJSQHg3xgqA9oU090Rva8ziIxH/xUrWXZthdZ5utiV1IitxXmU+X6mK7ta0guvSvapVMT0IkOT0jA76p380lTB/5A/85lVm0+u6COtMyj9NgP0OE/2T7ce1luZsbMvnb5ktwS5mbNxR6rNUpjoMc9zAXR+pOh3n4MAr6R9yf2k6E98IJo//Z1R0Zkm11iUOwH/5v/DD7rv+FCj5ei7h6qnQ7++90nuFmDP6713uU6sFdNvat70Lr2RBfnFDSSkVWHN+EgTZ5P8BkvHuA4gD9Ym88tOtlsk1NMljNrvE8rMrzdoeE9sJdLb3u6t3PkfQGkZA/fy6X3JzUnLppdoiH2g/3JRFEAABEMSURBVH/ATQ+3trVT7XSwNtHDzVEqkhp/Hl+A/Y1tIQG/u85NW8/gmmdyomFm4QmyJtXjT6yh0bOXlp5GWoFWP1abDVZtviS3hJJgl8o5mXPOvtp8wG81uwy6IDog1LtGOaRWYurwvVz6L0vQGbWiKfaDv1+ffoDrOjr58eRJ7HA42Nmyk0XZi6JYOBUJ7m6v1UTTr1fN/sY2vP7B9wHmpDmYV+Bj0uR6TFINTd59HHTvozbgobYbeoedT0tM62ub770Im+nIHLS/CcMY6G4dvpdL77L2Y2BGMQ+1LaFf7XyIXi69rx3p2uxyFoj94O/Xpx/AaQw3tXfw+8x01u5dy/cu/V6UC6jGqvcu15O9alzsqndz5HjXoHVFYHZOKvPzk8nJbkKchzju+5g9J6rZ0XkMXKHrz86c3RfwJbklzM6cPXFq896u4S+I9m9P9w3+PQwpJWf4Xi69r1OywaZDMMeK2A/+gTdDiI1b29r4fWY6rx94nX9c8o9kJGVEt4xqRD5/gAPNHcGmGldfTf5E5+C+2UkJNhZMTWdRQQbTcnqwJVshv+tENe+37MZ7LHSbtMS0vv7yJbklFOUURac27/dBR+PQF0T7L+tuHd3+ktKH7+XS/yajhKTInpeacGI/+CH0Zojqdcx68Stc5PHzflI3r37yKnedpxd5JxLrLtc2dvUL+D0NbfT4AoPWzUpJZFFwDtd5U504Uxto9u5lR8sGPmiq5LWDg0cFmZM5h5Ip1sXXktwSZmfNxiYRrM0aY7WRD3dBtHdZRyOYwec4iC2xX4AP0cul9z1HeuTOSZ3V4iP4+1v8efjgGW5tqeL9vFzW7l3LnQvu1EkmoqSxrTukLX5XnZuDwbtcBzpncjIL8zNYmJ/Jwvx0cid1Ud+9h6rm/6GyqZJ1u3bjG9BNMD0xPaRdvii3KLzf8Dydw/RFH3AXqb9n5H0h1rgtp+q6mJ4PyZO12UWdkfgLfhG47gmu/NelpPsDHHAdoPh3xeSn5utwzWP08kdH+ckf91LX2kVBVjIPXTuf5WWFIesEeu9y7XeH6656N01tgwMxwSbMm5rOwvwMqzZfkMHs3CSOdn1MZWMllU2VvLKzkqauppDtBGFu1tyQu2BnZc4aW23e74X2xhFCvR56XCPvC6y+5sP2cgmGeloe2HU0TRV58Rf8AIUX8Kfzrqara2/fovqOesrf+Q6gwzWfjpc/OsqjL1XT5bV6hhxt7WLlS1UcPt5BbrqzL+B317tD7nLtle5I4LxgU83C4OPcKam09ByjqqmKyqb/YsOuSvac2DO4Np+UHnJzVFFOEelJIzRvGAOdx09xg1Fvb5dGRjVGuj3p1L1c+ppd0kb7K1Uq4sQM9Z16glmyZInZsmVLWPd5zZpLqfcNHlo1PzGTP935TliPFasCAcMlq97kmHs0zRiQn+nsC3irXT6TaZOS8QR62NmyMxj0Vo2+uSt0lERBmJM1py/kS6aUMDNjZmhtvqd9mBuMBgwNMJox0hFImzJMf/R+IZ88SbsvqglLRLYaY5YMXB6fNX6gwese8j9svaeV32/4B66feglZ6QXWKHzOTOtnNH2UIzxJcjQEAobaE13sb2xj37F29h9rY39jOx83tvfV9IdyU2lBX8Cfl59OdpoDYwxH249S2bSNNR9bQb/3+F58JrQ2n5GUcbI2n72YIkcuad1uq0Z+vAEO/W5AqDdYwwWMhjNz+AuivaGelhc6RrpSMSR+a/zPnjfkcM29Eo1haUcnn2vr4OLubmxgzZTT+yHQ99Pvg6G1FvZWhA7/anfAZd+C+X9jPU9IspoHBj6fABfrxhLwNoElGc/TkruNpgQh12fIbjqfo/Jl3l25lC5fFzubd1LZVNlXo2/pDh18SxDmpkylxJFLiS2FEp9hZrsLaQ/W3juaGV2zi+PUvVx6H5NSw/DbUmriG67GH7fBX/GrxZSnGLr7Ba4zEOBzHT0cyspns7+1L2oK/Ibl7V0sd50g3z+KuxzHQuzWbez24IdBgsO60Dfsh0X/9XqfD7Ne/3XsSQRsSTR3w2GXl5pWPzWtHj457mV/i5c2r+AhAS8JeEjEQwIgTEl3cG5eOnOnpHFuXjrn5qUxb0o6qyse4d/b/xjye0w0hnn2QkhPZp/7IL4BXRQzjVDsNZR0tlHS1UlRj4fUU/07FNvJMdIHhXq/Wro2uygVQoN/oKq1VLzxEE9lpNCQYGeqz88KdyfLrv4JFN9KfXs9L3/yMi/vf5m6jjrAqplemreEmwuv4Mqsc0nydEK3C7parcc3vjv88fIWW23Lvh7r0e+xpnnze0bZ1W98BYB2m+C22XAlJOFOdOC2J+JOSMRtT8Btt+G22XnN3hMS+gPZjGGex0tJTw/FPR5KunuY4fPRF8/OrCEuiA5oT0/N1WYXpcZAg38oo2iPD5gA79e/z/r963nj8Bt4g804WY4srp99PZ+b9znmTZpnrfzk4pBxgfpkngP/sGP4chhjdR/s+0Do9+Fwqg+L4POArwdXWzvNrnZOuNtpbe+grb2D9s4OfHTjt3nwJXjw2rx4bT48dh++BD++REOP3dBtC9BuC9BuM7gF3ALtAoEzqT0bw7Mddhan5JOaXjB800ti8tiPoZQ6JQ3+MGjtbqXiYAUv7X+JfSf29S0vzinm5nk38zcdXWza+Niw3yLGwhhDh7cDt8fNie5WDhxvZn/TMQ4eb+Gou4XGjhOc6HbhpxOxd538sXWBvQuRsf99UxNTyUjKICMxjYykNDIS0shMTCEjIYUMu5MMu5Nf7vwtrbbBHxD5fsOf/vYUH3ZKqYjTXj1hkOXM4q7z7uLOBXeyq2UXL+5/kf86+F9UNVdR1VzFDyWRQHYm/uDVgfrEBMpzsjGpKVzp7cDd48blceHuceP2BH8GPHd5XLh6rJB39bjp9LVjOMVt/A6wOWC4xpaUhBQyHBlWgAd/Mh2Z1vP+yx2h76UnpY9q+r+0zhOUH1xPd7/wdwYMK2broNdKTVRa4z9DXb4uNhzawEv7X2Lrsa0ROYYJJGH8yRh/MomSSlpiOpOcWeSlTqIwczIzJ+WQlzZpUICnJ6WTaIv8naAVm77DUwfW02CDqQFYMftmll3xLxE/rlLq1LSpZxwUry7GDNPt0GF34rSlYSeFgD+Znh4HHV1J+LxOTCC5L9iNPxkTSGaSM5O52bnMz81j/tQszs1LZ96UNLJSdCRFpdToaFPPOMhIzMXlHTwaZMCbRfPulUNuk5vu6OsaOS8vTQNeKRVxGvxjZIyhsa2HHUdd7DjqZkedi8baK0nKfwmxnbyBywQS6Wm8VgNeKTVhaPCPgjHWHa07jrrYUWcF/c46F83tA8d8KSMAOHL/iCS2YrxZ9DRdi99dxoerro5G0ZVSapC4Dv6hhhO+oaSAg80d7KxzsbPOHazRu3B3D56UO8OZwKKCTBYXZrC4MJMfVOym0V2Gz10Wsl5hlvZVV0pNHHER/EMFPMCjL1XR5bW6Sh5t7eJba7fz0LrKISflzk5NYnFhMOQLMllcaI0s2X8CF2MIGaIYIDnR3nc8pZSaCGI2+B97uZo17x0e1MfmaGsXD6+rBBE8A6byCxgI+A0FmU4WFWayuCCTRQVWbT4vwzHiLF29k4+MNCmJUkpFU1SCX0SuA54C7MCzxphV4dz/Yy9X8/v3Dg/7vsdvGG60RwE2P3rVmI+9vKxQg14pNaGN+1jAImIHfgX8DbAQuENEFobzGGtOEfojKdD2eKVUjIvGIPAXAh8bYw4YYzzA88BN4TzAaG5Jy0pOJDnRHrJM2+OVUvEgGsFfCPQfwrI2uCyEiNwnIltEZEtTU9PAt89IcqKd8hsX8cTniijMSkawet488bkibaZRSsW8CXtx1xjzDPAMWEM2hGu/k1IS+e4Ni/oCXoNeKRVvolHjPwqc0+/1tOCysLn74ulDLv/UnMl89Pg1GvZKqbgWjeD/EJgnIrNEJAm4HfhDOA/w/eVF3H3xdOzB7pd2Ee6+eDprvnpJOA+jlFJnpXFv6jHG+ETkG8Afsbpz/tYYszPcx/n+8iK+v7wo3LtVSqmzXlTa+I0xrwOvR+PYSikV76LR1KOUUiqKNPiVUirOaPArpVSc0eBXSqk4c1bMuSsiTcChMW6eAzSHsThnAz3n+KDnHPvO9HxnGGNyBy48K4L/TIjIlqEmG45les7xQc859kXqfLWpRyml4owGv1JKxZl4CP5nol2AKNBzjg96zrEvIucb8238SimlQsVDjV8ppVQ/GvxKKRVnYjr4ReQ6EdkrIh+LyMpolyeSROQcEdkoIrtEZKeIrIh2mcaLiNhF5CMReS3aZRkPIpIlIutEZI+I7BaRmB9vXET+IfjveoeI/KeIOKNdpnATkd+KSKOI7Oi3bLKIbBCR/cHHSeE4VswG/3hM6j7B+IB/NMYsBC4Gvh7j59vfCmB3tAsxjp4C/tsYswAoIcbPXUQKgQeAJcaYxVjDud8e3VJFxHPAdQOWrQTeNMbMA94Mvj5jMRv8jMOk7hOJMabeGLMt+LwNKwxifqoxEZkGLAOejXZZxoOIZAKXA78BMMZ4jDGt0S3VuEgAkkUkAUgB6qJcnrAzxvwZOD5g8U3A6uDz1cDycBwrloN/VJO6xyIRmQmUAe9HtyTj4n8DDwOBaBdknMwCmoB/CzZvPSsiqdEuVCQZY44CPwUOA/WAyxjzp+iWatzkGWPqg88bgLxw7DSWgz8uiUga8CLw98YYd7TLE0kicj3QaIzZGu2yjKME4Hzg/xpjyoAOwvT1f6IKtmvfhPWhVwCkisjd0S3V+DNW3/uw9L+P5eCP+KTuE42IJGKF/hpjzEvRLs84+BRwo4jUYDXlLRWR30e3SBFXC9QaY3q/za3D+iCIZVcDB40xTcYYL/AScGmUyzRejolIPkDwsTEcO43l4I/4pO4TiYgIVrvvbmPMz6NdnvFgjHnUGDPNGDMT6+/7ljEmpmuCxpgG4IiIzA8uugrYFcUijYfDwMUikhL8d34VMX5Bu58/APcGn98LvBKOnUZlzt3xMF6Tuk8gnwK+CFSLyPbgsm8H5zdWseWbwJpgheYA8OUolyeijDHvi8g6YBtW77WPiMGhG0TkP4ErgBwRqQW+C6wC1orIV7CGpr81LMfSIRuUUiq+xHJTj1JKqSFo8CulVJzR4FdKqTijwa+UUnFGg18ppeKMBr+KWyLSHqb9lIvIg6NY7zkR+UI4jqnUmdDgV0qpOKPBr+KeiKSJyJsisk1EqkXkpuDymcEx758TkX0iskZErhaRd4Pjo1/YbzclIvKX4PKvBrcXEfllcE6IN4Ap/Y75uIh8GBxf/pngHalKjQsNfqWgG7jZGHM+cCXws35BPBf4GbAg+HMn8NfAg8C3++2jGFgKXAI8LiIFwM3AfKz5IO4hdHyZXxpj/io4vnwycH2Ezk2pQWJ2yAalToMAPxSRy7GGdy7k5PC3B40x1QAishNrUgwjItXAzH77eMUY0wV0ichGrPkgLgf+0xjjB+pE5K1+618pIg9jjS0/GdgJvBqxM1SqHw1+peAuIBe4wBjjDY722Tu1X0+/9QL9XgcI/f8zcOyTYcdCCU4b+H+wZpQ6IiLl/Y6nVMRpU49SkIk1rr9XRK4EZoxhHzeJiFNEsrEG2voQ+DNwW3BO4HysZiQ4GfLNwfkTtKePGlda41cK1gCvBptvtgB7xrCPKmAjkAP8izGmTkTWY7X778IaWvgvAMaYVhH5V2AH1qxKH575KSg1ejo6p1JKxRlt6lFKqTijwa+UUnFGg18ppeKMBr9SSsUZDX6llIozGvxKKRVnNPiVUirO/H94EjPXE2aStwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "def validationCurvetest(X, y, Xval, yval, Xtest, ytest):\n",
    "    lambda_vec = [0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]\n",
    "\n",
    "    # You need to return these variables correctly.\n",
    "    error_train = np.zeros(len(lambda_vec))\n",
    "    error_val = np.zeros(len(lambda_vec))\n",
    "    error_test = np.zeros(len(lambda_vec))\n",
    "\n",
    "    # ====================== YOUR CODE HERE ======================\n",
    "    X = np.concatenate([np.ones((y.shape[0], 1)), X], axis=1)\n",
    "    Xval = np.concatenate([np.ones((yval.shape[0], 1)), Xval], axis=1)\n",
    "    Xtest = np.concatenate([np.ones((ytest.shape[0], 1)), Xtest], axis=1)\n",
    "\n",
    "    for i in range(1, len(lambda_vec)+1):\n",
    "        lambda_ = lambda_vec[i-1]\n",
    "\n",
    "        theta,_ = trainLinearReg(linearRegCostFunction, X, y, lambda_)\n",
    "        print(\"for lambda: {}\".format(lambda_))\n",
    "        print(\"theta\", theta)\n",
    "        error_train[i-1],_ = linearRegCostFunction(X, y,theta, lambda_=0)\n",
    "        error_val[i-1], _ = linearRegCostFunction(Xval, yval,theta, lambda_=0)\n",
    "        error_test[i-1], _ = linearRegCostFunction(Xtest, ytest,theta, lambda_=0)\n",
    "     \n",
    "    return lambda_vec, error_train, error_val, error_test\n",
    "\n",
    "    \n",
    "    \n",
    "lambda_vec, error_train, error_val, error_test = validationCurvetest(X_poly, y, X_poly_val, yval, X_poly_test, ytest)\n",
    "pyplot.plot(lambda_vec, error_train, '-o', lambda_vec, error_val, '-o', lambda_vec, error_test, '-o', lw=2)\n",
    "pyplot.legend(['Train', 'Cross Validation', 'Test'])\n",
    "pyplot.xlabel('lambda')\n",
    "pyplot.ylabel('Error')\n",
    "\n",
    "print('lambda\\t\\tTrain Error\\tValidation Err\\tTest Error')\n",
    "for i in range(len(lambda_vec)):\n",
    "    print(' %f\\t%f\\t%f\\t%f' % (lambda_vec[i], error_train[i], error_val[i], error_test[i]))\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Normalized Training Example 1: \n",
      " [ 1.         -0.36214078 -0.75508669  0.18222588 -0.70618991  0.30661792\n",
      " -0.59087767  0.3445158  -0.50848117]\n",
      "Normalized Test Example 1:\n",
      " [ 1.         -0.94220394  0.33149914 -0.50128942 -0.03307354 -0.11950565\n",
      " -0.24013304  0.11556833 -0.33419137]\n",
      "Normalized Val Example 1:\n",
      " [ 1.         -0.38916543 -0.72149541  0.16876064 -0.69803716  0.30354037\n",
      " -0.58940525  0.34396872 -0.50824813]\n"
     ]
    }
   ],
   "source": [
    "p = 8\n",
    "\n",
    "# Map X onto Polynomial Features and Normalize\n",
    "X_poly = polyFeatures(X, p)\n",
    "X_poly, mu, sigma = utils.featureNormalize(X_poly)\n",
    "X_poly = np.concatenate([np.ones((m, 1)), X_poly], axis=1)\n",
    "\n",
    "# Map X_poly_test and normalize (using mu and sigma)\n",
    "X_poly_test = polyFeatures(Xtest, p)\n",
    "X_poly_test -= mu\n",
    "X_poly_test /= sigma\n",
    "X_poly_test = np.concatenate([np.ones((ytest.size, 1)), X_poly_test], axis=1)\n",
    "\n",
    "# Map X_poly_val and normalize (using mu and sigma)\n",
    "X_poly_val = polyFeatures(Xval, p)\n",
    "X_poly_val -= mu\n",
    "X_poly_val /= sigma\n",
    "X_poly_val = np.concatenate([np.ones((yval.size, 1)), X_poly_val], axis=1)\n",
    "\n",
    "print('Normalized Training Example 1: \\n', X_poly[0, :])\n",
    "\n",
    "\n",
    "print('Normalized Test Example 1:\\n', X_poly_test[0, :])\n",
    "\n",
    "\n",
    "print('Normalized Val Example 1:\\n', X_poly_val[0, :])\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "metadata": {},
   "outputs": [],
   "source": [
    "def learningCurvetest(X, y, Xval, yval,Xtest, ytest, lambda_=0):\n",
    "    \"\"\"\n",
    "    Generates the train and cross validation set errors needed to plot a learning curve\n",
    "    returns the train and cross validation set errors for a learning curve. \n",
    "    \n",
    "    In this function, you will compute the train and test errors for\n",
    "    dataset sizes from 1 up to m. In practice, when working with larger\n",
    "    datasets, you might want to do this in larger intervals.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array_like\n",
    "        The training dataset. Matrix with shape (m x n + 1) where m is the \n",
    "        total number of examples, and n is the number of features \n",
    "        before adding the bias term.\n",
    "    \n",
    "    y : array_like\n",
    "        The functions values at each training datapoint. A vector of\n",
    "        shape (m, ).\n",
    "    \n",
    "    Xval : array_like\n",
    "        The validation dataset. Matrix with shape (m_val x n + 1) where m is the \n",
    "        total number of examples, and n is the number of features \n",
    "        before adding the bias term.\n",
    "    \n",
    "    yval : array_like\n",
    "        The functions values at each validation datapoint. A vector of\n",
    "        shape (m_val, ).\n",
    "    \n",
    "    lambda_ : float, optional\n",
    "        The regularization parameter.\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    error_train : array_like\n",
    "        A vector of shape m. error_train[i] contains the training error for\n",
    "        i examples.\n",
    "    error_val : array_like\n",
    "        A vecotr of shape m. error_val[i] contains the validation error for\n",
    "        i training examples.\n",
    "    \n",
    "    Instructions\n",
    "    ------------\n",
    "    Fill in this function to return training errors in error_train and the\n",
    "    cross validation errors in error_val. i.e., error_train[i] and \n",
    "    error_val[i] should give you the errors obtained after training on i examples.\n",
    "    \n",
    "    Notes\n",
    "    -----\n",
    "    - You should evaluate the training error on the first i training\n",
    "      examples (i.e., X[:i, :] and y[:i]).\n",
    "    \n",
    "      For the cross-validation error, you should instead evaluate on\n",
    "      the _entire_ cross validation set (Xval and yval).\n",
    "    \n",
    "    - If you are using your cost function (linearRegCostFunction) to compute\n",
    "      the training and cross validation error, you should call the function with\n",
    "      the lambda argument set to 0. Do note that you will still need to use\n",
    "      lambda when running the training to obtain the theta parameters.\n",
    "    \n",
    "    Hint\n",
    "    ----\n",
    "    You can loop over the examples with the following:\n",
    "     \n",
    "           for i in range(1, m+1):\n",
    "               # Compute train/cross validation errors using training examples \n",
    "               # X[:i, :] and y[:i], storing the result in \n",
    "               # error_train[i-1] and error_val[i-1]\n",
    "               ....  \n",
    "    \"\"\"\n",
    "    # Number of training examples\n",
    "    m = y.size\n",
    "\n",
    "    # You need to return these values correctly\n",
    "    error_train = np.zeros(m)\n",
    "    error_val   = np.zeros(m)\n",
    "    error_test  = np.zeros(m)\n",
    "\n",
    "    # ====================== YOUR CODE HERE ======================\n",
    "    for i in range(1, m+1):\n",
    "        theta,_ = trainLinearReg(linearRegCostFunction, X[:i, :], y[:i], lambda_)\n",
    "        error_train[i-1],_ = linearRegCostFunction(X[:i, :], y[:i],theta, lambda_=0)\n",
    "        error_val[i-1], _ = linearRegCostFunction(Xval, yval,theta, lambda_=0)\n",
    "        error_test[i-1], _ = linearRegCostFunction(Xtest, ytest,theta, lambda_=0)\n",
    "        \n",
    "\n",
    "        \n",
    "    # =============================================================\n",
    "    return error_train, error_val,error_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial Regression (lambda = 3.000000)\n",
      "\n",
      "# Training Examples\tTrain Error\tCross Validation Error Test Error\n",
      "  \t1\t\t0.000000\t138.846777\t164.709076\n",
      "  \t2\t\t0.080704\t143.808317\t169.779712\n",
      "  \t3\t\t15.062852\t6.660747\t11.376190\n",
      "  \t4\t\t9.168031\t4.322521\t5.365247\n",
      "  \t5\t\t7.265773\t4.511018\t4.894375\n",
      "  \t6\t\t5.748444\t4.861816\t4.997470\n",
      "  \t7\t\t5.782694\t4.230118\t4.456939\n",
      "  \t8\t\t4.879505\t4.360617\t4.346039\n",
      "  \t9\t\t4.641373\t4.792851\t4.755793\n",
      "  \t10\t\t4.232230\t4.684224\t4.780203\n",
      "  \t11\t\t3.820539\t4.807689\t4.888737\n",
      "  \t12\t\t4.901372\t3.822929\t3.859903\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd5xU5dn/8c93aUuHBaRXpS3YAMXeoybWJMZoTGxkwUQTlTQ1xcRfoqZoNMmjgQUTYzSEx/6oUdGoWFCkKLCsoKAISO91l2Wu3x/3WRy2DLO7MztbrvfrNa+dU+ac65yZPdc5933u+8jMcM455+JlZToA55xzdY8nB+ecc+V4cnDOOVeOJwfnnHPleHJwzjlXjicH55xz5XhyqAWSPpF0RqbjiCfpP5KuSHLeOhd/qkjaLmlAXV6fpH9JujB6f6WkN9IU26uSvl2F+Rvs7yJTJM2UNCzTcYAnhyqJ/hl2Rf/gayT9XVKbTMdVHWb2RTN7sKbLifZBcbRPNkqaJmlIKmKsDWbWxsyWpnq5ZX4rpa8e8euL9t2vD7Ccw4DDgadSHWN9IylX0ixJm6LXS5JyE8yfI+kJSTskLZP0jTLTvxGN3yHpSUk5mf4s8Afgturuo1Ty5FB155lZG2AEMAr4WYbjqQt+F+2TnsBKYHKqVyCpaaqXWQvOi5JB6euzaixjHPCweWtVgM+Ai4AcoDPwNDAlwfz/AxQDXYHLgPtLz8qjvxOAb0XTdwL31YHPPg2cKqlbkvskfczMX0m+gE+AM+KGfw88E70/HygANgOvAkPLfg7oRvgxdIqbNgJYBzQDrgTeIJw9bAI+Br4YN28Pwo9nI/ARkBc37ZfA/wL/BLYB84FBwM3AWmA5cGbc/K8C347eHwz8F9gArAceBjpUtt1l9snfgV/HDX8J2FEm5seibfwY+H7ctJbAg9G2FgI/BlaUWe9PgHlAEdAUOAZ4K9rP7wOnxM1/JbA02v6Pgcui8YcArwFbou37d9xnDDgket8e+EcU6zJC4s+KW3al382Bfitl1weMBfYQDiLbgf+rZDlLgRPKbOMbccP3Rt/tVmA2cGINfxN3ADOj5T0F5MRN/1a0XzYAP43fRuBoYEb0vawC/gI0T+P/YlPgWmBnJdNbR/t2UNy4h4A7o/e3A4/ETTs4mr9tpj4bN24acEW69l2yL79yqCZJvQkHwrmSBgH/Am4AugDPAf8nqXn8Z8xsNeEf8OK40d8CppjZnmh4NLCIcGb0O2CyJEXTpgArCAfci4DbJZ0Wt6zzCD/EjsBc4AXC1WFPwqXqhMo2h3BQ6AEMBXoTDixVIqk1cCkhcSEpC/g/wkG8J3A6cIOks6KP3Ar0AwYAXwC+WcFiLwXOAToQzrSeBX5NOHv8IfCYpC7Ruv9EOGC3BY4D3ouW8f+AFwn7pRfw50o24c+EBDEAOBm4HLgqbnqi76ZKzGwiIQn/zsJVxXll54m2qX+0zsq8CxxB2B+PAP8rKTtuelV/E5cDVwPdgRLCPiUqvrmf8HvtAXQi7MtSe4EbCfvmWMJ3/d3Kgpa0OcHrpgTbi6TNwG7C93V7JbMNAkrMbHHcuPeB0vL8YdEwAGa2hOignsHPliokFCVmlCeHqnsy+nG+QTgbvR34OvCsmU2LDvJ/IJwVH1fB5x8kOghKakI4+D0UN32ZmeWb2d5o3u5A1ygZHQ/8xMx2m9l7wCTCP3Op183sBTMrIZwxdiGcsewhJJZ+kjqUDcjMPopiLzKzdcDdhINjsn4Y7ZNtwAmEAwjAUUAXM7vNzIotlLXnA5dE0y8GbjezTWa2guhAVMafzGy5me2K9ttzZvacmcXMbBowi5CkAWLAcEktzWyVmRVE4/cAfYEe0b4rV6EbfReXADeb2TYz+wS4K25boJLvJsF+eTLugPdkgvkqU/pdbatsBjP7p5ltMLMSM7sLaAEMjpulqr+Jh8xsgZntAH4OXBztm4sIV8nTzawomhaLi2O2mb0dxfEJIelU+hsysw4JXncm2ilm1oGQxK8jJLyKtCFc/cTbQji7L52+pZLpmfpsqW18/t1njCeHqrsw+gH3NbPvRgetHoTLbQDMLEa4ZO9ZweefAnIl9SecLW8xs5lx01fHLWdn9LZNtI6NZhZ/oFhWZh1r4t7vAtZHB7LS4dJl7UdSV0lTJK2UtJVQDNG5ku2vyB+if9h+0XpKD059gR7xZ4XALXx+QO1B2E+l4t9XNK4v8LUyyzsB6B4dzL4OXAOskvRsXMX4jwlXRzMlFUi6uoL1dCYU7S2LG1d2/1b23VTmwrgD3oUJ5qvM5uhv28pmkPRDSYWStkT7oz37f3dV/U3E7+9lhH3SmTLfVbS/N8TFMUjSM5JWR7+h26nab6hKovX/FfiHpIMqmGU70K7MuHZ8nmgTTc/UZ0u15fPvPmM8OaTGZ4QDFwBRUUNvQuXsfsxsNzCVcBb8Lfa/ajjQOnIkxR8o+lS0jmq4nVAWfqiZtYtiq3JxiZl9ClwP3CupJeFg8nGZs8K2ZlZ6pr+K/Ysmele02Lj3ywlntvHLa116phmdIX+BcEb/AeEqBTNbbWZ5ZtaDUMF7n6RDyqxnPZ9fYZRK1f6tTMJK5ugAuIT9ixz2kXQiIfFdDHSMEvQWqvHdxYn/DvoQ9sl6wne1b5qkVoSipVL3E/b5wOg3dEuiOMrcxVX2dUuSsWYBraj4JGwx0FTSwLhxhxPqBYn+7iu6Ubi9uEX0uUx9ttRQ4oqeMsWTQ2pMBc6RdLqkZsAPCBWob1Uy/z8IFYvnk2RyMLPl0fLukJStcIvjGMJZfk21JZzRbJHUE/hRdRcUFfV8RqhwnQlsk/QTSS0lNZE0XNJR0exTgZsldYzWe90BFv9P4DxJZ0XLypZ0iqRe0dXPBVE5fVG0PTEASV+TVJqENhEOyrH4BUdn01OB30hqK6kvMJ7U7N/KrCHUbyTyHJUXz7Ql1AusIxyQfkH5s9Kq+qbCLaOtCHUSj0b75lHgXEknRHVpt7H/8aMtoThle3TF9p1EK7H97+Iq+6qwHkHSFyQdGX337QjFn6U3M5Rd/g7gceA2Sa0lHQ9cwOf/bw8TfksnRr+Z24DHoyLFjHw22sZsYCShUjqjPDmkgJktIpxt/5lwlnUe4TbG4krmf5NwcJpjZssqmqcSlxKKbj4DngBuNbOXahB6qV8R7praQqjwfbyGy/s94Yy2KXAuocL0Y8K+mUQo+oDwj7EimvYS4QBUVNlCowR5AeGsdB3hSuJHhN9xFuFg/hnhbq6T+fwAdRTwjqTthLu9rreK2zZ8D9hBuEPoDUIF7wNV3fgqmEwoYkxUJzERuKySiu8XgOcJZ53LCJW0FRXNVcVDhDvQVgPZwPcBovqbawn7ZBXhoLwi7nM/BL5BKB7JB/5dwzgq0oFw48cWwhXVwcDZ0dU4km6R9J+4+b9LqPtbG33uO6X1UNHfawgH67WE5PbdOvDZ84BXrXq3PaeUzPz26UyQ9F/CLW2TMh1LXSHpO8AlZlaVyvAGT9IjwFQzq06ltqtHJL0DjDGzBRmPxZND7YuKVaYBvctUMDcqkroTilVmAAMJVy1/MbN7MhqYc46MtjqV9AnhMnQv4d7gUQpNyf9NKD75BLjYzDZlKsZUk/QgcCGhaKPRJoZIc8Itj/0Jd2dMYf/Wos65DMnolUOUHEaZ2fq4cb8j3LJ5Z9QYpqOZ/SRTMTrnXGNUFyukLyA0MCL6W537w51zztVApq8cPubzWwsnmNlESZuj+7VL2wtsKh0u89mxhNslad269cghQ+pNR6DOOVcnzJ49e72ZdaloWqZ7ujzBzFZGLRynSfogfqKZmaQKs5eFvmkmAowaNcpmzZqV/midc64BkVTprfQZLVYys5XR37WE+/aPBtZEd7GU3s2yNnMROucai5Wbd3Hen9/gTy9/mOlQ6oSMJYeo9WDb0vfAmcACQiOl0ieUXYE/5MQ5VwtmLNnA/JVbmLci490a1QmZLFbqCjwRNfxsSmgQ9rykd4GpksYQWn1enGAZzjmXEjOWhH4EjxnQ6QBzNg4ZSw5R9wXl+iw3sw2EvuCdc65WmBlvfLQOgOMPSVtnsvVKXbyV1TnnatWiNdtYs7WIg9q2YEi3SntIb1Q8OTjnGr3pi8NVw4kDu1BxH4eNjycH51yjN31x6KThpEFepFQq0+0cnHMu4648rh+9c1pxgtc37OPJwTnX6J2R25UzchM9Drzx8WIl55xz5XhycM41arc/V8jjc1ZQVLI306HUKV6s5JxrtFZu3sXE6Utp06Ip5x3eI9Ph1Cl+5eCca7ReWxRuYT324E40a+KHw3i+N5xzjdbLhWsAOG3IQRmOpO7x5OCca5R2Fe/ljY9C+4bTPTmU48nBOdcovfnReopKYhzeqz0HtcvOdDh1jicH51yj9PIHoUjp9KHevqEifreSc65RGtajPSP6bOMMTw4V8uTgnGuUvnlMX755TN9Mh1FnebGSc865cjw5OOcanYfeXsaClVsws0yHUmd5cnDONSorNu3k508u4OIJMygqiWU6nDrLk4NzrlF5fsFqAE4dchDZzZpkOJq6y5ODc65ReW7+KgC+NLx7hiOp2zw5OOcajdVbdjPn081kN8vilMFdMh1OnebJwTnXaDy/IFw1nDyoC61b+J38iWQ8OUhqImmupGei4f6S3pH0kaR/S2qe6Ridcw3Dc1F9w5cO9SKlA8l4cgCuBwrjhn8L/NHMDgE2AWMyEpVzrkGJxYx+nVrRuU0L74U1CQmvqyRlA+cCJwI9gF3AAuBZMyuo6col9QLOAX4DjJck4DTgG9EsDwK/BO6v6bqcc41bVpb43UWHE4sZWVnKdDh1XqXJQdKvCInhVeAdYC2QDQwC7owSxw/MbF4N1n8P8GOgbTTcCdhsZiXR8AqgZyXxjQXGAvTp06cGITjnGhNPDMlJdOUw08xurWTa3ZIOAqp9VJZ0LrDWzGZLOqWqnzezicBEgFGjRnkzR+dcpdZu3c2MpRs4M7cbLZt724ZkVFrnYGbPAkg6tJLpa81sVg3WfTxwvqRPgCmE4qR7gQ6SSpNWL2BlDdbhnHM8MXcl1095j588VpOCjsYlmQrp+yTNlPRdSe1TtWIzu9nMeplZP+AS4L9mdhnwCnBRNNsVwFOpWqdzrnF68r3PADjnML9LKVkHTA5mdiJwGdAbmC3pEUlfSGNMPyFUTn9EqIOYnMZ1OecaADOjoKCA119/nYKCgv061Fu8ZhuFq7bSLrupN3yrgqRuZTWzD4GfEQ7cJwN/kvSBpK+kIggze9XMzo3eLzWzo83sEDP7mpkVpWIdzrmGx8yYNGkShw4bxvDhwznppJMYPnw4hw4bxqRJkzAznpwbSqbPOaw7LZp6fUOyDthEUNJhwFWEW06nAeeZ2RxJPYAZwOPpDdE558ozM8aNG0d+fj4js7KYAAwAlgITFy0iLy+Pd2bOpGDA1wG44IgKb3x0lUim/fifgUnALWa2q3SkmX0m6Wdpi8w55xKYPHky+fn53Az8JhYj/gbVvFiMW4A//uctul12AT3aZ3N0v5wMRVo/HTA5mNnJCaY9lNpwnHPuwMyMe+6+m5FZWeUSA4CA24FnmzRl++YVnHfSSd6+oYoqrXOQ9H+SzpPUrIJpAyTdJunq9IbnnHPlLVy4kILCQsZWkBhKCbh22ft8POEazunjD/WpqkRXDnnAeOAeSRuBdYQW0v2AJcBfzMxvM3XO1bqNGzcCoY4hkdLpWzZtTGs8DVGlycHMVhO6tvixpH5Ad0LfSovNbGetROeccxXIyQn1B0sTzPO3keex6uO5sHEFnTp1qp3AGpCkOjQ3s0+AT9IaiXPOJSk3N5dhQ4aEu5LMyhUtfdSpF786Yxwq2smwl37N0KFDMxJnfVYXuux2zrmkmRmTJ09m0+bNzDbjp0DZztWmHHYmAFsLp3PjDd8ndPjsqsIfheScqzfi2zaMkBgA3AE8D4wj1DF8mNWUycNOA+D0/q24+mq/b6Y6PDk45+qN/do2RF1kPAD8EbgmmqfVwNF0ad2BLs33MOV/7vCrhmo6YLGSpHOjx3hulLRV0jZJW2sjOOecK7Vf2wbCraoiPCpyPlAAvAZ0HXkeANedfThZWV5yXl3JXDncA3wFmG/xvVk551wtKm3bMAEqbPSWCxR26Ues93BiRTsZ1npH7QfZgCSTHJYDCzwxOOcyKZm2DS33FHHc/Gn8Z+dWdm3tVjuBNVDJJIcfA89Jeg3Y10Oqmd2dtqicc66MZNo29Nu8ipOfu5d/AZ06/bBW4mqokimQ+w2wk9A6um3cyznnak1ubi7Dhg5lYlZWuVtXSxkwMSuL4bm53rahhpK5cuhhZsPTHolzziUgiRvGjycvL4+fwr5KaYAYYvw5N7Luo3eYs3gGk8aP97uUaiiZ5PCcpDPN7MW0R+OccwmMGTOGmTNnckd+Pi9kZTE2FmMA8PwhR/Pk8NMo6T2MMScN87YNKZBMsdJ3gOcl7fJbWZ1zmSSJCRMmMGnSJIoGD+Ya4Ezg4dHhoZRfOrgl+RP+6lcNKaCGcBPSqFGjbNasWZkOwzlXi8yMwsJCZny4hv83YyftWzbjrZtOo3ULb9ubLEmzzWxURdOS2ouSOgIDCZXSAJjZ9NSE55xzVSeJ3Nxc7pm1E9jJt47p64khhZJ5hvS3geuBXsB7wDGEZ0eflt7QnHMusSXrtjOtcA3Nm2ZxxXH9Mh1Og5JMncP1wFHAMjM7FTgS2JzWqJxzLgkPzViGGXx1RC+6tG2R6XAalGSuwXab2W5JSGphZh9IGlzTFUvKBqYDLaI4HjWzWyX1B6YAnYDZwLfMrLim63PONTw/Omsw3dpnc2Zu10yH0uAkc+WwQlIH4ElgmqSngGUpWHcRcJqZHQ4cAZwt6Rjgt8AfzewQYBOhXy3nnCundYumXHPywQzo0ibToTQ4B7xyMLMvR29/KekVoD2h+/Qaifpq2h4NNoteRqjL+EY0/kHgl8D9NV2fc67h2LyzmGZNsrwCOo0qvXKQlFP2RegZ9w0gJWlaUhNJ7wFrgWnAEmCzmZVEs6wAelby2bGSZkmatW7dulSE45yrJ+56cTEn/u4VXlq4JtOhNFiJ0u5swpm8gD6EIh4BHYBPgf41XbmZ7QWOiIqtngCGVOGzE4GJENo51DQW51z98NnmXfz73eXsicXo06lVpsNpsCq9cjCz/mY2AHgJOM/MOptZJ+BcIKVdaZjZZuAV4Figg6TSpNULWJnKdTnn6rf7Xv2I4r0xzjm0O4O6eh+g6ZJMhfQxZvZc6YCZ/Qc4rqYrltQlumJAUkvgC0AhIUlcFM12BfBUTdflnGsYPlm/gykzlyPB908fmOlwGrRkanM+k/Qz4J/R8GXAZylYd3fgQUlNCElqqpk9I2khMEXSr4G5wOQUrMs51wD84cVFlMSMi0b28quGNEsmOVwK3EqoEzBC24RLa7piM5tHaFBXdvxS4OiaLt8517C8v3wzz8xbRYumWYz/wqBMh9PgJXMr60ZCK2nnnMuYmBlDu7fjpEGd6dGhZabDafD8JmHnXL1wZJ+OPPu9EyjeG8t0KI1CMhXSzjmXMfGPFcjKEtnNmmQwmsbDk4Nzrk574M1PuGHKXNZs3Z3pUBqVAyYHSYMkvSxpQTR8WHT3knPOpdXabbu5Z9pinnzvMwo+25LpcBqVZK4c8oGbgT2w7y6jS9IZlHPOAfz2P4vYVlTCaUMO4rQh3vNqbUomObQys5llxpVUOKdzzqXI7GWbeGzOCpo3yeIX5+ZmOpxGJ5nksF7SwYQ2Dki6CFiV1qicc43anr0xfv7kAgDyTupPv86tMxxR45PMrazXEjq4GyJpJfAx8M20RuWca9TyX1/KwlVb6Z3TkmtPPSTT4TRKyTSCWwqcIak1kGVm29IflnOuMduycw8S3P7lQ2nV3JtjZcIB97qkFsBXgX5AU0kAmNltaY3MOddo3fyloVx6dB8vTsqgZFLyU8AWwvMditIbjnOuPjAzFi5cyMaNG8nJySE3N5fSE8eaiMWMrKywHE8MmZVMcuhlZmenPRLnXJ1nZkyePJl77r6bgsLCfeOHDR3KDePHM2bMmGoniU837GTMg+/yqwuGcdzBnVMVsqumZO5WekvSoWmPxDlXp5kZ48aNIy8vj+xFi5hAeLbvBCB70SLy8vIYN27cft1dJKtkb4wbp77Hh2u388g7n6Y6dFcNlV45SJpPuH21KXCVpKWEYiUBZmaH1U6Izrm6YPLkyeTn53Mz8JtYjPjrg7xYjFuAO/PzGT16NGPGjKnSsv/62hJmL9tEt3bZ/PrC4akM21WTKsvykvom+qCZLUtLRNUwatQomzVrVqbDcK7BMjMOHTaM7EWLeLdMYtg3DzAqK4viIUOYt2BB0sVL81Zs5iv3vUVJzPjnmNGcMNCLlGqLpNlmNqqiaYmeIb0sSgC/Ln0fPy5dwTrn6p6FCxdSUFjI2EoSA4QihbGxGAsWLqQwrj4ike1FJdww5T1KYsZVx/fzxFCHJFPnMCx+IHqs58j0hOOcq4s2btwIwIADzFc6fcOGDUkt9+bH57N0/Q6GdGvLT84eUv0AXcpVmhwk3SxpG3CYpK3RaxuwlnB7q3OukcjJyQFg6QHmK53eqVOnpJb7tZG96JPTivsuG+HPaahjKq1z2DeDdIeZ3VxL8VSL1zk4l17prHMo2RujaRN/tEwmVKvOoVRdTwzOufSTxA3jxzM7FuOnRL1wxjHgFmBOLMYN48cnTAwbthfx9tLPi508MdRN/q0455IyZswY8vLyuINwhRDfzmFUVhZ3Anl5eVx99dWVLqOoZC/X/HM238h/myfmrqidwF21JKpz6J/OFUvqLekVSQslFUi6PhqfI2mapA+jvx3TGYdzLjmSmDBhApMmTaJo8GCuAc4ErgGKBg9m0qRJTJgwodKrBjPjZ08s4N1PNnFQ22yO91bQdVqidg6zzWykpJfN7PSUr1jqDnQ3szmS2hL6broQuBLYaGZ3SroJ6GhmP0m0LK9zcK52mRmFhYVs2LCBTp06MXTo0APWMUx6fSm/fraQ7GZZPHrNcQzv2b6WonWVSVTnkKhvpSxJtwCDJI0vO9HM7q5JUGa2iuihQWa2TVIh0BO4ADglmu1B4FUgYXJwztUuSeTmJv90tmkL13D7c6Htw90XH+GJoR5IVOdwCbCXkEDaVvBKGUn9gCOBd4CuUeIAWA34g2Odq8dmfbKR6x6ZQ8zgxjMG8aVDu2c6JJeESq8czGwR8FtJ88zsP+kKQFIb4DHgBjPbGn9pamYmqcJyL0ljgbEAffr0SVd4zrkaatYki9YtmvKVEV35/un+VLf6Ipl2Du2BW4GTolGvAbeZ2ZYar1xqBjwDvFBaTCVpEXCKma2K6iVeNbPBiZbjdQ7O1W0rNu2kW7tsv221jqlROwfgAWAbcHH02gr8LQVBCZgMFJapv3gauCJ6fwXeGtu5emfN1t08M++zfcO9OrbyxFDPJPOwn4PN7Ktxw7+S9F4K1n088C1gftzybgHuBKZKGgMsIyQk51w9sXbrbi7Nf5uP1+8gS/I6hnoqmeSwS9IJZvYGgKTjgV01XXG0vMrufUv5rbPOufRbt62IS/PfZum60JnesQOS62PJ1T3JJIdrgH9EdQ8Am/i82Mc55wBYu203l+W/w5J1OxjctS0Pf3s0HVs3z3RYrpoOmBzM7H3gcEntouGtaY/KOVevLN+4k29OfodlG3YyqGsbHs4bTac2LTIdlquBZK4cAE8KzrmKmRnXPjKHZRt2MrxnO/5+1dF09sRQ7/ntA865GpHE7y46jDNzu/KvvGM8MTQQB0wOksp90xWNc841LovXbNv3fki3dky8fBRts5tlMCKXSslcOcxIcpxzro4zMwoKCnj99dcpKCjgQI1gK1vG/a8u4ax7pjN11vI0ROnqgkrrHCR1I3SE11LSkXx+22k7oFUtxOacSxEzY/Lkydxz990UFBbuGz9s6FBuGD+eMWPGJPXktuKSGD97cj5TZ4VnMWzaUZy2mF1mJaqQPovQfXYvIL4F8zZCYzXnXD1gZowbN478/HxGRg/pGUB43vPERYvIy8tj5syZCZ/FAKFx23WPzGXmJxvJbpbFHy8+gi96A7cGK1HHew8CD0r6qpk9VosxOedSaPLkyeTn53Mz8Jsyz3/Oi8VCtwT5+YwePZoxY8ZUuIx3lm7gun/NZd22Ig5q24JJV4zisF4daiN8lyHJdLx3K+UfGYuZ3ZauoKrKO95zrmJmxqHDhpG9aBHvlkkM++YhPOazeMgQ5i1YUO7qYW/M+OK901m8ZjvHDMjhT5ceyUFts2slfpdeNe14bzuwI3rtBb4I9EtZdM65tFm4cCEFhYWMrSQxQKhMHBuLsWDhQgrj6iNKNckSf/nGCK479RD+OWa0J4ZGIpkW0nfFD0v6A/BC2iJyzqXMxo0bgVDHkEjp9A0bNgDw2uJ1TF+8jp+fG572NqhrW354VsKe810Dk3QL6TitCJXUzrk6LicnBwiVz4mUTm/TviO/fLqAv7/1CQAnDuzMKYMPSlt8ru46YHKQNJ/P6xyaAF2AOlPf4JyrXG5uLsOGDg13JSWoc5iYlcXQY8/kRy9t4KO122maJW78wiBOHNiltkN2dUQyVw7nxr0vAdaYWUma4nHOpZAkbhg/nry8PH4K/Ib9+8k34EfNW/LxSZfTbuS5fLR2OwO6tOaerx/hdyM1csnUOSyTdDhwYjRqOjAvrVE551JmzJgxzJw5kzvy83khK4uxsdjn7Ryyslhy1JfpMPI8mmSJcScN4PunDyS7WZNMh+0yLJlipeuBPODxaNTDkiaa2Z/TGplzLiUkMWHCBEaPHs0f77qLawoLIaspxEoYNngwd1xxOu9nd+X7pw8it0e7TIfr6ohk2jnMA441sx3RcGtghpkdVgvxJcXbOTiXnB1Fe7jt0Xd4ZclW7r+gDyMOG5ZUtxmuYUrUziGZOgcR2jeU2kvlj/d0ztVBsZjx9Pufced/PmD11t0ArFKOJwZXqWSSw9+AdyQ9EQ1fCExOX0jOuVQxM15dtI7fvbCIwlXheV2H9mzPL88fxsi+HTMcnavLkl4Co/MAAB0RSURBVKmQvlvSq8AJ0airzGxuWqNyzqXELU/M518zQ7fa3dtnc+MZg7hoZC+ysvyKwSWWVCM4M5sDzElzLM65FCgq2UuLpuFuozOGduX5Bau59tRD+OYxff0uJJe06rSQThlJDxDaUaw1s+HRuBzg34T+mz4BLjazTZmK0bn6wMx4bfE67ntlCb1zWnHXxYcDcNqQg3j9J6fRpkVG/9VdPZTpZ0j/HTi7zLibgJfNbCDwcjTsnKtAyd4Yz85bxfl/eZMr//YuMz/ZyKuL1rKzOLRTleSJwVVLRn81ZjZdUr8yoy8ATonePwi8Cvyk1oJyrh7YsmsPD7+zjIdmLGPVlnD3Uec2Lfj2if355jF9adXcE4KrmWQawW2j/PMctgCzgB+Y2YH69Kqqrma2Knq/Guia4uU7V+9t2bmH37+wCDMY0Lk1Vx3fj6+N6u11Ci5lkjm9uAdYATxCaN9wCXAwoYL6AT4/y085MzNJFbbSkzQWGAvQp0+fdIXgXMZtLyrhmfc/480lG/jTJUcgiT6dWjH+jEEc2qs9Jw3s4ncfuZRLpoX0+2Z2eJlx75nZERVNq3IAoVjpmbgK6UXAKWa2SlJ34FUzS9iRvLeQdg2NmTHn0038+93lPP3eSnaXhP/TqeOO4ej+nTIcnWsoatpCeqeki4FHo+GLgN3R+8SZpXqeBq4A7oz+PpWGdThXJ+0sLmHi9KU8/f5nLF23Y9/43csXsH3ei1z17HpuvP57jBkzxls3u7RKJjlcBtwL3EdIBm8D35TUEriuJiuX9C9CsVRnSSuAWwlJYaqkMcAy4OKarMO5um7zzmI6tGoOQPMmWTz09jI2bC9m7/ZNtCn4L5fPe5GjNq7c14tqXl4eM2fOZMKECZ4gXNok00J6KXBeJZPfqMnKzezSSiadXpPlOlfXLV23nWkL1/DiwjXMX7mFmbecTodWzWnaJIsTWq9h4gN/5PpP3uMO2/8BPXmxGLcAd+bnM3r0aMaMGZOpTXANXDJ1Dl0IXXb3Iy6ZmNnVaY2sCrzOwdV1sZgxd/lmpi1cw7SFq1kSV2TUomkWk684ihMGdsbMOHTYMLIXLeLdBE9uG5WVRfGQIcxbsMCvHly11bTO4SngdeAl9u+d1TmXwK7ivbRsHm4t3bizmIv++hal52Ltspty2pCDOHNYN04a1GVfQ7WFCxdSUFjIBCrv+ljA2FiMaxYupLCwkNzc3LRvi2t8kkkOrcyswTZCW7JuO0vWbufMYd0yHYqr54pK9jL7k01M/3A9r3+4jg3bi5lx82lIonObFpx7WA86tW7OmbldOap/Ds2alO+gYOPGjQAMOMC6Sqdv2LAhtRvhXCSZ5PCMpC+Z2XNpj6aWbd5ZzOWTZ7Jqyy5u//KhXHK0t5dwVbN8404enb2CmR9vZO7yTezeE9s3rUXTLFZs2kXvnFYA/PnSIw+4vJycHCA8wjOR0umdOvltrS49kqlz2Aa0BoqAPYSrWjOzOvM8werWOZgZ9778Ife89CEA158+kOtPH+gNilyFNmwvYu6nm8lu1oQTBnYGYM6nm/jKfW/tm2do93acNLAzJw7swqh+HavcYtnrHFxtqlGdg5m1TX1IdYMkbjhjEJ3atOAXTy3g3pc/ZMHKLdz99SNo37JZpsNzGbSzuIQFK7fy/vLNvLdiM/NWbGb5xl0AnHBI533JYXiP9nz7hP6M6pfDUf060qlNixqtVxI3jB9PXl4ePwV+w/51DwbcAsyJxZg0frwnBpc2lV45SBpiZh9IGlHR9OgZD3VCKu5Wem3xOr7/r7ls2bWHvp1acffFR/iTshqJzTuLKVy1jSHd2tKxdWhv8IOp7/PYnBX7zdeqeROG92zPSQM7c91pA9MWj5kxbtw48vPzGZGVxdhYjAGwr53DnFiMvLw8b+fgaqy6Vw7jCX0X3VXBNANOS0FsdcbJg7rwzPdOYNxDs1m4aisbdxRnOiSXYrGYsWTddgpXb+ODVVspXLWVD1Zv29er6f2XjeCLh3YHYGTfjnyweiuH9+7AEb06cHjvDhxyUBua1EKRoyQmTJjA6NGj+eNdd3FNYeG+acMGD2bSD37A1Vdf7YnBpdUB6xzqg1S2cygq2cu0hWs497Ae+8a9v3wzh/VqXyf+Gc2MhQsXsnHjRnJycsjNza0TcdUlm3YUs2Tddpau20FRyV6+dWw/AIpLYgz9xfPsje3/m89ulsXgbu249pSD03rXWnW+OzOjsLCQDRs20KlTJ4YOHerft0uZGtU5SHoDeI3Q1uFNM9uW4vjqlBZNm+yXGBas3MKF973JiD4d+f7pAzlpYOeM/HOaGZMnT+aeu++mIP5McuhQbhg/vlH1tbM3ZsTM9t0K+tridTz13ko+3bCTJeu2s2nnnn3zdm7TfF9yaN40i9H9c2jVvCm53dsypHs7hnRrS99OrdN6RVCT706St2NwGZHMrazfAk4Evgr8XlIR8LqZ3ZjWyOqINVt3k9OqObOXbeKKB2ZyaM/2fPvE/pw1rFut9Z0fXwY9MiuLCfB5GfSiRQ2yr51YzHj74w2s2bqbVVt2s3zjLlZs2snyjTtZuXkXf/ja4VxwRE8APl63ncfnrNz32dbNm3DwQW0Y0Lk1A7q0YW/M9h38H8k7pla3ozF+d65hSKpYKeo6+2RCkjgV+NTMyj7eM2PS3X3Gtt17ePidT5n0+lLWbw91Ee1bNuPyY/vygzMT9iZeJZUVO0yaNIm8vDxupvK7V+4EJk2aVOf72tm0o5gVm3axfnsRq6MD/5otu1m1dTdtWjThvstGAmFfDP7Z8xTvjVW4nJ9+aSh5J4WmYB+t3c7sZRvpndOKg7u04aC2LerMgbYhfXeu4UlUrJRMO4clwHrCw35eB94zs4r/YzOktvpW2r1nL4/NWcGUmcuZv3ILVx7Xj1+ePwwIB713Pt7AqH45dK7i7YyJih2uv/FG7r37brIXL077fe/Vrc8oKtnL3E83s2F7MRt2FLF+ezHrtxexYXsRG7YXc9MXhzCqX2jc9fsXPuB/XllS4XJyWjdnzs+/sG/42ofnIEH39tn06tiK3jkt6d2xFb06ttrXLUVd5m0WXF1X076V/gScAFwKHAm8Jmm6mVX8H96AZTdrwmWj+3LZ6L4sWLllv7YQLxSs5qbH5wMwoEtrRvTpyJBubRkcvQ5qm13hMg9U7DB27FiAtPW1E4sZ24v2MOnBf5L/t4f4YPYbsDc8nH7wqRcx4qyL6DdwKFt2lbB5VzGbd+5h8849DOvRjomXh9/UruK9XDLx7UrXsWLTLkb1C+/75rQmt3s7OrdtQde2LejWPju82oW/8f7nsgrvoi4nFZX06ajo936SXH2WTCO4e4F7JbUBrgJ+CfQC6v6pWxoN79l+v+E22U057uBOzPl0E0vX7djvQS2dWjdndtwZ8W+f/4AshaKpebPe5p+vzueqPofxnb176Lt5FQft2AzAZU2acUWbdjylLNpILFUWMWVhEnuVxZD1y/Yts6jv4bRu3ZHH5q5i5qaW7C6JsbN4L9t3l3BEnw6cf3ioZF/42VaunzKX7UUlbNtdwvaikmgJXeHUH/LrpUsZvf5TlgJ3terNW5va8NbM5eW2PydqDwDQLrsZR/fPIadVczq1aU6nNi3oEv3t1Lo5g7p+3o7y4qN6c/FRvau93+OlopI+nRX93k+Sq8+SuVvpLsKVQxtgBvALQvGSi3PuYT0497AeFJfEWPDZFgpWbuGD1dtYtHrbfhXXe2PGpNeXsmdvaXFeBw76ys/4L/Bf4JfT/sqVc54B4LnBJzDrnBvpSSibLmvpb88jK3oY3yMnX0nn7gOZPH83zF+433wXF/Xalxwk+HDt9v2mx4p20rp4Fz2LdnBhVhOGReP7LnqLv2z8jFd3b+Pbl3+DL3/pTDq0akaHls3p0Przq6asLDF13LHV3HPVk4qK3nRXFns/Sa4+S6bO4SLC3UlraiekqqtPz3MoLonx71nL2byjmCUrVvPPqY9xZHZbclq0prhJM8bNfIyzF88A4MncU/jdyZez0owmsRh9zMiyGCIMP/3gDWTv3YMBuSdfwZ7eA/ny+eeS3awJ2c2yaNm8Ke2ymzK0ezuOPyR097B7z16WbdhJ2+ymtG7RhONGHknLRR/UqEw8E20vUlHRm+7KYq9zcHVdojoHzOyAL+B84A/R67xkPlObr5EjR1p9NH36dANsGpgleN0YjiF2M1iszLQY2E3R9EmTJlVp/QsWLDDAJhxg/X+Nll9QULDf52OxmOXn59uwoUONaB7Ahg0davn5+RaLxVK5u/Zb77ChQ21kVla5/RG/X0ZkZdnw3NwK40jFMpKRn5+flu/OuVQAZlllx/3KJuybAe4AXgaujl7TgNsP9LnafNXX5JDswfn+uAPviKws+yvYi9FBe0RWlgGWl5dX5QNYssnpxWjd06dP3/fZWCxmeXl5BtjIrCybEC1nQjRc3ZiSUdOklqplJCN+P6Xyu3MuFWqaHOYBWXHDTYB5B/pcbb7qa3Koytlr6dl4RWfpkyZNqtbBpSYHyEyeEdckqaVyGcmKxWI2adKklH53zqVCKpJDTtxwjieH1KnqQTYWi1lBQYFNnz7dCgoKanRgqW7RSm0VyVSmPl05xEvld+dcKtQ0OVwKLAP+DjwIfAx8/UCfq81XfU4OmS52qM4VQCYOrPHqU52Dc3VZjZJD+Dzdo0rp84FuyXymNl/1OTmYZbbYoTrJqTaLZCqTimItryx2jV21kgMwItGrss+l6gWcDSwCPgJuSjRvfU8OpTJV7FDV5JTpK4fSmGt6xZXpqzbnMq26yeGVBK//Vva5VLyiSu8lhDZJzYH3gdzK5m8oySHTkk1OdaVIJhVXXF5Z7BqzRMkh0WNCLzazqZIGmNmBGnmmlKRjgV+a2VnR8M0AZnZHRfPXp0ZwDUVd6m3UrOYPxEnFMpyrb6rb8d5NwFTgUUJRUm3qCcR36LMCGB0/g6SxhMeY0qdPn9qLzAEwZswYZs6cyR35+byQ4DnHV199ddpjScUDcfyhOs7tL1Fy2CDpRaC/pKfLTjSz89MX1oGZ2URgIoQrh0zG0hj5c46da9gSJYdzCFcMDwF31U44+6wE4rvu7BWNc3WIJMaMGcPVV1/tRTLONTCVJgczKwbelnScma2rxZgA3gUGSupPSAqXAN+o5RhckrxIxrmGJ5nnOdR2YsDMSiRdB7xAuHPpATMrqO04nHOusUrmSXAZYWbPAc9lOg7nnGuMshJNlNRE0o21FYxzzrm6IWFyMLO9hL6VnHPONSLJFCu9KekvwL+BfQ9GNrM5aYvKOedcRiWTHI6I/t4WN86A01IfjnPOubogmbuVTq2NQJxzztUdCescACR1lTRZ0n+i4VxJ6e0sxznnXEYdMDkQHvLzAtAjGl4M3JCugJxzzmVeMsmhs5lNBWIQGqgBe9MalXPOuYxKJjnskNSJUAmNpGOALWmNyjnnXEYlc7fSeOBp4GBJbwJdgK+lNSrnnHMZlUxyKABOBgYTnumyiOSuOJxzztVTyRzkZ5hZiZkVmNkCM9sDzEh3YM455zKn0isHSd0IT2RrKelIPn8SZDugVS3E5pxzLkMSFSudBVxJeNDO3XHjtxEeEeycc66BSvSwnweBByV91cweq8WYnHPOZVgy3Wc8JukcYBiQHTf+tso/5Zxzrj5LpvuMvwJfB75HqHf4GtA3zXE555zLoGTuVjrOzC4HNpnZr4BjgUHpDcs551wmJZMcdkV/d0rqAewBuqcvJOecc5mWTCO4ZyR1AH4PzCF0o5Gf1qicc85lVKJ2DjcAbwF3RJ3tPSbpGSDbzLxvJeeca8ASXTn0Au4BhkiaD7xJSBZv1UZgzjnnMqfSOgcz+6GZHQd0A24GNgJXAQskLazJSiV9TVKBpJikUWWm3SzpI0mLJJ1Vk/U455yrnmTqHFoSusxoH70+A+bXcL0LgK8AE+JHSsoFLiG0qegBvCRpkJn58yOcc64WJapzmEg4SG8D3iEUJ91tZptqulIzK4zWUXbSBcAUMysCPpb0EXA03tGfc87VqkS3svYBWgCrgZXACmBzmuPpCSyPG14RjStH0lhJsyTNWrduXZrDcs65xiVR30pnK5zaDwOOA34ADJe0kdCN962JFizpJUJ9RVk/NbOnahBzaXwTgYkAo0aNspouzznn3OcS1jmYmREqoDcTHg26BTiXUNSTMDmY2RnViGcl0DtuuFc0zjnnXC2qtFhJ0vclTZH0KfAaISl8QKhIzklTPE8Dl0hqIak/MBCYmaZ1Oeecq0SiK4d+wP8CN5rZqlSuVNKXgT8Tnkf9rKT3zOwsMyuQNBVYCJQA1/qdSs45V/sUSo7qt1GjRtmsWbMyHYZzztUrkmab2aiKpiXT8Z5zzrlGxpODc865cjw5OOecK8eTg3POuXI8OTjnnCvHk4NzzrlyPDk455wrx5ODc865cjw5OOecK8eTg3POuXI8OTjnnCvHk4NzzrlyPDk455wrx5ODc865cjw5OOecK8eTg3POuXI8OTjnnCvHk4NzzrlyPDk455wrx5ODc865cjw5OOecK8eTg3POuXIykhwk/V7SB5LmSXpCUoe4aTdL+kjSIklnZSI+55xr7DJ15TANGG5mhwGLgZsBJOUClwDDgLOB+yQ1yVCMzjnXaGUkOZjZi2ZWEg2+DfSK3l8ATDGzIjP7GPgIODoTMTrnXGPWNNMBAFcD/47e9yQki1IronHlSBoLjI0Gt0talKb4OgPr07Ts2lDf44f6vw31PX6o/9tQ3+OH9GxD38ompC05SHoJ6FbBpJ+a2VPRPD8FSoCHq7p8M5sITKxRkEmQNMvMRqV7PelS3+OH+r8N9T1+qP/bUN/jh9rfhrQlBzM7I9F0SVcC5wKnm5lFo1cCveNm6xWNc845V4sydbfS2cCPgfPNbGfcpKeBSyS1kNQfGAjMzESMzjnXmGWqzuEvQAtgmiSAt83sGjMrkDQVWEgobrrWzPZmKMZSaS+6SrP6Hj/U/22o7/FD/d+G+h4/1PI26PMSHeeccy7wFtLOOefK8eTgnHOuHE8OlZD0vaiLjwJJv4sbX6+695D0A0kmqXM0LEl/irZhnqQRmY6xIg2lixVJZ0dxfiTppkzHcyCSekt6RdLC6Ld/fTQ+R9I0SR9GfztmOtZEJDWRNFfSM9Fwf0nvRN/DvyU1z3SMiUjqIOnR6H+gUNKxtf0deHKogKRTCa21DzezYcAfovH1qnsPSb2BM4FP40Z/kXAX2EBCI8L7MxBaMup9FytRXP9D2Oe5wKVR/HVZCfADM8sFjgGujWK+CXjZzAYCL0fDddn1QGHc8G+BP5rZIcAmYExGokrevcDzZjYEOJywLbX6HXhyqNh3gDvNrAjAzNZG4+tb9x5/JNwyHH/XwQXAPyx4G+ggqXtGokuggXSxcjTwkZktNbNiYAoh/jrLzFaZ2Zzo/TbCQaknIe4Ho9keBC7MTIQHJqkXcA4wKRoWcBrwaDRLXY+/PXASMBnAzIrNbDO1/B14cqjYIODE6DL0NUlHReN7Asvj5qu0e49Mk3QBsNLM3i8zqd5sQ5yrgf9E7+tT/PUp1nIk9QOOBN4BuprZqmjSaqBrhsJKxj2Ek6JYNNwJ2Bx3slHXv4f+wDrgb1HR2CRJranl76Au9K2UEYm69yDslxzCZfVRwFRJA2oxvKQcYBtuIRQp1Vnp7mLFVZ+kNsBjwA1mtjVqjwSAmZmkOnkPvKRzgbVmNlvSKZmOp5qaAiOA75nZO5LupUwRUm18B402OSTq3kPSd4DHo249ZkqKETq9qlPde1S2DZIOJZx9vB/9U/cC5kg6mjq0DY2gi5X6FOs+kpoREsPDZvZ4NHqNpO5mtioqhlxb+RIy6njgfElfArKBdoTy+w6SmkZXD3X9e1gBrDCzd6LhRwnJoVa/Ay9WqtiTwKkAkgYBzQm9IdaL7j3MbL6ZHWRm/cysH+HHNsLMVhO24fLorqVjgC1xl6p1RgPpYuVdYGB0p0xzQkX60xmOKaGofH4yUGhmd8dNehq4Inp/BfBUbceWDDO72cx6Rb/7S4D/mtllwCvARdFsdTZ+gOj/dLmkwdGo0wm9RtTqd9BorxwO4AHgAUkLgGLgiujMtS5271FVzwFfIlTk7gSuymw4lapPXaxUyMxKJF0HvAA0AR4ws4IMh3UgxwPfAuZLei8adwtwJ6F4dQywDLg4Q/FV10+AKZJ+Dcwlquytw74HPBydVCwl/J9mUYvfgXef4ZxzrhwvVnLOOVeOJwfnnHPleHJwzjlXjicH55xz5XhycM45V44nB1drJHWTNEXSEkmzJT0naZCkU0p7z8w0SbdJStg4L0Xr6SDpuylYzquSUvrQ+UTLjHoKrbS3AEnNJU2X5LfJ13OeHFytiBpXPQG8amYHm9lIQk+rdaqPHjP7hZm9VAur6gBUKTlEDRcz9j8raRjQxMyWVjZP1MHgy8DXay0wlxaeHFxtORXYY2Z/LR1hZu+b2evRYJu4/usfjpIJkn4h6V1JCyRNjBv/qqTfSpopabGkE6PxrSRNVXgewRNR54mjomlnSpohaY6k/436D9qPpL9Luih6/4mkX0Xzz5c0pIL5n5V0WPR+rqRfRO9vk5QnqY2kl+OWUdor653AwZLek/T76DM/irZ1nqRfReP6KTwP4h/AAvbvjqNsLOW2T+F5Ev8bN8++q7Rk9kcZlxG1ypXUV+G5Ap0lZUl6XVJpX15PRvO6esyTg6stw4HZCaYfCdxAeO7BAEJLXYC/mNlRZjYcaEnoa6lUUzM7OvrcrdG47wKboucR/BwYCaDwsKOfAWeY2QhgFjA+ibjXR/PfD/ywgumvE3rwbU9osV0a94nAdGA38OVoGacCd0UJ7iZgiZkdYWY/ig6sAwndfB8BjJR0UrSsgcB9ZjbMzJZVFGSC7XsJGK3QqyeEM/op1dwfxxN9h1Ecv432yw+AhWb2YjTfAkKHla4e83JBV1fMNLMVAFG3Df2AN4BTJf0YaEXoKbcA+L/oM6Wdws2O5gc4gdDRGma2QNK8aPwxhMTzZnTx0RyYkURc8ev4SgXTXwe+D3wMPAt8QVIroL+ZLVLoxO726EAfI3QVXVFR2pnRa2403IaQFD4FlkXP3kikwu2LuvB4HjhP0qOE5xz8GDi5ovkPsI7uhK6kATCzSZK+BlxDSGil4/dKKpbUNnomhKuHPDm42lLA5x2fVaQo7v1eoKmkbOA+YJSZLZf0S0JPm2U/s5cD/5YFTDOzS6sU9YHX8S4witD/zTRC7715fH6VdBnQBRhpZnskfVJmG+Lju8PMJuw3MjxTYUcScSbavinAdcBGYJaZbYuuXqq6P3bFxx4lwdKHMLUB4hNBC8JVk6unvFjJ1Zb/Ai0kjS0dIemw0rqCSpQeiNZH5eGJkkupN4k6JFN4vOWh0fi3geMlHRJNa63Q426NRBWwy4GvEc68XycUP02PZmlPeL7AHoXHz/aNxm8D2sYt6gXg6tJyf0k9JR1UhVASbd9rhOcD5BESxYHmr0whcEjc8G8Jz9n4BZBfOlJSJ0Jx3J4qxO/qGE8OrlZEvdp+GThD4VbWAuAOwhOtKvvMZsJBZwHh4PluEqu6D+giaSHwa8IVyxYzWwdcCfwrKmqaAZSrYK6m1wkJYFf0vlf0F8LBc5Sk+cDlwAcAZraBUKSzQNLvo/L6R4AZ0byPsn/ySCjR9kW91j5DeJb1MweaP4FngVMAJJ1MqFf4rZk9DBRLKu3h99RoXlePea+srkGR1ARoZma7JR1MqJAdHJ3huxqQ1JLwXITjE3WTLulx4CYzW1xrwbmU8zoH19C0Al6JKoIFfNcTQ2qY2S5JtxIq1T+taB6F5w886Ymh/vMrB+ecc+V4nYNzzrlyPDk455wrx5ODc865cjw5OOecK8eTg3POuXL+P46KqxvyGCmGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhU5dn48e89M8kkmYQMkIAILoC7sinFqnXXat2wr7WKWNe61FZt+7pgayv99W2rrdXuKi4Fl1ap1l1bN6h7FVxYBEEUJbIYluzrzNy/P56TMAwJBJKZk5ncn+uaa85+7nNmuc/znHOeI6qKMcYY0x0BvwMwxhiT/SyZGGOM6TZLJsYYY7rNkokxxphus2RijDGm2yyZGGOM6basTyYislxEjvE7jmQi8qyInNvFaXtd/D1FROpEZITfcaSDiBwqIh/6HYcfRKRcRBaLSKHXP1tEvp2G9ewqIioioS5Of4SIVPR0HH2ZiIwWkde7Mm2vSSben2qj9we0RkSmi0ix33FtD1X9mqrO6O5yvH3Q4u2T9SLyvIjs1RMxZoKqFqvqxz293N6QgFX1FVXdM13LF5HjRORlEakVkUoR+Y+InJKu9W2jKcB0VW30OxC/icgPRORjEakRkZUicuuWkp+IHO0l4gYRmSUiuySNC4vIPd6yVovID/2eV1XnAVUicvLW9kWvSSaek1W1GNgfGA9c73M8vcGvvX0yFPgcuLunV9DVI7++RESCPq77G8A/gHuBYcBg4KfAVn/QHSxLRKTHfuciEgbOBe7vqWVmuSeA/VW1H7AfMAa4oqMJRaQM+CfwE2AAMAd4KGmSqcDuwC7AkcA1InK8n/N6HgAu2eqeUNVe8QKWA8ck9f8GeMrrPgVYCFQBs4G9U+cDdgAagIFJ4/YHKoE84DzgVeBmYAPwCfC1pGl3xH0x1gMfARcljZuK+3HfD9QC84E9gOuAL4AVwFeTpp8NfNvrHgm8BKwD1nofTLSz7U7ZJ9OB/0vqPwGoT4n5EW8bPwGuSBpXCMzwtnURcA1QkbLea4F5QDMQAr4MvO7t5/eBI5KmPw/42Nv+T4DJ3vDdgP8A1d72PZQ0jwK7ed2luD/HSuBT3IFCIGnZnX42W/uuJA0P4I6al3n7eyYwIGn8P4DVXqwvA/um7OvbgGeAetx3ajlwlbePqnE/wAJv+iM62J8dTuuNvwZYBawEvp28b1K2QYDPgKu3sP1TgfuT+nf1lhdK+v79AngNaPQ+5zkpy/gB8ITXHfb2/WfAGuB2oLCTdR8GfJQybDbb9n2/2ttP9biDo8HAs7jv1gtA/5Ttutjbb6uAq1K+49O978wH3nKTP5O270KtN/7raf4PG+jF/5dOxl8MvJ7UH/E+n728/pVs+j/yc+BBP+f1+od604e3tP29rWQCgIjshPvjfFdE9gD+DnwfKMf92J8UkfzkeVR1Ne5L/c2kwd/C7ZRWr/9A4EOgDPg1cLeIiDfuQaAC9wf9DeCXInJU0rJOBu4D+gPvAv/G/XkNBf4fcEdnmwP8ylvu3sBOuD+DbSIiEWASLtHhHW0+ifvTHwocDXxfRI7zZrkB92McARwLnN3BYicBJwJR3A/6aeD/cEcvVwGPePXjEeAPuD/4EuBg4D1vGT8HnsPtl2HAHzvZhD/iEsoI4HDgHOD8pPFb+my66nLgVG/5O+L+ZP6cNP5Z3BHYIOAd3B9dsrNwf8IluOQG7vt0PDAcGI1LfJ3pcFrvKO+HuAS1Gy4RdWZP3Hfk4S1M0xXfwv2JlOCSw54isnvS+LOAv3ndN+IOjsZ68Q3FlYQ6Mgr3OXWmK9/303DfyT1wv6tngR/hft8BNj+yPxL3uX0VuDapivMGXPIaCRyHKzElWwYcivve/Qy4X0SGdBi0yFkiUrWF186dbrCbtwaXPMfQ+X/BvrjfKwCqWu/FuK+I9AeGJI/3uvf1eV5U9XOgFffd7Fw6M/U2ZvXlQB3uqPhT4C+4I4+fADOTpgvgqnuOSJrvGK/7DOA1rzuIOwqd4PWfR9IRFVCEO+rZAfeFjwMlSeN/hasXBvdjeD5p3MlerEGvv8RbVjT1SK2D7TwVeDdlu7dUMmny9kkCd8Q+2ht3IPBZyvTXAX/1uj8Gjksa9202P5K+IKn/WuC+lOX9G/cDjXgxnEbKESuutDENGNZB/Ir7cwoCLcA+SeMuAWZv7bPZwnelo5LJIuDopP4huB9BqINpo946SpP29b0drOfspP5fA7d73Ud0sD87m/Ye4FdJ43aj85LJId64gtRxSdNMZeslk/+XMs/9wE+97t1xR+tFuD//emBk0rQHAZ90su4fk3TUup3f98lJ/Y8AtyX1Xw48lrJde6Xs17uTvuPHJ427OPkz6SCW94CJnY3v7svbrz/fwvf2buDGlGGved//nVI/d1zCXe7nvEnDPgcO29L297aSyamqGlXVXVT1MnUn+HbEJRcAVDWBq1Ya2sH8jwP7iMhw3A6pVtW3ksavTlpOg9dZ7K1jvarWJk37aco61iR1NwJrVTWe1N+2rE2IyGAReVBEPveOXu7HHX131c2qGsX9sBrZeHSwC7Bj8tET7uhusDd+R9x+apPc3dGwXYDTU5b3FWCIuiOZM4BLgVUi8nTShQDX4P6Q3hKRhSJyQQfrKcNVNX6aNCx1/3b22WyLXYBHk+JfhDtIGCwiQRG5UUSWeZ/D8qTY2nS0j1YndTdsJabOpu3KZ9Fmnffe4RH0Nkhdx99wJVFwpZLHvP1cjksqc5P227+84R3ZgDt46lAXv++pv6XU/tR9nLwtn+L2J2y+X5O/X4jIOSLyXtJ27ddBLD1GVZfiquP/0skkdUC/lGH9cIm9Lqk/dZyf87YpwR1Qdqq3JZOOrMT9SQDuhCIum36eOqGqNuHqyc/GFfPv24Z1DBCR5B/Jzh2tYzv8Epf5R6k7SXc27s93m6jqZ8CVwO/FXZK5Anf0GE16lajqCd4sq3DVTm126mixSd0rcCWT5OVFVPVGb/3/VtVjcX9yi4E7veGrVfUiVd0RV9r4i4jslrKetbgSwi5Jw3pq/yZbgauKS96GAnXF9LOAibiqplJccoZNPwslPbryWbT5ELcdp21hmnpcAmizQwfTpG7L80C5iIzFJZW2Kq61uD/wfZP2Wam6iz46Mg9XPdWZHvm+p0jeXzvjfq/g9mvqOAC8q5XuBL6HO48aBRZ0FouITBZ31WRnr06ruVKEcNVuHVmIqwZrW2fEm3ahqm7wtmdM0vRjvHn8nBcRGQrks+XqzaxIJjOBE71L2/KA/8WdMO7s2ud7ccW3U+hiMlHVFd7yfiUiBSIyGriQnrlipQSX/au9D+Xq7V2Qqj6P+yFdDLwF1IrItSJS6B157yciX/ImnwlcJyL9vfV+byuLvx84WdwlqUFvPxwhIsO8o82J3pew2dueBICInC4ibX+UG3B/JImUuONePL8QkRLvh/5Durd/87wY214h3LmBX3jLb7sfYqI3fYkX+zrcH/Evu7HubTUTOF9E9haRIlzVbYfU1Sn8EPiJiJwvIv1EJCAiXxGRad5k7wGHicjOIlKKq97cInXnDf+Bu7BlAC65tJX07wRuFZFB4P48ks69pXoLiHrfqY702Pc9yU9EpEhE9sWdZ2u7Ein5Oz4MV0XWJoL7LlYCiMj5uJJJh1T1AXWXsnf2+qyj+UTk20n7bR/cZ/FiJ6t5FNhPRE4TkQLceal5qrrYG38vcL23PXsBF+GqX/2cF9w5yJdUtbmz/QdZkExU9UPc0c0fcUdRJ+MuIW7pZPrXcH9m76jqpx1N04lJuKPVlbidf4OqvtCN0Nv8DHdVWTXuBPc/u7m83+CqlkLASbiTpp/g9s1duKNucBcFVHjjXsCd0O30y+Al1Im4qrJK3NHx1bjvSAD3B7cSd7Xb4cB3vFm/BPxXROpwV8NdqR3fW3I57oj6Y9zJ7b/hziVsr2dwR9Rtr6nA770YnhORWuBN3LklcD+YT3GloQ+8cRmhqs/iLmCYhbuAom3dHX4eqvowrlrxAtw+X4O7MOJxb/zzuD/UecBc4KkuhvI3XMnsH6oaSxp+bVtcXtXUC3RystX73U2n4ws6oOe/7+CuFvwI9yd9s6o+l7SuT3Hf8edIOnhU1Q+A3wJv4PbfKNx5gp52CDBfROpx38lncL8hALyq38leTJW4EucvcAdeBwJnJi3rBtyJ8U9x2/wbVf2Xn/N6JuMO1LZIvJMrOUVEXgL+pqp3+R1LbyEi3wHOVNXD/Y6lrxORvXFVLuGUP/WsICLlwCvAOLUbF3OaV0tzh6oetNVpcy2ZeNU8zwM7pZxQ71PEXQI5AndktjvuKPFPqvo7XwPro0Tk67ij1iLc/T8JVT3V36iM6Tlpq+YSd3v+FyKyIGnYAHFNgiz13vt7w0VE/iAiH4nIPBHZfzvXOQNXRP9+X04knnzc9e61uJvIHqfzq0xM+l2Cu8F1Ge4Ks+9seXJjskvaSiYichjuRNy9qrqfN+zXuEtwbxSRKbg7Xa8VkRNwdeon4Orzfq+qB3a2bGOMMb1L2komqvoy7mRtsom4Ij7e+6lJw+9V503c1SLdvc7eGGNMhmS6gb/BqrrK617NxhvshrLpzUcV3rBVpBCRi3GXxhKJRA7Ya6+saUS3e1bPg8IBUDqMZVXLyAvmsXNJVy99N8aYjebOnbtWVTu7MXW7+NZarKqqiGxzHZuqTsM138H48eN1zpw5PR5br/THA2CHUXD6dC7894W0Jlq592v3+h2VMSYLici23DbRJZm+z2RNW/WV9/6FN/xzNr2TdRg9f3d0douUQ/1aAErDpVQ1b7FlA2OMyahMJ5Mn2Niy57l4N2F5w8/xrur6Mq5Nrc2quPq0SBnUVwIQDUepbq72OSBjjNkonZcG/x13j8OeIlIhIhfimro+VkSW4u7EvdGb/BncndEf4Zp2uCxdcWWtSPlmySTX7hEyxmSvtJ0zUdVJnYw6uoNpFfhuumLJCZFyaFgP8Ril4VLiGqe2tZZ++amNgRrTPa2trVRUVNDU1OR3KKabCgoKGDZsGHl5eWlflz2uNVtEygGFxvX0L+gPQHVTtSUT0+MqKiooKSlh1113Rbb5+WSmt1BV1q1bR0VFBcOHD0/7+np9Q4/GE/Eew1BfSTQcBWBD8wYfAzK5qqmpiYEDB1oiyXIiwsCBAzNWwrRkki0i3iXh9ZWUhl3DwHZFl0kXSyS5IZOfoyWTbNGWTOo2lkzsii5jTG9hySRbJJVM2pKJlUxMLlq3bh1jx45l7Nix7LDDDgwdOrS9v6Wlw8cYtZszZw5XXHFFhiI1yewEfLYoiEIgBPWVlOSXEJCAJROTkwYOHMh7770HwNSpUykuLuaqq65qHx+LxQiFOv7rGj9+POPHj89InGZTVjLJFoEAFLkbFwMSoF9+P6vmMn3Geeedx6WXXsqBBx7INddcw1tvvcVBBx3EuHHjOPjgg/nwQ/d48tmzZ3PSSScBLhFdcMEFHHHEEYwYMYI//OEPfm5CzrOSSTZJalIlGo5aycSk3c+eXMgHK2t6dJn77NiPG07ed5vnq6io4PXXXycYDFJTU8Mrr7xCKBTihRde4Ec/+hGPPPLIZvMsXryYWbNmUVtby5577sl3vvOdjNxz0RdZMskmKU2qVDVZMjF9x+mnn04wGASgurqac889l6VLlyIitLa2djjPiSeeSDgcJhwOM2jQINasWcOwYcMyGXafYckkm0TKYf3HgEsmq+qt+TKTXttTgkiXSCTS3v2Tn/yEI488kkcffZTly5dzxBFHdDhPOBxu7w4Gg8RisXSH2WfZOZNsYi0HGwO4ksnQoUMBmD59ur/BGMCSSXaJlEFrPbTUW8vBpk+75ppruO666xg3bpyVNnqJtD0DPhP61MOxAN65D574Hlz5PndVvMDv3/k9b09+m4JQgd+RmRyyaNEi9t57b7/DMD2ko89TROaqao9eQ20lk2xSPMi916+1JlWMMb2KJZNsktTYY/+w13KwVXUZY3oBSybZpIPGHq3lYGNMb2DJJJsUbd4MvVVzGWN6A0sm2SS/CPKLoX7txpaDm6yayxjjP0sm2ca7C95KJsaY3sSSSbaJlEN9JXnBPIpCRZZMTE5avXo1Z555JiNHjuSAAw7ghBNOYMmSJWld54wZM5g0adImw9auXUt5eTnNzc0dzjN9+nS+973vAXD77bdz7733bjbN8uXL2W+//ba47uXLl/O3v/2tvT8bm9K3ZJJtUhp7tKu5TK5RVb7+9a9zxBFHsGzZMubOncuvfvUr1qxZs8l0PX2z4te//nWef/55Ghoa2oc9/PDDnHzyyZs0y9KZSy+9lHPOOWe71p2aTMaPH591rRxbMsk2kTKo+wKAaIG1HGxyz6xZs8jLy+PSSy9tHzZmzBgOPfRQZs+ezaGHHsopp5zCPvvsQ1NTE+effz6jRo1i3LhxzJo1C4CFCxcyYcIExo4dy+jRo1m6dCn19fWceOKJjBkzhv3224+HHnpok/X269ePww8/nCeffLJ92IMPPsikSZN48sknOfDAAxk3bhzHHHPMZokNXJP3N998MwBz585lzJgxjBkzhj//+c/t0yxfvpxDDz2U/fffn/3335/XX38dgClTpvDKK68wduxYbr311k2a0l+/fj2nnnoqo0eP5stf/jLz5s1rX19vamLfGnrMNpFB0LAWEglrht6k37NTYPX8nl3mDqPgazd2OnrBggUccMABnY5/5513WLBgAcOHD+e3v/0tIsL8+fNZvHgxX/3qV1myZAm33347V155JZMnT6alpYV4PM4zzzzDjjvuyNNPPw249r1STZo0iQceeIAzzjiDlStXsmTJEo466ihqamp48803ERHuuusufv3rX/Pb3/620xjPP/98/vSnP3HYYYdx9dVXtw8fNGgQzz//PAUFBSxdupRJkyYxZ84cbrzxRm6++WaeeuopwD2Xpc0NN9zAuHHjeOyxx3jppZc455xz2h8e1pua2LeSSbaJlIMmoHGDNfZo+qQJEyYwfPhwAF599VXOPvtsAPbaay922WUXlixZwkEHHcQvf/lLbrrpJj799FMKCwsZNWoUzz//PNdeey2vvPIKpaWlmy37xBNP5LXXXqOmpoaZM2dy2mmnEQwGqaio4LjjjmPUqFH85je/YeHChZ3GV1VVRVVVFYcddhgA3/rWt9rHtba2ctFFFzFq1ChOP/10Pvjgg61u76uvvtq+jKOOOop169ZRU1PTHm84HKasrKy9iX2/WMkk20Q2vdfEkolJqy2UINJl33335eGHH+50fHJT9J0566yzOPDAA3n66ac54YQTuOOOOzjqqKN45513eOaZZ7j++us5+uij+elPf7rJfIWFhRx//PE8+uijPPjgg9xyyy0AXH755fzwhz/klFNOYfbs2UydOnW7tu3WW29l8ODBvP/++yQSCQoKuteuXm9qYt9KJtkm6S74aDhKbUstsYS1mmpyx1FHHUVzczPTpk1rHzZv3jxeeeWVzaY99NBDeeCBBwBYsmQJn332GXvuuScff/wxI0aM4IorrmDixInMmzePlStXUlRUxNlnn83VV1/NO++80+H6J02axC233MKaNWs46KCDgE2bvJ8xY8YW449Go0SjUV599VWA9vjaljNkyBACgQD33Xcf8XgcgJKSEmpraztcXvI2zp49m7KyMvr167fFGPxgySTbdNCkSk1Lzz5W1Rg/iQiPPvooL7zwAiNHjmTffffluuuuY4cddths2ssuu4xEIsGoUaM444wzmD59OuFwmJkzZ7LffvsxduxYFixYwDnnnMP8+fPbT8r/7Gc/4/rrr+9w/cceeywrV67kjDPOQEQAd7L79NNP54ADDqCsrGyr2/DXv/6V7373u4wdO5bkltkvu+wyZsyYwZgxY1i8eHF7KWv06NEEg0HGjBnDrbfeusmypk6dyty5cxk9ejRTpkzZajLzizVBn23qKuHm3eBrv+Hp8qFMeWUKj5/6OCNKR/gdmckR1gR9brEm6E3HigYAYi0HG2N6FUsm2SYQhKKBrpqrwGs5uMlaDjbG+MuSSTbymlRpb+zRSibGGJ9ZMslGxeXW2KMxplexZJKNvJJJUaiIUCBkycQY4ztLJtnIa+xRRKyxR2NMr2B3wGejSBk010Brk90Fb3LOunXrOProowHXFH0wGKS83N1f9dZbb5Gfn7/F+WfPnk1+fj4HH3xw2mM1G/mSTETkB8C3AQXmA+cDQ4AHgYHAXOBbqtriR3y9XtuNiw1rLZmYnDNw4MD2hgynTp1KcXExV111VZfnnz17NsXFxZZMMizj1VwiMhS4AhivqvsBQeBM4CbgVlXdDdgAXJjp2LJGSpMqVU2WTExumzt3LocffjgHHHAAxx13HKtWrQLgD3/4A/vssw+jR4/mzDPPZPny5dx+++3ceuutjB07tsMmWEx6+FXNFQIKRaQVKAJWAUcBZ3njZwBTgdt8ia63a08ma63lYJNWN711E4vXL+7RZe41YC+unXBtl6dXVS6//HIef/xxysvLeeihh/jxj3/MPffcw4033sgnn3xCOBymqqqKaDTKpZdeus2lGdN9GU8mqvq5iNwMfAY0As/hqrWqVLWtxcIKYGhH84vIxcDFADvvvHP6A+6NUloOrm6uRlXb2xEyJpc0NzezYMECjj32WADi8ThDhgwBXJtWkydP5tRTT+XUU0/1M8w+L+PJRET6AxOB4UAV8A/g+K7Or6rTgGng2uZKR4y9XnI1V2mUmMaob62nOL/Y37hMztmWEkS6qCr77rsvb7zxxmbjnn76aV5++WWefPJJfvGLXzB/fg8/yMt0mR+XBh8DfKKqlaraCvwTOASIikhbchsGfO5DbNkhvxhChVD3RXvLwVbVZXJVOBymsrKyPZm0traycOFCEokEK1as4Mgjj+Smm26iurqaurq6LTbnbtLHj2TyGfBlESkSVy9zNPABMAv4hjfNucDjPsSWHUTa7zWxJlVMrgsEAjz88MNce+21jBkzhrFjx/L6668Tj8c5++yz25//fsUVVxCNRjn55JN59NFH7QR8hvlxzuS/IvIw8A4QA97FVVs9DTwoIv/nDbs707FllUiZazm4wLUcbCUTk4uSn2j48ssvbza+7QFUyfbYYw/mzZuXzrBMB3y5mktVbwBuSBn8MTDBh3CyU6Qcale1V3NtaLaWg40x/rHmVLKVVXMZY3oRSybZyqvm6pdXgiBWzWV6VDY/gdVslMnP0ZJJtoqUQ6KVYEsdJfkldhe86TEFBQWsW7fOEkqWU1XWrVtHQUFBRtZnDT1mq6S74K3lYNOThg0bRkVFBZWVlX6HYrqpoKCAYcOGZWRdlkyyVcpd8FbNZXpKXl4ew4cP9zsMk2WsmitbFQ9y7/VfEC2wZGKM8Zclk2yV2nKwJRNjjI8smWSrooHu3VoONsb0ApZMslUwDwr7t5dMGmONtMTtWWLGGH9YMslmkfL2ZALWpIoxxj+WTLKZdxe8tRxsjPGbJZNs5t0Fb02qGGP8Zskkm1k1lzGml7Bkks0i5dC4gWiee8LihiZrOdgY4w9LJtnMu9ckGo8DVs1ljPGPJZNs5iWTcGMVhaFCq+YyxvjGkkk2S7oL3m5cNMb4yZJJNrOWg40xvYQlk2yW1HKwlUyMMX6yZJLNCkohkAf1lfQP97eSiTHGN5ZMspnIJnfBb2i2S4ONMf6wZJLtku6Cr2muIZ6I+x2RMaYPsmSS7ZLugleU2pZavyMyxvRBlkyyXfEga+zRGOM7SybZLlLmHt1rycQY4yNLJtkuUg6xJqKBMGBNqhhj/GHJJNultM9lJRNjjB8smWS7tmQSc4/stWRijPGDJZNs590FX9xUR0hClkyMMb6wZJLtvJKJNKylX7ifJRNjjC8smWS7oo3tc1ljj8YYv1gyyXZ5BRDu195ysJVMjDF+sGSSC7y74K3lYGOMXyyZ5IJIOdR9Qf+C/lQ1WTIxxmSeL8lERKIi8rCILBaRRSJykIgMEJHnRWSp997fj9iyUqSsvUmVquYqVNXviIwxfYxfJZPfA/9S1b2AMcAiYArwoqruDrzo9ZuuSGrssTXRSmOs0e+IjDF9TMaTiYiUAocBdwOoaouqVgETgRneZDOAUzMdW9aKlEPDOqJ5/QC7cdEYk3l+lEyGA5XAX0XkXRG5S0QiwGBVXeVNsxoY3NHMInKxiMwRkTmVlZUZCrmXi5QDSilBwJKJMSbz/EgmIWB/4DZVHQfUk1Klpa7Sv8OKf1WdpqrjVXV8eXl52oPNCt5d8NFEArBkYozJPD+SSQVQoar/9fofxiWXNSIyBMB7/8KH2LJTe/tcrYC1HGyMybyMJxNVXQ2sEJE9vUFHAx8ATwDnesPOBR7PdGxZqy2ZtDYDsKHJngVvjMmskE/rvRx4QETygY+B83GJbaaIXAh8CnzTp9iyT/EgAEqb6wErmRhjMs+XZKKq7wHjOxh1dKZjyQkFUZAgoYZ1lOSV2DkTY0zG2R3wuSAQaH98rzWpYozxgyWTXBEpb2/s0aq5jDGZZskkV0TKXGOPBVYyMcZkniWTXJHUpIolE2NMplkyyRVeNVf/cH9LJsaYjLNkkisiZdBSR2moiPrWelrjrX5HZIzpQ7aaTEQkICIHZyIY0w1tNy6Ka5+rusVOwhtjMmeryURVE8CfMxCL6Y6Iu3Ex6prnsodkGWMyqqvVXC+KyGkiImmNxmw/r2RSGo8D1tijMSazuppMLgH+AbSISI2I1IpITRrjMtuqreVgr30uu9fEGJNJXWpORVVL0h2I6aa2ZNLSAFjJxBiTWV1um0tETsE9IRFgtqo+lZ6QzHbJj0BehGhTHQAbmq3lYGNM5nSpmktEbgSuxDUV/wFwpYj8Kp2Bme0QKaOwYQPhYNiquYwxGdXVkskJwFjvyi5EZAbwLnBdugIz28G7C7600JpUMcZk1rbctBhN6i7t6UBMD0hq7NGSiTEmk7paMvkl8K6IzAIEd+5kypZnMRkXKYOV7xIN72nVXMaYjNpqMhGRAJAAvgx8yRt8rff4XdObFA+ChrWU5pfyUfVHfkdjjOlDtppMVDUhIteo6kzcc9pNbxUph7E0gKkAAB7ySURBVESMaKjQSibGmIzq6jmTF0TkKhHZSUQGtL3SGpnZdu3tc+VR1VxFQhM+B2SM6Su6es7kDO/9u0nDFBjRs+GYbmm7cREhoQlqW2opDdu1EsaY9OvqOZMpqvpQBuIx3dFWMom7Ekl1c7UlE2NMRnS11eCrMxCL6a62ZBJzzzKxy4ONMZli50xySeEAQChtbQIsmRhjMsfOmeSSYAiKBhBtrges5WBjTOZ0tdXg4ekOxPSQSDnRRpdErGRijMmULVZzicg1Sd2np4z7ZbqCMt0QKaekYQMBCbChyVoONsZkxtbOmZyZ1J3aqOPxPRyL6QmRcgL17i54q+YyxmTK1pKJdNLdUb/pDSLlUFdJadhaDjbGZM7Wkol20t1Rv+kNIuXQXE00v5+VTIwxGbO1E/BjvGe9C1CY9Nx3AQrSGpnZPm13wYcKWWUlE2NMhmwxmahqMFOBmB7i3bhYGshnkSUTY0yGbMvDsUw2aLsLnqBVcxljMsaSSa5pq+ZKQFO8icZYo88BGWP6Aksmuaa9scc4YHfBG2Myw5JJrgmXQKiAaGszYHfBG2Myw7dkIiJBEXlXRJ7y+oeLyH9F5CMReUhE8v2KLauJQKSc0hZXvWXJxBiTCX6WTK4EFiX13wTcqqq7ARuAC32JKhdEyog21QKWTIwxmeFLMhGRYcCJwF1evwBHAQ97k8wATvUjtpwQKSfa4Nrlqm6ycybGmPTzq2TyO+AaoO0h5QOBKlWNef0VwNCOZhSRi0VkjojMqaysTH+k2ShSTrR+PWAlE2NMZmQ8mYjIScAXqjp3e+ZX1WmqOl5Vx5eXl/dwdDkiUkZefSWRvIglE2NMRnT14Vg96RDgFBE5AdckSz/g90BUREJe6WQY8LkPseWGSDnEW4jm97NkYozJiIyXTFT1OlUdpqq74pq4f0lVJwOzgG94k50LPJ7p2HJGW5MqoSJLJsaYjOhN95lcC/xQRD7CnUO52+d4slfbXfCBsN20aIzJCD+qudqp6mxgttf9MTDBz3hyRmQQAKUSYkWzPW3RGJN+valkYnpKe2OPAavmMsZkhCWTXFQ0EIBoPEFtSy2xRGwrMxhjTPdYMslFoXwoiBKNtQDW2KMxJv0smeSqSDnRlibAkokxJv0smeSqSDnR5nrA7oI3xqSfJZNcFSmjtLEGsGRijEk/Sya5KlJOtMG1z2XVXMaYdLNkkqussUdjTAZZMslVxeUUqRKSkCUTY0zaWTLJVZFyBOifX2zJxBiTdpZMclVbY4/BQqqaLJkYY9LLkkmuamtSJZBvJRNjTNpZMslVbS0HE7CruYwxaedrq8EmjQqiEAhRmlCqYlYyMcakl5VMcpWIuzw4Hqe6uRpV9TsiY0wOs2SSyyJlRFubiWmM+tZ6v6MxxuQwSya5LFJOtKURgA32kCxjTBpZMsllkUFEm2oBa1LFGJNelkxyWaSMaIM7+W6XBxtj0smSSS6LlFPa0gBYMjHGpJclk1wWKScaTwBWzWWMSS9LJrksUk6/RAJBrGRijEkrSya5LFJGECgJFrChya7mMsakjyWTXOa1z9U/WGDVXMaYtLJkksu89rlK7Zkmxpg0s2SSy/IKIb+EqIqVTIwxaWXJJNcVlxNNxK1kYoxJK2s1ONdFyimN1VNFg9+RGGNymJVMcl2knGhLE42xRprjzX5HY4zJUZZMcl2kjGiTazHYHt9rjEkXSya5LlJOtKkGsCZVjDHpY8kk10XKicZjgDWpYoxJH0smuS5SRqnXPpeVTIwx6WLJJNdFyokmLJkYY9Ir48lERHYSkVki8oGILBSRK73hA0TkeRFZ6r33z3RsOSni7jMBq+YyxqSPHyWTGPC/qroP8GXguyKyDzAFeFFVdwde9PpNd0UGEVYotCZVjDFplPFkoqqrVPUdr7sWWAQMBSYCM7zJZgCnZjq2nFTYHyRAaSDfkokxJm18PWciIrsC44D/AoNVdZU3ajUwuJN5LhaROSIyp7KyMiNxZrVAAIrK6E/AkokxJm18SyYiUgw8AnxfVWuSx6mqAtrRfKo6TVXHq+r48vLyDESaAyLllCbsBLwxJn18SSYikodLJA+o6j+9wWtEZIg3fgjwhR+x5aRIGdF4zE7AG2PSxo+ruQS4G1ikqrckjXoCONfrPhd4PNOx5axIOaWtzVYyMcakjR+tBh8CfAuYLyLvecN+BNwIzBSRC4FPgW/6EFtuipQTXdVATTPEE3GCgaDfERljckzGk4mqvgpIJ6OPzmQsfUakjGhrE0ohtS21RAuifkdkjMkxdgd8XxAppzTublzc0LzB52CMMbnIkklfUDyI/l6TKnYS3hiTDpZM+oJIOVFr7NEYk0aWTPqCSBmlXvtclkyMMelgyaQvSCqZ3P/WIt75zM6bGGN6liWTviA/QiRYSEBh3qqV/M9fXuece95i7qeWVIwxPcOSSR/QGk+wVksojAc4blQJU762Fws+r+a0217nW3f/15KKMabbLJn0Ab96ZjErW4sZGAgQymvk0sNH8so1R3Ld1/Zi4cqapKSy3u9QjTFZypJJjnv8vc+557VPKOq/A2Wq7SfgI+EQlxw+klevPZIfnbAXi1bVcNptb3D2Xf/l7eWWVIwx28aSSQ77YGUN1z4yjwnDB7Db8OFEY62bXc1VlB/i4sNG8vI1R/LjE/Zm8eoaTr/9DSbf9SZvfWJJxRjTNZZMclRVQwuX3D+HaGE+fz5rfwLFg4i2NHZ602JRfoiLDhvBK9ccxfUn7s2Hq+v45h1vcNadb/Lfj9dlOHpjTLaxZJKD4gnlygffY011M7edvT/lJeH2JlWqmqtwj4vpWGF+kG8fOoJXrjmS60/cm6Vf1HHGtDeZNO1N3rSkYozphCWTHPS7F5bwnyWVTD1lX8bt3N8NjJQTTcRpTbSyYO2CrS4jOan89KR9+KiyjjOnvcmZ097gjWWWVIwxm7JkkmOeW7iaP770EWeM34lJE3baOCJSxiENTfQLFXHWM2fx/Vnf58P1H251eQV5QS74ynBeueZIbjh5Hz6urGfSnW9yxh1v8PqytVss5Rhj+g7J5j+D8ePH65w5c/wOo9dYVlnHxD+9xsjyCA9dchAFeUnPLVm9AG4/hNr/mcb9up77Ft5HbWstx+5yLJeNuYzd+u/WpXU0tcZ58K3PuO0/y1hT08yE4QP4/tG7c9DIgbjnnhljejsRmauq43t0mZZMckNdc4xT//waG+pbePLyr7BjtHDTCWrXwG/3gBNuhgkXUd1czX0f3Mf9i+6nobWB43c9nkvHXsqI0hFdWl9Ta5yH3l7BbbOXsbqmiQm7DuDKY3bnYEsqxvR6lkxSWDJxVJXLHniH5z5Yw30XTuDgkWWbTxSPwc8HwuFT4Mjr2gdXNVUx44MZPLDoAZrjzZww/AQuHXMpu/TbpUvrbmqNM3POCv4yyyWVL+3anyuP3oNDdrOkYkxvZckkhSUT57bZy7jpX4u5/sS9+fahWyhZ3DQc9v06nHTLZqPWN61n+oLp/H3x32lNtHLSiJO4ZMwl7FSyUwcL2lxzLM7MORX8ZdZHrKpuYkRZhL2GlLBbeTEjBxWz26BiRpYXb1r1ZozxhSWTFJZM4JWllZx7z1ucOHpH/nDm2C2XBv40Acr3hDPu63SStY1ruWfBPcz8cCbxRJyJu03kotEXMbR4aJfiaY7FeXhuBbMWV7Ksso5P19WT8L5iIjCsfyG7lbvk0v4qL6G0KG9bNtsY0w2WTFL09WSyYn0DJ//pVQaXFPDodw+mKD+05RmmnwSJOFzw7FaXXdlQyd0L7mbmhzNRlP/Z7X+4aPRF7BDZYZtibI7FWb62gY++qHOvSve+rLKOlliifbqy4jC7DYqw+6CSTRLNoJKwVZcZ08MsmaToy8mkqTXOabe9zmfrG3jye19h17LI1mf6x3nuqq7Lu77PVtev5q75d/HI0kcQhNN2P41vj/o2gyODtz943I2VFRuSkkxSoqltirVPVxIOtVeTuVKMe99pQBHBgCUZY7aHJZMUfTWZqCr/+4/3efTdz7n73PEctVcX/9ifuRrmzYQpn27zOlfWreTO+Xfy2NLHCEiAb+75TS4cdSFlhR2c7O8GVaWytrk9uSxdszHRVNY2t0+XHwowoizCrgMj7DSgkJ0GFLFT/yKG9S9kWP8iCvPt3IwxnbFkkqKvJpN731jOTx9fyA+O2YMrj9m96zP+59cw6xdwwXOwwyjIL9rmdVfUVjBt3jSeWPYEeYE8ztjzDM7f73wGFg7c5mVtq+rGVldFllSK+XRdPRUbGmlOqjIDV22204DC9gTTlmx2GlDIjtFC8oJ2v67puyyZpOiLyWTO8vWcOe1NDt+jnDvPGU9gW6p6Fj0JD53tuiXoTsYPGQNDxrr3HUZBuLhLi/qs5jPumHcHT338FOFgmEl7TeK8fc+jf0H/7diq7mkrzazY0EjFhgZWrG9gxfpGVmxooGJDIyurGoklNn7PAwI79CtgWFKCce8u8QzuV2BVaCanWTJJ0deSyZqaJk7646sUh0M89t1DKC3cjiugqj+HVe/Byvdg1fuuu26NN1KgbPeNyWXHsbDDaCjo1+niPqn+hNvfv51nP3mWwlAhk/eezLn7nktpuHT7NjINYvEEq2uaNkkwFesbWLHBJZ01tU0k/wzygsLQaKGXXIoYGi1oL8kotE+r6Cbzqbp+be/fdBr1BqaOb+sOBoTSwjwGRPLpX5RP/0g+/Yvy6B/JpyQcsgsRTI+xZJKiLyWTlliCSXe+yaJVNTx62SHsuUNJzy28dvWmyWXle1C7cuP4ASM3JpchY2HIaCjctASyrGoZt79/O/9a/i+K84o5euejKckvoSBUQEGwgIJQAYWhQgpDhZsNKwgWUJjnvYcKCQfDBAOZO+fRHIuzsqrJlWi8BFOxocGVdNY3sK6+pUfXJwLS3i2INyyW2DQ5JQsFhGjRxuTSvygp6aQknv5F+QwoyqekILRtJVfTZ1gySdGXkslPH1/AvW98yp/OGsdJo3dM/wrrvtg0uayaB9WfbRzff9dNq8iGjIXIQJZsWMLt79/Ou1+8S3OsmcZ4I7FErNPVdCY/kN9pAkoeLgiKktDEJi9VJa5xVOMkEjESiRiaiBNPtJJIbByuGieeiJPQOJqIe/O7/kQigXrLCwfyKAyGKQwVUhSKUJhXRGF+CUX5/SgM96OwIEpROEpRfoSiUCFFeRGKQkUUhQqJ5BcRyYtQECogIJ2fq0kklNqmGBsaWljf0EJVQwvr61u99xY2NLSyob6FDQ1tL9efXIWXLCDQvyifqJd4ol6SKS4IEQoKeYGAew8GyAsKoYD3HgwQCrjhIW94fsi9hwKQT4xQogGJ15OI16CxOjTeQDxeSyzW4F7xBkQUCQgSCLrylwiKgAS8d0El4GXWtu6kaUS84ZI0vG1+2t/d5x0nkfA+a/U+R+9zTbSN0ziqCeKJGKpKwvvsVRMkaJsm9XuUIK5xAhIgWjCAAZHBDCzZiYGlOzGg3y4MKBxIJC+yTaXGWDzBuvoWKmubWVvX7L23sLaumbxggClf26vLy9pe6UgmW7kxwfQGD8+t4N43PuWSw0ZkJpEAFA+C3Y91rzb162D1+5uWYj54fOP40p3YY8gYbhkyFvac4O5pScRojTfTFGumKd5EU7yZxngTTfEWGuPN7j3RQlO8haZEK02JVhq1lcZEK02JOE0tMZqaq2nU9TRqnEZNsIE4TSRoVEVRBCWgSlDxuhMEVBFVgu4viIC6JrKFtulcfwBF1P0QxOtvmzYAiCogtAg0BAJUidAYEBokQENAaAxs24n8QglSGMijMJBPUbCAolABhaEiivIiFOQVEcSVVIIiBBAvDkFCUNoPov2E3UiOD+KJBPGYEovF3Xs8TmtMicfixGJxYl53a3Wc9esSrE3EgBaQVlRaQWJoIEZCYqgkSEicWCBBXJRWSdAaUFoFWgWaRWgWoSXHSjyi2r5PA5t0u+9MQoS6Tj7rsMIACRKVMKWBQiJSTFj6EZRS4vEBNMQHsL5lIGvqoqyrK6CqId5hCTSSH2SPnqxxyDBLJr3cgs+r+dGj8zl45ECuPm5Pf4OJDISRR7lXm8YNrtSyykswK9+DxU9tMlue9+rwZyJBCISSXh30B/NShnkvCUIoH0IFEPTe2/pDYQiGOxmW9GqfJnVYUncwBLFmaK7d9NVSR6Kphqam9TQ0bqCxuYqG5loaW6ppaKmjsbWBhtYGGmKNNCaaaYg305BopdFLTA0iLiFJgHUBoUmEBEJCIEHbK7k/ZZwICsTBO4rvQNB7hTsZrUpYIYxQgBBGCBMgX/IpkCD5hMiXEPmSR14gjzzJJxTIJyT5BCVMIFBAQMIEpBCRQggUghSiFBJPBGmNxYnF48TiCZfY4nEv2cVJxOPEvXHxeIxY3CXDeCJGPJ5wiZ3ExoOAtgMHb7jrBlV3GJAgQEKDIEIwkE8oGCQUDBEK5BEKhQgF88gLhsgLhcjPyycUzCM/5F55+SH3HsojPy9Efl4e+fkh8kMh8vJCBFFq1q9kXdVyqusrqG36gsbWdTTEq2mmnqZgE43BOjYEq/k4KKwPBoklfyb5wAD32jEBUQ0SDRQwIBhhQLgfg4rKKC8ezA6luwCHbOlX2GtZNVcvtr6+hZP/+CoAT3zvEAYWd/KP0Ns0VUNdpfsT3ixR5G3a39dOKqtCa2N7MqK5xktOddDa4FXjBL19k/Qusvmwtv0nQVQCJCTgJZgAcVFUhDjiDROXdAICgTwKwlHC4Qh5gd7ZjI2q0hxL0NyaoCkWp6k1TlNrwnuP0xzzur335uTxMdfdHEuep61/0+maWzcupyXl8vLOFOYFKSvJp7w4TFlxmLKSsOsuCVNenM+gIigP1BKOraGhsYL1tZ+zrm4V6xsqWd+8gXUtNayP1bMu0cx6jbM+ADVBd45wWByevWB+OnctYNVcfUosnuCKv79LZV0zD196UPYkEoCCUvcymxNx9/fkFwHda0Vgk8WysRCSC0SEgrwgBXlBSslMwkskNClJbUw6zbEE8YQyMJJPWUmYSH6wC+dIhgB7ALDr1lYca6a1bjXrN3xCQ0tt9zfEJ5ZMeqmbn1vCqx+t5dffGM3oYVG/wzEm5wUCQmF+MPOtJ4TC5EV3YXC0a4996K3sNuBe6Jn5q7j9P8uYfODOfHN815qAN8YYP1ky6WWWrqnlqn+8z7ido9xw8r5+h2OMMV1iyaQXqWlq5ZL75lKUH+K2yQeQH7KPxxiTHXrVv5WIHC8iH4rIRyIyxe94MimRUH740Pt8tr6Bv0zenx1KC/wOyRhjuqzXnIAXkSDwZ+BYoAJ4W0SeUNUP/I1sy9ydtJDw2mVKtLfPlDQ8sWl/wmvAqX088NDbK3hh0RqmnrwPE4YP8HuzjDFmm/SaZAJMAD5S1Y8BRORBYCLgazI55MaXWF/f0nGS6OFbdL4+bijnHrxrzy7UGGMyoDclk6HAiqT+CuDA1IlE5GLgYq+3WUQWZCC2jPgdlP3uTNb6HUcPKYOc2RbIre3JpW2B3NqeTG1Lj1+H3JuSSZeo6jRgGoCIzOnpuzj9lEvbk0vbArm1Pbm0LZBb25PN29KbTsB/DiTfVDHMG2aMMaaX603J5G1gdxEZLiL5wJnAEz7HZIwxpgt6TTWXqsZE5HvAv3FNDN2jqgu3Mtu09EeWUbm0Pbm0LZBb25NL2wK5tT1Zuy1Z3WqwMcaY3qE3VXMZY4zJUpZMjDHGdFvWJpNcaXpFRHYSkVki8oGILBSRK/2OqbtEJCgi74rIU1ufuncTkaiIPCwii0VkkYgc5HdM3SEiP/C+ZwtE5O8ikjXt9ojIPSLyRfK9ZSIyQESeF5Gl3nt/P2PcFp1sz2+879o8EXlURLLm+RNZmUySml75GrAPMElE9vE3qu0WA/5XVfcBvgx8N4u3pc2VwCK/g+ghvwf+pap7AWPI4u0SkaHAFcB4Vd0Pd6HLmf5GtU2mA8enDJsCvKiquwMvev3ZYjqbb8/zwH6qOhpYAlyX6aC2V1YmE5KaXlHVFqCt6ZWso6qrVPUdr7sW92c11N+otp+IDANOBO7yO5buEpFS4DDgbgBVbVHVKn+j6rYQUCgiIaAIWOlzPF2mqi8D61MGTwRmeN0zgFMzGlQ3dLQ9qvqcqsa83jdx99tlhWxNJh01vZK1f8BtRGRXYBzwX38j6ZbfAdcAXXugdu82HKgE/upV290lIhG/g9peqvo5cDPwGbAKqFbV5/yNqtsGq+oqr3s1PfksZP9dADzrdxBdla3JJOeISDHwCPB9Va3xO57tISInAV+o6ly/Y+khIWB/4DZVHQfUk13VKJvwzidMxCXJHYGIiJztb1Q9R919Djlxr4OI/BhXBf6A37F0VbYmk5xqekVE8nCJ5AFV/aff8XTDIcApIrIcV/V4lIjc729I3VIBVKhqW0nxYVxyyVbHAJ+oaqWqtgL/BA72OabuWiMiQwC89y98jqfbROQ84CRgsmbRjYDZmkxypukVERFcnfwiVb3F73i6Q1WvU9Vhqror7jN5SVWz9shXVVcDK0RkT2/Q0fj8SIRu+gz4sogUed+7o8niCwo8TwDnet3nAo/7GEu3icjxuGriU1S1we94tkVWJhPvBFVb0yuLgJldaHqltzoE+BbuKP4973WC30GZdpcDD4jIPGAs8Euf49luXgnrYeAdYD7u9581zXeIyN+BN4A9RaRCRC4EbgSOFZGluJLXjX7GuC062Z4/ASXA895/we2+BrkNrDkVY4wx3ZaVJRNjjDG9iyUTY4wx3WbJxBhjTLdZMjHGGNNtlkyMMcZ0myUTs11EREXkt0n9V4nI1B5a9nQR+UZPLGsr6zndawl4VsrwXUXkrO1c5utdmOauHGjMs523vxZsfUqTyyyZmO3VDPyPiJT5HUgyrwHDrroQuEhVj0wZvivQYTLZ2vJVdat3lKvqt1U1m29+NGYzlkzM9orhbnj7QeqI1JKFiNR570eIyH9E5HER+VhEbhSRySLylojMF5GRSYs5RkTmiMgSr82vtuek/EZE3vae93BJ0nJfEZEn6OAOdRGZ5C1/gYjc5A37KfAV4G4R+U3KLDcCh3o3jf1ARM4TkSdE5CXgRREpFpEXReQdb7kTk9aVvK2zZeOzUB7w7jrHGz6+bXoR+YWIvC8ib4rIYG/4SK9/voj8X9tyO9i2s739956I3OHtoy95+6dARCLinl+yX2dxeyWLxd7ntsSL9RgReU3cc0ImeNNNFZH7ROQNb/hFHcTT2Wc0RERe9uJcICKHdrQ9Joupqr3stc0voA7oBywHSoGrgKneuOnAN5Kn9d6PAKqAIUAY157az7xxVwK/S5r/X7iDnd1xbWQVABcD13vThIE5uEYLj8A1wji8gzh3xDUjUo5ruPEl4FRv3Gzcsz1S5zkCeCqp/zwvhgFefwjo53WXAR+x8Qbg5G2txrUbF8Dd6fyV1PXiGiY82ev+ddL2PQVM8rovbVtuSpx7A08CeV7/X4BzvO7/w7UQ/Gfgui3FjSuJxYBRXqxzgXu8cROBx7x5pgLvA4Xe/Cu8/bsrsMCbprPP6H+BH3vDg0CJ399he/Xsa1uqBIzZhKrWiMi9uAcuNXZxtrfVazJcRJYBbU2gzweSq5tmqmoCWCoiHwN7AV8FRieVekpxyaYFeEtVP+lgfV8CZqtqpbfOB3DPKHmsi/G2eV5V2549IcAvReQwXFP7Q3FNn69OmectVa3w1vse7k/31ZRpWnCJA9yf+LFe90FsfDbH33CJIdXRwAHA216hp5CNDR3+P1wbdk24z2dLcYNrAHK+F+tC3AOnVETme3G3eVxVG4FG71zTBOC9pPGdfUZvA/eIa9T0MVVNnsfkAEsmprt+h2vr6a9Jw2J4VagiEgDyk8Y1J3UnkvoTbPp9TG3nR3F/hper6r+TR4jIEbiSSTolL38yrqRzgKq2imsluaPH3yZva5yOf2+tqqpbmaYzAsxQ1Y6exjcQKAbyvNjqtxJ3dz6X1Jg2+4wAvCR2IjBdRG5R1Xu3vHkmm9g5E9Mt3tH6TNzJ7DbLcUfMAKfg/tC21ekiEvDOo4wAPsQ17Pkd7+gWEdlDtv6wqreAw0WkTNzjnicB/9nKPLW4xvY6U4p7bkuriBwJ7NKF7dlWbwKned2dPVr3ReAbIjII2p+H3hbLHcBPcM/DuKkH457onYsZiKvKeztlfIefkRfXGlW9E/cUzmxuyt90wEompif8FteKc5s7gcdF5H3cuY/tKTV8hksE/YBLVbVJRO7CVbm8453MrmQrj2lV1VUiMgWYhTtqflpVt9ZM+Twg7sU/HdiQMv4B4EmvCmgOsHhbNqyLvg/cL+4hSf/CnX/ZhKp+ICLXA895JcBW4LsicjiuxPM3L4G+LiJH9VDc83D7sgz4uaquFPeE0DadfUZHAFeLSCvufNs527Fu04tZq8HG9EIiUgQ0euctzsSdjJ+4tfnSHNNU3IUAHZ2/MX2clUyM6Z0OAP7kHd1X4Z4HbkyvZSUTY4wx3WYn4I0xxnSbJRNjjDHdZsnEGGNMt1kyMcYY022WTIwxxnTb/wcznMBPq7PRjwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "lambda_ = 3\n",
    "theta = utils.trainLinearReg(linearRegCostFunction, X_poly, y,\n",
    "                             lambda_=lambda_, maxiter=55)\n",
    "\n",
    "# Plot training data and fit\n",
    "pyplot.plot(X, y, 'ro', ms=10, mew=1.5, mec='k')\n",
    "\n",
    "utils.plotFit(polyFeatures, np.min(X), np.max(X), mu, sigma, theta, p)\n",
    "\n",
    "pyplot.xlabel('Change in water level (x)')\n",
    "pyplot.ylabel('Water flowing out of the dam (y)')\n",
    "pyplot.title('Polynomial Regression Fit (lambda = %f)' % lambda_)\n",
    "pyplot.ylim([-20, 50])\n",
    "\n",
    "pyplot.figure()\n",
    "error_train, error_val, error_test = learningCurvetest(X_poly, y, X_poly_val, yval, X_poly_test, ytest, lambda_)\n",
    "pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_val, np.arange(1, 1+m), error_test)\n",
    "\n",
    "pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)\n",
    "pyplot.xlabel('Number of training examples')\n",
    "pyplot.ylabel('Error')\n",
    "pyplot.axis([0, 13, 0, 100])\n",
    "pyplot.legend(['Train', 'Cross Validation', 'Test'])\n",
    "\n",
    "print('Polynomial Regression (lambda = %f)\\n' % lambda_)\n",
    "print('# Training Examples\\tTrain Error\\tCross Validation Error Test Error')\n",
    "for i in range(m):\n",
    "    print('  \\t%d\\t\\t%f\\t%f\\t%f' % (i+1, error_train[i], error_val[i], error_test[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5 Optional (ungraded) exercise: Plotting learning curves with randomly selected examples\n",
    "\n",
    "In practice, especially for small training sets, when you plot learning curves to debug your algorithms, it is often helpful to average across multiple sets of randomly selected examples to determine the training error and cross validation error.\n",
    "\n",
    "Concretely, to determine the training error and cross validation error for $i$ examples, you should first randomly select $i$ examples from the training set and $i$ examples from the cross validation set. You will then learn the parameters $\\theta$ using the randomly chosen training set and evaluate the parameters $\\theta$ on the randomly chosen training set and cross validation set. The above steps should then be repeated multiple times (say 50) and the averaged error should be used to determine the training error and cross validation error for $i$ examples.\n",
    "\n",
    "For this optional (ungraded) exercise, you should implement the above strategy for computing the learning curves. For reference, the figure below  shows the learning curve we obtained for polynomial regression with $\\lambda = 0.01$. Your figure may differ slightly due to the random selection of examples.\n",
    "\n",
    "![](Figures/learning_curve_random.png)\n",
    "\n",
    "*You do not need to submit any solutions for this optional (ungraded) exercise.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = y.shape[0]\n",
    "error_train = np.zeros(m)\n",
    "error_val   = np.zeros(m)\n",
    "error_test  = np.zeros(m)\n",
    "\n",
    "lambda_ = 0.01\n",
    "no_iter = 50\n",
    "for i in range(0, no_iter):\n",
    "    for j in range(1, y.shape[0]+1):\n",
    "        n = j\n",
    "        index = np.random.choice(X.shape[0], n, replace=False) \n",
    "        theta,_ = trainLinearReg(linearRegCostFunction, X_poly[index], y[index], lambda_)\n",
    "        temp,_ = linearRegCostFunction(X_poly[index], y[index],theta, lambda_=0)\n",
    "        error_train[j-1] +=  temp\n",
    "        temp, _ = linearRegCostFunction(X_poly_val, yval,theta, lambda_=0)\n",
    "        error_val[j-1] += temp\n",
    "        temp, _ = linearRegCostFunction(X_poly_test, ytest,theta, lambda_=0)\n",
    "        error_test[j-1] += temp\n",
    "\n",
    "print(error_train)\n",
    "error_train = error_train/no_iter\n",
    "error_val = error_train/no_iter\n",
    "error_test = error_train/no_iter\n",
    "print(error_train)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.67293931e-13 1.07770057e-08 6.37734151e-03 3.70051818e-03\n",
      " 3.73096353e-03 6.90593236e-02 1.00990816e-01 1.54928063e-01\n",
      " 2.12578478e-01 1.99194828e-01 2.17361557e-01 2.21532101e-01]\n",
      "[138.84901847 123.64411017  19.94876402   8.94930585   8.55762914\n",
      "   5.96381874   8.89789996  28.3833515   15.3278164   15.54898137\n",
      "   7.77547154  16.93958258]\n",
      "[1.67294060e-13 2.17943146e-01 6.73821343e-03 9.77629258e-02\n",
      " 1.51629028e-01 1.18213448e-01 1.79356850e-01 2.43573932e-01\n",
      " 3.52250248e-01 3.85890983e-01 3.97718773e-01 4.42980611e-01]\n",
      "[506.0301248  154.04797183  33.518398    59.19617868  16.10631146\n",
      "  40.80277281  15.08838525  67.4792249   36.33637318  30.82482792\n",
      "  23.49695292  33.89193739]\n",
      "[1.67294188e-13 2.17975410e-01 1.33587230e-02 9.81015742e-02\n",
      " 1.61396778e-01 1.26706427e-01 2.57453043e-01 3.53667977e-01\n",
      " 4.81656179e-01 6.01558755e-01 6.07071345e-01 6.64378188e-01]\n",
      "[873.21123113 297.58259187 141.62177549  72.95243969  22.12097915\n",
      "  51.39748281  24.1660468   81.73968989  42.64104585  38.39742386\n",
      "  38.38324781  50.83939983]\n",
      "[1.67302670e-13 2.18200645e-01 1.39357298e-02 1.03082679e-01\n",
      " 2.44924987e-01 2.41899125e-01 2.71276570e-01 5.80798320e-01\n",
      " 6.70534910e-01 8.22634104e-01 8.24435671e-01 8.85816376e-01]\n",
      "[967.97059609 418.36629119 151.15199212  91.9666609   82.61714463\n",
      "  62.54483026  43.42472135  96.68875473  57.09009689  54.97250287\n",
      "  46.16010952  67.78418229]\n",
      "[1.97659446e-13 2.18783706e-01 1.44797754e-02 1.54951547e-01\n",
      " 3.03440364e-01 2.52920919e-01 3.85663838e-01 6.98417663e-01\n",
      " 8.89547303e-01 9.72864627e-01 9.66497966e-01 1.10731701e+00]\n",
      "[1097.26676309  437.73693893  165.75213982  121.67963124   95.06213215\n",
      "   66.83153791   61.84247413  116.84852918   72.99496168   85.13374778\n",
      "   76.93057385   84.72779641]\n",
      "[1.97659446e-13 2.18785423e-01 1.59770220e-02 2.35721325e-01\n",
      " 3.66774155e-01 3.19359954e-01 4.85374710e-01 8.06668997e-01\n",
      " 1.00111507e+00 1.15954618e+00 1.18208582e+00 1.32882853e+00]\n",
      "[1408.19802447  512.74911489  196.13808707  131.69081501  116.40865665\n",
      "   78.7262344    67.81807206  131.55327992  101.21129768  100.4101315\n",
      "   92.30136929  101.67027011]\n",
      "[1.97661745e-13 2.18791887e-01 1.61627632e-02 2.36059974e-01\n",
      " 3.92023023e-01 4.59202465e-01 7.07687437e-01 8.91654208e-01\n",
      " 1.05465589e+00 1.35410981e+00 1.39137822e+00 1.55033555e+00]\n",
      "[1547.18749272  588.24438772  286.67629671  145.44708198  140.28208281\n",
      "   98.75926211  153.87410815  159.84279446  119.58819647  115.07710802\n",
      "  107.19891608  118.57766646]\n",
      "[2.02958524e-13 2.24501308e-01 3.92026298e-02 3.24365263e-01\n",
      " 5.19468634e-01 6.10318693e-01 8.08693095e-01 1.01267886e+00\n",
      " 1.17650895e+00 1.53297279e+00 1.62918394e+00 1.77180464e+00]\n",
      "[1670.10487273  613.44447583  317.8829283   154.71397379  147.74182519\n",
      "  112.87394876  192.09965677  184.61249561  131.47775874  130.50794034\n",
      "  124.54049201  135.5283019 ]\n",
      "[2.08255303e-13 2.25423794e-01 5.00101691e-02 3.24742028e-01\n",
      " 5.42375228e-01 6.81183208e-01 9.04842644e-01 1.14114615e+00\n",
      " 1.37332912e+00 1.73590892e+00 1.84473394e+00 1.99330700e+00]\n",
      "[1793.02225273  644.03628538  361.74634524  169.86867308  172.87863106\n",
      "  130.5094979   197.69801179  210.01325752  144.59304759  146.60604901\n",
      "  139.91183419  152.47004981]\n",
      "[5.16608859e-13 2.25423794e-01 5.58312924e-02 3.25296722e-01\n",
      " 5.46707640e-01 7.38575236e-01 9.42908166e-01 1.25066738e+00\n",
      " 1.57325177e+00 1.93345665e+00 2.02509725e+00 2.21479373e+00]\n",
      "[1924.82922915  766.88340924  372.71235036  180.65438374  193.18643227\n",
      "  148.28035689  204.82394636  229.29648513  159.28474838  162.41489296\n",
      "  155.62916676  169.41667866]\n",
      "[5.20787880e-13 2.25649029e-01 5.60170336e-02 3.76339232e-01\n",
      " 5.71767361e-01 9.20321872e-01 1.06172439e+00 1.46129986e+00\n",
      " 1.70798831e+00 2.13101291e+00 2.26287379e+00 2.43630345e+00]\n",
      "[2007.28170838  887.66710857  463.25056001  193.92568862  211.04605562\n",
      "  202.73498031  235.56980253  241.91841933  179.69236555  178.22280352\n",
      "  172.97656185  186.36131981]\n",
      "[5.51144657e-13 2.26433744e-01 6.57398839e-02 3.84469838e-01\n",
      " 6.12465505e-01 1.09246732e+00 1.07063610e+00 1.55751165e+00\n",
      " 1.92414879e+00 2.32870748e+00 2.50490564e+00 2.65780992e+00]\n",
      "[2136.57787539 1000.59874003  487.85124015  240.74023379  255.18633777\n",
      "  211.63587271  245.30607942  247.80524783  194.37875519  194.18076299\n",
      "  188.76592238  203.30101822]\n",
      "[5.51144785e-13 2.26453347e-01 6.60820543e-02 4.00966114e-01\n",
      " 6.47534411e-01 1.11329895e+00 1.22285060e+00 1.60047837e+00\n",
      " 2.12399330e+00 2.50757508e+00 2.63996440e+00 2.87930016e+00]\n",
      "[2503.75898172 1092.04445609  499.98844094  273.28190897  378.4105969\n",
      "  237.13299394  256.26816086  262.91811777  209.09817505  209.44852428\n",
      "  211.17045348  220.24641818]\n",
      "[7.18438716e-13 2.44006358e-01 7.80142603e-02 4.32777820e-01\n",
      " 7.78399620e-01 1.12437192e+00 1.35810964e+00 1.74766304e+00\n",
      " 2.21653273e+00 2.65723474e+00 2.85557187e+00 3.10079328e+00]\n",
      "[2642.60800019 1214.90569486  545.80764032  291.20179047  384.70832549\n",
      "  246.17195472  269.45678264  274.83877678  227.32382584  219.87436649\n",
      "  226.53704002  237.1568924 ]\n",
      "[7.58192683e-13 4.61949493e-01 2.39528527e-01 4.37799361e-01\n",
      " 8.17480125e-01 1.12599562e+00 1.52762735e+00 1.90562652e+00\n",
      " 2.38810768e+00 2.84639101e+00 3.09764012e+00 3.32228862e+00]\n",
      "[2776.40212486 1245.30955652  563.49254734  300.80808882  397.5562865\n",
      "  259.10022331  289.92905743  279.95118567  241.08652061  234.54356689\n",
      "  242.32132015  254.10053416]\n",
      "[9.25486615e-13 4.61993299e-01 2.39906291e-01 4.72238188e-01\n",
      " 8.20531909e-01 1.17296316e+00 1.65122199e+00 2.02791241e+00\n",
      " 2.48498551e+00 3.08045698e+00 3.33890847e+00 3.54374602e+00]\n",
      "[2915.25114332 1379.77919641  579.11161774  368.97680042  405.60148204\n",
      "  266.2960888   309.99793743  287.65597197  255.53827006  242.44009292\n",
      "  259.30709597  271.05158878]\n",
      "[9.65240582e-13 5.63274079e-01 2.40811429e-01 4.94555090e-01\n",
      " 8.92455327e-01 1.30926496e+00 1.72217127e+00 2.18501757e+00\n",
      " 2.73608864e+00 3.18321282e+00 3.57668382e+00 3.76524414e+00]\n",
      "[3049.04526799 1397.2552294   588.41224024  389.53918458  414.00094677\n",
      "  297.16976996  349.63651237  301.19594089  271.03984338  271.16692166\n",
      "  276.65855102  287.99868066]\n",
      "[1.00499455e-12 5.63274079e-01 2.49517512e-01 4.99060114e-01\n",
      " 9.19726570e-01 1.37158588e+00 1.81443612e+00 2.37634865e+00\n",
      " 2.82850846e+00 3.31548233e+00 3.81796440e+00 3.98671839e+00]\n",
      "[3182.83939267 1520.10235327  675.7725257   533.89254914  431.18276208\n",
      "  371.71320297  377.23350517  314.52229922  276.98154154  291.63535726\n",
      "  293.64222336  304.95110167]\n",
      "[1.00499455e-12 5.64058794e-01 2.50336599e-01 5.03544835e-01\n",
      " 9.65249331e-01 1.50978158e+00 1.85328024e+00 2.44489103e+00\n",
      " 2.96933471e+00 3.51140799e+00 4.05574343e+00 4.20822003e+00]\n",
      "[3493.77065404 1633.03398473  700.05739029  542.61712255  438.19740721\n",
      "  382.96380166  386.67785998  339.52131444  292.20817032  306.22139374\n",
      "  310.98801687  321.86033617]\n",
      "[1.17228848e-12 5.69768214e-01 2.61278877e-01 5.30827251e-01\n",
      " 9.74861134e-01 1.52544811e+00 1.87596521e+00 2.65573100e+00\n",
      " 3.12323952e+00 3.61340707e+00 4.23609387e+00 4.42968850e+00]\n",
      "[3632.61967251 1658.23407284  733.00913098  600.77750437  449.89180571\n",
      "  409.33842133  410.83467019  347.59537316  315.30759693  334.24460983\n",
      "  326.70478366  338.80862672]\n",
      "[1.21204245e-12 5.70451562e-01 2.74268077e-01 5.62650958e-01\n",
      " 1.01445619e+00 1.66453887e+00 1.94464655e+00 2.73300565e+00\n",
      " 3.24330053e+00 3.76301339e+00 4.47388333e+00 4.65117210e+00]\n",
      "[3766.41379718 1681.61865044  773.08826821  619.20938416  463.0138352\n",
      "  418.92282317  419.18104001  374.03608422  321.01689472  348.18014122\n",
      "  344.0596758   355.7587032 ]\n",
      "[1.21205093e-12 5.76160983e-01 2.75094694e-01 5.69259107e-01\n",
      " 1.08452375e+00 1.84524354e+00 2.04566744e+00 2.86443628e+00\n",
      " 3.38144744e+00 3.94016685e+00 4.65427160e+00 4.87269315e+00]\n",
      "[3861.17316214 1706.81873855  814.11827625  629.01364004  479.93483675\n",
      "  433.36024975  428.07027859  380.15199157  334.27619153  361.26263145\n",
      "  359.77340714  372.66205486]\n",
      "[1.21205093e-12 5.82072664e-01 2.84540122e-01 5.71546665e-01\n",
      " 1.10173720e+00 1.96041719e+00 2.19354092e+00 2.98738214e+00\n",
      " 3.54747075e+00 4.07887429e+00 4.86987084e+00 5.09415640e+00]\n",
      "[4172.10442351 1824.69574032  901.3493825   636.15234821  508.88746493\n",
      "  438.67294988  442.08554998  408.72149883  365.69158238  366.96989687\n",
      "  375.13980476  389.61191092]\n",
      "[1.21205108e-12 5.82086960e-01 2.85283053e-01 6.29738373e-01\n",
      " 1.11040864e+00 2.09952375e+00 2.26305043e+00 3.09724126e+00\n",
      " 3.68471919e+00 4.25105911e+00 5.01191782e+00 5.31563554e+00]\n",
      "[4294.94182013 2036.57159025  959.41164966  663.23889627  520.0501167\n",
      "  453.80064076  450.19168679  423.09890836  375.97854733  379.82992046\n",
      "  405.91500934  406.5295524 ]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.37934501e-12 5.82086971e-01 2.85760262e-01 8.19395868e-01\n",
      " 1.13380632e+00 2.11613237e+00 2.42689020e+00 3.17855169e+00\n",
      " 3.83854554e+00 4.50928171e+00 5.25319915e+00 5.53714178e+00]\n",
      "[4433.79083859 2160.21570043  970.57432429  677.01209729  546.96543721\n",
      "  472.93770843  462.12431491  448.61921021  389.41461671  395.59626695\n",
      "  422.90248614  423.47286885]\n",
      "[1.37934731e-12 5.82087129e-01 2.86110983e-01 9.09987775e-01\n",
      " 1.21919745e+00 2.18608914e+00 2.45148032e+00 3.30784395e+00\n",
      " 4.04080701e+00 4.64796611e+00 5.47054143e+00 5.75862750e+00]\n",
      "[4572.78030685 2297.19003869  992.36355381  733.41059545  562.44380612\n",
      "  481.30769933  467.18051909  475.34166507  403.9599822   401.30402323\n",
      "  430.68015632  440.41770473]\n",
      "[1.41910127e-12 5.82087947e-01 2.86772763e-01 9.50413473e-01\n",
      " 1.29433326e+00 2.25561059e+00 2.62958222e+00 3.47929653e+00\n",
      " 4.17643238e+00 4.84719470e+00 5.61262628e+00 5.98011974e+00]\n",
      "[4706.57443152 2394.06325495 1013.15893987  757.1858658   570.9390018\n",
      "  487.53959977  493.85693457  482.51615769  409.73346444  416.8538126\n",
      "  461.43782514  457.36469011]\n",
      "[1.72745483e-12 5.82088370e-01 3.63099023e-01 9.56281088e-01\n",
      " 1.29500977e+00 2.29238813e+00 2.74882191e+00 3.63639767e+00\n",
      " 4.34236968e+00 4.98601378e+00 5.75471991e+00 6.20160265e+00]\n",
      "[4838.38140794 2531.29193597 1039.72679493  767.10451503  586.4467365\n",
      "  514.7237203   517.13781804  496.05530316  441.16704686  444.71367931\n",
      "  492.20023377  474.31466753]\n",
      "[1.72745483e-12 5.82107972e-01 3.63726768e-01 9.65956731e-01\n",
      " 1.38328569e+00 2.45494430e+00 2.79282699e+00 3.81959928e+00\n",
      " 4.59868573e+00 5.24994095e+00 5.89683929e+00 6.42311142e+00]\n",
      "[5149.31266931 2622.73765203 1098.02550877  849.14219966  633.7923497\n",
      "  570.34166447  540.06024172  503.12695942  455.64255253  460.94680575\n",
      "  522.9541979   491.25814742]\n",
      "[1.75781161e-12 5.82151778e-01 4.19340248e-01 9.70977774e-01\n",
      " 1.40997481e+00 2.63397799e+00 2.85723112e+00 4.01503013e+00\n",
      " 4.73432075e+00 5.39734705e+00 6.07720455e+00 6.64465183e+00]\n",
      "[5278.60883632 2757.20729192 1124.44901855  858.74824945  652.78546825\n",
      "  583.81568973  557.16557864  514.81404859  461.42057271  483.45060963\n",
      "  538.6755837   508.20218365]\n",
      "[1.75781175e-12 5.82151835e-01 4.19916395e-01 9.71302070e-01\n",
      " 1.47263269e+00 2.71830875e+00 2.90095128e+00 4.18837527e+00\n",
      " 4.87649787e+00 5.50013392e+00 6.21925008e+00 6.86616367e+00]\n",
      "[5401.44623293 2907.06418999 1149.79133018  873.78984883  683.1030118\n",
      "  592.30080886  574.19269351  542.27748866  488.08233136  512.15625594\n",
      "  569.45089638  525.14633268]\n",
      "[1.76310853e-12 5.82151846e-01 4.23647114e-01 1.02318499e+00\n",
      " 1.48089817e+00 2.75181173e+00 2.91756383e+00 4.37038426e+00\n",
      " 5.07439189e+00 5.73893604e+00 6.43662510e+00 7.08766155e+00]\n",
      "[5524.36361294 3030.70830016 1265.50998044  903.48005091  712.78449541\n",
      "  598.41213166  599.45471232  571.28355213  501.16554216  526.68839841\n",
      "  577.22183775  542.05494416]\n",
      "[1.80286250e-12 5.82152269e-01 4.23934887e-01 1.09578590e+00\n",
      " 1.50433715e+00 2.83614096e+00 2.97318056e+00 4.47938811e+00\n",
      " 5.24590918e+00 5.96632681e+00 6.62386251e+00 7.30916494e+00]\n",
      "[5658.15773761 3167.93698118 1312.45092328  963.09076416  738.13923865\n",
      "  606.89733748  605.7277103   590.89689926  515.29421849  541.85804162\n",
      "  592.96118949  558.99898692]\n",
      "[1.80287098e-12 5.82152691e-01 4.24189100e-01 1.13815609e+00\n",
      " 1.58586021e+00 2.96694151e+00 3.02211253e+00 4.49628499e+00\n",
      " 5.40012298e+00 6.11598769e+00 6.80417612e+00 7.53063948e+00]\n",
      "[5752.91710257 3305.16566221 1358.53180624  985.32156863  764.31591122\n",
      "  625.50333909  624.07987444  615.46023119  541.21305506  552.2853689\n",
      "  608.68094584  575.91092958]\n",
      "[1.80287098e-12 5.82937406e-01 4.77957363e-01 1.13852840e+00\n",
      " 1.73120542e+00 2.97223969e+00 3.14799211e+00 4.60528234e+00\n",
      " 5.56590673e+00 6.26559212e+00 6.99141420e+00 7.75215118e+00]\n",
      "[6063.84836394 3418.09729367 1382.56801528  996.51982025  777.12710368\n",
      "  631.63395851  638.61147026  635.07448094  555.44633636  566.21680135\n",
      "  624.42036109  592.81881935]\n",
      "[1.80287328e-12 5.82981211e-01 4.81111305e-01 1.13899102e+00\n",
      " 1.74081475e+00 3.00777413e+00 3.30762550e+00 4.66090136e+00\n",
      " 5.72543600e+00 6.40913968e+00 7.17865202e+00 7.97365686e+00]\n",
      "[6202.83783219 3552.56693356 1445.06173676 1010.36720373  788.81097297\n",
      "  644.42638654  650.58196487  649.37037094  577.80828907  588.21546175\n",
      "  640.15989334  609.76196377]\n",
      "[1.97016721e-12 5.82995507e-01 4.88530699e-01 1.13981996e+00\n",
      " 1.79073272e+00 3.03588329e+00 3.45404942e+00 4.81662830e+00\n",
      " 5.94315261e+00 6.58131889e+00 7.36584740e+00 8.19513843e+00]\n",
      "[6341.68685066 3764.4427835  1530.49560104 1020.11178772  836.78302208\n",
      "  649.80152487  661.08177428  676.0832814   592.56374749  601.07628155\n",
      "  655.88517977  626.71266138]\n",
      "[1.97016951e-12 5.84482995e-01 4.88784968e-01 1.16376033e+00\n",
      " 1.79747811e+00 3.05668630e+00 3.53440907e+00 5.03248197e+00\n",
      " 6.01366056e+00 6.73194212e+00 7.55307914e+00 8.41662330e+00]\n",
      "[6480.67631892 3877.00714381 1576.57667484 1050.76419086  842.68579053\n",
      "  672.89865871  672.42259981  688.48365371  611.00242208  632.36331342\n",
      "  671.61737566  643.66320385]\n",
      "[1.97016951e-12 6.85763776e-01 4.99660184e-01 1.24279164e+00\n",
      " 1.80222498e+00 3.12278185e+00 3.66549280e+00 5.19303302e+00\n",
      " 6.09190883e+00 6.92108847e+00 7.73343564e+00 8.63813457e+00]\n",
      "[6791.60758029 3894.48317681 1601.26690711 1061.40722992  847.21162259\n",
      "  703.25116942  698.95598018  715.87340585  625.45225302  647.02807187\n",
      "  687.33285669  660.60718274]\n",
      "[1.97016951e-12 6.86318553e-01 5.02019011e-01 1.24693607e+00\n",
      " 1.88065521e+00 3.12808469e+00 3.72012982e+00 5.41731390e+00\n",
      " 6.19311510e+00 7.10835571e+00 7.87547700e+00 8.85960011e+00]\n",
      "[7102.53884166 3949.26010751 1780.36120774 1071.04092333  853.64224049\n",
      "  709.38254627  716.78615162  730.89647042  646.48318923  654.63055789\n",
      "  718.11063247  677.55290589]\n",
      "[1.97016964e-12 7.02559927e-01 5.07075574e-01 1.43722789e+00\n",
      " 2.06905075e+00 3.22575561e+00 3.87307170e+00 5.48004751e+00\n",
      " 6.39098020e+00 7.34538666e+00 8.08483659e+00 9.08108341e+00]\n",
      "[7252.20016541 4032.26260394 1795.94031576 1085.53354499  863.84636931\n",
      "  726.80341943  730.27582668  755.21638575  661.97801525  662.0812407\n",
      "  732.99590121  694.49958038]\n",
      "[1.97434866e-12 7.04047415e-01 5.07082846e-01 1.44583989e+00\n",
      " 2.08651534e+00 3.28252004e+00 4.11309971e+00 5.61313791e+00\n",
      " 6.61459628e+00 7.53264706e+00 8.26295091e+00 9.30258900e+00]\n",
      "[7334.65264465 4144.82696425 1932.33441812 1119.30567883  957.1828207\n",
      "  740.30796329  742.93427487  766.5796994   671.95695878  669.6861168\n",
      "  747.55564712  711.44231976]\n",
      "[1.97435714e-12 7.05534903e-01 5.07095432e-01 1.44612394e+00\n",
      " 2.11702311e+00 3.30058765e+00 4.11970478e+00 5.73436059e+00\n",
      " 6.77528972e+00 7.72726200e+00 8.50422510e+00 9.52409456e+00]\n",
      "[7429.41200961 4257.39132456 2055.87133838 1134.94630402  972.38637909\n",
      "  759.50172095  751.76421421  775.9370414   682.11921251  684.33419575\n",
      "  764.53892422  728.38556816]\n",
      "[1.97436562e-12 7.05539864e-01 5.33480579e-01 1.44856265e+00\n",
      " 2.12976410e+00 3.38535535e+00 4.21159382e+00 6.02569866e+00\n",
      " 6.97305899e+00 7.99121708e+00 8.68234770e+00 9.74555418e+00]\n",
      "[7524.17137457 4351.8032523  2085.48862286 1149.82052643  996.26105797\n",
      "  874.00932205  777.57210947  783.54222081  697.63800925  700.5723662\n",
      "  779.10024353  745.33525294]\n",
      "[1.97436575e-12 7.06125895e-01 5.87432959e-01 1.44898998e+00\n",
      " 2.25573925e+00 3.49806647e+00 4.33518787e+00 6.10105972e+00\n",
      " 7.06543142e+00 8.18704823e+00 8.92362121e+00 9.96704886e+00]\n",
      "[7673.83269832 4446.90830086 2105.89136059 1162.86546025 1002.20727278\n",
      "  881.47186319  797.64104636  801.85122771  725.04366741  715.19844212\n",
      "  796.0827836   762.28261646]\n",
      "[2.28271931e-12 7.06125953e-01 5.87445545e-01 1.44941345e+00\n",
      " 2.26293284e+00 3.50686013e+00 4.45976190e+00 6.21261057e+00\n",
      " 7.30834379e+00 8.37880975e+00 9.10397729e+00 1.01885319e+01]\n",
      "[7805.63967474 4596.76519892 2229.42828086 1177.34872595 1011.33577093\n",
      "  891.31060829  819.15105809  825.80774221  741.97941185  730.3017039\n",
      "  811.79585495  779.2326877 ]\n",
      "[2.28689833e-12 7.22367327e-01 5.96355003e-01 1.44972398e+00\n",
      " 2.47974642e+00 3.56564427e+00 4.58661762e+00 6.30087340e+00\n",
      " 7.49357130e+00 8.55767499e+00 9.24606625e+00 1.04099801e+01]\n",
      "[7888.09215397 4679.76769536 2267.46853512 1191.89409145 1022.11103258\n",
      "  908.14342389  830.04395332  835.68049345  755.06044986  745.7320927\n",
      "  842.55632373  796.18058065]\n",
      "[2.28689847e-12 7.28279008e-01 5.96848724e-01 1.45007052e+00\n",
      " 2.54153374e+00 3.57361607e+00 4.60987687e+00 6.40981035e+00\n",
      " 7.69860507e+00 8.79251148e+00 9.47932885e+00 1.06314660e+01]\n",
      "[8010.92955059 4797.64469712 2282.47297803 1205.55018172 1046.20165567\n",
      "  914.09467595  853.62895924  856.85011309  770.6438437   761.45281878\n",
      "  859.19957613  813.13069333]\n",
      "[2.45419241e-12 7.83735840e-01 5.96851443e-01 1.45032050e+00\n",
      " 2.62491160e+00 3.61341645e+00 4.75197215e+00 6.60842123e+00\n",
      " 7.86067406e+00 8.97418103e+00 9.65967458e+00 1.08529607e+01]\n",
      "[8149.77856906 4816.59396875 2421.48179387 1220.63140139 1052.43396511\n",
      "  938.9751205   866.63227814  894.3399887   794.1831338   775.02252546\n",
      "  874.92017258  830.07986632]\n",
      "[2.45419254e-12 7.84331493e-01 6.05558147e-01 1.59013547e+00\n",
      " 2.64787348e+00 3.66230676e+00 4.78765363e+00 6.71235402e+00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 7.98296751e+00 9.18449898e+00 9.89294739e+00 1.10744643e+01]\n",
      "[8299.43989281 4943.08031494 2508.84175562 1248.78066725 1105.87954185\n",
      "  971.45912706  872.24465347  912.60072599  806.47353321  782.30516641\n",
      "  891.56542074  847.02363175]\n",
      "[2.45419254e-12 7.84331493e-01 6.05558147e-01 1.59013547e+00\n",
      " 2.64787348e+00 3.66230676e+00 4.78765363e+00 6.71235402e+00\n",
      " 7.98296751e+00 9.18449898e+00 9.89294739e+00 1.10744643e+01]\n",
      "[8299.43989281 4943.08031494 2508.84175562 1248.78066725 1105.87954185\n",
      "  971.45912706  872.24465347  912.60072599  806.47353321  782.30516641\n",
      "  891.56542074  847.02363175]\n",
      "[4.90838507e-14 1.56866299e-02 1.21111629e-02 3.18027094e-02\n",
      " 5.29574695e-02 7.32461352e-02 9.57530726e-02 1.34247080e-01\n",
      " 1.59659350e-01 1.83689980e-01 1.97858948e-01 2.21489286e-01]\n",
      "[165.98879786  98.8616063   50.17683511  24.97561334  22.11759084\n",
      "  19.42918254  17.44489307  18.25201452  16.12947066  15.64610333\n",
      "  17.83130841  16.94047264]\n"
     ]
    }
   ],
   "source": [
    "#choosing i examples from training and calculating training error for those i examples\n",
    "m = y.shape[0]\n",
    "error_train = np.zeros(m)\n",
    "error_val   = np.zeros(m)\n",
    "error_test  = np.zeros(m)\n",
    "\n",
    "lambda_ = 0.01\n",
    "no_iter = 50\n",
    "for i in range(0, no_iter):\n",
    "    for j in range(1, y.shape[0]+1):\n",
    "        n = j\n",
    "        index = np.random.choice(X.shape[0], n, replace=False)\n",
    "#         print(index)\n",
    "        theta,_ = trainLinearReg(linearRegCostFunction, X_poly[index], y[index], lambda_)\n",
    "        temp1,_ = linearRegCostFunction(X_poly[index], y[index],theta, lambda_=0)\n",
    "        error_train[j-1] +=  temp1\n",
    "        temp2, _ = linearRegCostFunction(X_poly_val, yval,theta, lambda_=0)\n",
    "        error_val[j-1] += temp2\n",
    "        \n",
    "    print(error_train)\n",
    "    print(error_val)\n",
    "#         temp, _ = linearRegCostFunction(X_poly_test, ytest,theta, lambda_=0)\n",
    "#         error_test[j-1] += temp\n",
    "\n",
    "\n",
    "print(error_train)\n",
    "print(error_val)\n",
    "error_train = error_train/no_iter\n",
    "error_val = error_val/no_iter\n",
    "error_test = error_test/no_iter\n",
    "print(error_train)\n",
    "print(error_val)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# X_poly, y, , yval, X_poly_test, ytest, lambda_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[  1.38915437],\n",
       "        [-29.15297922],\n",
       "        [-34.70626581],\n",
       "        [-15.93675813],\n",
       "        [  7.01350208],\n",
       "        [-48.05882945],\n",
       "        [-44.38375985],\n",
       "        [ 36.18954863],\n",
       "        [ 15.30779289],\n",
       "        [ 37.49218733]]),\n",
       " array([ 3.74017167,  1.17325668,  2.61418439,  2.13431051,  7.62765885,\n",
       "         2.80896507,  3.73169131, 34.35910918, 14.71026831, 36.83795516]))"
      ]
     },
     "execution_count": 269,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n = 10\n",
    "index = np.random.choice(X.shape[0], n, replace=False) \n",
    "X[index], y[index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 288,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial Regression (lambda = 0.010000)\n",
      "\n",
      "# Training Examples\tTrain Error\tCross Validation Error Test Error\n",
      "  \t1\t\t0.000000\t165.988798\t0.000000\n",
      "  \t2\t\t0.015687\t98.861606\t0.000000\n",
      "  \t3\t\t0.012111\t50.176835\t0.000000\n",
      "  \t4\t\t0.031803\t24.975613\t0.000000\n",
      "  \t5\t\t0.052957\t22.117591\t0.000000\n",
      "  \t6\t\t0.073246\t19.429183\t0.000000\n",
      "  \t7\t\t0.095753\t17.444893\t0.000000\n",
      "  \t8\t\t0.134247\t18.252015\t0.000000\n",
      "  \t9\t\t0.159659\t16.129471\t0.000000\n",
      "  \t10\t\t0.183690\t15.646103\t0.000000\n",
      "  \t11\t\t0.197859\t17.831308\t0.000000\n",
      "  \t12\t\t0.221489\t16.940473\t0.000000\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd5xU5bnA8d+zy7JLWVhg6QsLooLShdBWFCzR2DBXjWI3icQSS7zWXBPJvYm9xFQ1FjBiIfaaiMomoCICIkWahbI0kd5Z2Of+8Z7ZPTvMbJvZOTOzz/fzmc+cPs97ZuY85z3lPaKqGGOMMbHICDoAY4wxqc+SiTHGmJhZMjHGGBMzSybGGGNiZsnEGGNMzCyZGGOMiVnKJxMRWS4iJwQdh5+IvCMil9Rw2qSLP15EZIeIHBJ0HPVBREaKyJKg4wiCiLQVkcUi0sTrLxaRn9bD53QTERWRRjWcfpSIlMQ7joZMRPqJyEc1mTZpkom3Ud3tbYDWi8gEEWkedFx1oao/UNWJsS7HWwf7vHWySUSmiEiveMSYCKraXFW/jvdykyEBq+o0Ve1ZX8sXkZNE5D8isl1ENojIv0XkjPr6vFq6FZigqruDDiRo4twjIhu91z0iIlVMf76IrBCRnSLyqoi09o37uYjMEpG9IjIhwrzHe0l8l4hMFZFC37hsEXlSRLaJyDoRuSEe86rqPGCLiJxe3bpImmTiOV1VmwNHAYOB2wOOJxnc662TzsBq4Il4f0BN9/waEhHJDPCzzwb+ATwNFADtgV8D1f6hIyxLRCRu/3MRyQYuAZ6J1zJT3DjgTKA/0A/3Hf0s0oQi0ht4FLgI953uAv7im2QN8FvgyQjz5gMvA78CWgOzgBd8k4wHDgMKgdHAzSJycqzzeiZFK1MlqpoUL2A5cIKv/z7gTa/7DGAhsAUoBo4Inw/o4H05bXzjjgI2AFnApcB04H5gM/AN8APftJ2A14FNwJfA5b5x43F/7meA7cB84HDgNuBbYBXwfd/0xcBPve4ewAfARuA774vJi1busHUyAfitr/8UYGdYzC95ZfwGuNY3rgkw0SvrIuBmoCTsc28B5gF7gUbAMOAjbz1/DozyTX8p8LVX/m+AC7zhhwL/BrZ65XvBN48Ch3rdLXEbxw3ACtyOQoZv2VG/m+p+K77hGbi95q+89T0ZaO0b/w9gnRfrf4DeYev6r8DbwE7cb2o5cKO3jrbi/oA53vSjIqzPiNN6428G1uI2GD/1r5uwMgiwEripivKPB57x9XfzltfI9/v7HfAhsNv7nmeFLeMXwOted7a37lcC64FHgCZRPvsY4MuwYcXU7vd+k7eeduJ2jtoD7+B+W+8BrcLKNc5bb2uBG8N+4xO838wX3nL930not7DdG//DethufQSM8/X/BJgRZdo7gWd9/T2AfUBu2HS/xdX8/MPGAR/5+pt5320vr38NlbdB/wc8H+u8Xn9nb/rsqtZFstVMABCRLrgN52cicjjwHHA90Bb3Z39DRBr751HVdbgf9Y98gy/CrZRSr38osATIB+4FnvBVSZ8HSnAb6LOBO0XkON+yTgf+DrQCPgP+hdt4dQb+F7fHEbE4wF3eco8AuuA2BrUiIs2AsbhEh7e3+QZuo98ZOB64XkRO8ma5A/dnPAQ4EbgwwmLHAqcCebg/9Fu4H3Jr3IbxJe/4eDPgD7gNfC4wApjrLeP/gHdx66UA+GOUIvwRl1AOAY4FLgYu842v6rupqWtwe4nH4tb3ZuDPvvHv4PbA2gFzcBs6v/NxG+FcXHID93s6GeiO2/O8tIrPjzitt5d3Ay5BHYpLRNH0xP1GXqximpq4CLcRycUlh54icphv/PnAs1733bidowFefJ1xNaFI+uK+p2hq8ns/C/ebPBz3v3oH+CXu/50BXBs2/Wjc9/Z94BbfIc47cBvkHsBJuBqT31fASNzv7jfAMyLSMWLQ7vDTlipeXaOUtzfuPxjyuTes2mlV9StcMjk8yvRVzbvTK19vEWkFdKwijljmRVVXA6W432Z08c7UMWT45cAO3F7xClz1rwmuajbZN10G7nDPKN98J3jd5wIfet2ZuL3QIV7/pfj2qICmuL2eDrgf/AF8ewi4P8QErdgTnOIbd7oXa6bXn+stKy98Ty1COc8EPgsrd1U1kz3eOinD7bH388YNBVaGTX8b8JTX/TVwkm/cTzl4T/rHvv5bgL+HLe9fuD9oMy+GswjbY8XVNh4DCiLEr7iNUybuT3Okb9zPgOLqvpsqfiuRaiaLgON9/R1xf4JGEabN8z6jpW9dPx3hcy709d8LPOJ1j4qwPqNN+yRwl2/coUSvmRR543LCx/mmGU/1NZP/DZvnGeDXXvdhuL31priN/06gh2/a4cA3UT77f/Dttdbx936Br/8l4K++/muAV8PK1StsvT7h+42f7Bs3zv+dRIhlLjAm2vi6vHDbDX98h3kxS4Rp3weuCBtWvi3zDYtUM3kCuDts2Ie4/06X8N8MLlkvj3XesDiPqWpdJFvN5ExVzVPVQlW9St0Jvk645AKAqpbhDit1jjD/a8CRItIdt0K2qupM3/h1vuXs8jqbe5+xSVW3+6ZdEfYZ633du4HvVPWArz+0rEpEpL2IPC8iq0VkG+5PnR+l/JHcr6p5uD/Wbir2DgqBTv69J9zeXXtvfCfcegrxd0caVgicE7a8o4GO6vZkzgWuANaKyFu+CwFuxm2QZorIQhH5cYTPyccdalzhGxa+fqN9N7VRCLzii38R7s/eXkQyReRuEfnK+x6W+2ILibSO1vm6d1UTU7Rpa/JdhGz03iPuQddC+Gc8i6uJgquVvOqt57a4pDLbt97+6Q2PZDNu5ymiGv7ew/9L4f3h69hflhW49QkHr1f/7wsRuVhE5vrK1SdCLLHaAbTw9bcAdqi3Ba5m2tD02yNMW5t5d/j6Iy03lnlDcnE7lFElWzKJZA1uIwG4E4q4bLo6fEJV3YM7Tn4hrpr/91p8RmsR8f9Jukb6jDq4E5f5+6pqCy+22h6+QVVXAtcBD4u7JHMVbu8xz/fKVdVTvFnW4g47hXSJtFhf9ypczcS/vGaqerf3+f9S1RNxG7nFwN+84etU9XJV7YSrbfxFRA4N+5zvcDWEQt+weK1fv1W4Q3H+MuSoq6afD4zBHWpqiUvOUPm7iLQBiIeafBchS3DlOKuKaXbiEkBIhwjThJdlCtBWRAbgkkroENd3uA14b986a6nuoo9I5lH1YZm4/N7D+NdXV9z/Fdx6DR8HgHe10t+An+POo+YBC6LFIiIXiLtqMtor2mGuhbiT7yH9vWHVTivusvlsYGmU6auatxnu8N5CVd2MWxfR4ohlXkSkM9CYqg9vpkQymQyc6l3algX8N+6EcbRrn5/GVd/OoIbJRFVXecu7S0RyRKQf7kRaPK5YycVl/63el3JTXRekqlNwf6RxwExgu4jcIiJNvD3vPiLyPW/yycBtItLK+9yfV7P4Z4DTxV2Smumth1EiUuDtbY7xfoR7vfKUAYjIOSIS2lBuxm1IysLiPuDF8zsRyfX+6DcQ2/rN8mIMvRrhzg38LnTZo3e+Z4w3fa4X+0bchvjOGD67tiYDl4nIESLSFHfoNiJvj/YG4FcicpmItBCRDBE5WkQe8yabCxwjIl1FpCXu8GaV1J03/AfuwpbWuOQSqun/DXhIRNqB23j4zr2Fmwnkeb+pSOL2e/f5lYg0FXc11GVUXInk/40X4A6RhTTD/RY3AIjIZbiaSUSqOkndpezRXiujzPo0cIO3zjrhtk8Tokw7CfcfG+n9l/4XeDl0REREGolIDu6wcKbvdw3wCtBHRM7ypvk1ME9VF/viuN1bF72Ay31xxDIvuHOQH6jq3mjrD1IgmajqEtzezR9xe1Gn4y4h3hdl+g9xG7M5qroi0jRRjMXtra7Brfw7VPW9GEIP+Q3uqrKtuBPcL8e4vPtwh5YaAafhTpp+g1s3j+P2usH9UEu8ce/hTuhG/TF4CXUM7lDZBtze8U2430gGbgO3Bne127HAld6s3wM+EZEduKvhrtPI95Zcg9uj/hp3cvtZIlwCWQtv4/aoQ6/xwMNeDO+KyHZgBu7cErg/zApcbegLb1xCqOo7uAsYpuIuoAh9dsTvQ1VfxB1W/DFuna/HHUd/zRs/BbdBnQfMBt6sYSjP4mpm/1DV/b7ht4Ti8g5NvUeUk63e/24CkS/ogPj/3sFdLfgl7pzD/ar6ru+zVuB+4+/i23lU1S+AB4CPceuvL+48Qbw9irsQZj6u5vMWvotxvFrNSC+mhbhDxZNwV4HmAlf5lnU77rd8K2797vaGoaobcLXV3+F22oYC5/nmvQN3Un0Fbn3dp6r/jHVezwW4HbUqSeRDe6lNRD7AXYL3eNCxJAsRuRI4T1WPDTqWhk5EjsBteLLDNuopQUTaAtOAgWo3LqY17yjNo6o6vNpp0y2ZeId5pgBdwk6oNyjiLoE8BLdndhhuj+lPqvr7QANroETkh7jaVFPc/T9lqnpmsFEZEz/1dphL3O3534rIAt+w1uKaBFnmvbfyhouI/EFEvhSReSJyVB0/cyKuin59Q04knsa46vZ23E1kr1H5bluTWD/DHdr4CneF2ZVVT25Maqm3momIHIM7Efe0qvbxht2LuwT3bhG5FXen6y0icgrumPopuON5D6vq0GjLNsYYk1zqrWaiqv/Bnaz1G4Or4uO9n+kb/rQ6M3BXi8R6nb0xxpgESXQDf+1Vda3XvY6KG+w6U/nmoxJv2FrCiMg43KWx5OTkDOraNdrl36kjZ88GGu3fwbamhWRkJP0FdjVSVlaWNmWB9CpPOpUF0qs8iSrL0qVLv1PVaDem1klgrcWqqopIrY+xqepjuOY76Nmzpy5ZkgaPlJj5N3j7Rj4edhfDTz476Gjiori4mFGjRgUdRtykU3nSqSyQXuVJVFlEpDa3TdRIotP5+tDhK+/9W2/4airfyVpA/O+OTl7tjgSg2c7lwcZhjDF1lOhk8joVLXtegncTljf8Yu+qrmG4NrUOOsSVtjr2A8mk5dbF1U9rjDFJqN4Oc4nIc7iWVfPFPUrzDlxT15NF5Ce4uy1DzcW/jbuS60tcA3mXHbTAdJadC50G0HJrtCZ9jDEmudVbMlHVsVFGHR9hWgWurq9YUkLhCFp8/Fco3QNZOUFHYxqw0tJSSkpK2LNnT9Ch1EjLli1ZtGhR0GHERbzLkpOTQ0FBAVlZWXFbZjT2uNZkUVhExkd/hNWzoNvRQUdjGrCSkhJyc3Pp1q0bUuvnkyXe9u3byc2N2ip+SolnWVSVjRs3UlJSQvfu3eOyzKqkx/V06aDrMBSBFdEaQzYmMfbs2UObNm1SIpGY6ESENm3aJKyGackkWTRpxc5m3WBFfTRsakztWCJJD4n8Hi2ZJJEteUfCqplwoLT6iY0xJolYMkkiW1v2htJdsGZu0KEYE5iNGzcyYMAABgwYQIcOHejcuXN5/759ER9jVG7WrFlce+21CYrU+NkJ+CSyJa+361jxIXT5XtUTG5Om2rRpw9y5bodq/PjxNG/enBtvvLF8/P79+2nUKPKma/DgwQwePDghcZrKrGaSREob50H+4XYS3pgwl156KVdccQVDhw7l5ptvZubMmQwfPpyBAwdywgknEGpWqbi4mNNOOw1wiejHP/4xo0aN4pBDDuEPf/hDkEVIe1YzSTaFI2DBy1B2ADIyg47GNHC/eWMhX6zZFtdlHtmpBXec3rvW85WUlPDRRx+RmZnJtm3bmDZtGo0aNeL111/nl7/8JS+99NJB8yxevJipU6eyfft2evbsyZVXXpmQey4aIksmyaawCGZPgPULoGP/oKMxJmmcc845ZGa6HaytW7dyySWXsGzZMlSVAwcORJzn1FNPJTs7m+zsbNq1a8f69espKChIZNgNhiWTZFM4wr2v+MiSiQlcXWoQ9aVZs2bl3b/61a8YPXo0r7zyCgsWLCg/tBUuOzu7vDszM5P9+/fXe5wNlZ0zSTYtCyCv0O43MaYKW7dupXPnzgBMmjQp4GgMWDJJToVFrmZST49UNibV3Xzzzdx2220MHDjQahtJwg5zJaNuRfD5s7BhCbTrFXQ0xgRm/PjxEYcPHz6cpUuXAq49q/vuuw+AUaNGlT9cKnzeBQsW1FeYBquZJKfy8yZ2qMsYkxosmSSjVt0ht6Pdb2KMSRmWTJKRiKudrPjQzpsYY1KCJZNkVVgE29fC5m+CjsQYY6plySRZFRa5dzvUZYxJAZZMklXbntC0DSy3k/DGmORnySRZ+c+bGNPArFu3jvPOO48ePXowaNAgTjnllPJLgevLxIkTGTt2bKVh3333HW3btmXv3r0R55kwYQI///nPAXjkkUd4+umnD5pm+fLl9OnTp8rPXr58Oc8++2x5fyo2pW/JJJkVFsGWFbC1JOhIjEkYVeWHP/who0aN4quvvmL27NncddddrF+/vtJ08b5Z8Yc//CFTpkxh165d5cNefPFFTj/99ErNskRzxRVXcPHFF9fps8OTyeDBg1OulWNLJsms/H6Tj4ONw5gEmjp1KllZWVxxxRXlw/r378/IkSMpLi5m5MiRnHHGGRx55JHs2bOHK6+8kr59+zJw4ECmTp0KwMKFCxkyZAgDBgygX79+LFu2jJ07d3LqqafSv39/+vTpwwsvvFDpc1u0aMGxxx7LG2+8UT7s+eefZ+zYsbzxxhsMHTq0vMn78MQG7ibJ+++/H4DZs2fTv39/+vfvz5///OfyaZYvX87IkSM56qijOOqoo/joI3dO9NZbb2XatGkUFRXx0EMPVWpKf9OmTZx55pn069ePYcOGMW/evPLPS6Ym9u0O+GTWvg9kt4QV06HfOUFHYxqid26FdfPju8wOfeEHd0cdvWDBAgYNGhR1/Jw5c1iwYAHdu3fngQceQESYP38+ixcv5vvf/z5Lly7lkUce4brrruOCCy5g3759HDhwgLfffptOnTrx1ltvAa59r3Bjx45l0qRJnHvuuaxZs4alS5dy3HHHsW3bNmbMmIGI8Pjjj3PvvffywAMPRI3xsssu409/+hPHHHMMN910U/nwdu3aMWXKFHJycli2bBljx45l1qxZ3H333dx///0899xz5ObmUlxcXD7PHXfcwcCBA3n11Vf54IMPuPjii8sfHpZMTexbzSSZZWRC12F2RZcxPkOGDKF79+4ATJ8+nXPPPReAXr16UVhYyNKlSxk+fDh33nkn99xzDytWrKBJkyb07duXKVOmcMsttzBt2jRatmx50LJPPfVUPvzwQ7Zt28bkyZM566yzyMzMpKSkhJNOOom+ffty3333sXDhwqjxbdmyhS1btnDMMccAcNFFF5WPKy0t5fLLL6dv376cc845fPHFF9WWd/r06eXLOO6449i4cSPbtm0rjzc7O5v8/PzyJvaDYjWTZFc4Apb9C3ZsgOZtg47GNDRV1CDqS+/evXnxxRejjvc3RR/N+eefz9ChQ3nrrbc45ZRTePTRRznuuOOYM2cOb7/9NrfffjvHH388v/71ryvN16RJE04++WReeeUVnn/+eR588EEArrnmGm644QbOOOMMiouLo7YZVp2HHnqI9u3b8/nnn1NWVkZOTk6dlhOSTE3sW80k2ZXfb2JXdZmG4bjjjmPv3r089thj5cPmzZvHtGnTDpp25MiRTJ48GYClS5eycuVKevbsyddff80hhxzCtddey5gxY5g3bx5r1qyhadOmXHjhhdx0003MmTMn4uePHTuWBx98kPXr1zN8+HCgcpP3EydOrDL+vLw88vLymD59OlC5ifytW7fSsWNHMjIy+Pvf/17+UK/c3Fy2b98ecXkjR44sX0ZxcTH5+fm0aNGiyhiCYMkk2XUaAFlN7VCXaTBEhFdeeYX33nuPHj160Lt3b2677TY6dOhw0LRXXXUVZWVl9O3bl3PPPZcJEyaQnZ3N5MmT6dOnDwMGDGDBggVcfPHFzJ8/v/yk/G9+8xtuv/32iJ9/4oknsmbNGs4991xEBHAnu8855xwGDRpEfn5+tWV46qmnuPrqqxkwYADqaxLpqquuYuLEifTv35/FixeX17L69etHZmYmI0aM4KGHHqq0rPHjxzN79mz69evHrbfeWm0yC4poCrf91LNnT12yZEnQYcRNcXFxefPZlTw9BnZuhCunJzymuopalhSVTuWpriyLFi3iiCOOSFxAMdq+fTu5ublBhxEX9VGWSN+niMxW1cHx/ByrmaSCwiL3TPjdm4OOxBhjIrJkkgoKRwAKK2cEHYkxxkRkySQVdB4MmY3tJLxJmFQ+/G0qJPJ7tGSSCrJyXEKxk/AmAXJycti4caMllBSnqmzcuDHmy49ryu4zSRWFI2D6Q7B3O2Snx8lGk5wKCgooKSlhw4YNQYdSI3v27EnYBrO+xbssOTk5FBQUxG15VbFkkiq6FcG0+2HVTDj0+KCjMWksKyur/A7zVFBcXMzAgQODDiMuUrksdpgrVRQMAcm0Q13GmKQUSDIRkV+IyEIRWSAiz4lIjoh0F5FPRORLEXlBRBoHEVvSym7ubmC0ZGKMSUIJTyYi0hm4Fhisqn2ATOA84B7gIVU9FNgM/CTRsSW9whGwehaU7g46EmOMqSSow1yNgCYi0ghoCqwFjgNCrbtNBM4MKLbkVXg0HNgHq2cHHYkxxlQSSHMqInId8DtgN/AucB0ww6uVICJdgHe8mkv4vOOAcQBt27YdFGrkLR3s2LGD5s2bRx3fqHQHRR9eyPJuY1nR7dwERlZ71ZUl1aRTedKpLJBe5UlUWUaPHh335lQSfjWXiLQCxgDdgS3AP4CTazq/qj4GPAauba50aS8Jatj+05d96J6xhu5JXu50assK0qs86VQWSK/ypHJZgjjMdQLwjapuUNVS4GWgCMjzDnsBFACrA4gt+XUrcpcHHygNOhJjjCkXRDJZCQwTkabi2nc+HvgCmAqc7U1zCfBaALElv8IRULoL1swNOhJjjCmX8GSiqp/gTrTPAeZ7MTwG3ALcICJfAm2AJxIdW0roOsK9WztdxpgkEsgd8Kp6B3BH2OCvgSEBhJNamreF/MNdMjn6+qCjMcYYwO6AT02FRa45+rIDQUdijDGAJZPUVFgEe7e5B2YZY0wSsGSSigpD502saRVjTHKwZJKKWnaGVt1geeo8E94Yk94smaSqwiJXM7EHGBljkoAlk1RVOAJ2b4INS4KOxBhjLJmkrPLzJnaoyxgTPEsmqapVd8jtZCfhjTFJwZJJqhJxtRM7b2KMSQKWTFJZ4QjYvhY2fxN0JMaYBs6SSSrrdrR7X27tdBljgmXJJJXlHw5N29h5E2NM4CyZpLLy8yZWMzHGBMuSSaorLIItK2BrSdCRGGMaMEsmqa6wyL3boS5jTIAsmaS69r0hu6Ud6jLGBMqSSarLyISuw6xmYowJlCWTdNCtCL5bCju+DToSY0wDZckkHdh5E2NMwCyZpIOO/SGrqSUTY0xgLJmkg8ws6DLEkokxJjCWTNJF4dHumfC7NwcdiTGmAbJkki4KRwAKK2cEHYkxpgGyZJIuOg+CzGy738QYEwhLJukiKwcKBlsLwsaYQFgySSeFI2Dt57B3e9CRGGMaGEsm6aRwBOgBWDUz6EiMMQ2MJZN0UjAEJNMuETbGJJwlk3SS3Rw6DbST8MaYhLNkkm4KR8Dq2VC6O+hIjDENiCWTdFNYBAf2uYRijDEJYskk3XQdBohdImyMSShLJummSR506GPnTYwxCWXJJB0VFrnLg/fvCzoSY0wDEUgyEZE8EXlRRBaLyCIRGS4irUVkiogs895bBRFbWigcAft3uxsYjTEmAYKqmTwM/FNVewH9gUXArcD7qnoY8L7Xb+qi/GFZ04ONwxjTYCQ8mYhIS+AY4AkAVd2nqluAMcBEb7KJwJmJji1tNMuH/J5286IxJmFEVRP7gSIDgMeAL3C1ktnAdcBqVc3zphFgc6g/bP5xwDiAtm3bDpo8eXKiQq93O3bsoHnz5nFZ1uFL/kK7b6cx/ehn3F3xCRbPsiSDdCpPOpUF0qs8iSrL6NGjZ6vq4LguVFUT+gIGA/uBoV7/w8D/AVvCpttc3bIOP/xwTSdTp06N38Lm/UP1jhaqa+bGb5m1ENeyJIF0Kk86lUU1vcqTqLIAszTO2/YgzpmUACWq+onX/yJwFLBeRDoCeO/fBhBb+ug63L3b/SbGmARIeDJR1XXAKhHp6Q06HnfI63XgEm/YJcBriY4trbTsDK262f0mxpiEaBTQ514DTBKRxsDXwGW4xDZZRH4CrAB+FFBs6aOwCJa8A6ogEnQ0xpg0FkgyUdW5uHMn4Y5PdCxprbAI5k6CDYuh3RFBR2OMSWN2B3w6Kxzh3u1QlzGmnlkySWetukFuJ7vfxBhT7yyZpDMR6FbkkkmC7ycyxjQslkzSXeEI2L4WNn0ddCTGmDRmySTdlbfTZYe6jDH1x5JJuss/HJrmWzIxxtSrapOJiGSIyIhEBGPqgYg71GUtCBtj6lG1yURVy4A/JyAWU18Ki2DLStiyKuhIjDFpqqaHud4XkbO81nxNqgndb7Ly42DjMMakrZomk58B/wD2icg2EdkuItvqMS4TT+17Q05Lu3nRGFNvatSciqrm1ncgph5lZLpWhK0FYWNMPalx21wicgbuCYkAxar6Zv2EZOpF4QhY+k/Y8S00bxd0NMaYNFOjw1wicjfuaYhfeK/rROSu+gzMxJndb2KMqUc1PWdyCnCiqj6pqk8CJwOn1l9YJu469oesZpZMjDH1ojY3Lfqfx94y3oGYepaZBV2G2El4Y0y9qGkyuRP4TEQmiMhEYDbwu/oLy9SLwiJYvxC2rQk6EmNMmqnRHfBAGTAMeBl4CRiuqi/Uc2wm3vqe7e6In/GXoCMxxqSZmt4Bf7OqrlXV173XugTEZuKtdXfo/V8w6ynYvTnoaIwxaaSmh7neE5EbRaSLiLQOveo1MlM/jr4e9u2ATx8POhJjTBqp6X0m53rvV/uGKXBIfMMx9a5DXzjs+zDjrzDsamjcNOiIjDFpoKbnTG5V1e5hL0skqeroX8CujfDZM0FHYoxJEzU9Z3JTAmIxidJ1OHQZCh/9EQ6UBh2NMazXW64AABb+SURBVCYN2DmThkgEjr4Btq6EBS8FHY0xJg3YOZOG6rDvQ7sjYfrvoe+PIMMeummMqbsabUEinC+xcyapLiMDiq6HDYtg2b+CjsYYk+KqTCYicrOv+5ywcXfWV1AmQfr8F7TsCtMeBNWgozHGpLDqaibn+bpvCxt3cpxjMYmWmQVF10LJTGsA0hgTk+qSiUTpjtRvUtGAC6BpPkx/KOhIjDEprLpkolG6I/WbVNS4KQy7Ar6cAuvmBx2NMSZFVZdM+oee+Q7087pD/X0TEJ9JhO9dDo1zrXZijKmzKpOJqmaqagtVzVXVRl53qD8rUUGaetYkDwZfBgtfgU1fBx2NMSYF2c0Fxhl2FWQ0cnfFG2NMLVkyMU6LjjDgfPhsEmxfH3Q0xpgUY8nEVBhxLZSV2sOzjDG1ZsnEVGjTA44cA7OehD1bg47GGJNCAksmIpIpIp+JyJtef3cR+UREvhSRF0SkcVCxNWhH/wL2brOHZxljaiXImsl1wCJf/z3AQ6p6KLAZ+EkgUTV0HftDj+Pdw7NKdwcdjTEmRQSSTESkADgVeNzrF+A44EVvkonAmUHEZnC1k50bYO6koCMxxqQI0QAa+BORF4G7gFzgRuBSYIZXK0FEugDvqGqfCPOOA8YBtG3bdtDkyZMTFXa927FjB82bNw86DFBl4Ge30HjfFmYO+SuakVnrRSRNWeIkncqTTmWB9CpPosoyevTo2ao6OJ7LrOnzTOJGRE4DvlXV2SIyqrbzq+pjwGMAPXv21FGjar2IpFVcXEzSlKfjHfD8+Rybvwn6nVP99GGSqixxkE7lSaeyQHqVJ5XLEsRhriLgDBFZDjyPO7z1MJAnIqHkVgCsDiA2E3L4D6BtL9fEijVPb4ypRsKTiarepqoFqtoN18T9B6p6ATAVONub7BLgtUTHZnxCD8/6diEsezfoaIwxSS6Z7jO5BbhBRL4E2gBPBByP6Xs2tCiwBiCNMdUKNJmoarGqnuZ1f62qQ1T1UFU9R1X3BhmbwT08a8Q1sPJjWPFx0NEYY5JYMtVMTDI66iJo0tpqJ8aYKlkyMVVr3AyGXQnL/gXrFgQdjTEmSVkyMdX73k8hqxl8+HDQkRhjkpQlE1O9pq3dw7MWvASblwcdjTEmCVkyMTUz/GqQDHt4ljEmIksmpmZadIL+58Fnz8COb4OOxhiTZCyZmJorug7273UtChtjjI8lE1Nz+YfBkWe4Z53Yw7OMMT6WTEztFF3vHp4166mgIzHGJBFLJqZ2Oh8Fh4xyz4kv3RN0NMaYJGHJxNTe0TfAjvXw+bNBR2KMSRKWTEztdT8GOh3lbmI8sD/oaIwxScCSiak9Efdo383L4YtXg47GGJMELJmYuul1GrQ5DKb/3h6eZYyxZGLqKCMDjr4e1s+HL98POhpjTMAsmZi66/sjaNEZpj8YdCTGmIBZMjF116gxDP85rPgQVn4SdDTGmABZMjGxOepiaNIKPvx90JEYYwJkycTEJrs5DPkZLHkb1n8RdDTGmIBYMjGxG/ozyGpqD88ypgGzZGJi17Q1DLoU5v8DtqwMOhpjTAAsmZj4GH61u5nRHp5lTINkycTER8sC6HcuzHkadmwIOhpjTIJZMjHxE3p41sxHg47EGJNglkxM/LTtCb1OhZmPkbl/V9DRGGMSyJKJia+jb4A9WykoeR3KyoKOxhiTII2CDsCkmYJBcMhoun/9HNz7NnQdDoUjoOsI6DQAMrOCjtAYUw8smZj4+9HTLHr1AY5ouglWfAxL/+mGZzWFgu9BYREUDofOg6Fx02BjNcbEhSUTE385LVjfYTRHjBrl+nd8Cys+qngV3wUoZGS5xwAXjnAJpssQyGkZZOTGmDqyZGLqX/N20PtM9wLYvQVWfeIaiFzxkbs3ZfpDIBnQvo9XcxnhDpE1bxts7MaYGrFkYhKvSR4cfpJ7AezbCSWzvJrLhzD7Kfjkr25c/uEVNZfCEe5+FmNM0rFkYoLXuBkccqx7AezfB2vnVtRcFrwMsye4cS27esnFSzBterg7740xgbJkYpJPo8bu/EmXIe5Z82UHYP3CiprLl+/BvOfdtE3zoctQb/qh7oqxrCbBxm9MA2TJxCS/jEzo2M+9hl3hnjm/8UtYPh1WzXTnX5a85U2bBR37u8TSdah7z+0QbPzGNAAJTyYi0gV4GmgPKPCYqj4sIq2BF4BuwHLgR6q6OdHxmRQgAvmHudfgy9ywnd9VJJZVM2HWEzDjz25cXlev9uLVYNr1hkzbjzImnoL4R+0H/ltV54hILjBbRKYAlwLvq+rdInIrcCtwSwDxmVTULB96neJe4M67rJsPq2a4BPPNNNdEPkBWM3dzZZeh0GUYFAx2FwUYY+os4clEVdcCa73u7SKyCOgMjAFGeZNNBIqxZGLqqlFjlzAKBrnm8VVh6ypf7eUTmPYg6AE3fdsjKs67dBlqJ/aNqSVR1eA+XKQb8B+gD7BSVfO84QJsDvWHzTMOGAfQtm3bQZMnT05YvPVtx44dNG/ePOgw4iIVypK5fze525fRYttiWm5dTItti8navxOA0ka5bG3Zi20terG15RGsJ5+cvPYBRxwfqfDd1EY6lSdRZRk9evRsVR0cz2UGlkxEpDnwb+B3qvqyiGzxJw8R2ayqrapaRs+ePXXJkiX1HWrCFBcXMyp013iKS8mylJXBd0srzrus+gQ2LqsYn9vJtYzc7gho28t775lyd+2n5HdThXQqT6LKIiJxTyaBnIUUkSzgJWCSqr7sDV4vIh1Vda2IdAS+DSI204BlZEC7Xu416BI3bOdGWD2Lr2a8SY/m+2DDIpj1FOzfXTFfi86+5NLLe/WEnBbBlMOYAARxNZcATwCLVPVB36jXgUuAu7331xIdmzEHadYGDj+JVWuy6RHaYyw7AFtWwLeLYYP3+nYRfPoh7N9TMW+LApeYyhONV5PJrqfDGKqwdzvs3gx7trj33VsO6j9s3QZo/rVruqbdEZCdWz/xmAYliJpJEXARMF9E5nrDfolLIpNF5CfACuBHAcRmTPUyMqH1Ie4VunoMXJLZvLwiuWxY7BLON9PgwN6K6Vp29Q6X9XIJpl0vyPclmdLdYUng4IQQuX9rxQUFkWQ2hpw82u/ZDm++XTG8VTeXWNr39l593LCMzDiutBSzb5f7DtfPh3ULYMtKdzizaWto0tq9V+pu47obcCvYQVzNNR2IdpnM8YmMxZi4ysh0V4G16eGeOBlSdgA2fePVYhZV1Gi++Tcc2FcxXdN82Lejcu3mIOIuY87Jc+9NWkFeoXsP9efkRejPc48AEGH61KmMGtjDtSqwfoH3vhCWvA3qPdCsURNXawkll1Ciadq6XlZdYFRh+1qXMEKJY9182PRVxbponOuS63dLYNdm2Ls1+vIa5XgJpg00bXVwsilPQL7xOS3T4spBu3PLmPqWkQn5h7rXEadVDD+wHzZ/U1GL2bbaHXKKmhBaQXYLd24nFiLuRs68rtDzBxXDS3e7ONZ/UZFolrwNn/29YprcTtD+yMpJps1h7lLsZLd/n7vAYt18V7bQ+66NFdPkdYX2faHPWdChjytjXmHldX6g1NUEd22C3Zvc/OXdmyp3r1/oundvrkhO4STTfbdN29BH8yBFLyawZGJMUDIbVdzJzxlBR+PaNOs00L1CVN3zaPw1mG8Xwjf/qahVZWS5w3bte0O7IyuSTPP2sSe+utq1qXLSWLfAJcqyUje+UY6refU8BTr0da/2vWt2ZV5mlnusQvN2NY+nrMzVaMKTza6NlbpLN++oW3mTgCUTY0x0IpDb3r0O9R2FPlDq2kfzHypbPh3mvVB5/qymLkllNfPem7hWorOaeOOauvMMoe5I46uaRzJosqvEtSy9fkHFYartaypiaN7B1TIOPb4icbTukdgmdTIyvJpmK3cYNIolxcV0TFxUcWXJxBhTe5lZbs++3RHQ9+yK4bs3Vxwm2/UdlO5yJ7NLd0PpTu99t9sTL11dMWzfLjcttb3vTRiKwkwgo5F7/k33ka521KGPO2RlD1hLCEsmxpj4adIKuhW5V22puosPSnf7kpDvdVBS2gX797J4/V56jTrbXYLdKDv+ZTI1YsnEGJMcRCoOhVHzq8bWFRfTq2P/+ovL1EhAZ8eMMcakE0smxhhjYmbJxBhjTMwsmRhjjImZJRNjjDExs2RijDEmZpZMjDHGxMySiTHGmJhZMjHGGBMzSybGGGNiZsnEGGNMzCyZGGOMiZklE2OMMTGzZGKMMSZmlkyMMcbEzJKJMcaYmFkyMcYYEzNLJsYYY2JmycQYY0zMLJkYY4yJmSUTY4wxMbNkYowxJmaWTIwxxsTMkokxxpiYWTIxxhgTM0smxhhjYmbJxBhjTMwsmRhjjImZJRNjjDExS6pkIiIni8gSEflSRG4NOh5jjDE10yjoAEJEJBP4M3AiUAJ8KiKvq+oXwUZmAFTVe/cNCx/nm35/mbJvfxnKwfP5+yONj7bc8mnC5q28vMrzRprfPx8HzecfVRHbxt1lrN6yu3JcNSxTteXxTa9hy1ENDdfKsYUNU2/5FfMfvMzQ5y/edIDsrzZWXhdR1kP4Z0QqR/j3EYq5tmWP9vmRx1esjC/W7Gfr3NUHfx8Rfh8HdfunD1uARuiJFE91sRKl7P51Ehq2uqSUUaSmpEkmwBDgS1X9GkBEngfGAIEmk6K7P2Dzrn1A5I1RtB9m5e1VDabH/bjk3bdrFZ/UaurInx39jx6jd9+J04KSxL8/CDqC+Jk5I+gI4mve3KAjiIv8JsKvgg6ijpIpmXQGVvn6S4Ch4ROJyDhgnNe7V0QWJCC2RMkHvgs6iDhJp7JAepUnncoCaVSeFZAv4xNSlsJ4LzCZkkmNqOpjwGMAIjJLVQcHHFLcpFN50qkskF7lSaeyQHqVJ5XLkkwn4FcDXXz9Bd4wY4wxSS6ZksmnwGEi0l1EGgPnAa8HHJMxxpgaSJrDXKq6X0R+DvwLyASeVNWF1cz2WP1HllDpVJ50KgukV3nSqSyQXuVJ2bJI+OVwxhhjTG0l02EuY4wxKcqSiTHGmJilbDJJl6ZXRKSLiEwVkS9EZKGIXBd0TLESkUwR+UxE3gw6lliJSJ6IvCgii0VkkYgMDzqmWIjIL7zf2QIReU5EcoKOqaZE5EkR+dZ/b5mItBaRKSKyzHtvFWSMtRGlPPd5v7V5IvKKiOQFGWNtpGQy8TW98gPgSGCsiBwZbFR1th/4b1U9EhgGXJ3CZQm5DlgUdBBx8jDwT1XtBfQnhcslIp2Ba4HBqtoHd6HLecFGVSsTgJPDht0KvK+qhwHve/2pYgIHl2cK0EdV+wFLgdsSHVRdpWQywdf0iqruA0JNr6QcVV2rqnO87u24jVXnYKOqOxEpAE4FHg86lliJSEvgGOAJAFXdp6pbgo0qZo2AJiLSCGgKrAk4nhpT1f8Am8IGjwEmet0TgTMTGlQMIpVHVd9V1f1e7wzc/XYpIVWTSaSmV1J2AxwiIt2AgcAnwUYSk98DNwNlQQcSB92BDcBT3mG7x0WkWdBB1ZWqrgbuB1YCa4GtqvpusFHFrL2qrvW61wHtgwwmzn4MpEwDd6maTNKOiDQHXgKuV9VtQcdTFyJyGvCtqs4OOpY4aQQcBfxVVQcCO0mtwyiVeOcTxuCSZCegmYhcGGxU8aPuPoe0uNdBRP4Hdwh8UtCx1FSqJpO0anpFRLJwiWSSqr4cdDwxKALOEJHluEOPx4nIM8GGFJMSoERVQzXFF3HJJVWdAHyjqhtUtRR4GRgRcEyxWi8iHQG8928DjidmInIpcBpwgabQjYCpmkzSpukVERHcMflFqvpg0PHEQlVvU9UCVe2G+04+UNWU3fNV1XXAKhHp6Q06noAfiRCjlcAwEWnq/e6OJ4UvKPC8DlzidV8CvBZgLDETkZNxh4nPUNVdQcdTGymZTLwTVKGmVxYBk2vQ9EqyKgIuwu3Fz/VepwQdlCl3DTBJROYBA4A7A46nzrwa1ovAHGA+7v+fMs13iMhzwMdATxEpEZGfAHcDJ4rIMlzN6+4gY6yNKOX5E5ALTPG2BY8EGmQtWHMqxhhjYpaSNRNjjDHJxZKJMcaYmFkyMcYYEzNLJsYYY2JmycQYY0zMLJmYOhERFZEHfP03isj4OC17goicHY9lVfM553gtAU8NG95NRM6v4zI/qsE0j6dBY57lvPW1oPopTTqzZGLqai/wXyKSH3Qgfl4DhjX1E+ByVR0dNrwbEDGZVLd8Va32jnJV/amqpvLNj8YcxJKJqav9uBvefhE+IrxmISI7vPdRIvJvEXlNRL4WkbtF5AIRmSki80Wkh28xJ4jILBFZ6rX5FXpOyn0i8qn3vIef+ZY7TUReJ8Id6iIy1lv+AhG5xxv2a+Bo4AkRuS9slruBkd5NY78QkUtF5HUR+QB4X0Sai8j7IjLHW+4Y32f5y1osFc9CmeTddY43fHBoehH5nYh8LiIzRKS9N7yH1z9fRH4bWm6Esl3orb+5IvKot46+562fHBFpJu75JX2ixe3VLBZ739tSL9YTRORDcc8JGeJNN15E/i4iH3vDL48QT7TvqKOI/MeLc4GIjIxUHpPCVNVe9qr1C9gBtACWAy2BG4Hx3rgJwNn+ab33UcAWoCOQjWtP7TfeuOuA3/vm/yduZ+cwXBtZOcA44HZvmmxgFq7RwlG4Rhi7R4izE64Zkba4hhs/AM70xhXjnu0RPs8o4E1f/6VeDK29/kZAC687H/iSihuA/WXdims3LgN3p/PR4Z+La5jwdK/7Xl/53gTGet1XhJYbFucRwBtAltf/F+Bir/u3uBaC/wzcVlXcuJrYfqCvF+ts4Elv3BjgVW+e8cDnQBNv/lXe+u0GLPCmifYd/TfwP97wTCA36N+wveL7qs0hAWMqUdVtIvI07oFLu2s426fqNRkuIl8BoSbQ5wP+w02TVbUMWCYiXwO9gO8D/Xy1npa4ZLMPmKmq30T4vO8Bxaq6wfvMSbhnlLxaw3hDpqhq6NkTAtwpIsfgmtrvjGv6fF3YPDNVtcT73Lm4je70sGn24RIHuI34iV73cCqezfEsLjGEOx4YBHzqVXqaUNHQ4f/i2rDbg/t+qoobXAOQ871YF+IeOKUiMt+LO+Q1Vd0N7PbONQ0B5vrGR/uOPgWeFNeo6auq6p/HpAFLJiZWv8e19fSUb9h+vEOoIpIBNPaN2+vrLvP1l1H59xjezo/iNobXqOq//CNEZBSuZlKf/Mu/AFfTGaSqpeJaSY70+Ft/WQ8Q+f9WqqpazTTRCDBRVSM9ja8N0BzI8mLbWU3csXwv4TEd9B0BeEnsVGCCiDyoqk9XXTyTSuyciYmJt7c+GXcyO2Q5bo8Z4AzcBq22zhGRDO88yiHAElzDnld6e7eIyOFS/cOqZgLHiki+uMc9jwX+Xc0823GN7UXTEvfcllIRGQ0U1qA8tTUDOMvrjvZo3feBs0WkHZQ/Dz0Uy6PAr3DPw7gnjnGP8c7FtMEdyvs0bHzE78iLa72q/g33FM5UbsrfRGA1ExMPD+BacQ75G/CaiHyOO/dRl1rDSlwiaAFcoap7RORx3CGXOd7J7A1U85hWVV0rIrcCU3F7zW+panXNlM8DDnjxTwA2h42fBLzhHQKaBSyuTcFq6HrgGXEPSfon7vxLJar6hYjcDrzr1QBLgatF5FhcjedZL4F+JCLHxSnuebh1mQ/8n6quEfeE0JBo39Eo4CYRKcWdb7u4Dp9tkpi1GmxMEhKRpsBu77zFebiT8WOqm6+eYxqPuxAg0vkb08BZzcSY5DQI+JO3d78F9zxwY5KW1UyMMcbEzE7AG2OMiZklE2OMMTGzZGKMMSZmlkyMMcbEzJKJMcaYmP0/PXMiJIZZMFoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pyplot.plot(np.arange(1, 1+m), error_train, np.arange(1, 1+m), error_val)\n",
    "\n",
    "pyplot.title('Polynomial Regression Learning Curve (lambda = %f)' % lambda_)\n",
    "pyplot.xlabel('Number of training examples')\n",
    "pyplot.ylabel('Error')\n",
    "pyplot.axis([0, 13, 0, 100])\n",
    "pyplot.legend(['Train', 'Cross Validation', 'Test'])\n",
    "pyplot.grid()\n",
    "\n",
    "print('Polynomial Regression (lambda = %f)\\n' % lambda_)\n",
    "print('# Training Examples\\tTrain Error\\tCross Validation Error Test Error')\n",
    "for i in range(m):\n",
    "    print('  \\t%d\\t\\t%f\\t%f\\t%f' % (i+1, error_train[i], error_val[i], error_test[i]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}