Report-Diabetic_Retinopathy_Augmented-96x96,64,128-FlowFromDataFrame.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Diabetic Retinopathy Detection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Jupyter Notebook uses a neural network to classify retinal images for various stages of diabetic retinopathy, a disease which can lead of blindness.\n",
    "\n",
    "Reference:\n",
    "* Kaggle Competition: [APTOS 2019 Blindness Detection](https://www.kaggle.com/c/aptos2019-blindness-detection)\n",
    "* My Convolutional Neural Network (CNN) is based upon the CNN defined by: [SmallerVGGNet by Adrian Rosebrock, PyImageSearch](https://www.pyimagesearch.com/2018/04/16/keras-and-convolutional-neural-networks-cnns)\n",
    "\n",
    "## Approach\n",
    "* The dataset is provided as a part of the Kaggle competition:\n",
    "    * 3662 training images, with an associated CSV file containing ID code and label class for each image\n",
    "    * 1928 testing images (which I refer to as \"official test\" images), with an associated CSV file containing only the ID code for each image. Note: a subset of a broader set of testing images that will be used to calculate official scoring for the competition\n",
    "    * Each image was a color image, but with size/resolution ranging from 474x358 to 3388x2588 pixels.\n",
    "    \n",
    "* I divided the 3662 training images into 3 sets using sklearn `test_train_split`:\n",
    "    * 2197 (60%) for training\n",
    "    * 732 (20%) for validation\n",
    "    * 732 (20%) for evaluation of model accuracy.\n",
    "    \n",
    "* <font color=\"blue\"><b>REFACTORED</b></font>: Now using the Keras `ImageDataGenerator` method `flow_from_dataframe` to access the training, validation, and testing images.\n",
    "    * The train.csv file provides the image `id_code` and associated `diagnosis`.  This file is read into a dataframe.\n",
    "    * `flow_from_dataframe` is then used to ready the images associated with the `id_code` entries in the dataframe.\n",
    "        \n",
    "* I defined and refined the Convolutional Neural Network:\n",
    "    * Defined the CNN model as a class based initially upon `SmallerVGGNet` (Adrian Rosebrock) and subsequently revised to refine the model's accuracy\n",
    "    * Used Keras `DataImageGenerator` and `flow_from_dataframe` to instantiate image generators linked to the training, validation, and Testing directories\n",
    "    * Initialized the model using the defined class and compiled it (specifying the Adam optimizer)\n",
    "    * Fit the model using the training image generator, with validation using the model using the validation image generator\n",
    "    * The Fit History, providing accuracy and loss at each epoch, was plotted\n",
    "    * The fitted model was then used to predict the class for each of the Official Test images\n",
    "    \n",
    "* To improve the robustness of the model, the `DataImageGenerator` instantiated for training was revised to add augmentation to introduce random rotations, translations, and other transformations of the training images used to fit the model.\n",
    "    \n",
    "## Results\n",
    "* Refer to the Figure provided below\n",
    "* For the neural network definition specified in this notebook, the fitting history shows a maximum training accuracy of 0.702 at epoch 8 (the 9th epoch iteration).  The fitted model yielded an accuracy of 0.5829 with the validation data, with a major dip in validation accurance and loss at epoch 5.  Accuracy and loss decrease with epoch 6 for both training and validation data, suggesting that overfitting may be an issue after this iteration.\n",
    "* Across all of the CNN models I attempted, the performance of the model against the official test data was poor, with Kaggle indicating a maximum accurancy of 0.486 for the \"Private\" dataset that is not available while the competition is in-progress.  However, while this is sufficient for any meaningful real-world usage, it's reassuring that the model at least performed better than a random guess of the image class (e.g., 1/5 = 0.200)!\n",
    "\n",
    "## Opportunities for Further Investigation\n",
    "* Further investigation of the effects of changes in each layer of the CNN is needed.\n",
    "* I did not check for skew in the distribution of classes in the training images.  Taking this into account during sample selection and during fitting could improve results\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Figure: Diabetic Retinopathy Detection - Model Fitting History |\n",
    "|----------|\n",
    "| ![Figure: Diabetic Retinopathy Detection - Model Fitting History is Loading...](docs/Figure-DPA_Fit_History.png \"Figure: Diabetic Retinopathy Detection - Model Fitting History\") |\n",
    "| ![Table: Diabetic Retinopathy Detection - Model Fitting History is Loading...](docs/Table-DPA_Fit_History.png \"Table: Diabetic Retinopathy Detection - Model Fitting History\") |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "nbpresent": {
     "id": "828c2e12-b1c6-4994-8f55-ce86373b6c97"
    },
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "# %matplotlib notebook\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D    # Support 3D graphing\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pandas.plotting import table\n",
    "import math\n",
    "import random\n",
    "import shutil\n",
    "import os\n",
    "\n",
    "# from imutils import paths\n",
    "# import argparse\n",
    "# import pickle\n",
    "# import cv2\n",
    "\n",
    "from pprint import pprint\n",
    "\n",
    "# Visualization\n",
    "import graphviz\n",
    "import pydotplus\n",
    "from IPython.display import Image\n",
    "\n",
    "# Machine Learning - Data Preparation and Pre-Processing\n",
    "from sklearn.model_selection import train_test_split # Split data into training and testing samples\n",
    "from sklearn.model_selection import cross_val_score  # Score a model using k-fold or other cross validation\n",
    "\n",
    "from sklearn.preprocessing import OneHotEncoder   # Convert categorical integer features (X) to One-Hot encoded values\n",
    "from sklearn.preprocessing import LabelEncoder    # Convert categorical labeled values to categorical integer values\n",
    "from sklearn.preprocessing import LabelBinarizer  # Convert categorical labeled values to Binary encoded values\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler  # Scale numerical features to standard normal distribution\n",
    "from sklearn.preprocessing import MinMaxScaler    # Scale numerical values based upon mix/max values\n",
    "\n",
    "# Machine Learning - Sci-Kit Learn - Models - Regression\n",
    "from sklearn.linear_model import LinearRegression  # TBD\n",
    "from sklearn.linear_model import Lasso             # TBD\n",
    "from sklearn.linear_model import Ridge             # TBD\n",
    "from sklearn.linear_model import ElasticNet        # TBD\n",
    "\n",
    "# Machine Learning - Sci-Kit Learn - Models - Classification\n",
    "from sklearn.linear_model import LogisticRegression   # Logistic Regression Classifier\n",
    "from sklearn import tree                              # Decision Tree Classifier\n",
    "from sklearn.ensemble import RandomForestClassifier   # Random Forest Classifier\n",
    "from sklearn import svm                               # Support Vector Machine Classifier\n",
    "from sklearn.neighbors import KNeighborsClassifier    # K-Nearest Neighbors (KNN)\n",
    "\n",
    "# Machine Learning - GridSearch for Hyper-Parameter tuning\n",
    "from sklearn.model_selection import GridSearchCV      # Grid Search\n",
    "\n",
    "# Machine Learning - Quantify Model Performance\n",
    "from sklearn.metrics import mean_squared_error    # Mean Squared Error (MSE) metric\n",
    "from sklearn.metrics import r2_score              # R-squared (Coefficient of Determination) metric\n",
    "from sklearn.metrics import confusion_matrix      # Generate a confusion matrix (actual vs. predicted counts)\n",
    "from sklearn.metrics import classification_report # Calculate metrics for prediction performance\n",
    "from sklearn.metrics import accuracy_score        # Calculate the precision: (Tp+Tn) / (Tp + Fp + Tn + Fn) => Ability Correctly Predict samples\n",
    "from sklearn.metrics import precision_score       # Calculate the precision: Tp / (Tp + Fp) => Ability to avoid false negatives\n",
    "from sklearn.metrics import recall_score          # Calculate the recall: Tp / (Tp + Fn) => Ability to find all positive samples\n",
    "from sklearn.metrics import f1_score              # Calculate the F1 score: 2*(precision*recall)/(precision+recall)\n",
    "\n",
    "# Machine Learning - Dataset Generation\n",
    "from sklearn.datasets import make_regression     # Generate linear data\n",
    "from sklearn.datasets import make_s_curve        # Generate nonlinear data\n",
    "from sklearn.datasets import make_blobs          # Generate blobs for classification\n",
    "from sklearn.datasets import make_circles        # Generate circles for classification\n",
    "from sklearn.datasets import load_iris           # Sample multi-class dataset for classification\n",
    "from sklearn.datasets import make_classification # Generate datasets for classification\n",
    "\n",
    "# Machine Learning - Keras (Tensorflow) - Models\n",
    "from keras.models import Sequential               # Sequential model serving as foundation for neural network\n",
    "# from keras.layers import Dense                    # Nodes for specifying input, hidden, and output layers\n",
    "from keras.layers.core import Activation\n",
    "from keras.layers.core import Flatten\n",
    "from keras.layers.core import Dropout\n",
    "from keras.layers.core import Dense\n",
    "\n",
    "# Machine Learning - Keras (Tensorflow) - Convolutional Neural Networks (Image Classification)\n",
    "from keras.layers.normalization import BatchNormalization\n",
    "from keras.layers.convolutional import Conv2D\n",
    "from keras.layers.convolutional import MaxPooling2D\n",
    "from keras import backend as K\n",
    "\n",
    "# Machine Learning - Keras (Tensorflow) - Optimizers\n",
    "from keras.optimizers import SGD\n",
    "from keras.optimizers import Adam\n",
    "\n",
    "# Machine Learning - Keras (Tensorflow) - Encoding\n",
    "from keras.utils import to_categorical            # One-Hot Encoder provided through Keras\n",
    "\n",
    "# Machine Learning - Keras (Tensorflow) - Other related Tools\n",
    "from keras.utils import plot_model                # Plot a neural network model\n",
    "from keras.models import load_model               # Load a saved machine learning model\n",
    "from keras.preprocessing import image             # Loads an image for application of machine learning\n",
    "from keras.preprocessing.image import img_to_array # Converts an image to a numpy array\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "# Machine Learning - Keras (Tensorflow) -  Dataset Generation\n",
    "from keras.datasets import mnist                  # Images: Handwritten digits 0-9 (28x28 grayscale, 60K train, 10K test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Locate the Training and Testing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using Main Data Scource Folder './data/'\n"
     ]
    }
   ],
   "source": [
    "# Determine the operating enviroment (Kaggle vs. Local PC)\n",
    "# based upon the presence of the main data source folder\n",
    "main_data_src_folder_kaggle = \"../input/aptos2019-blindness-detection/\"\n",
    "main_data_src_folder_localpc = \"./data/\"\n",
    "\n",
    "if os.path.exists(main_data_src_folder_kaggle):\n",
    "    # Use the Kaggle data file folder\n",
    "    main_data_src_folder = main_data_src_folder_kaggle\n",
    "    \n",
    "    # On Kaggle, by default don't save augmented images\n",
    "    flag_save_aug = False\n",
    "\n",
    "else:\n",
    "    # Use the Local PC data file folder\n",
    "    main_data_src_folder = main_data_src_folder_localpc\n",
    "\n",
    "    # On Local PC, by default don't save augmented images\n",
    "    flag_save_aug = False\n",
    "\n",
    "print(f\"Using Main Data Scource Folder '{main_data_src_folder}'\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using Training and Testing Folders and Files:\n",
      "\tTraining Labels File: './data/train.csv'\n",
      "\tTraining Image Directory: './data/train_images/'\n",
      "\tTesting Labels File: './data/test.csv'\n",
      "\tTesting Image Directory: './data/test_images/'\n",
      "\tTemporary Augmented Image Directory: './data/temp_aug_images/'\n",
      "\tFlag to Save Augmented Images: False\n"
     ]
    }
   ],
   "source": [
    "# Set the key file and directory pointers\n",
    "# CSV file with list of training images and labels\n",
    "train_labels_file = main_data_src_folder + \"train.csv\"\n",
    "\n",
    "# Training images - Source of training images in a single folder for training/validation/testing\n",
    "dir_train_src = main_data_src_folder + \"train_images/\"\n",
    "\n",
    "# CSV file with list of testing images\n",
    "test_labels_file = main_data_src_folder + \"test.csv\"\n",
    "\n",
    "# Testing images - Source of training images in a single folder for training/validation/testing\n",
    "dir_test_src = main_data_src_folder + \"test_images/\"\n",
    "\n",
    "# Temporary folder for optionally outputing images generated during model training, validation, and testing\n",
    "#  On Kaggle must be in the local (i.e., \"working\") directory, since input/data directory is not writable\n",
    "#  On Local PC, place in the data directory so it will be ignored by Git (via a .gitignore entry for `data` folder)\n",
    "#\n",
    "#  NOTE: Will override and set to 'None' by default\n",
    "\n",
    "if os.path.exists(main_data_src_folder_kaggle):\n",
    "    # Use the Kaggle current (i.e., \"working\") directory\n",
    "    dir_temp_aug_images = \"./temp_aug_images/\"\n",
    "\n",
    "else:\n",
    "    # Use the Local PC data directory\n",
    "    dir_temp_aug_images = main_data_src_folder_localpc + \"temp_aug_images/\"\n",
    "    \n",
    "# Summary\n",
    "print(f\"Using Training and Testing Folders and Files:\")\n",
    "print(f\"\\tTraining Labels File: '{train_labels_file}'\")\n",
    "print(f\"\\tTraining Image Directory: '{dir_train_src}'\")\n",
    "print(f\"\\tTesting Labels File: '{test_labels_file}'\")\n",
    "print(f\"\\tTesting Image Directory: '{dir_test_src}'\")\n",
    "print(f\"\\tTemporary Augmented Image Directory: '{dir_temp_aug_images}'\")\n",
    "print(f\"\\tFlag to Save Augmented Images: {flag_save_aug}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id_code</th>\n",
       "      <th>diagnosis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>000c1434d8d7</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>001639a390f0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0024cdab0c1e</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>002c21358ce6</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>005b95c28852</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3657</td>\n",
       "      <td>ffa47f6a7bf4</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3658</td>\n",
       "      <td>ffc04fed30e6</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3659</td>\n",
       "      <td>ffcf7b45f213</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3660</td>\n",
       "      <td>ffd97f8cd5aa</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3661</td>\n",
       "      <td>ffec9a18a3ce</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3662 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           id_code  diagnosis\n",
       "0     000c1434d8d7          2\n",
       "1     001639a390f0          4\n",
       "2     0024cdab0c1e          1\n",
       "3     002c21358ce6          0\n",
       "4     005b95c28852          0\n",
       "...            ...        ...\n",
       "3657  ffa47f6a7bf4          2\n",
       "3658  ffc04fed30e6          0\n",
       "3659  ffcf7b45f213          2\n",
       "3660  ffd97f8cd5aa          0\n",
       "3661  ffec9a18a3ce          2\n",
       "\n",
       "[3662 rows x 2 columns]"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load into a dataframe\n",
    "all_train_labels_df = pd.read_csv(train_labels_file)\n",
    "\n",
    "# Limit subsequent pre-processing to only 15 images initially\n",
    "# all_train_labels_df = all_train_labels_df[0:100]\n",
    "\n",
    "# Process all training data\n",
    "all_train_labels_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id_code</th>\n",
       "      <th>diagnosis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>000c1434d8d7.png</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>001639a390f0.png</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0024cdab0c1e.png</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>002c21358ce6.png</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>005b95c28852.png</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            id_code  diagnosis\n",
       "0  000c1434d8d7.png          2\n",
       "1  001639a390f0.png          4\n",
       "2  0024cdab0c1e.png          1\n",
       "3  002c21358ce6.png          0\n",
       "4  005b95c28852.png          0"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert the id_code entries to filenames by adding .png extension\n",
    "all_train_labels_df['id_code'] = all_train_labels_df['id_code'] + \".png\"\n",
    "all_train_labels_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>diagnosis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>count</td>\n",
       "      <td>3662.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>mean</td>\n",
       "      <td>1.126980</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>std</td>\n",
       "      <td>1.298409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>min</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25%</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50%</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75%</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>max</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         diagnosis\n",
       "count  3662.000000\n",
       "mean      1.126980\n",
       "std       1.298409\n",
       "min       0.000000\n",
       "25%       0.000000\n",
       "50%       1.000000\n",
       "75%       2.000000\n",
       "max       4.000000"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_train_labels_df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Look for any entries where the labels are not a number\n",
    "invalid_list = all_train_labels_df['diagnosis'].map(math.isnan)\n",
    "if sum(invalid_list) > 0:\n",
    "    print(f\"WARNING: Dropping {sum(invalid_list)} rows with labels having value NaN\")\n",
    "    print(all_train_labels_df[ invalid_list ])\n",
    "\n",
    "    # Drop rows with invalid labels\n",
    "    all_train_labels_df = all_train_labels_df[ invalid_list == False ].reset_index(drop=True)\n",
    "    all_train_labels_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id_code      object\n",
       "diagnosis     int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_train_labels_df.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert the diagnosis labels from integer to string to permit later\n",
    "# use of class_mode='categorical' with flow_from_dataframe\n",
    "all_train_labels_df['diagnosis'] = all_train_labels_df['diagnosis'].map(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id_code      object\n",
       "diagnosis    object\n",
       "dtype: object"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_train_labels_df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Training Data into Training, Validation, and Test Subsets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "# One-hot encode the diagnosis\n",
    "# NOTE: No need to one-hot encode the diagnosis label since that\n",
    "# will be handled internally by flow_from_dataframe with class_mode='categorical'\n",
    "# y_all_train_encoded = to_categorical(all_train_labels_df['diagnosis'])\n",
    "# y_all_train_encoded = all_train_labels_df['diagnosis']\n",
    "# y_all_train_encoded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2197,) (2197,)\n",
      "(732,) (732,)\n",
      "(733,) (733,)\n"
     ]
    }
   ],
   "source": [
    "# Perform test_train_split twice to create training (60%), validation (20%), and testing (20%) subsets\n",
    "X_train, X_remain, y_train, y_remain = train_test_split(\n",
    "                        all_train_labels_df['id_code'],\n",
    "                        all_train_labels_df['diagnosis'],\n",
    "                        test_size = 0.4,\n",
    "                        random_state=1\n",
    "                        )\n",
    "\n",
    "X_valid, X_test, y_valid, y_test = train_test_split(\n",
    "                        X_remain,\n",
    "                        y_remain,\n",
    "                        test_size = 0.5,\n",
    "                        random_state=1\n",
    "                        )\n",
    "\n",
    "print( X_train.shape, y_train.shape )\n",
    "print( X_valid.shape, y_valid.shape )\n",
    "print( X_test.shape, y_test.shape )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the Convolutional Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the classification model for the Convolutional Neural Network\n",
    "# REFERENCE: PyImageSearch (Adrian Rosebrock)\n",
    "#            https://www.pyimagesearch.com/2018/04/16/keras-and-convolutional-neural-networks-cnns/\n",
    "class SmallerVGGNet:\n",
    "    @staticmethod\n",
    "    def build(width, height, depth, classes):\n",
    "        # initialize the model along with the input shape to be\n",
    "        # \"channels last\" and the channels dimension itself\n",
    "        model = Sequential()\n",
    "        inputShape = (height, width, depth)\n",
    "        chanDim = -1\n",
    "\n",
    "        # if we are using \"channels first\", update the input shape\n",
    "        # and channels dimension\n",
    "        if K.image_data_format() == \"channels_first\":\n",
    "            inputShape = (depth, height, width)\n",
    "            chanDim = 1\n",
    "            \n",
    "        # CONV => RELU => POOL\n",
    "        model.add(Conv2D(32, (3, 3), padding=\"same\", input_shape=inputShape ) )\n",
    "        \n",
    "        model.add(Activation(\"relu\"))\n",
    "        model.add(BatchNormalization(axis=chanDim))\n",
    "        \n",
    "        model.add(MaxPooling2D(pool_size=(3, 3)))\n",
    "        model.add(Dropout(0.25))\n",
    "        \n",
    "        # (CONV => RELU) * 2 => POOL\n",
    "        model.add(Conv2D(64, (3, 3), padding=\"same\"))\n",
    "        model.add(Activation(\"relu\"))\n",
    "        model.add(BatchNormalization(axis=chanDim))\n",
    "        model.add(Conv2D(64, (3, 3), padding=\"same\"))\n",
    "        model.add(Activation(\"relu\"))\n",
    "        model.add(BatchNormalization(axis=chanDim))\n",
    "        model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "        model.add(Dropout(0.25))\n",
    "        \n",
    "        # (CONV => RELU) * 2 => POOL\n",
    "        model.add(Conv2D(128, (3, 3), padding=\"same\"))\n",
    "        model.add(Activation(\"relu\"))\n",
    "        model.add(BatchNormalization(axis=chanDim))\n",
    "        model.add(Conv2D(128, (3, 3), padding=\"same\"))\n",
    "        model.add(Activation(\"relu\"))\n",
    "        model.add(BatchNormalization(axis=chanDim))\n",
    "        model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "        model.add(Dropout(0.25))\n",
    "        \n",
    "        # first (and only) set of FC => RELU layers\n",
    "        model.add(Flatten())\n",
    "        model.add(Dense(1024))\n",
    "        model.add(Activation(\"relu\"))\n",
    "        model.add(BatchNormalization())\n",
    "        model.add(Dropout(0.5))\n",
    "\n",
    "        # softmax classifier\n",
    "        model.add(Dense(classes))\n",
    "        model.add(Activation(\"softmax\"))\n",
    "\n",
    "        # return the constructed network architecture\n",
    "        return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Image Generators linked to the Image Training, Validation, and Testing subfolders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Removing the existing temporary augmented images directory './data/temp_aug_images/'\n",
      "Creating a new tempoary augmented images directory './data/temp_aug_images/'\n"
     ]
    }
   ],
   "source": [
    "# Before fitting the model (which will generate new augmented images),\n",
    "# delete and the temporary directory dir_temp_aug_images and all of its contents\n",
    "if os.path.exists(dir_temp_aug_images):\n",
    "    # Remove the target directory and all of it's files and subdirectories\n",
    "    print(f\"Removing the existing temporary augmented images directory '{dir_temp_aug_images}'\")\n",
    "    shutil.rmtree(dir_temp_aug_images)\n",
    "    pass\n",
    "\n",
    "else:\n",
    "    print(f\"A temporary augmented images directory '{dir_temp_aug_images}' is not present, so nothing to delete\")\n",
    "\n",
    "# Create the the temporary directory dir_temp_aug_images\n",
    "if not os.path.exists(dir_temp_aug_images):\n",
    "    print(f\"Creating a new tempoary augmented images directory '{dir_temp_aug_images}'\")\n",
    "    os.mkdir(dir_temp_aug_images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Covert the images from -\n",
    "# Source Images: Width 3216 x Height 2136 x Depth 3 (RGB)\n",
    "# Analysis Images: Width 96 x Height 96 x Depth 3 (RGB)     -- Just to get this to run initially\n",
    "image_dims = (96, 96, 3)         # Width, Height, Depth (RGB = 3, Grayscale = 1)\n",
    "n_classes = all_train_labels_df['diagnosis'].nunique()  # Number of classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 2197 validated image filenames belonging to 5 classes.\n"
     ]
    }
   ],
   "source": [
    "# Create a Image Data Generator for the training data\n",
    "train_datagen = ImageDataGenerator(\n",
    "        rotation_range=20,             # Range (degrees) of rotation to apply\n",
    "        zoom_range=0.15,               # Range of zoom to apply: Up to +/- (1-zoom_range)\n",
    "        width_shift_range=0.2,         # Shift the image by a fraction of total width \n",
    "        height_shift_range=0.2,        # Shift the image by a fraction of total height\n",
    "        shear_range=0.15,              # Range (degrees) of shear angle (FYI: cotangent shear angle = shear factor)\n",
    "        horizontal_flip=True,          # Perform random horizontal flips of the images\n",
    "        vertical_flip=True,            # Perform random vertical flips of the images\n",
    "        rescale=1.0/255.0,                 # Rescale the RGB color values from 0-255 to 0.0-1.0\n",
    "        fill_mode=\"nearest\"            # Interpolation method\n",
    "    )\n",
    "\n",
    "# Define generator batch size and step size\n",
    "train_n_images = len(X_train)\n",
    "train_n_batch_size = 32\n",
    "train_n_step_size = train_n_images // train_n_batch_size\n",
    "\n",
    "# Create dataframes with the training data\n",
    "train_df = pd.DataFrame( {'id_code': X_train, 'diagnosis': y_train }).reset_index(drop=True)\n",
    "\n",
    "# Associated the generator with the training images\n",
    "train_generator = train_datagen.flow_from_dataframe(\n",
    "    dataframe=train_df,                         # Dataframe with filenames of images and column for labels\n",
    "    directory=dir_train_src,                    # Directory containing images used for training, validation, and testing\n",
    "    x_col='id_code',                            # Column containing filenames of images in directory dir_train_src\n",
    "    y_col='diagnosis',                          # Column containing the labels for the images\n",
    "    target_size=(image_dims[1], image_dims[0]), # The (height, width) to which each image will be resized\n",
    "    color_mode='rgb',                           # Images will be color ('rgb') (3 color channels) vs. 'grayscale'\n",
    "#     classes=n_classes,                          # The number of classes\n",
    "    class_mode='categorical',                   # Performs one-hot encoding of the string label in y_col\n",
    "    batch_size=train_n_batch_size,              # Number of images to be used per batch\n",
    "    shuffle=True,                               # Shuffle the samples randomly\n",
    "    seed=1,                                     # Random number seed to use for shuffle\n",
    "    save_to_dir=dir_temp_aug_images if flag_save_aug else None, # Directory to save aug images if flag is True\n",
    "    save_prefix='train_aug_',                   # Prefix to apply to any saved images\n",
    "    save_format='png',                          # File format to appy to any saved images\n",
    "    interpolation='bicubic'                     # Interpolation method used to resize samples\n",
    "    )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 732 validated image filenames belonging to 5 classes.\n"
     ]
    }
   ],
   "source": [
    "# Create a Image Data Generator for the validation data\n",
    "valid_datagen = ImageDataGenerator(\n",
    "        rotation_range=20,             # Range (degrees) of rotation to apply\n",
    "        zoom_range=0.15,               # Range of zoom to apply: Up to +/- (1-zoom_range)\n",
    "        width_shift_range=0.2,         # Shift the image by a fraction of total width \n",
    "        height_shift_range=0.2,        # Shift the image by a fraction of total height\n",
    "        shear_range=0.15,              # Range (degrees) of shear angle (FYI: cotangent shear angle = shear factor)\n",
    "        horizontal_flip=True,          # Perform random horizontal flips of the images\n",
    "        vertical_flip=True,            # Perform random vertical flips of the images\n",
    "        rescale=1.0/255.0,                 # Rescale the RGB color values from 0-255 to 0.0-1.0\n",
    "        fill_mode=\"nearest\"            # Interpolation method\n",
    "    )\n",
    "\n",
    "# Define generator batch size and step size\n",
    "valid_n_images = len(X_valid)\n",
    "valid_n_batch_size = train_n_batch_size\n",
    "valid_n_step_size = valid_n_images // valid_n_batch_size\n",
    "\n",
    "# Create dataframes with the validation data\n",
    "valid_df = pd.DataFrame( {'id_code': X_valid, 'diagnosis': y_valid }).reset_index(drop=True)\n",
    "\n",
    "# Associated the generator with the validation images\n",
    "valid_generator = valid_datagen.flow_from_dataframe(\n",
    "    dataframe=valid_df,                         # Dataframe with filenames of images and column for labels\n",
    "    directory=dir_train_src,                    # Directory containing images used for training, validation, and testing\n",
    "    x_col='id_code',                            # Column containing filenames of images in directory dir_train_src\n",
    "    y_col='diagnosis',                          # Column containing the labels for the images\n",
    "    target_size=(image_dims[1], image_dims[0]), # The (height, width) to which each image will be resized\n",
    "    color_mode='rgb',                           # Images will be color ('rgb') (3 color channels) vs. 'grayscale'\n",
    "#     classes=n_classes,                          # The number of classes\n",
    "#     class_mode='other',                         # Labels are specified by y_col ('other')\n",
    "    class_mode='categorical',                   # Performs one-hot encoding of the string label in y_col\n",
    "    batch_size=valid_n_batch_size,              # Number of images to be used per batch\n",
    "    shuffle=True,                               # Shuffle the samples randomly\n",
    "    seed=1,                                     # Random number seed to use for shuffle\n",
    "    save_to_dir=dir_temp_aug_images if flag_save_aug else None, # Directory to save aug images if flag is True\n",
    "    save_prefix='valid_aug_',                   # Prefix to apply to any saved images\n",
    "    save_format='png',                          # File format to appy to any saved images\n",
    "    interpolation='bicubic'                     # Interpolation method used to resize samples\n",
    "    )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 733 validated image filenames belonging to 5 classes.\n"
     ]
    }
   ],
   "source": [
    "# # Create a Image Data Generator for the testing data\n",
    "# test_datagen = ImageDataGenerator(\n",
    "#         rotation_range=20,             # Range (degrees) of rotation to apply\n",
    "#         zoom_range=0.15,               # Range of zoom to apply: Up to +/- (1-zoom_range)\n",
    "#         width_shift_range=0.2,         # Shift the image by a fraction of total width \n",
    "#         height_shift_range=0.2,        # Shift the image by a fraction of total height\n",
    "#         shear_range=0.15,              # Range (degrees) of shear angle (FYI: cotangent shear angle = shear factor)\n",
    "#         horizontal_flip=True,          # Perform random horizontal flips of the images\n",
    "#         vertical_flip=True,            # Perform random vertical flips of the images\n",
    "#         rescale=1.0/255.0,             # Rescale the RGB color values from 0-255 to 0.0-1.0\n",
    "#         fill_mode=\"nearest\"            # Interpolation method\n",
    "#     )\n",
    "\n",
    "# Create the Image Data Generator for the testing data but with no augmentation\n",
    "test_datagen = ImageDataGenerator()\n",
    "\n",
    "# Define generator batch size and step size\n",
    "test_n_images = len(X_test)\n",
    "test_n_batch_size = 1\n",
    "test_n_step_size = test_n_images // test_n_batch_size\n",
    "\n",
    "# Create dataframes with the testing data\n",
    "# test_df = pd.DataFrame( {'id_code': X_test }).reset_index(drop=True)\n",
    "test_df = pd.DataFrame( {'id_code': X_test, 'diagnosis': y_test }).reset_index(drop=True)\n",
    "\n",
    "\n",
    "# Associated the generator with the testing images\n",
    "# NOTE: These are testing images to potentially be used for hyperparameter tuning.\n",
    "#       The 'official' testing images have no label data\n",
    "test_generator = test_datagen.flow_from_dataframe(\n",
    "    dataframe=test_df,                          # Dataframe with filenames of images and column for labels\n",
    "    directory=dir_train_src,                    # Directory containing images used for training, validation, and testing\n",
    "    x_col='id_code',                            # Column containing filenames of images in directory dir_train_src\n",
    "    y_col='diagnosis',                          # Column containing the labels for the images\n",
    "    target_size=(image_dims[1], image_dims[0]), # The (height, width) to which each image will be resized\n",
    "    color_mode='rgb',                           # Images will be color ('rgb') (3 color channels) vs. 'grayscale'\n",
    "    class_mode='categorical',                   # Performs one-hot encoding of the string label in y_col\n",
    "    batch_size=test_n_batch_size,               # Number of images to be used per batch\n",
    "    shuffle=False,                              # Shuffle the samples randomly\n",
    "    seed=1,                                     # Random number seed to use for shuffle\n",
    "    save_to_dir=dir_temp_aug_images if flag_save_aug else None, # Directory to save aug images if flag is True\n",
    "    save_prefix='test_aug_',                    # Prefix to apply to any saved images\n",
    "    save_format='png',                          # File format to appy to any saved images\n",
    "    interpolation='bicubic'                     # Interpolation method used to resize samples\n",
    "    )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set parameters needed to fit the CNN model\n",
    "# initialize the number of epochs to train for, initial learning rate,\n",
    "# batch size, and image dimensions\n",
    "# n_epochs = 10\n",
    "n_epochs = 1\n",
    "init_learning_rate = 1e-3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the model\n",
    "model = SmallerVGGNet.build(\n",
    "    width=image_dims[0],\n",
    "    height=image_dims[1],\n",
    "    depth=image_dims[2],\n",
    "    classes=n_classes)\n",
    "\n",
    "opt = Adam( lr=init_learning_rate, decay=init_learning_rate / n_epochs )\n",
    "\n",
    "model.compile(loss=\"categorical_crossentropy\", optimizer=opt, metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_6 (Conv2D)            (None, 96, 96, 32)        896       \n",
      "_________________________________________________________________\n",
      "activation_8 (Activation)    (None, 96, 96, 32)        0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_7 (Batch (None, 96, 96, 32)        128       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_4 (MaxPooling2 (None, 32, 32, 32)        0         \n",
      "_________________________________________________________________\n",
      "dropout_5 (Dropout)          (None, 32, 32, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_7 (Conv2D)            (None, 32, 32, 64)        18496     \n",
      "_________________________________________________________________\n",
      "activation_9 (Activation)    (None, 32, 32, 64)        0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_8 (Batch (None, 32, 32, 64)        256       \n",
      "_________________________________________________________________\n",
      "conv2d_8 (Conv2D)            (None, 32, 32, 64)        36928     \n",
      "_________________________________________________________________\n",
      "activation_10 (Activation)   (None, 32, 32, 64)        0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_9 (Batch (None, 32, 32, 64)        256       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_5 (MaxPooling2 (None, 16, 16, 64)        0         \n",
      "_________________________________________________________________\n",
      "dropout_6 (Dropout)          (None, 16, 16, 64)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_9 (Conv2D)            (None, 16, 16, 128)       73856     \n",
      "_________________________________________________________________\n",
      "activation_11 (Activation)   (None, 16, 16, 128)       0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_10 (Batc (None, 16, 16, 128)       512       \n",
      "_________________________________________________________________\n",
      "conv2d_10 (Conv2D)           (None, 16, 16, 128)       147584    \n",
      "_________________________________________________________________\n",
      "activation_12 (Activation)   (None, 16, 16, 128)       0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_11 (Batc (None, 16, 16, 128)       512       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_6 (MaxPooling2 (None, 8, 8, 128)         0         \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 8, 8, 128)         0         \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 8192)              0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1024)              8389632   \n",
      "_________________________________________________________________\n",
      "activation_13 (Activation)   (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_12 (Batc (None, 1024)              4096      \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 5)                 5125      \n",
      "_________________________________________________________________\n",
      "activation_14 (Activation)   (None, 5)                 0         \n",
      "=================================================================\n",
      "Total params: 8,678,277\n",
      "Trainable params: 8,675,397\n",
      "Non-trainable params: 2,880\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/1\n",
      " - 426s - loss: 1.9779 - acc: 0.4697 - val_loss: 1.8613 - val_acc: 0.5639\n"
     ]
    }
   ],
   "source": [
    "# Fit/Train the model (using the train_generator)\n",
    "hist = model.fit_generator(\n",
    "    generator=train_generator,\n",
    "    steps_per_epoch=train_n_step_size,\n",
    "    validation_data=valid_generator,\n",
    "    validation_steps=valid_n_step_size,\n",
    "    epochs=n_epochs,\n",
    "    verbose=2\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>epoch</th>\n",
       "      <th>val_loss</th>\n",
       "      <th>val_acc</th>\n",
       "      <th>loss</th>\n",
       "      <th>acc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.8613</td>\n",
       "      <td>0.5639</td>\n",
       "      <td>1.9779</td>\n",
       "      <td>0.4697</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   epoch  val_loss  val_acc    loss     acc\n",
       "0      0    1.8613   0.5639  1.9779  0.4697"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_history_df = pd.DataFrame( hist.history, columns=list(hist.history.keys()) )\n",
    "fit_history_df = fit_history_df.round(4)\n",
    "fit_history_df['epoch'] = fit_history_df.index\n",
    "fit_history_df = fit_history_df[ ['epoch'] + list(hist.history.keys()) ]\n",
    "fit_history_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss: 1.8916\n",
      "Acc: 0.5554\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model (using the valid_generator)\n",
    "valid_eval_info = model.evaluate_generator(\n",
    "    generator=valid_generator,\n",
    "    steps=valid_n_step_size\n",
    "    )\n",
    "\n",
    "# Print the evaluation results (Accuracy and Loss)\n",
    "for m,v in zip(model.metrics_names, valid_eval_info):\n",
    "    print(f\"{m.title()}: {v:0.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Eval - Accuracy: 0.5553977272727273, Loss: 1.8916238031604073\n"
     ]
    }
   ],
   "source": [
    "valid_eval_acc = valid_eval_info [ model.metrics_names.index('acc') ]\n",
    "valid_eval_loss = valid_eval_info [ model.metrics_names.index('loss') ]\n",
    "print(f\"Validation Eval - Accuracy: {valid_eval_acc}, Loss: {valid_eval_loss}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss: 8.1346\n",
      "Acc: 0.4952\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model (using the test_generator)\n",
    "test_generator.reset()\n",
    "test_eval_info = model.evaluate_generator(\n",
    "    generator=test_generator,\n",
    "    steps=test_n_step_size\n",
    "    )\n",
    "\n",
    "# Print the evaluation results (Accuracy and Loss)\n",
    "for m,v in zip(model.metrics_names, test_eval_info):\n",
    "    print(f\"{m.title()}: {v:0.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Testing Eval - Accuracy: 0.495225102319236, Loss: 8.13462653361759\n"
     ]
    }
   ],
   "source": [
    "test_eval_acc = test_eval_info [ model.metrics_names.index('acc') ]\n",
    "test_eval_loss = test_eval_info [ model.metrics_names.index('loss') ]\n",
    "print(f\"Testing Eval - Accuracy: {test_eval_acc}, Loss: {test_eval_loss}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABHYAAAFHCAYAAADEL8svAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZBV1Z3A8d9jid00pNltG7BlCRiJEDGoRJhQmRnHTMxgVAIicUklmRpnmOjErYSxiSMUrmMMk4xLhQghxiVxjcSoUXEtHCyIKMMioVnaoEjYlKWXM38kvglhayLbgc+nqqt899x+7x7qet7tb/d7r5BSCgAAAADy0+xAHwAAAAAAfxlhBwAAACBTwg4AAABApoQdAAAAgEwJOwAAAACZarGrwdLS0t9t3rz5yP11MEC+SkpKGjdv3nxIx+LDYY7A3mG9AJrKegE0RUlJyapNmzZV7GissKuPOy8UCsnHoQNNUSgU4lBfLw6HOQJ7h/UCaCrrBdAUf1wrCjsaU4YBAAAAMiXsAAAAAGRK2AEAAADIlLADAAAAkClhBwAAACBTwg4AAABApoQdAAAAgEwJOwAAAACZEnYAAAAAMiXsAAAAAGRK2AEAAADIlLADAAAAkClhBwAAACBTwg4AAABApoQdAAAAgEwJOwAAAACZEnYAAAAAMiXssE8tXbo0CoVC1NfXH+hDAQDY64455ph46qmnDvRhAHAYE3aAg86zzz4bXbt23e1+LqbZH5p6PgIAwIEg7AAAAABkStg5DNXW1sbZZ58dnTp1iu7du8dtt90WERHjx4+Pc845J0aMGBFt2rSJAQMGxNy5c4vfN3/+/Bg6dGi0bds2+vbtG4888khxbNOmTfHtb387qqqqory8PAYPHhybNm0qjk+fPj2OPvro6NixY0yYMGH/TRYAYD/YsmVLXHLJJVFZWRmVlZVxySWXxJYtWyIiYvXq1XHGGWdE27Zto3379jFkyJBobGyMiIjrr78+unTpEm3atIk+ffrE008/fSCnAewHkyZNip49e0abNm3iuOOOiwcffLA4duedd8YnP/nJ4thrr70WERHLly+Ps846Kzp16hQdOnSIf/mXfzlQh89BSNg5zDQ2NsaXvvSl6N+/f6xcuTKefvrpuPXWW+OJJ56IiIiHH344hg8fHmvWrIlRo0bFmWeeGXV1dVFXVxdf+tKX4rTTTot33nknvve978V5550XCxYsiIiIyy67LGbPnh0vvfRSrFmzJm644YZo1uz/T68XXnghFixYEE8//XRce+21MX/+/AMyf/avSZMmxTnnnLPNtm9961vxr//6rzFlypTik1aPHj3i9ttv/0iP5YKa3dnX5+OuLtIiXKjBoW7ChAnxyiuvxJw5c2Lu3Lkxa9asuO666yIi4uabb46uXbvGu+++G6tWrYqJEydGoVCIBQsWxOTJk+PVV1+NDRs2xBNPPBHHHHPMgZ0IsM/17Nkznn/++Vi3bl1UV1fH6NGj4+233477778/xo8fH1OnTo3169fHI488Eh06dIiGhoY444wzoqqqKpYuXRorV66MkSNHHuhpcDBJKe306w/DHEpeeeWV1K1bt222TZw4MV144YWpuro6nXzyycXtDQ0NqaKiIs2cOTPNnDkzHXnkkamhoaE4PnLkyFRdXZ0aGhpSSUlJmjNnznaP99vf/jZFRFq+fHlx28CBA9M999yzD2bHgbSj9WLp0qWptLQ0rVu3LqWUUn19faqoqEgvv/xyeuyxx9LixYtTY2NjevbZZ1NpaWmaPXt2SimlZ555JnXp0mW3j1lVVZWefPLJlFJK//7v/55OPvnktGrVqvTOO++kQYMGpXHjxqWUUrrqqqvSP/7jP6atW7emrVu3ppkzZ6bGxsb0v//7v6lr165p5cqVKaU/nK+LFy/eozmSj319Pt53331p5cqVqaGhIf30pz9NrVq1SrW1tcWxysrKNGvWrNTY2JgWLVqUli5dmurr61O/fv3SJZdckjZu3Jg2bdqUnn/++X33j8B+Y704fHz4XNSjR4/0i1/8orj9l7/8Zaqqqkop/eE56h/+4R/SokWLtvneRYsWpU6dOqUnn3wybd26dX8eNgcR6wX9+/dPDz30UDrttNPSrbfeut34Sy+9lDp27Jjq6uoOwNFxsPjjWrHDduMvdg4zNTU1UVtbG23bti1+TZw4MVatWhUREd26dSvu26xZs+jatWvU1tZGbW1tdOvWbZu/wqmqqoqVK1fG6tWrY/PmzdGzZ8+dPm5FRUXxv1u1ahUbN27cB7PjYFNVVRUDBgyIhx56KCIifv3rX0erVq3ilFNOiS9+8YvRs2fPKBQK8bnPfS5OO+20eP755//ix5o+fXpcc8010blz5+jUqVNUV1fHtGnTIiKiZcuW8fbbb0dNTU20bNkyhgwZEoVCIZo3bx5btmyJN998M+rq6uKYY47Z5XlM3vb1+Th8+PCorKyMZs2axYgRI+ITn/hEzJo1KyIi7rrrrrjiiiti4MCBUSgUolevXlFVVRWzZs2K2trauPHGG6OsrCxKSkpi8ODBe33uwL5XW1sbVVVVxdtVVVVRW1sbERGXX3559OrVK0477bTo0aNHTJo0KSIievXqFbfeemuMHz8+OnfuHCNHjix+D3Domjp1anz6058u/jw2b968WL16dSxfvnyH16LLly+PqqqqaNGixQE4WnIg7BxmunXrFt27d4+1a9cWvzZs2BCPP/54RPxh0fhQY2NjrFixovjSluXLlxdfvhIRsWzZsujSpUt07NgxSkpK4q233trv8+HgN2rUqLjnnnsiIuInP/lJjBo1KiIiZsyYEaecckq0b98+2rZtG48//nisXr36L34cF9Q0xb48H3d2kRYRLtTgMFBZWRk1NTXF28uWLYvKysqIiGjTpk3cfPPNsWTJknj00UfjlltuKb70d9SoUfHCCy9ETU1NFAqFuPLKKw/I8QP7R01NTXzjG9+IyZMnx3vvvRdr166NT33qU5FSim7duu3wZ6pu3brFsmXLor6+/gAcMTkQdg4zJ510Unz84x+P66+/PjZt2hQNDQ0xb968ePXVVyMiYvbs2fHzn/886uvr49Zbb40jjjgiTjnllDj55JOjrKwsbrjhhqirq4tnn302Hn300Rg5cmQ0a9Ysvva1r8W//du/RW1tbTQ0NMTLL79cfH8TDm/Dhw+PZ599NlasWBEPPvhgjBo1KrZs2RJnn312XHbZZbFq1apYu3Zt/P3f//2HLwH9i7igpin21fm4q4u0iHChBoeBc889N6677rp49913Y/Xq1XHttdfG6NGjIyLisccei8WLF0dKKT7+8Y9H8+bNo3nz5rFgwYL49a9/HVu2bImSkpIoLS2N5s2bH+CZAPvS+++/H4VCITp16hQREVOmTIl58+ZFRMTXv/71uOmmm2L27NmRUorFixdHTU1NnHTSSXHUUUfFVVddFe+//35s3rw5XnzxxQM5DQ4yws5hpnnz5vHoo4/GnDlzonv37tGxY8f4+te/HuvWrYuIiGHDhsW9994b7dq1i2nTpsXPf/7zaNmyZXzsYx+LRx55JGbMmBEdO3aMiy++OKZOnRrHHntsRETcdNNNcfzxx8fAgQOjffv2ceWVV27z1z0cvjp16hRDhw6Niy66KLp37x6f/OQnY+vWrbFly5bo1KlTtGjRImbMmBG/+tWvPtLjuKCmKfbV+biri7QIF2pwOBg3blx85jOfiX79+sXxxx8fAwYMiHHjxkVExKJFi+Jv/uZvonXr1jFo0KC4+OKLY+jQobFly5a46qqromPHjlFRURHvvPNOTJw48QDPBNiXjjvuuPj2t78dgwYNiiOPPDJef/31OPXUUyPiD7+AGjt2bIwaNSratGkTZ555ZqxZs6b4M9zixYvj6KOPjq5du8a99957gGfCQWVnb76TvHnyYae6ujqdd955B/owyNSu1oupU6emiEg33HBDcdvkyZNT586dU3l5eRo9enQaMWJEGjt2bErpL3vz5E2bNqUxY8akioqKVFFRkcaMGZM2bdqUUkrplltuSVVVValVq1apS5cu6dprr00ppTR37tw0cODA1Lp169SuXbv0xS9+sfhGyns6R/Kxr87Hq6++OrVr1y516NAhXXrppemv/uqv0p133lkc/8EPfpB69+6dysrKUt++fdNrr72WUkqppqYmDRs2LLVv3z516NAhjRkzZi/PmAPBegE0lfUCaIrYxZsnF9Iu/tS8UCikXY1zaBk/fnwsXrw4fvzjHx/oQyFDhULhI72UKgeHwxyBvcN6ATSV9QJoij+uFYUdjXkpFgAAAECm/MUOsFfsi982LVu2LI477rgdjr355ptx9NFH79XH2x2/UTu8HWznIwc36wXQVNYLoCl29Rc7wg6wVxwOFyWHwxyBvcN6ATSV9QJoCi/FAgAAADgECTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJCpFrsaLCkpaSwUCuIPsFslJSVRKBQO9GHsU4fDHIG9w3oBNJX1AmiKkpKSxp2NFVJKO/3GQqGQdjUO8KFCoRCH+npxOMwR2DusF0BTWS+ApvjjWrHDCuyvcQAAAAAyJewAAAAAZErYAQAAAMiUsAMAAACQKWEHAAAAIFPCDgAAAECmhB0AAACATAk7AAAAAJkSdgAAAAAyJewAAAAAZErYAQAAAMiUsAMAAACQKWEHAAAAIFPCDgAAAECmhB0AAACATAk7AAAAAJkSdgAAAAAyJewAAAAAZErYYTtr1qyJL3/5y1FWVhZVVVXxk5/8ZIf7pZTiyiuvjA4dOkSHDh3iiiuuiJTSfj5acjJ58uT4zGc+E0cccURceOGFO90vpRTjxo2LLl26RHl5eQwdOjTeeOONbfZ56qmnYsCAAVFWVhbdunWL++67rzj2zW9+M/r06RPNmjWLH/3oR9t8309/+tPo06dPlJeXR+fOneOCCy6I9evX781pkoGmrnPjx4+Pli1bRuvWrYtfS5YsKY43NDTEuHHjorKyMtq0aRMnnHBCrF27NiJ2f67Nnz8/Pv/5z0d5eXn06tUrHnzwwX07aWCPNfV5a8uWLXHppZdGZWVltGvXLi6++OKoq6srjv/pGtK6deto3rx5jBkzJiIipk+fvs1Yq1atolAoxOzZsyMiYu3atXHBBRdE586do3PnzjF+/Ph9OWXgL9TUa4sPbd26NY499tjo2rXrNtt3dW3xUdYaDm3CDtv553/+5/jYxz4Wq1atiunTp8c//dM/bfdDdUTEHXfcEQ899FDMnTs3fvOb38Rjjz0Wt99++wE4YnJRWVkZ48aNi6997Wu73O/++++PH/7wh/H888/HmjVrYtCgQfHVr361OP7mm2/GqFGjYsKECbFu3bqYM2dOnHjiicXx/v37x/e///0YMGDAdvd96qmnxosvvhjr1q2LJUuWRH19fYwbN27vTZIsNHWdi4gYMWJEbNy4sfjVo0eP4lh1dXW89NJL8fLLL8f69etj2rRpUVJSEhG7Ptfq6+tj2LBhccYZZ8SaNWvijjvuiNGjR8fChQv3/eSBJmvq89akSZPif/7nf2LevHmxcOHCeO211+K6664rjv/pGrJq1aooLS2N4cOHR0TEeeedt83497///ejRo0fxOezSSy+NDz74IJYuXRqzZs2KadOmxZQpU/bdpIG/yJ5cW0RE3HjjjdG5c+fttu/q2uKjrDUc4lJKO/36wzCHk40bN6aWLVumBQsWFLeNHj06XXnlldvtO2jQoHT77bcXb991113p5JNP3i/HycFnT9aLsWPHpgsuuGCn45MmTUrDhw8v3p43b1464ogjirfPPffcNG7cuN0+zqmnnpqmTJmy0/ENGzakr371q+kLX/hCk47bmnho2JN1rrq6Op133nk7vJ81a9aksrKytHjx4t0+5p+fa6+//noqKytLjY2NxX3+9m//tknnNXmwXhxadve8deKJJ6b77ruveHv69Ompa9euO9z3Rz/6Uerevfs2////qaFDh6bx48cXb3fo0CHNmjWreHvChAlp8ODBezgDDmbWi/ztybVFSiktWbIkHXvssenxxx9PXbp0KW7f3bXF3lxryM8f14odtht/scM2Fi5cGM2bN4/evXsXt/Xv33+HtfmNN96I/v3773Y/2FMjR46MxYsXx8KFC6Ouri7uvvvuOP3004vjr7zySkREHH/88XHUUUfF6NGjY82aNU2+/xdeeCHKy8ujTZs28bOf/SwuueSSvT4HDl57ss5FRDz66KPRvn376Nu3b/zgBz8obn/99dejRYsW8cADD0RFRUX07t07/uu//mub793ZuZZ28LLVlFLMmzdvb0wR2M/S//9StHh7xYoVsW7duu32vfvuu+P888+PQqGw3VhNTU3MnDkzzj///O3u/0//21oBB5c9vbYYM2ZMTJw4MUpLS7fZvrtri7211nDoEXbYxsaNG6O8vHybbeXl5bFhw4bd7lteXh4bN270Pjt8ZEcddVQMGTIk+vTpE6WlpXH//ffHf/7nfxbHV6xYEdOmTYuf/exnsWjRoti0adMevX548ODBsW7dulixYkVcfvnlccwxx+yDWXCw2pN17itf+UrMnz8/3n333bjzzjvj2muvjXvuuScionghtXDhwvjtb38bDzzwQIwfPz6efPLJ4vfv7Fw79thjo3PnznHjjTdGXV1d/OpXv4rnnnsuPvjgg303cWCf+cIXvhDf/e534913343f/e53cdttt0VEbPf/9LJly+K5556LCy64YIf3M3Xq1BgyZEh07969uO3000+PSZMmxYYNG2Lx4sXxwx/+0FoBB5k9ubZ48MEHo76+Pr785S9vN7a7a4u9tdZw6BF22Ebr1q23eyPZ9evXR5s2bXa77/r166N169aqMB/Zd77znXj11Vdj+fLlsXnz5qiuro7Pf/7zxSet0tLSuOiii6J3797RunXruPrqq+Pxxx/f48fp0qVLnH766TFy5Mi9PQUOYnuyzh133HFRWVkZzZs3j89+9rPxrW99Kx544IGIiOJv2a655pooLS2Nfv36xciRI3d4Lv75udayZct46KGH4he/+EVUVFTEzTffHF/5yle2ewNFIA9jx46NE044IT796U/HZz/72TjzzDOjZcuW271/xtSpU2Pw4MHbhJs/H//zH8Ruu+22KC0tjU984hMxbNiwOPfcc60VcJBp6rXF+++/H1dccUV873vf2+H97O7aYm+tNRx6hB220bt376ivr49FixYVt82dOzf69u273b59+/aNuXPn7nY/2FNz586NESNGRNeuXaNFixZx4YUXxu9///t48803IyKiX79+ey0g1tfXx1tvvbVX7os87Mk69+cKhULxrxL79etX3NYUf36u9evXL5577rl477334oknnoglS5bESSedtCdTAQ4SpaWlMXny5Fi5cmUsWbIkOnToECeeeGI0b958m/12FG4+9OKLL0ZtbW2cc84522xv3759TJ8+PX73u9/FG2+8EY2NjdYKOMg09dpi0aJFsXTp0hgyZEhUVFTEWWedFW+//XZUVFTE0qVLd3ttsTfWGg5Nwg7bKCsri7POOiuuueaaeP/99+PFF1+Mhx9+eJtPJPrQ+eefH7fcckusXLkyamtr4+abb97lR4FCfX19bN68ORoaGqKhoSE2b94c9fX12+03cODAuP/++2PVqlXR2NgY06ZNi7q6uujVq1dERFx00UUxZcqUWLJkSXzwwQdx/fXXxxlnnFH8/q1bt8bmzZsjpRR1dXWxefPmaGxsjIg/fKzssmXLIqUUNTU1MXbs2Pjrv/7r/fMPwEFhT9a5hx9+OH7/+99HSilmzZoVt912WwwbNiwiInr27BlDhgyJCRMmxJYtW2L+/Plx7733Fs/F3Z1rv/nNb2Lz5s3xwQcfxE033RRvv/22NRQOMk193vrwWiilFK+88kr8x3/8R3znO9/ZZp+XXnopVq5cudNPqLn77rvj7LPP3u43/G+99Va899570dDQEDNmzIg77rjDpznCQaap1xaf+tSnYvny5TFnzpyYM2dO3HXXXXHkkUfGnDlzolu3bru9ttgbaw2HqJ29q3LyqViHrffeey8NGzYstWrVKnXr1i1Nnz49pZTSzJkzU1lZWXG/xsbGdPnll6d27dqldu3apcsvv9y7rh/GmrJeVFdXp4jY5qu6ujrV1NSksrKyVFNTk1JKadOmTeniiy9OFRUVqU2bNumEE05IM2bM2Oa+rrnmmtSxY8fUsWPHNHr06LRmzZri2Oc+97ntHueZZ55JKaV09dVXpy5duqRWrVqlLl26pG984xtp9erVe22O5KGp69zIkSNT+/btU1lZWerTp0/67ne/u839rFixIv3d3/1dKisrS927d0///d//XRzb3bl22WWXpbZt26aysrJ0+umnp0WLFu3jWbM/WS8ODU193nruuedSVVVVKi0tTb17904//vGPt7uvb37zm2n06NE7fJxNmzal8vLy9NRTT203du+996ajjjoqlZaWpv79+6df/vKXe3eSHHDWi0NDU68t/tQzzzyzzadipbTra4uPutaQt9jFp2IV0i7e6LZQKKRdjQN86E9fonKoOhzmCOwd1gugqawXQFP8ca3Y4ev0vBQLAAAAIFPCDgAAAECmhB0AAACATAk7AAAAAJkSdgAAAAAyJewAAAAAZErYAQAAAMiUsAMAAACQKWEHAAAAIFPCDgAAAECmhB0AAACATAk7AAAAAJkSdgAAAAAyJewAAAAAZErYAQAAAMiUsAMAAACQKWEHAAAAIFPCDgAAAECmWuxqsKSkZFWhUDhyfx0MkK+SkpLGQqFwSMfiw2GOwN5hvQCaynoBNEVJScmqnY0VUkr781gAAAAA2EuUYQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU8IOAAAAQKaEHQAAAIBMCTsAAAAAmRJ2AAAAADIl7AAAAABkStgBAAAAyJSwAwAAAJApYQcAAAAgU4PHjuAAAAAGSURBVP8HFT0TKHk8VhoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Table: Fit History\n",
    "# fig_table_1 = plt.figure(figsize=(15,12))\n",
    "# fig_table_1 = plt.figure(figsize=(15,10))\n",
    "fig_table_1 = plt.figure()\n",
    "\n",
    "# Create axes for this table, but with no visible frame\n",
    "ax_table = fig_table_1.add_subplot(111, frame_on=False)\n",
    "\n",
    "# Hide the x and y axes\n",
    "ax_table.xaxis.set_visible(False)\n",
    "ax_table.yaxis.set_visible(False)\n",
    "\n",
    "# Create the table\n",
    "results_table = plt.table(\n",
    "            cellText=fit_history_df.values,\n",
    "#             colWidths = [0.2]*len(table_df.columns),\n",
    "#             rowLabels=table_df.index,\n",
    "            colLabels=fit_history_df.columns,\n",
    "            cellLoc = 'center',\n",
    "            rowLoc = 'center',\n",
    "            loc='top')\n",
    "\n",
    "# Set the fontsize to something readable\n",
    "results_table.auto_set_font_size(False)\n",
    "results_table.set_fontsize(12)\n",
    "\n",
    "# Set the column widths automatically\n",
    "results_table.auto_set_column_width(True)\n",
    "\n",
    "# Scale the table\n",
    "results_table.scale(4,4)\n",
    "\n",
    "# Show the resulting table\n",
    "plt.show()\n",
    "        \n",
    "# Save the image - use it for Summary of Analysis at the top of the notebook\n",
    "fig_table_1.savefig('docs/Table-DPA_Fit_History.png', bbox_inches='tight') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAJcCAYAAAC8DwN/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXwW1dn/8c9FAAGV1YpgELBgkS0sUdSibIpoVUR8FEoVrYq1P9dWn2Jxxa2lrrRWi9WKiuDyCLVV3EkRV6xABUREChrZdwKEJVy/P84k3IQ7IeQOyZB836/X/crMmTMz1z0n6JVzzsyYuyMiIiIi8VCtogMQERERkV2UnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5E6nCzKyFmbmZVS9B3UvMbFopzjHEzN7ax31+a2Z/3ddzlRUzyzGzoyvq/CJStSk5EzlAmNkiM9tmZocVKp8ZJVgtKiay3ZK8nITPLAB3H+fufRPqupm1SljvaWbZicdz93vd/fL9FOtu54/K7jCz5xLOf4i7L9zLcfaIOw4sWGhmcys6FhEpHSVnIgeW/wKD81fMrANQu+LC2UP9KLE5xN0zKjqYOCtJb2UpnQIcDhxtZsftp3MktR+/k0iVouRM5MDyLHBxwvpQ4JnECmZWz8yeMbOVZrbYzG4xs2rRtjQzu9/MVpnZQuAnSfZ90syWmtn3Zna3maWlEnDicKiZTY2KZ0W9a0OByUDThB63pok9WQm9ckPN7Nso9hEJx69tZmPNbK2ZfWlm/5tqj1Zi75qZnWlmc81sY3RNbjSzg4uI+yAze9jMlkSfh83soOg4Pc0s28x+Y2bLgL+Z2WwzOzvhvDWi79cphfCHAn8HXo+WE79XQzP7WxTbWjOblLCtf9QLu8HMvjGzflH5IjM7NaFesra5zMy+Bd6Lyl8ys2Vmtt7MpppZu4T9a5vZA9Hv5nozmxaVvWZm1xSK9z9mdm4K10LkgKTkTOTA8jFQ18yOjZKmC4HnCtX5I1APOBroQUjmLo22XQGcBXQGMoHzC+07FtgBtIrq9AXKbHjR3U+JFjOi3rWxwBnAkoQetyVF7N4d+BHQB7jNzI6Nym8HWhC+72nAz8oq3siTwJXufijQHnjP3TcVEfcI4ASgE5ABHA/cknCsI4CGQHNgGCGxToz3TGCpu88sTaBmVofQpuOizyAzq5lQ5VmgDtCO0Lv2ULTf8VEsNwH1Cb1vi/bh1D2AY4HTo/XJQOvoHJ9HseS7H+gKnES4Fv8L7CT87hVcCzPLAI4kJJkiVYqSM5EDT37v2WnAPOD7/A0JCdvN7r7R3RcBDwAXRVUuAB529+/cfQ1wX8K+jQkJx/XuvsndVxD+5z1oH2JbZWbros+Npf6Gyd3p7lvcfRYwi5D8QPhO97r7WnfPBkaX4FifJ8S5DhheTN3tQFszqxud4/Ni6g4BRrr7CndfCdzJrmsPIQm53d23uvsWQmJ9ppnVjbZfRGjf0joP2Aq8BfwTqE7UO2pmTQjt+4voe2x3939F+10GPOXub7v7Tnf/3t3n7cN574h+Z7YAuPtT0e/fVuAOICPqla0G/By4LjpHnrt/GNX7O9DazFonXIsX3H1bCtdD5ICk5EzkwPMs8FPgEgoNaQKHATWBxQlliwk9EABNge8KbcvXHKgBLE1IWv5C6P0oqcPcvX70uX8f9iuJZQnLm4FDouXC3ylxuShdEuKsD/yumLoDCT1ai83sX2Z2YjF1m7LntW+asL7S3XPzV6Letg+AgWZWn5A8JfYyFTCzyQlDqEOKOP9Q4EV33xElPK+wa2izGbDG3dcm2a8Z8E0x32tvCq55NHT+u2hodAO7euAOiz61kp0rivdF4GdREjeY1BJVkQOWJm+KHGDcfbGZ/ZeQMFxWaPMqQk9PcyD/br2j2NW7tpTwP2IStuX7jtDrcpi77yjruIvhKe6/FEhn1/dtVkzdfebu04H+ZlYDuJqQQDQjedxLCNd+TrR+VFRWcLgk+4wlDB1XBz5y9++T1MHdzyguTjNLB3oDx5vZwKi4DlDLwh2+3wENzay+u68rtPt3wA+LOPSm6Dj5jkgWXsLyT4H+wKmExKwesBYwwu9nbnSuWUmOM5aQkE0DNrv7R0XEJFKpqedM5MB0GdA7mvtUwN3zCMnDPWZ2qJk1B37FrnlpLwLXmlm6mTUgYTjP3ZcShsMeMLO6ZlbNzH5oZj3KOPblhPlhieuNzKxeKY/3InCzmTUwsyMJCVSZMLOaFp7TVs/dtwMbgLxoc7K4xwO3mNkPooToNvacE1jYJKALcB179oTui4uA+YR5eZ2izzFANjA4at/JwJ+ja1XDzPLnAD4JXGpmfaJ2P9LM2kTbZhLmrtUws2TzFAs7lJDkryYkdffmb3D3ncBTwIPRDRRpZnZi/k0TUTK2kzAUr14zqbKUnIkcgNz9G3f/rIjN1xB6OxYSeiCeJ/wPEeAJ4E1Cr8XnhGGvRBcThkXnEno7XgaalGnwYQ7S2Gjo9IJobtN4YGFU1rT43fcwkpCA/Bd4hxDz1jKM9yJgUTRE9wuiSetFxH038BnwH+ALwjW+u7iDR/O0/g9oyZ7tsS+GAn9292WJH+Bxdg1tXkToWZ0HrACuj2L4lHDTyEPAeuBfhB5AgFsJPV1rCXPont9LHM8QhnO/J/wefVxo+42EazMdWAP8nt3/X/QM0IG9J7UilZa5pzqiICISH2Z2FTDI3cu6x2+/MbPbgGPcvazvND3gmNnFwDB3717RsYhUFPWcicgBzcyamNmPo+G4HwG/BiZWdFwlZWYNCcPUYyo6looWPQrkl+haSBVXIcmZmT1lZivMbHYR29uY2UdmtnU/3I4vIpVLTcJdpRsJD0H9O/DnCo2ohMzsCsJk/MnuPnVv9SszMzsdWEmYy7e3oVORSq1ChjWjSag5wDPu3j7J9sMJ8x3OBdbuh1vyRURERGKpQnrOor8Q1xSzfUV0+/r28otKREREpOId8M85M7NhhNegULt27a7NmpXpI44qtZ07d1KtmqYdxonaJJ7ULvGjNokftcm+mz9//ip3/0Hh8gM+OXP3MUSTRzMzM/2zz4p6uoAUlpWVRc+ePSs6DEmgNokntUv8qE3iR22y78xscbJypbgiIiIiMaLkTERERCRGKmRY08zGAz2Bw8wsG7id8MJl3P1xMzuC8JTtusBOM7seaOvuGyoiXhEREZHyUiHJmbsP3sv2ZYQXGYuIiIhUKRrWFBEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxEiFJGdm9pSZrTCz2UVsNzMbbWYLzOw/ZtalvGMUERERqQgV1XP2NNCvmO1nAK2jzzDgsXKISURERKTCVUhy5u5TgTXFVOkPPOPBx0B9M2tSPtGJiIiIVJzqFR1AEY4EvktYz47KlhauaGbDCL1rNG7cmKysrPKIr1LIycnR9YoZtUk8qV3iR20SP2qTshPX5MySlHmyiu4+BhgDkJmZ6T179tyPYVUuWVlZ6HrFi9okntQu8aM2iR+1SdmJ692a2UCzhPV0YEkFxSIiIiJSbuKanL0KXBzdtXkCsN7d9xjSFBEREalsKmRY08zGAz2Bw8wsG7gdqAHg7o8DrwNnAguAzcClFRGniIiISHmrkOTM3QfvZbsD/6+cwhERERGJjbgOa4qIiIhUSUrORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjFRIcmZm/czsKzNbYGbDk2xvbmbvmtl/zCzLzNIrIk4RERGR8lbuyZmZpQGPAmcAbYHBZta2ULX7gWfcvSMwErivfKMUERERqRgV0XN2PLDA3Re6+zZgAtC/UJ22wLvR8pQk20VEREQqpeoVcM4jge8S1rOBboXqzAIGAo8AA4BDzayRu68ufDAzGwYMA2jcuDFZWVn7I+ZKKScnR9crZtQm8aR2iR+1SfyoTcpORSRnlqTMC63fCPzJzC4BpgLfAzuSHczdxwBjADIzM71nz55lFmhll5WVha5XvKhN4kntEj9qk/hRm5SdikjOsoFmCevpwJLECu6+BDgPwMwOAQa6+/pyi1BERESkglTEnLPpQGsza2lmNYFBwKuJFczsMDPLj+1m4KlyjlFERESkQpR7cubuO4CrgTeBL4EX3X2OmY00s3Oiaj2Br8xsPtAYuKe84xQRERGpCBUxrIm7vw68XqjstoTll4GXyzsuERERkYqmNwSIiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxEhKyZmZXW1mDcoqGBEREZGqLtWesyOA6Wb2opn1MzMri6BEREREqqqUkjN3vwVoDTwJXAJ8bWb3mtkPyyA2ERERkSon5Tln7u7AsuizA2gAvGxmo4raJ+pl+8rMFpjZ8CTbjzKzKWY2w8z+Y2ZnphqniIiIyIEg1Tln15rZv4FRwAdAB3e/CugKDCxinzTgUeAMoC0w2MzaFqp2C/Ciu3cGBgF/TiVOERERkQNF9RT3Pww4z90XJxa6+04zO6uIfY4HFrj7QgAzmwD0B+YmHgKoGy3XA5akGKeIiIjIAcHCqGQpdzY7AZjj7huj9UOBtu7+STH7nA/0c/fLo/WLgG7ufnVCnSbAW4Qh0oOBU93930UcbxgwDKBx48ZdJ0yYUOrvU9Xk5ORwyCGHVHQYkkBtEk9ql/hRm8SP2mTf9erV69/unlm4PNWes8eALgnrm5KUFZbsjs7CGeJg4Gl3f8DMTgSeNbP27r5zjx3dxwBjADIzM71nz577EH7VlpWVha5XvKhN4kntEj9qk/hRm5SdVG8IME/oeouSp70lfNlAs4T1dPYctrwMeDE65kdALcIQqoiIiEillmpytjC6KaBG9LkOWLiXfaYDrc2spZnVJEz4f7VQnW+BPgBmdiwhOVuZYqwiIiIisZdqcvYL4CTge0KPWDei+V9FcfcdwNXAm8CXhLsy55jZSDM7J6r2a+AKM5sFjAcu8VQmx4mIiIgcIFKac+buKwg9X/u63+vA64XKbktYngv8OJXYRERERA5EKSVnZlaLMD+sHWHoEQB3/3mKcYmIiIhUSakOaz5LeL/m6cC/CJP7N6YalIiIiEhVlWpy1srdbwU2uftY4CdAh9TDEhEREamaUk3Otkc/15lZe8LT/FukeEwRERGRKivVh9COMbMGhHdhvgocAtyaclQiIiIiVVSpkzMzqwZscPe1wFTg6DKLSkRERKSKKvWwZvQ2gKv3WlFERERESizVOWdvm9mNZtbMzBrmf8okMhEREZEqKNU5Z/nPM/t/CWWOhjhFRERESiXVNwS0LKtARERERCT1NwRcnKzc3Z9J5bgiIiIiVVWqw5rHJSzXAvoAnwNKzkRERERKIdVhzWsS182sHuGVTiIiIiJSCqnerVnYZqB1GR9TREREpMpIdc7ZPwh3Z0JI9NoCL6YalIiIiEhVleqcs/sTlncAi909O8VjioiIiFRZqSZn3wJL3T0XwMxqm1kLd1+UcmQiIiIiVVCqc85eAnYmrOdFZSIiIiJSCqkmZ9XdfVv+SrRcM8VjioiIiFRZqSZnK83snPwVM+sPrErxmCIiIiJVVqpzzn4BjDOzP0Xr2UDStwaIiIiIyN6l+hDab4ATzOwQwNx9Y9mEJSIiIlI1pTSsaWb3mll9d89x941m1sDM7i6r4ERERESqmlTnnJ3h7uvyV9x9LXBmiscUERERqbJSTc7SzOyg/BUzqw0cVEx9ERERESlGqjcEPAe8a2Z/i9YvBcameEwRERGRKivVGwJGmdl/gFMBA94AmpdFYCIiIiJVUarDmgDLCG8JGAj0Ab7c2w5m1s/MvjKzBWY2PMn2h8xsZvSZb2brkh1HREREpLIpVc+ZmR0DDAIGA6uBFwiP0uhVgn3TgEeB0wjPRZtuZq+6+9z8Ou5+Q0L9a4DOpYlTRERE5EBT2p6zeYResrPdvbu7/5HwXs2SOB5Y4O4Lo9c9TQD6F1N/MDC+lHGKiIiIHFBKO+dsIKHnbIqZvUFIsKyE+x4JfJewng10S1bRzJoDLYH3ijqYmQ0DhgE0btyYrKysEoYhOTk5ul4xozaJJ7VL/KhN4kdtUnZKlZy5+0RgopkdDJwL3AA0NrPHgInu/lYxuydL4ryIuoOAl929yF45dx8DjAHIzMz0nj17luAbCEBWVha6XvGiNokntUv8qE3iR21SdlK6IcDdN7n7OHc/C0gHZgJ7TPAvJBtolrCeDiwpou4gNKQpIiIiVUhZ3K0JgLuvcfe/uHvvvVSdDrQ2s5ZmVpOQgL1auJKZ/QhoAHxUVjGKiIiIxF2ZJWcl5e47gKuBNwmP3XjR3eeY2UgzOyeh6mBggrsXNeQpIiIiUumk+oaAUnH314HXC5XdVmj9jvKMSURERCQOyr3nTERERESKpuRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRipkOTMzPqZ2VdmtsDMhhdR5wIzm2tmc8zs+fKOUURERKQiVC/vE5pZGvAocBqQDUw3s1fdfW5CndbAzcCP3X2tmR1e3nGKiIiIVISK6Dk7Hljg7gvdfRswAehfqM4VwKPuvhbA3VeUc4wiIiIiFaLce86AI4HvEtazgW6F6hwDYGYfAGnAHe7+RrKDmdkwYBhA48aNycrKKut4K62cnBxdr5hRm8ST2iV+1CbxozYpOxWRnFmSMi+0Xh1oDfQE0oH3zay9u6/bY0f3McAYgMzMTO/Zs2eZBluZZWVloesVL2qTeFK7xI/aJH7UJmWnIoY1s4FmCevpwJIkdf7u7tvd/b/AV4RkTURERKRSq4jkbDrQ2sxamllNYBDwaqE6k4BeAGZ2GGGYc2G5RikiIiJSAco9OXP3HcDVwJvAl8CL7j7HzEaa2TlRtTeB1WY2F5gC3OTuq8s7VhEREZHyVhFzznD314HXC5XdlrDswK+ij4iIiEiVoTcEiIiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYqRCbggQEYm7m266iQ0bNhSsjx8/HoC6devyhz/8oaLCEpEqQD1nIiJJJCZmJSkXESkrSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMRCSJunXr7lO5iEhZ0aM0RESSSHxcRlZWFj179qy4YESkSlHPmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI3rOmYhICZhBy5awcGFYz82FOnXgBz+A5cvh1Vdh7lwYPrx0x//vf2HQIFizBpo0gWXLYOdOuPzyPY/57bfQti3ccQe0aAH/8z/w3nvw6KMwezZs3QoHHwzusG0bNGwImzdDzZphm3v4LtnZ4Xjffw+rVsExx4RtK1bAunWlvVIikiolZyIiJbRsWUieGjaEUaPgoIN2bTvnnPAprd/8Bm64ISRaDRvCjTeGsuOOC8dt23ZX3RtugDPOCAni6NHQrRvcfz8MGAATJkDnzvDnP8PJJ4fl++6DjIyQfA0dCg8+GH7ecAP06QNffAH9+8NXX8Ef/wgzZpT+e1Rm27dvJzs7m9zc3IoOJZbq1avHl19+WdFhxFKtWrVIT0+nRo0aJaqv5ExEpIQ6dYJ77oEHHoCxY6FnT/j887DtttvgiSfgiCNCT9Tpp8M334SEJyMj9Gx98QUMHgyffhp63fK5h+3PPx+2tWkD06bBrbeG3rS//31XcjZpEhx9dOgZe/PNkMD9/vchofrnP2Hy5HC+k08O9b/5Bnr1gs8+g2OPhTfeCMnZqafC//1fSM7Gjw9xQVi+885yu6QHlOzsbA499FBatGiBmVV0OLGzceNGDj300IoOI3bcndWrV5OdnU3Lli1LtI/mnImIlNCvfgUvvBCG/JYsCQlYviZNQs/VjBkhiZsyJSRY//oXfPQRTJwIl14ahhXr1AnJ0uWXh31Xr4b69aF69ZDYHX10+AmQnr5redOmkIjdfjssXRriOOss2LIFGjQIx7/iCvj445B8dekSyl99FVq1CkOeixfDjh0hyfvuu3DcF14IydnixWF4tXfv8rumB5Lc3FwaNWqkxEz2iZnRqFGjfepxVXImIlJC558Pa9eGJK1z5923bd4ceqXat4fp08N8sWrVwnq1anDRRdCjB3z9daifmQl//WtYdt91nPzlxP//5y/ffnsYiqxTJ/SanX32rn3mzYOrrgrbV64MvWfTpoXkbOTIkKydeWZIDk8+OcxVq14dPvkkHK99+zAkev75kJZW5peu0lBiJqWxr783GtYUESmho48O87uefhrGjYOf/nRX4vTXv4aEbPbskOBMnbprv7y80DO2ZEny4x52WOgJa9UqDEM2bAgbN4aEacgQaNo01PvkkzB0OmRI6P26774w923nzpCgXXRR6F3buDHMRXv33RDPwQeHGwD+8Y8QS05OGJbduhXefz8kj926wX/+E3rqsrLCkK2IVAz1nImIlNCyZfDb38K554YesOoJf97WqRPmdAEsWLCrfP36cMfk1Klh+PLll/c8rhmceCJcdx00bx564X7+85BwTZiw60aD4cPh+OPDMGa3blC3brhBITMTDjkkJF+zZ4e7Mnv3DslctWrw8MNhblzHjmE+29SpIaGrWTN8rroqDGc2bBh6237967Bd4mnixImYGfPmzavoUEply5Yt9OjRg1mzZtGpUyc6depEw4YNadmyJZ06deLUU0/dp+OdfvrpbNy4sdg6I0aMYMqUKaWKNzc3lx49epCXl1eq/UtDyZmISAl16hQm3L/ySuh5atVq17batcNQ4yGHhARpwwY46SQ4/PCw/Zhj4Kab4IILYNGi3eecAYwZA88+G+aXdewYhki//ho6dAhzwgAeeggaNw5J4Zo1oUftjTdCYlinTjh29+6hV23hQvjRj0L5ffeFpGz27DBP7aSTYPv2cOyjjgqP0WjUKMTYuHHo5fvss3K7rJXWuHHjaNGiBdWqVaNFixaMGzeuTI47fvx4unfvzoQJE8rkeEXZX8nIU089xXnnnUdGRgYzZ85k5syZnHPOOfzhD39g5syZvPPOO7vV37FjR7HHe/PNN/d6I8I999xDr169ShVvrVq1OOWUU3g52V9W+0mFJGdm1s/MvjKzBWa2x1OBzOwSM1tpZjOjz+XJjiMiUp4K3xBw5ZXhOWcQhhp/8YswZNi7dxiqnDYN/v3v8MiNiRPh5pvhhz8M870KO/rocKfmkUfCa6+FocVjjoHnngtzxgD+939DD9fmzWH+2JIlYVL/qFGhV+6pp8Ldm/k9e717w913w/z5MHNmmEs2Zw785S+hh27JknCuY48NzzZr2zYc/9//3nWzgJTOuHHjGDZsGIsXL8bdWbx4McOGDUs5QcvJyeGDDz7gySef3CM5GzVqFB06dCAjI4Ph0cPxFixYwKmnnkpGRgZdunThm2++ISsri7POOqtgv6uvvpqnn34agBYtWjBy5Ei6d+/OSy+9xBNPPMFxxx1HRkYGAwcOZPPmzQAsX76cAQMGkJGRQUZGBh9++CF33XUXjzzySMFxR4wYwejRo5Nem/79+xf7Pd955x1OPfVUBg0aROdogufZZ59N165dadeuHX/Nn7AJpKens27dOhYsWED79u257LLLaNeuHWeccUbBJPyf/exnTJo0qaD+HXfcQefOnenYsSPz588HYMWKFfTp04cuXbrwy1/+kiOPPJJ10QP/zj333DJLrkui3OecmVka8ChwGpANTDezV919bqGqL7j71eUdn4hIUc4/PzwfbG83BHz/fehJK3xDwJVXhsdwwO43BJRU377hZoOTTgoT/Q87LDwWY+bMEE/16uFRGJdfHoZSX3019JpBGPL88Y/D0OX48eHOzvr14bTTwpy0I4+EF18MD6Y96aTdh2xlT9dffz0zZ84scvvHH3/M1q1bdyvbvHkzl112GU888UTSfTp16sTDDz9c7HknTZpEv379OOaYY2jYsCGff/45Xbp0YfLkyUyaNIlPPvmEOnXqsGbNGgCGDBnC8OHDGTBgALm5uezcuZPv9pJ516pVi2nTpgGwevVqrrjiCgBuueUWnnzySa655hquvfZaevTowcSJE8nLyyMnJ4e6dety8cUXc91117Fz504mTJjAp59+utuxt23bxsKFC2mR7C+UQj7++GPmzp3LUUcdBcDYsWNp2LAhmzdvJjMzk00kV7sAACAASURBVIEDB9KgQYPd9vnqq68YP348HTp04LzzzmPSpEkMGjRoj2M3btyYGTNmMHr0aB588EEef/xxbrvtNvr168dNN93EP//5Tx577LGC+hkZGXz88cd7jbmsVETP2fHAAndf6O7bgAlA8Sm0iEhM5N8Q8Otf714+cWJIcGbPDs8OSxyJycsLw51F3RCwL0aMCMnYlCkhlieegGeeCXdo1qsXet9+8pMw/NqlSximhDB3bfDgENcrr8CFF8Jll4W5aO+/Hx7Jceed4Zlq69ZB69apx1qVFU7M9lZeUuPHjy9INgYNGsT48eOB0NN06aWXUid6gF7Dhg3ZuHEj33//PQMGDABC0lUn8QF7RbjwwgsLlmfPns3JJ59Mhw4dGDduHHPmzAHgvffe46qrrgIgLS2NevXq0bx5cxo1asSMGTN466236Ny5M40aNdrt2KtWraJ+/fol+q4nnnhiQWIG8NBDD5GRkcGJJ55IdnY233zzzR77tGrVig4dOgDQtWtXFi1alPTY55133h51pk2bVnBtzzrrrN2GSqtXr46ZsWXLlhLFnqqK+NvoSCAxbc8GuiWpN9DMTgHmAze4e9JU38yGAcMgZMJZWVllG20llpOTo+sVM2qTeMrJyQGcrKx/ccEFh7J9+1E0ajSHKVPS2bbtKLKyPmTFiq7Ur7+NrKwvmDkzk23bapKV9SE5OWls29adP/zhU0aPbs2ddy6lR4+VRZ4rN/cEPvjg32zZksamTR3IyppesC3caVmdevV28MUXdfnwwx9x2WWfcdNNR3DMMfUYPXoDmZl1+fjjeTzySFuOO24NWVnLyMlJ4913T2DYsI948MH6HHHEUSxYMJPp02vQoMF2vv22Ns8804HHHvuc++8/lE2bmrNixUxWrCiHi1tKFfFvpV69egUTz++6665i67Zr1y5pD1WzZs34xz/+UeR+xU1sX716Ne+99x5ffPEFZkZeXh5mxq233srWrVvZunXrbvtv2LABd9/jmNu2bWPbtm0F5Rs3biQ3N5eNGzfi7rvtM3ToUJ5//vmC5Oz9998vqLdx40a2bdtWcNy8vDyGDBnCmDFjWLFiBYMHD97j3Dt27GDLli17lG/fvn238s2bN3PQQQcVrE+ZMoUpU6bw9ttvU7t2bfr27cuaNWt2iyUnJ4caNWoU7LNjx46C75V4fHcv+P65ubkF1y2/BzAxto0bN5IWPVsm/7rtbQ5cUXJzc0v+O5vfEOX1Af4H+GvC+kXAHwvVaQQcFC3/AnivJMfu2rWrS8lNmTKlokOQQtQm8TRlyhQH91q1dn1OP939oYfcDz881Bkxwr1ePfeTTnLv2NH9Bz8I5Zde6l6zZlj+9tuwvHy5+/Tp7pddFsrff989Lc39yCPDz1q13GvXdq9ePRy/e/dQb/Ro9yOOcD/2WPe2bUM9cDcL5+/Rw33yZPdNm9wbNnRfty7s97e/uV94YVgeOtT9scfCcvfu4Vht2rinp4efffq4L1q0ny9oGaiIfytz584tcd3nnnvO69Sp40DBp06dOv7cc8+V+vyPP/64Dxs2bLeyU045xadOneqTJ0/2E0880Tdt2uTu7qtXr3Z3927duvnEiRPd3T03N9c3bdrk3377rTdv3txzc3N93bp13qJFC//b3/7m7u7Nmzf3lStXFhy/UaNGvnz5ct+2bZufeuqpPnToUHd3v/DCC/2hhx5yd/cdO3b4+vXrfcOGDb5161Y/5phjvGXLlr5jx46k3yM9Pd23bNmyW9nQoUP9pZdeKlh/++23vX///gXrL7/8sp977rnu7j579myvWbOmv//+++7ufuSRR/ratWv966+/9oyMjIJ97rvvPr/rrrvc3X3IkCEF1yG/vrv7Rx995H369HF392HDhvn999/v7u6vvfaaAwX1li1b5u3atUv6fUoq2e8P8JknyWcqoucsG2iWsJ4O7NbZ7+6rE1afAH5fDnGJiBQr2YjG9deHn3ffHT6FPfVU+AA0axaeLQbhLs7EOWfVq+96EXlRrrkmfCDMJ7vwwvCojpkzw1Dqpk3h0RgQbhDId8kl4QNhSDbf++8Xfz4pvSFDhgBhUvy3337LUUcdxT333FNQXhrjx48vmOifb+DAgTz//PM89thjzJw5k8zMTGrWrMmZZ57Jvffey7PPPsuVV17JbbfdRo0aNXjppZc4+uijueCCC+jYsSOtW7cumHCfzF133UW3bt1o3rw5HTp0KOhVeuSRRxg2bBhPPvkkaWlpPPbYY7Rv356aNWvSq1cv6tevX9DjVFjfvn2ZNm3aPj0y4yc/+QljxowhIyODNm3a0K1bsgG31Nx555389Kc/Zdy4cfTu3ZvGjRtz8MEHA6Hn7ic/+UmZn7NIyTK2/fkhDKUuBFoCNYFZQLtCdZokLA8APi7JsdVztm/USxM/apN4yu85SyYtzf34490PPjh83nknlE+b5t6gQejdatDA/cMPQ/kXX7g3abKrB+7xx0PPWc2a7j/6kftBB4Ver6jjo0h9+4ZPvkaN3P/yl9S/64Ei7j1nVdGGDRs8Ly/PMzIyfP78+UXW+/zzz/1nP/tZOUZWMlu2bPHt27e7u/v777/viTnFOeec419//XVKx9+XnrNyvyHA3XcAVwNvAl8CL7r7HDMbaWbRoxa51szmmNks4FrgkvKOU0SksNq1d32uu25Xed264REaP/1p+ACcdx4MHBh62/r3D+sQHoHRrVsoX78ezjgjlG/bFibk5+aGZ5ONGBHKE4+ZaOnS8PiNfA0bhpesi1SUefPm0apVK/r06UPrYu4o6dy5M7169SrXh7qWxKJFizjuuOPo2LEjN9xwA3/5y1+AcBPH+eefT6vEBxvuZxVys7S7vw68XqjstoTlm4GbyzsuEZHiFHWjVn4i9fDDu4YqV66E/Ec+PfpouFsTwtDl7NlhuWbN8BDYb7+FGjXCMCWEx2/kv4Pz+eeTnzPxfZz5qumx4lKB2rRpw8KFC0tU9+c///l+jmbftWnThhkzZuxRftBBB3HRRReVayz6pywiUkKJPWf9+u0qTzUpmjUrPLE/X1ra7o/iSKZp0/Bcsho1QpKXna3HX4hUFkrORERKaMuWXZ833thVfu+94eevf73rjQGHHx4eVgtw7bVwxBFhOT09PMgWwlDm3m4CKEpmZniF03//Gx5su2VL8uFPETnwKDkTESlC/rsRe/fuDTg1a+4o6Dk74YRQJy8PPvwwvLz88cfhxhtD+e23h/dlmoW7Ne+4I5Q//HC409IsvNYp8ea7Nm2gVi14662QuEHRc84++yzMM2vZMiSFtWsXPQQqIgcWJWciIkkUfjcihAdl5uWF8cbEO/k7dAhzwK64Au6/P5TdcUd4Ar976Cm7/fZQfv31cM45oXzr1tDrlpERtuXfENC48a6yoixdGl50vn17SOTS03VDgEhloeRMRCSJESNGFLzkeZc6NG3aii1bdk32h/DCcwi9Yiujh/8XviFg+fKwnJ0NY8eG5fwbAqD4GwKS9YjphoCqa+LEiZgZ8+bNq+hQSmXLli306NGDvLw8WrZsyVeF/qq4/vrrGTVqVJH7L1q0iPbt2wPw2Wefce211yat16JFC1atWlVsLPfmz0mInHTSSSX5CkkNGjSIr/P/4aZIr7YVEUni22+/LVF5Who0b576+RITq7S0ou8Mzde0KSTeGLdmjW4IiJObbrqJDRs27FFet25d/vCHP6R07PHjx9O9e3cmTJjAHfnj5ftBXl5ekQ+STcVTTz3FeeedR1paGoMGDWLChAncHnUt79y5k5dffpkPPvigRMfKzMwkMzOz1LHce++9/Pa3vy1Y//DDD0t9rKuuuopRo0YV+WL7faG/s0REkkh84fLeyivihoBf/hKmToUNG8LP9et3vQVAKl6yxKy48pLKycnhgw8+4Mknn2TChAm7bRs1ahQdOnQgIyOj4E0CCxYs4NRTTyUjI4MuXbrwzTffkJWVxVlnnVWw39VXX83T0asjWrRowciRI+nevTsvvfQSTzzxBMcddxwZGRkMHDiwoDd5+fLlDBgwgIyMDDIyMvjwww+56667eCShS3nEiBGMHj16j+8wbtw4+vfvD8DgwYN3+x5Tp06lRYsWNG/enEWLFnHyySfTpUsXunTpkjRxSvwuq1evpm/fvnTu3Jkrr7yyYDoCwLnnnkvXrl1p164dY8aMAWD48OFs2bKFTp06Fby54ZDomTfuzk033UT79u3p0KEDL7zwQsH5evbsyfnnn0+bNm0YMmRIwXlOPvlk3nnnnVK/ezORes5ERJK45557GDZsWKGhzc0sWVKD2rXDnLCPPw6lW7eG55i5h8n+AC+/DGefHYYwa9eG114L5a+/DqedFib+m4Whz7Zti44j/2aAwkOb/ftD9+7QqFE4zq237np1k5SPBx54YI+yrl270rNnz73um5OTU/CQ03y//vWv97rfpEmT6NevH8cccwwNGzbk888/p0uXLkyePJlJkybxySefUKdOHdasWQOE10gNHz6cAQMGkJuby86dO5O+kD1RrVq1mDZtGhASniuuuAKAW265hSeffJJrrrmGa6+9lh49ejBx4sSCF4bXrVuXiy++mOuuu46dO3cyYcIEPv30092OvW3bNhYuXEiLFi0A6NixI9WqVWPWrFlkZGQwYcIEBg8eDMDhhx/O22+/Ta1atfj6668ZPHgwn332WZFx33nnnXTv3p3bbruN1157rSAJg9Bb17BhQ7Zs2cJxxx3HwIED+d3vfsef/vQnZs6cucexXnnlFWbOnMmsWbNYtWoVxx13HKeccgoAM2bMYM6cOTRt2pQf//jHfPDBB3Tv3p1q1arRqlUrZs2aRdeuXYu9xnuj5ExEJIk9343Yssh3Iz7+OPzoR7uXde8Oa9fuedz27cNk/sJyc3ct//Ofu5aLuwPz7beL+wZSGY0fP57roxe6Dho0iPHjx9OlSxfeeecdLr30UurUqQNAw4YN2bhxI99//z0DBgwAQtJVEhfmT34EZs+ezS233MK6devIycnh9NNPB+C9997jmWeeASAtLY169epRrVo1GjVqxIwZM1i+fDmdO3emUaNGux171apV1K9ff7ey/N6zdu3a8fe//52RI0cC4Qacq6++mpkzZ5KWlsb8+fOLjXvq1Km88sorQHgXZ4MGDQq2jR49mokTJwLw3Xff8fXXX+8RW6Jp06YxePBg0tLSaNy4MT169GD69OnUrVuX448/nvT0dAA6derEokWL6N69OxASyiVLlig5ExHZX4YMGcKQIUMKhjJEEpWkp6sohxxyyD7vv3r1at577z1mz56NmZGXl4eZMWrUKNwdM9utvie7awSoXr06O3fuLFjPTfzLAApe9g1wySWXMGnSJDIyMnj66afJysoqNsbLL7+cp59+mmXLliV9C0Dt2rX3ON/gwYPp27cvPXr0oGPHjhx++OEAPPTQQzRu3JhZs2axc+fOEiWXha8BhKHId955h48++og6derQs2fPPWIorKhrB+GNAfnS0tJ2G8bMzc2ldu3ae41zbzTnTEQkBTt27NlrJrI/vPzyy1x88cUsXryYRYsW8d1339GyZUumTZtG3759eeqppwqG4desWUPdunVJT09n0qRJQHhH5ObNm2nevDlz585l69atrF+/nnfffbfIc27cuJEmTZqwfft2xo0bV1Dep08fHnvsMSDcOJA/l27AgAG88cYbTJ8+vaCXLVGDBg3Iy8vbLTn64Q9/SKNGjRg+fHjBkCbA+vXradKkCdWqVePZZ5/d67s4TznllIIYJ0+ezNqo63r9+vU0aNCAOnXqMG/ePD7On48A1KhRg+2Jr+dIONYLL7xAXl4eK1euZOrUqRx//PHFnh9g/vz5tGvXbq/19kbJmYiISBmrW7fuPpWXxPjx4wuGKPMNHDiQ559/nn79+nHOOeeQmZlJp06duD964N6zzz7L6NGj6dixIyeddBLLli2jWbNmXHDBBXTs2JEhQ4bQuXPnIs9511130a1bN0477TTatGlTUP7II48wZcoUOnToQNeuXZkzZw4ANWvWpFevXlxwwQVF3unZt2/fgjlt+QYPHsy8efN2+36//OUvGTt2LCeccALz58/frUcvmdtvv52pU6fSpUsX3nrrrYKbd/r168eOHTvo2LEjt956KyfkP0EaGDZsWMF1SDRgwAA6duxIRkYGvXv3ZtSoURyRf1dPEZYvX07t2rVp0qRJsfVKworrujvQZGZmenGTBWV3GqqJH7VJPKld4qci2uTLL7/k2GOPLddzHkg2btzIwQcfTJcuXXjppZdoXcSzXWbMmMGDDz7Is88+W84R7l8PPfQQdevW5bLLLku6Pdnvj5n92933eBaIes5EREQkZfPmzaNVq1b06dOnyMQMoHPnzvTq1Wuvw5QHmvr16zM0/zk5KdINASIiIpKyNm3asDDxycjFSHazwIHu0ksvLbNjqedMRESkhCrTVCApP/v6e6PkTEREpARq1arF6tWrlaDJPnF3Vq9eXeLnzIGGNUVEREokPT2d7OxsVua/3V52k5ubu08JSFVSq1atggfXloSSMxERkRKoUaMGLVu2rOgwYisrK6vYx3JIyWlYU0RERCRGlJyJiIiIxIiSMxEREZEYqVRvCDCzlcDiio7jAHIYsKqig5DdqE3iSe0SP2qT+FGb7Lvm7v6DwoWVKjmTfWNmnyV7bYRUHLVJPKld4kdtEj9qk7KjYU0RERGRGFFyJiIiIhIjSs6qtjEVHYDsQW0ST2qX+FGbxI/apIxozpmIiIhIjKjnTERERCRGlJyJiIiIxIiSs0rOzBqa2dtm9nX0s0ER9YZGdb42s6FJtr9qZrP3f8SVXyptYmZ1zOw1M5tnZnPM7HflG33lYmb9zOwrM1tgZsOTbD/IzF6Itn9iZi0Stt0clX9lZqeXZ9yVXWnbxcxOM7N/m9kX0c/e5R17ZZXKv5Vo+1FmlmNmN5ZXzAcyJWeV33DgXXdvDbwbre/GzBoCtwPdgOOB2xMTBjM7D8gpn3CrhFTb5H53bwN0Bn5sZmeUT9iVi5mlAY8CZwBtgcFm1rZQtcuAte7eCngI+H20b1tgENAO6Af8OTqepCiVdiE8APVsd+8ADAWeLZ+oK7cU2yTfQ8Dk/R1rZaHkrPLrD4yNlscC5yapczrwtruvcfe1wNuE/+FgZocAvwLuLodYq4pSt4m7b3b3KQDuvg34HEgvh5gro+OBBe6+MLqWEwhtkyixrV4G+piZReUT3H2ru/8XWBAdT1JX6nZx9xnuviQqnwPUMrODyiXqyi2VfyuY2bnAQkKbSAkoOav8Grv7UoDo5+FJ6hwJfJewnh2VAdwFPABs3p9BVjGptgkAZlYfOJvQ+yb7bq/XOLGOu+8A1gONSrivlE4q7ZJoIDDD3bfupzirklK3iZkdDPwGuLMc4qw0qld0AJI6M3sHOCLJphElPUSSMjezTkArd7+h8PwBKd7+apOE41cHxgOj3X3hvkco7OUa76VOSfaV0kmlXcJGs3aEYbW+ZRhXVZZKm9wJPOTuOVFHmpSAkrNKwN1PLWqbmS03sybuvtTMmgArklTLBnomrKcDWcCJQFczW0T4XTnczLLcvSdSrP3YJvnGAF+7+8NlEG5VlQ00S1hPB5YUUSc7SojrAWtKuK+UTirtgpmlAxOBi939m/0fbpWQSpt0A843s1FAfWCnmeW6+5/2f9gHLg1rVn6vEibGEv38e5I6bwJ9zaxBNOm8L/Cmuz/m7k3dvQXQHZivxKxMlLpNAMzsbsJ/+K4vh1grs+lAazNraWY1CRP8Xy1UJ7Gtzgfe8/Dk7leBQdEdai2B1sCn5RR3ZVfqdomG+l8Dbnb3D8ot4sqv1G3i7ie7e4vo/yMPA/cqMds7JWeV3++A08zsa+C0aB0zyzSzvwK4+xrC3LLp0WdkVCb7R6nbJOoVGEG4Y+pzM5tpZpdXxJc40EXzYq4mJL1fAi+6+xwzG2lm50TVniTMm1lAuDFmeLTvHOBFYC7wBvD/3D2vvL9DZZRKu0T7tQJujf5tzDSzZHM6ZR+k2CZSCnp9k4iIiEiMqOdMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIVAlmlpfweIWZZlZmt/qbWQszm11WxxORqk1vCBCRqmKLu3eq6CBERPZGPWciUqWZ2SIz+72ZfRp9WkXlzc3sXTP7T/TzqKi8sZlNNLNZ0eek6FBpZvaEmc0xs7fMrHaFfSkROaApORORqqJ2oWHNCxO2bXD344E/EV4xQ7T8jLt3BMYBo6Py0cC/3D0D6ALMicpbA4+6eztgHTBwP38fEamk9IYAEakSzCzH3Q9JUr4I6O3uC82sBrDM3RuZ2Sqgibtvj8qXuvthZrYSSHf3rQnHaAG87e6to/XfADXc/e79/81EpLJRz5mICHgRy0XVSWZrwnIemtMrIqWk5ExEBC5M+PlRtPwhMChaHgJMi5bfBa4CMLM0M6tbXkGKSNWgv+xEpKqobWYzE9bfcPf8x2kcZGafEP5gHRyVXQs8ZWY3ASuBS6Py64AxZnYZoYfsKmDpfo9eRKoMzTkTkSotmnOW6e6rKjoWERHQsKaIiIhIrKjnTERERCRG1HMmIiIiEiNKzkRERERiRMmZiOyz6EXfbmZ7vePbzC4xs2l7q5dkvyFm9tY+7vNbM/vrvp6rrJhZjpkdXVHnF5HKQcmZSCUXvTtym5kdVqh8ZpRgtaiYyHZL8nISPrMA3H2cu/dNqOv5772M1nuaWXbi8dz9Xne/fD/Futv5o7I7zOy5hPMf4u4L93KcPeKuaMm+m4hUHCVnIlXDf9n1/C7MrAMQpxdz148Sm0Oid1ZKEUrSWykiBzYlZyJVw7PAxQnrQ4FnEiuYWT0ze8bMVprZYjO7xcyqRdvSzOx+M1tlZguBnyTZ90kzW2pm35vZ3WaWlkrAicOhZjY1Kp4V9a4NBSYDTRN63Jom9mQl9MoNNbNvo9hHJBy/tpmNNbO1Zvalmf1vqj1aiT1QZnammc01s43RNbnRzA4uIu6DzOxhM1sSfR42s4Oi4/Q0s2wz+42ZLQP+ZmazzezshPPWiL5fp1TiT/J9qkW/B4vNbEX0+1Ev2lbLzJ4zs9Vmts7MpptZ42jbJWa2MPru/zWzIWUZl0hlp+RMpGr4GKhrZsdGSdOFwHOF6vwRqAccDfQgJHP5T8W/AjgL6AxkAucX2ncssANoFdXpC5TZ8KK7nxItZkS9a2OBM4AlCT1uS4rYvTvwI6APcJuZHRuV3w60IHzf04CflVW8kSeBK939UKA98J67byoi7hHACUAnIAM4Hrgl4VhHAA2B5sAwQmKdGO+ZhBezJ74BoSxcEn16Ea7TIcCfom1DCb8vzYBGwC+ALVECOho4I/ruJwFlHZdIpabkTKTqyO89Ow2YB3yfvyEhYbvZ3Te6+yLgAeCiqMoFwMPu/p27rwHuS9i3MSHhuN7dN7n7CuAhdr2XsiRWRb0v68zsxlJ/w+TudPct7j4LmEVIfiB8p3vdfa27ZxMSir35PCHOdcDwYupuB9qaWd3oHJ8XU3cIMNLdV7j7SuBOdl17gJ3A7e6+1d23EBLrM23Xez0vIrRvWRsCPOjuC909B7gZGBQNrW4nJGWt3D3P3f/t7hsS4m1vZrXdfam7z9kPsYlUWkrORKqOZ4GfEnpCnim07TCgJrA4oWwxcGS03BT4rtC2fM2BGsDShKTlL8Dh+xDbYe5eP/rcvw/7lcSyhOXNhN4f2PM7JS4XpUtCnPWB3xVTdyChR2uxmf3LzE4spm5T9rz2TRPWV7p7bv5K1Nv2ATDQzOoTkuNxyQ5sZpMThlD3dXgxWVzVgcaE36c3gQnRUOwoM6sR9Q5eSOhJW2pmr5lZm308r0iVpuRMpIpw98WEGwPOBF4ptHkVoSekeULZUezqXVtKGL5K3JbvO2AruydYdd29XVnGn0SqrzdZCqQnrDcrqmJpuPt0d+9PSFInAS/mb0pSfQl7XvvEYdpk+4wlDG3+D/CRu3+fpA7ufkbCEGrSBK4YyeLaASx39+3ufqe7tyUMXZ5FNK/R3d9099OAJoRe2if28bwiVZqSM5Gq5TKgd9S7UcDd8wjJwz1mdqiZNQd+xa55aS8C15pZupk1IGE4z92XAm8BD5hZ3WgS+Q/NrEcZx76cMO8pcb1R/gT1UngRuNnMGpjZkcDVqQaYz8xqWnhOWz133w5sAPKizcniHg/cYmY/sPDIk9vYc05gYZOALsB17NkTWho1o0n++Z+0KK4bzKylmR0C3Au84O47zKyXmXWI6m0gJPd5ZtbYzM6J5p5tBXISvruIlICSM5EqxN2/cffPith8DbAJWAhMA54Hnoq2PUEYwpoFfM6ePW8XE4ZF5wJrgZcJvSZl6Q5gbDR0eoG7zyMkDwujsqbF776HkUA2oTfxHULMW8sw3ouARWa2gTDE9zOAIuK+G/gM+A/wBeEa313cwaO5Z/8HtGTP9iiNOfz/9u4+zqr6vPf+5xJQQB60QdFIYDR6FCQIOtUqJBnlbtTYqKFWgyRU1NBYE7QmJjbmxErk3J5j0qqNp4Yk6smRW4+PDY0ay1EnRGM0IA8KJIUgKEJ8wCiMgDpy3X/sLR1khhmY2bMXw+f9es2L2eu31trX7IuRr7/1BBubfE2i1P//Dcym9DltovT3BEoXKdxDKZgtAX5BKVDuAXyV0qzb65QuLvnbDqhP2m344HNJAiLiIuBzmdnRM34VExHfBv5LZnb0laaSqsiZM0m7pYg4MCJGlw/DHk5ptuf+atfVVhHxJ5QOU0+vdi2SOpbhTNLuak9KV5WuBx4Ffgr8z6pW1EYR8UVKF2I8lJmzW1tf0q7Fw5qSJEkF4syZJElSgXSpB+gOGDAga2pqql3GLuOtt95i7733rnYZasKe5fr3tAAAIABJREFUFJN9KR57Ujz2ZMfNnTv3tczc74PLu1Q4q6mpYc6clu4SoA+qr6+nrq6u2mWoCXtSTPaleOxJ8diTHRcRK5tb7mFNSZKkAjGcSZIkFUjFwllE3BIRr0TEcy2M7xsR90fEwoh4OiKGNxn7u4hYFBHPRcQdEdGzUnVKkiQVSSXPObsN+D4tP/Ptm8D8zPxsRBwB3ASMLT/jbgowLDM3RsRdwOfK+5MkqUt69913WbVqFZs2bap2KTulf//+LFmypNplFFLPnj0ZNGgQPXr0aNP6FQtnmTk7Imq2s8ow4P8tr/vbiKiJiIFN6uoVEe8CvSk9o02SpC5r1apV9O3bl5qaGiKi2uXssPXr19O3b99ql1E4mcnatWtZtWoVBx98cJu2qebVmguAccDjEXEsMAQYlJlzI+K7wAuUHr7775n57y3tJCImA5MBBg4cSH19fcUL7yoaGhr8vArGnhSTfSmertiT/v3786EPfYiGhoZql7JT3nvvPdavX1/tMgppzz335I033mjz39lqhrNrgRsiYj7wLDAPaIyIfYEzgIOBN4C7I+LzmXl7czvJzOmUny1XW1ubXsbbdl72XDz2pFhmzJjBlVdeyQsvvMDgwYOZNm0aEyZMqHZZomv+rixZsoR+/fpVu4yd5szZ9vXs2ZNRo0a1ad2qhbPMXAdMAojS/O3z5a+Tgecz89Xy2H3ACUCz4UySKmHGjBlMnjyZDRs2ALBy5UomT54MYECTVFFVu5VGROwTEXuWX14IzC4HtheAP4uI3uXQNhbwDENJnerKK6/cEszet2HDBq688soqVSRtbcaMGdTU1LDHHntQU1PDjBkz2r3PPn36dEBl25eZnHTSSaxcuZKRI0cycuRIDjjgAA466KAtr995550272/SpEn87ne/2+46N910U7s+n7Fjx/Lmm2/u9PY7qmIzZxFxB1AHDIiIVcBVQA+AzLwZGAr8JCLeAxYDF5THnoqIe4BngEZKhzunV6pOSWrOCy+8sEPLpc60K8/sPvjggxx11FEMGTKE+fPnA/AP//AP9OnTh6997WvbrJ+ZZCZ77NH8fNKtt97a6ntefPHF7ar53HPP5eabb+Yb3/hGu/bTVpW8WnN8K+NPAoe1MHYVpTAnSVUxePBgVq7c9skqgwcPrkI12t1ceumlW4JLc37961/z9ttvb7Vsw4YNXHDBBfzwhz9sdpuRI0dy/fXX73AtK1eu5Pzzz+fVV19lv/3249Zbb2Xw4MHcfffdXH311XTr1o3+/fvzwAMPsGjRIiZNmsQ777zD5s2buffeeznssK3/qX8/WG7PsmXLOPPMMxkzZgxPPfUUP/vZz7j66qt55pln2LhxI+eccw7f/va3ARgzZgzf//73GT58OAMGDOBLX/oSDz30EL179+anP/0p+++/P9/61rcYMGAAl156KWPGjGHMmDE8+uijvPnmm9x6662ccMIJvPXWW0ycOJFly5YxbNgwli5dyo9+9CNGjhzJGWecwdixYzstnPmEAElqxrRp0+jdu/dWy3r37s20adOqVJH0nz4YzFpb3h5f/vKXmThxIgsXLmTChAlMmTIFgKlTp/Lwww+zYMECZs6cCcDNN9/MJZdcwvz585kzZw6DBg3aZn9PPPEExxxzTKvvu3jxYi644ALmzZvHQQcdxLXXXsucOXNYsGABs2bNYvHixdts8+abb/LJT36SBQsWcPzxx3PLLbc0u+/M5Omnn+a6665j6tSpAPzzP/8zBxxwAAsWLOCKK65g3rx5W9YfMGAA69ev54033mj9A+sAXerB55LUUd4/NOTVmqqG1ma4ampqmp3ZHTJkSIffYuTJJ5/kvvvuA+ALX/gCX//61wEYPXo05513HmeffTbjxo2jW7duHH/88UybNo1Vq1Yxbty4bWbNAF5//fU2XdX50Y9+lD/90z/d8vqOO+7gxz/+MY2NjaxevZrFixczbNiwrbbp1asXp556KgDHHHMMv/zlL5vd97hx47ass2LFCgAef/zxLTNjRx11FEceeeRW2+y3336sWbOGffbZp9Xa28uZM0lqwYQJE1ixYgWPPvooK1asMJipMKo5s/v+DXJvvvlmrrnmGl588UVGjhzJ2rVrOffcc5k5cya9evXi5JNP5tFHH91m++7du7N58+ZW32fvvffe8v3SpUu54YYbePTRR1m4cCGnnHJKs09S2HPPPbd8361bNxobG5vd91577bXNOpm53Xo2bdpEr169Wq27IxjOJEnaxUyYMIHp06czZMgQIoIhQ4Ywffr0ivwPxAknnMCdd94JlM4XGzNmDAC///3vOe6445g6dSoDBgzgpZdeYvny5RxyyCFMmTKF008/nYULF26zv8MPP5zly5fvUA3r1q2jb9++9OvXjzVr1vDwww+3/wf7gDFjxnDXXXcB8Oyzz2512HTz5s289tprfOQjH+nw922OhzUlSdoFTZgwocPD2IYNG7Y6T+yyyy7jxhtv5Pzzz+e6667bckEAwOWXX87SpUvJTMaOHcvHPvYxbrrpJm6//XZ69OjBAQccsOWk/aZOO+006uvrOfTQQ9tc19FHH82wYcMYPnw4hxxyCKNHj27/D/sBX/nKV5g4cSIjRozg6KOPZvjw4fTv3x+Ap59+mjFjxtCtW7cOf9/mRGvTeLuS2tranDNnTrXL2GV0xTts7+rsSTHZl+Lpij1ZsmQJQ4cOrXYZO62tTwhYs2YNEydOZNasWZ1QVds1NjbS2NhIz549Wbp0KZ/61KdYunQp3bt35+KLL+bss8/mk5/85E7vv7n+RsTczKz94LrOnEmSpE5z4IEH8sUvfpF169YV6nFVDQ0NjB07lsbGRjKTH/zgB3TvXopJo0aNalcw21GGM0mS1KnOPvvsapewjX322Ye5c+c2O3bhhRd2ai1eECBJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiTtYi6//HL+5m/+Zpuvyy+/vF377dOnTwdV2LLM5KSTTmLdunXU1dVtc0PZ66+/nr/927/d7j7er3P16tWcddZZza5TV1dHa7fXuv7669mwYcOW15/+9Kd3+vmZX/va15p9IsLOMJxJkrSLWbdu3Q4tL5IHH3yQo446in79+jF+/PgtTx9435133sn48ePbtK8Pf/jD3HPPPTtdywfD2YMPPrjTz878yle+wrXXXrvTtTTlrTQkSSqg733ve9ssO+aYY9p0892GhgZ+8IMfbLXsq1/96k7VsXLlSs4//3xeffXVLU8IGDx4MHfffTdXX3013bp1o3///jzwwAMsWrSISZMm8c4777B582buvffebR5+PmPGDCZPngzAWWedxbe+9S3efvtt9tprL1asWMHq1asZM2YMDQ0NnHHGGfzxj3/k3Xff5ZprruGMM87Yal8rVqzgL/7iL3juuefYuHEjkyZNYvHixQwdOpSNGzduWe+iiy7iN7/5DRs3buSss87i6quv5sYbb2T16tWceOKJDBgwgMcee4yamhrmzJnDgAED+Md//EduueUWoHQrjUsvvZQVK1Zw6qmnMmbMGH71q19x0EEH8dOf/pRevXoxZMgQ1q5dyx/+8AcOOOCAnfqs3+fMmSRJatGXv/xlJk6cyMKFC5kwYQJTpkwBYOrUqTz88MMsWLCAmTNnAqWHoV9yySXMnz+fOXPmbPUoqPc98cQTHHPMMQB86EMf4thjj+XnP/85UJo1O+ecc4gIevbsyf33388zzzzDY489xle/+tXtPpz8X/7lX+jduzcLFy7kyiuv3OqeZdOmTWPOnDksXLiQX/ziFyxcuJApU6bw4Q9/mMcee4zHHntsq33NnTuXW2+9laeeeopf//rX/PCHP2TevHlA6SHsF198MYsWLWKfffbh3nvv3bLd0UcfzRNPPLEzH/NWnDmTJKmAdnamC0rnZLVn+6aefPJJ7rvvPgC+8IUv8PWvfx2A0aNHc95553H22Wczbtw4unXrxvHHH8+0adNYtWoV48aN22bWDOD111/f6jFP7x/aPOOMM7jzzju3zFZlJt/85jeZPXs2e+yxBy+99BIvv/xyi7NSs2fP3hIcR4wYwYgRI7aM3XXXXUyfPp3GxkbWrFnD4sWLtxr/oMcff5zPfvaz7L333gCMGzeOX/7yl5x++ukcfPDBjBw5EijNZK5YsWLLdvvvvz+rV69u9TNtjTNnkiSpzSICKM2SXXPNNbz44ouMHDmStWvXcu655zJz5kx69erFySef3OwJ8t27d2fz5s1bXp955pk88sgjPPPMM2zcuJGjjz4aKB3+fPXVV5k7dy7z589n4MCBbNq0qU21NfX888/z3e9+l0ceeYSFCxdy2mmntbqf7c3Q7bXXXlu+79atG42NjVteb9q0iV69em13321hOJMkaRfT0jMpK/GsyhNOOGHLSfszZsxgzJgxAPz+97/nuOOOY+rUqQwYMICXXnqJ5cuXc8ghhzBlyhROP/10Fi5cuM3+Dj/8cJYvX77ldZ8+fairq+P888/f6kKAN998k/33358ePXrw2GOPsXLlyu3W+YlPfIIZM2YA8Nxzz21573Xr1rH33nvTv39/Xn75ZR566KEt2/Tt25f169c3u69//dd/ZcOGDbz11lvcf//9fPzjH2/1s/qP//gPhg8f3up6rfGwpiRJu5jrrruuIvvdsGHDVueJXXbZZdx4442cf/75XHfddVsuCIDS7TyWLl1KZjJ27Fg+9rGPcdNNN3H77bfTo0cPDjjgAL797W9v8x6nnXYa9fX1HHrooVuWjR8/nnHjxm115eaECRP4zGc+Q21tLSNHjuSII47Ybu0XXXQRkyZNYsSIEYwcOZJjjz0WgKOOOopRo0Zx5JFHcsghhzB69Ogt20yePJlTTz2VAw88cKvzzo4++mjOO++8Lfu48MILGTVq1FaHMD/o3XffZdmyZdTW1m63zraI7U3d7Wpqa2uztXua6D/V19e36aofdR57Ukz2pXi6Yk+WLFnC0KFDq13GTlu/fv1W55K1ZM2aNUycOJFZs2Z1QlWd5/2LF77zne80O95cfyNibmZuk+Y8rClJkjrNgQceyBe/+MVd4p5sO6KxsbHDLsLwsKYkSepUZ599drVL6HB/9Vd/1WH7cuZMkqSC6EqnGuk/7WhfDWeSJBVAz549Wbt2rQGti8lM1q5dS8+ePdu8jYc1JUkqgEGDBrFq1SpeffXVapeyUzZt2rRDAWR30rNnz2afltASw5kkSQXQo0cPDj744GqXsdPq6+sZNWpUtcvoEjysKUmSVCCGM0mSpAKpWDiLiFsi4pWIeK6F8X0j4v6IWBgRT0fE8CZj+0TEPRHx24hYEhHHV6pOSZKkIqnkzNltwCnbGf8mMD8zRwATgRuajN0A/DwzjwCOApZUqkhJkqQiqVg4y8zZwOvbWWUY8Eh53d8CNRExMCL6AZ8Aflweeycz36hUnZIkSUVS0WdrRkQN8LPM3OYR7RHx34CemXlZRBwL/Ao4DngPmA4spjRrNhe4JDPfauE9JgOTAQYOHHhM04emavsaGhro06dPtctQE/akmOxL8diT4rEnO+7EE09s9tma1Qxn/SgdvhwFPAscAVwI9AB+DYzOzKci4gZgXWb+19bezwef75iu+ODgXZ09KSb7Ujz2pHjsyY5r6cHnVbvPWWauAyYBREQAz5e/egOrMvOp8qr3AFdUpUhJkqROVrVbaZSvyNyz/PJCYHZmrsvMPwAvRsTh5bGxlA5xSpIkdXkVmzmLiDuAOmBARKwCrqJ0yJLMvBkYCvwkIt6jFL4uaLL5V4AZ5fC2nPIMmyRJUldXsXCWmeNbGX8SOKyFsfnANsdgJUmSujqfECBJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKpCKhbOIuCUiXomI51oY3zci7o+IhRHxdEQM/8B4t4iYFxE/q1SNkiRJRVPJmbPbgFO2M/5NYH5mjgAmAjd8YPwSYEllSpMkSSqmioWzzJwNvL6dVYYBj5TX/S1QExEDASJiEHAa8KNK1SdJklRE3av43guAccDjEXEsMAQYBLwMXA98Hejb2k4iYjIwGWDgwIHU19dXqt4up6Ghwc+rYOxJMdmX4rEnxWNPOk41w9m1wA0RMR94FpgHNEbEXwCvZObciKhrbSeZOR2YDlBbW5t1da1uorL6+nr8vIrFnhSTfSkee1I89qTjVC2cZeY6YBJARATwfPnrc8DpEfFpoCfQLyJuz8zPV6tWSZKkzlK1W2lExD4RsWf55YXA7Mxcl5l/n5mDMrOGUlB71GAmSZJ2FxWbOYuIO4A6YEBErAKuAnoAZObNwFDgJxHxHrAYuKBStUiSJO0qKhbOMnN8K+NPAoe1sk49UN9xVUmSJBWbTwiQJEkqEMOZJElSgRjOJEmSCsRwJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTJEkqEMOZJElSgRjOJEmSCsRwJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTJEkqEMOZJElSgRjOJEmSCsRwJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTJEkqkDaFs4j4aETsVf6+LiKmRMQ+lS1NkiRp99PWmbN7gfci4lDgx8DBwP9XsaokSZJ2U20NZ5szsxH4LHB9Zv4dcGDlypIkSdo9tTWcvRsR44G/Bn5WXtajMiVJkiTtvtoaziYBxwPTMvP5iDgYuH17G0TELRHxSkQ818L4vhFxf0QsjIinI2J4eflHIuKxiFgSEYsi4pId+YEkSZJ2ZW0KZ5m5ODOnZOYdEbEv0Dczr21ls9uAU7Yz/k1gfmaOACYCN5SXNwJfzcyhwJ8BF0fEsLbUKUmStKtr69Wa9RHRLyL+BFgA3BoR/7i9bTJzNvD6dlYZBjxSXve3QE1EDMzMNZn5THn5emAJcFBb6pQkSdrVdW/jev0zc11EXAjcmplXRcTCdr73AmAc8HhEHAsMAQYBL7+/QkTUAKOAp1raSURMBiYDDBw4kPr6+naWtftoaGjw8yoYe1JM9qV47Enx2JOO09Zw1j0iDgTOBq7soPe+FrghIuYDzwLzKB3SBCAi+lC6hcelmbmupZ1k5nRgOkBtbW3W1dV1UHldX319PX5exWJPism+FI89KR570nHaGs6mAg8DT2TmbyLiEGBpe964HLgmAUREAM+Xv4iIHpSC2YzMvK897yNJkrQraVM4y8y7gbubvF4O/GV73rj8hIENmfkOcCEwu3zoNCjd6HZJZm73vDZJkqSupq0XBAwq3/bilYh4OSLujYhBrWxzB/AkcHhErIqICyLiSxHxpfIqQ4FFEfFb4FTg/VtmjAa+AJwUEfPLX5/eqZ9OkiRpF9PWw5q3Unpc01+VX3++vOzPW9ogM8dvb4eZ+SRwWDPLHweijXVJkiR1KW29Ce1+mXlrZjaWv24D9qtgXZIkSbultoaz1yLi8xHRrfz1eWBtJQuTJEnaHbU1nJ1P6TYafwDWAGdRvtJSkiRJHaetj296ITNPz8z9MnP/zDyT0g1kJUmS1IHaOnPWnMs6rApJkiQB7QtnXlEpSZLUwdoTzrLDqpAkSRLQyn3OImI9zYewAHpVpCJJkqTd2HbDWWb27axCJEmS1L7DmpIkSepghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBVKxcBYRt0TEKxHxXAvj+0bE/RGxMCKejojhTcZOiYjfRcSyiLiiUjVKkiQVTSVnzm4DTtnO+DeB+Zk5ApgI3AAQEd2Am4BTgWHA+IgYVsE6JUmSCqNi4SwzZwOvb2eVYcAj5XV/C9RExEDgWGBZZi7PzHeAO4EzKlWnJElSkXSv4nsvAMYBj0fEscAQYBBwEPBik/VWAce1tJOImAxMBhg4cCD19fWVqrfLaWho8PMqGHtSTPaleOxJ8diTjlPNcHYtcENEzAeeBeYBjUA0s262tJPMnA5MB6itrc26urqOr7SLqq+vx8+rWOxJMdmX4rEnxWNPOk7VwllmrgMmAUREAM+Xv3oDH2my6iBgdacXKEmSVAVVu5VGROwTEXuWX14IzC4Htt8Ah0XEweXxzwEzq1WnJElSZ6rYzFlE3AHUAQMiYhVwFdADIDNvBoYCP4mI94DFwAXlscaI+DLwMNANuCUzF1WqTkmSpCKpWDjLzPGtjD8JHNbC2IPAg5WoS5Ikqch8QoAkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSoQw5kkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSoQw5kkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSoQw5kkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAqGs4i4paIeCUinmthvH9E/FtELIiIRRExqcnY/ygvWxIRN0ZEVLJWSZKkIqj0zNltwCnbGb8YWJyZRwF1wPciYs+IOAEYDYwAhgN/CnyysqVKkiRVX0XDWWbOBl7f3ipA3/KsWJ/yuo3l5T2BPYG9gB7Ay5WsVZIkqQgiMyv7BhE1wM8yc3gzY32BmcARQF/gnMx8oDz2XeBCIIDvZ+aVLex/MjAZYODAgcfceeedFfgpuqaGhgb69OlT7TLUhD0pJvtSPPakeOzJjjvxxBPnZmbtB5d3r0YxTZwMzAdOAj4KzIqIXwL7A0OBQeX1ZkXEJ8ozcVvJzOnAdIDa2tqsq6vrjLq7hPr6evy8isWeFJN9KR57Ujz2pONU+2rNScB9WbIMeJ7SLNpngV9nZkNmNgAPAX9WxTolSZI6RbXD2QvAWICIGAgcDiwvL/9kRHSPiB6ULgZYUrUqJUmSOklFD2tGxB2UrsIcEBGrgKsondxPZt4MfAe4LSKepXRu2Tcy87WIuIfSoc5nKV0c8PPM/LdK1ipJklQEFQ1nmTm+lfHVwKeaWf4e8DeVqkuSJKmoqn1YU5IkSU0YziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQg3atdgCQV0eWXX866deu2vL7jjjsA6NevH9ddd121ypK0G3DmTJKa0TSYtWW5JHWUioWziLglIl6JiOdaGO8fEf8WEQsiYlFETGoyNjgi/j0ilkTE4oioqVSdkiRJRVLJmbPbgFO2M34xsDgzjwLqgO9FxJ7lsZ8A12XmUOBY4JUK1ilJklQYFQtnmTkbeH17qwB9IyKAPuV1GyNiGNA9M2eV99OQmRsqVackSVKRVPOCgO8DM4HVQF/gnMzcHBH/BXgjIu4DDgb+L3BFZr7X3E4iYjIwGWDgwIHU19d3Ru1dQkNDg59XwdiTXYM9qj5/V4rHnnScaoazk4H5wEnAR4FZEfHLck0fB0YBLwD/BzgP+HFzO8nM6cB0gNra2qyrq6t03V1GfX09fl7FYk+K44EHHmj25P9+/frZowLwd6V47EnHqWY4mwRcm5kJLIuI54EjgFXAvMxcDhAR/wr8GS2EM0mqhKa3y/AfHUmdqZq30ngBGAsQEQOBw4HlwG+AfSNiv/J6JwGLq1KhJElSJ6vYzFlE3EHpKswBEbEKuAroAZCZNwPfAW6LiGeBAL6Rma+Vt/0a8Ej5YoG5wA8rVackSVKRVCycZeb4VsZXA59qYWwWMKISdUmSJBWZTwiQJEkqEMOZJElSgRjOJEmSCiRKd7LoGiLiVWBltevYhQwAXqt2EdqKPSkm+1I89qR47MmOG5KZ+31wYZcKZ9oxETEnM2urXYf+kz0pJvtSPPakeOxJx/GwpiRJUoEYziRJkgrEcLZ7m17tArQNe1JM9qV47Enx2JMO4jlnkiRJBeLMmSRJUoEYziRJkgrEcNbFRcSfRMSsiFha/nPfFtb76/I6SyPir5sZnxkRz1W+4q6vPT2JiN4R8UBE/DYiFkXEtZ1bfdcSEadExO8iYllEXNHM+F4R8X/K409FRE2Tsb8vL/9dRJzcmXV3dTvbl4j484iYGxHPlv88qbNr76ra87tSHh8cEQ0R8bXOqnlXZjjr+q4AHsnMw4BHyq+3EhF/AlwFHAccC1zVNDBExDigoXPK3S20tyffzcwjgFHA6Ig4tXPK7loiohtwE3AqMAwYHxHDPrDaBcAfM/NQ4J+A/17edhjwOeBI4BTgf5b3p3ZqT18o3QD1M5n5MeCvgf/dOVV3be3syfv+CXio0rV2FYazru8M4H+Vv/9fwJnNrHMyMCszX8/MPwKzKP2DQ0T0AS4DrumEWncXO92TzNyQmY8BZOY7wDPAoE6ouSs6FliWmcvLn+WdlHrTVNNe3QOMjYgoL78zM9/OzOeBZeX9qf12ui+ZOS8zV5dHot9qAAAEJ0lEQVSXLwJ6RsRenVJ119ae3xUi4kxgOaWeqA0MZ13fwMxcA1D+c/9m1jkIeLHJ61XlZQDfAb4HbKhkkbuZ9vYEgIjYB/gMpdk37bhWP+Om62RmI/Am8KE2bqud056+NPWXwLzMfLtCde5OdronEbE38A3g6k6os8voXu0C1H4R8X+BA5oZurKtu2hmWUbESODQzPy7D54/oO2rVE+a7L87cAdwY2Yu3/EKRSufcSvrtGVb7Zz29KU0GHEkpcNqn+rAunZn7enJ1cA/ZWZDeSJNbWA46wIy8/9paSwiXo6IAzNzTUQcCLzSzGqrgLomrwcB9cDxwDERsYLS35X9I6I+M+vQdlWwJ++bDizNzOs7oNzd1SrgI01eDwJWt7DOqnIg7g+83sZttXPa0xciYhBwPzAxM39f+XJ3C+3pyXHAWRHxP4B9gM0RsSkzv1/5snddHtbs+mZSOjGW8p8/bWadh4FPRcS+5ZPOPwU8nJn/kpkfzswaYAzwHwazDrHTPQGIiGso/Yfv0k6otSv7DXBYRBwcEXtSOsF/5gfWadqrs4BHs3Tn7pnA58pXqB0MHAY83Ul1d3U73Zfyof4HgL/PzCc6reKub6d7kpkfz8ya8r8j1wP/zWDWOsNZ13ct8OcRsRT48/JrIqI2In4EkJmvUzq37Dflr6nlZaqMne5JeVbgSkpXTD0TEfMj4sJq/BC7uvJ5MV+mFHqXAHdl5qKImBoRp5dX+zGl82aWUbow5orytouAu4DFwM+BizPzvc7+Gbqi9vSlvN2hwH8t/27Mj4jmzunUDmhnT7QTfHyTJElSgThzJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTtFuIiPea3F5hfkR02KX+EVETEc911P4k7d58QoCk3cXGzBxZ7SIkqTXOnEnarUXEioj47xHxdPnr0PLyIRHxSEQsLP85uLx8YETcHxELyl8nlHfVLSJ+GBGLIuLfI6JX1X4oSbs0w5mk3UWvDxzWPKfJ2LrMPBb4PqVHzFD+/ieZOQKYAdxYXn4j8IvMPAo4GlhUXn4YcFNmHgm8AfxlhX8eSV2UTwiQtFuIiIbM7NPM8hXASZm5PCJ6AH/IzA9FxGvAgZn5bnn5mswcEBGvAoMy8+0m+6gBZmXmYeXX3wB6ZOY1lf/JJHU1zpxJEmQL37e0TnPebvL9e3hOr6SdZDiTJDinyZ9Plr//FfC58vcTgMfL3z8CXAQQEd0iol9nFSlp9+D/2UnaXfSKiPlNXv88M9+/ncZeEfEUpf9hHV9eNgW4JSIuB14FJpWXXwJMj4gLKM2QXQSsqXj1knYbnnMmabdWPuesNjNfq3YtkgQe1pQkSSoUZ84kSZIKxJkzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSqQ/x/MyqlXQRsfdgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot Epoch history\n",
    "fig1 = plt.figure(figsize=(10,10))\n",
    "\n",
    "# Create a single plot of all results\n",
    "ax1 = fig1.add_subplot( 2,1,1 )\n",
    "ax2 = fig1.add_subplot( 2,1,2 )\n",
    "\n",
    "# X-axis\n",
    "x_vals = range(len(hist.history['acc']))\n",
    "\n",
    "# Plots\n",
    "ax1.set_ylim( ymin=0.9*min(hist.history['acc'] + hist.history['val_acc']), ymax=1.1)\n",
    "\n",
    "ax1.plot( x_vals, hist.history['acc'], label='Accuracy (Training)', c='k', marker='o', linestyle='-')\n",
    "ax1.plot( x_vals, hist.history['val_acc'], label='Accuracy (Validation)', c='0.4', marker='s', linestyle='--')\n",
    "\n",
    "ax2.plot( x_vals, hist.history['loss'], label='Loss (Training)', c='k', marker='o', linestyle='-')\n",
    "ax2.plot( x_vals, hist.history['val_loss'], label='Loss (Validation)', c='0.4', marker='s', linestyle='--')\n",
    "\n",
    "# Add text note on the max and min accuracy and loss points\n",
    "acc_min_idx = np.argmin(hist.history['acc'])\n",
    "ax1.text( x=acc_min_idx, y=hist.history['acc'][acc_min_idx]*1.02, c='b',\n",
    "          s=f\"Min: {hist.history['acc'][acc_min_idx]:.4f}\\nEpoch: {acc_min_idx}\" )\n",
    "\n",
    "acc_max_idx = np.argmax(hist.history['acc'])\n",
    "ax1.text( x=acc_max_idx, y=hist.history['acc'][acc_max_idx]*1.02, c='b',\n",
    "          s=f\"Max: {hist.history['acc'][acc_max_idx]:.4f}\\nEpoch: {acc_max_idx}\" )\n",
    "\n",
    "loss_min_idx = np.argmin(hist.history['loss'])\n",
    "ax2.text( x=loss_min_idx, y=hist.history['loss'][loss_min_idx]*1.02, c='b',\n",
    "          s=f\"Min: {hist.history['loss'][loss_min_idx]:.4f}\\nEpoch: {loss_min_idx}\" )\n",
    "         \n",
    "loss_max_idx = np.argmax(hist.history['loss'])\n",
    "ax2.text( x=loss_max_idx, y=hist.history['loss'][loss_max_idx]*1.02, c='b',\n",
    "          s=f\"Max: {hist.history['loss'][loss_max_idx]:.4f}\\nEpoch: {loss_max_idx}\" )\n",
    "\n",
    "# If populated, plot the loss and accuracy from the test samples\n",
    "try:\n",
    "    loss_val = m_score_list[0]\n",
    "    acc_val = m_score_list[1]\n",
    "\n",
    "    ax1.text( x=min(x_vals), y=acc_val*1.02, c='darkgreen', s=f\"{acc_val:.4f}\")\n",
    "    ax1.hlines(y=acc_val, xmin=min(x_vals), xmax=max(x_vals),\n",
    "              label='Accuracy (Testing)', color='g', linewidth=1, linestyle=':')    \n",
    "    \n",
    "    ax2.text( x=min(x_vals), y=loss_val*0.97, c='darkgreen', s=f\"{loss_val:.4f}\")\n",
    "    ax2.hlines(y=loss_val, xmin=min(x_vals), xmax=max(x_vals),\n",
    "              label='Loss (Testing)', color='g', linewidth=1, linestyle=':')    \n",
    "except:\n",
    "    pass\n",
    "    \n",
    "ax1.legend(loc='lower right')\n",
    "ax1.set_xlabel(\"Epoch\")\n",
    "ax1.set_ylabel(\"Accuracy\")\n",
    "ax1.set_title(\"Model Fitting History - Accuracy\")\n",
    "ax1.grid(True)\n",
    "\n",
    "ax2.legend(loc='upper right')\n",
    "ax2.set_xlabel(\"Epoch\")\n",
    "ax2.set_ylabel(\"Loss\")\n",
    "ax2.set_title(\"Model Fitting History - Loss\")\n",
    "ax2.grid(True)\n",
    "         \n",
    "# Save the image\n",
    "fig1.savefig(\"docs/Figure-DPA_Fit_History.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "733/733 [==============================] - 107s 146ms/step\n"
     ]
    }
   ],
   "source": [
    "# Predict the output (using the test_generator)\n",
    "test_generator.reset()\n",
    "pred_onehot = model.predict_generator(test_generator,\n",
    "                             steps=test_n_step_size,\n",
    "                             verbose=1)\n",
    "\n",
    "predicted_class = np.argmax(pred_onehot, axis=1)\n",
    "# predicted_class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ensure the predicted labels are all strings for comparison with the actual label\n",
    "predicted_y = [ str(x) for x in predicted_class ]\n",
    "# predicted_y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The actual labels are already strings\n",
    "# actual_y = [ str(x) for x in y_test ]\n",
    "actual_y = list(y_test)\n",
    "# actual_y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[293,   0,  71,   0,   0],\n",
       "       [ 14,   0,  62,   0,   0],\n",
       "       [ 30,   0, 156,   0,   0],\n",
       "       [ 24,   0,  16,   0,   0],\n",
       "       [ 15,   0,  52,   0,   0]], dtype=int64)"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generate the confusion matrix for this prediction\n",
    "test_cm = confusion_matrix(actual_y, predicted_y )\n",
    "test_cm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6125511596180082"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_accuracy = accuracy_score(actual_y, predicted_y )\n",
    "test_accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6125511596180082"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_precision = precision_score(actual_y, predicted_y, average='micro' )\n",
    "test_precision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6125511596180082"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_recall = recall_score(actual_y, predicted_y, average='micro' )\n",
    "test_recall"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6125511596180082"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_f1 = f1_score(actual_y, predicted_y, average='micro' )\n",
    "test_f1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the prediction results to a file\n",
    "\n",
    "# Get the list of test image id codes used by the test generator to make the predictions\n",
    "# test_id_code_list = [ (x.split(\"\\\\\"))[1].replace(\".png\",\"\") for x in test_generator.filenames ]\n",
    "test_id_code_list = [ x.replace(\".png\",\"\") for x in test_generator.filenames ]\n",
    "\n",
    "# Associate the test image id codes and the predictions in a dataframe\n",
    "# NOTE: Using `predicted_class` (which has integer labels) vs. `predicted_y` (which has string labels)\n",
    "results_df = pd.DataFrame({\"id_code\": test_id_code_list, \"diagnosis\": predicted_class})\n",
    "results_df.to_csv(\"./data/prediction_results.csv\",index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the model to a file\n",
    "model.save(\"./saved_model.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predict the Class for Official Test Images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the Saved Convolutional Neural Network Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load a saved model\n",
    "# model = load_model(\"./saved_model-96x96,64,128.h5\")\n",
    "model = load_model(\"./saved_model.h5\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the Model Fit History"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAJcCAYAAAC8DwN/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXwW1dn/8c9FAAGV1YpgELBgkS0sUdSibIpoVUR8FEoVrYq1P9dWn2Jxxa2lrrRWi9WKiuDyCLVV3EkRV6xABUREChrZdwKEJVy/P84k3IQ7IeQOyZB836/X/crMmTMz1z0n6JVzzsyYuyMiIiIi8VCtogMQERERkV2UnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5E6nCzKyFmbmZVS9B3UvMbFopzjHEzN7ax31+a2Z/3ddzlRUzyzGzoyvq/CJStSk5EzlAmNkiM9tmZocVKp8ZJVgtKiay3ZK8nITPLAB3H+fufRPqupm1SljvaWbZicdz93vd/fL9FOtu54/K7jCz5xLOf4i7L9zLcfaIOw4sWGhmcys6FhEpHSVnIgeW/wKD81fMrANQu+LC2UP9KLE5xN0zKjqYOCtJb2UpnQIcDhxtZsftp3MktR+/k0iVouRM5MDyLHBxwvpQ4JnECmZWz8yeMbOVZrbYzG4xs2rRtjQzu9/MVpnZQuAnSfZ90syWmtn3Zna3maWlEnDicKiZTY2KZ0W9a0OByUDThB63pok9WQm9ckPN7Nso9hEJx69tZmPNbK2ZfWlm/5tqj1Zi75qZnWlmc81sY3RNbjSzg4uI+yAze9jMlkSfh83soOg4Pc0s28x+Y2bLgL+Z2WwzOzvhvDWi79cphfCHAn8HXo+WE79XQzP7WxTbWjOblLCtf9QLu8HMvjGzflH5IjM7NaFesra5zMy+Bd6Lyl8ys2Vmtt7MpppZu4T9a5vZA9Hv5nozmxaVvWZm1xSK9z9mdm4K10LkgKTkTOTA8jFQ18yOjZKmC4HnCtX5I1APOBroQUjmLo22XQGcBXQGMoHzC+07FtgBtIrq9AXKbHjR3U+JFjOi3rWxwBnAkoQetyVF7N4d+BHQB7jNzI6Nym8HWhC+72nAz8oq3siTwJXufijQHnjP3TcVEfcI4ASgE5ABHA/cknCsI4CGQHNgGCGxToz3TGCpu88sTaBmVofQpuOizyAzq5lQ5VmgDtCO0Lv2ULTf8VEsNwH1Cb1vi/bh1D2AY4HTo/XJQOvoHJ9HseS7H+gKnES4Fv8L7CT87hVcCzPLAI4kJJkiVYqSM5EDT37v2WnAPOD7/A0JCdvN7r7R3RcBDwAXRVUuAB529+/cfQ1wX8K+jQkJx/XuvsndVxD+5z1oH2JbZWbros+Npf6Gyd3p7lvcfRYwi5D8QPhO97r7WnfPBkaX4FifJ8S5DhheTN3tQFszqxud4/Ni6g4BRrr7CndfCdzJrmsPIQm53d23uvsWQmJ9ppnVjbZfRGjf0joP2Aq8BfwTqE7UO2pmTQjt+4voe2x3939F+10GPOXub7v7Tnf/3t3n7cN574h+Z7YAuPtT0e/fVuAOICPqla0G/By4LjpHnrt/GNX7O9DazFonXIsX3H1bCtdD5ICk5EzkwPMs8FPgEgoNaQKHATWBxQlliwk9EABNge8KbcvXHKgBLE1IWv5C6P0oqcPcvX70uX8f9iuJZQnLm4FDouXC3ylxuShdEuKsD/yumLoDCT1ai83sX2Z2YjF1m7LntW+asL7S3XPzV6Letg+AgWZWn5A8JfYyFTCzyQlDqEOKOP9Q4EV33xElPK+wa2izGbDG3dcm2a8Z8E0x32tvCq55NHT+u2hodAO7euAOiz61kp0rivdF4GdREjeY1BJVkQOWJm+KHGDcfbGZ/ZeQMFxWaPMqQk9PcyD/br2j2NW7tpTwP2IStuX7jtDrcpi77yjruIvhKe6/FEhn1/dtVkzdfebu04H+ZlYDuJqQQDQjedxLCNd+TrR+VFRWcLgk+4wlDB1XBz5y9++T1MHdzyguTjNLB3oDx5vZwKi4DlDLwh2+3wENzay+u68rtPt3wA+LOPSm6Dj5jkgWXsLyT4H+wKmExKwesBYwwu9nbnSuWUmOM5aQkE0DNrv7R0XEJFKpqedM5MB0GdA7mvtUwN3zCMnDPWZ2qJk1B37FrnlpLwLXmlm6mTUgYTjP3ZcShsMeMLO6ZlbNzH5oZj3KOPblhPlhieuNzKxeKY/3InCzmTUwsyMJCVSZMLOaFp7TVs/dtwMbgLxoc7K4xwO3mNkPooToNvacE1jYJKALcB179oTui4uA+YR5eZ2izzFANjA4at/JwJ+ja1XDzPLnAD4JXGpmfaJ2P9LM2kTbZhLmrtUws2TzFAs7lJDkryYkdffmb3D3ncBTwIPRDRRpZnZi/k0TUTK2kzAUr14zqbKUnIkcgNz9G3f/rIjN1xB6OxYSeiCeJ/wPEeAJ4E1Cr8XnhGGvRBcThkXnEno7XgaalGnwYQ7S2Gjo9IJobtN4YGFU1rT43fcwkpCA/Bd4hxDz1jKM9yJgUTRE9wuiSetFxH038BnwH+ALwjW+u7iDR/O0/g9oyZ7tsS+GAn9292WJH+Bxdg1tXkToWZ0HrACuj2L4lHDTyEPAeuBfhB5AgFsJPV1rCXPont9LHM8QhnO/J/wefVxo+42EazMdWAP8nt3/X/QM0IG9J7UilZa5pzqiICISH2Z2FTDI3cu6x2+/MbPbgGPcvazvND3gmNnFwDB3717RsYhUFPWcicgBzcyamNmPo+G4HwG/BiZWdFwlZWYNCcPUYyo6looWPQrkl+haSBVXIcmZmT1lZivMbHYR29uY2UdmtnU/3I4vIpVLTcJdpRsJD0H9O/DnCo2ohMzsCsJk/MnuPnVv9SszMzsdWEmYy7e3oVORSq1ChjWjSag5wDPu3j7J9sMJ8x3OBdbuh1vyRURERGKpQnrOor8Q1xSzfUV0+/r28otKREREpOId8M85M7NhhNegULt27a7NmpXpI44qtZ07d1KtmqYdxonaJJ7ULvGjNokftcm+mz9//ip3/0Hh8gM+OXP3MUSTRzMzM/2zz4p6uoAUlpWVRc+ePSs6DEmgNokntUv8qE3iR22y78xscbJypbgiIiIiMaLkTERERCRGKmRY08zGAz2Bw8wsG7id8MJl3P1xMzuC8JTtusBOM7seaOvuGyoiXhEREZHyUiHJmbsP3sv2ZYQXGYuIiIhUKRrWFBEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxEiFJGdm9pSZrTCz2UVsNzMbbWYLzOw/ZtalvGMUERERqQgV1XP2NNCvmO1nAK2jzzDgsXKISURERKTCVUhy5u5TgTXFVOkPPOPBx0B9M2tSPtGJiIiIVJzqFR1AEY4EvktYz47KlhauaGbDCL1rNG7cmKysrPKIr1LIycnR9YoZtUk8qV3iR20SP2qTshPX5MySlHmyiu4+BhgDkJmZ6T179tyPYVUuWVlZ6HrFi9okntQu8aM2iR+1SdmJ692a2UCzhPV0YEkFxSIiIiJSbuKanL0KXBzdtXkCsN7d9xjSFBEREalsKmRY08zGAz2Bw8wsG7gdqAHg7o8DrwNnAguAzcClFRGniIiISHmrkOTM3QfvZbsD/6+cwhERERGJjbgOa4qIiIhUSUrORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjFRIcmZm/czsKzNbYGbDk2xvbmbvmtl/zCzLzNIrIk4RERGR8lbuyZmZpQGPAmcAbYHBZta2ULX7gWfcvSMwErivfKMUERERqRgV0XN2PLDA3Re6+zZgAtC/UJ22wLvR8pQk20VEREQqpeoVcM4jge8S1rOBboXqzAIGAo8AA4BDzayRu68ufDAzGwYMA2jcuDFZWVn7I+ZKKScnR9crZtQm8aR2iR+1SfyoTcpORSRnlqTMC63fCPzJzC4BpgLfAzuSHczdxwBjADIzM71nz55lFmhll5WVha5XvKhN4kntEj9qk/hRm5SdikjOsoFmCevpwJLECu6+BDgPwMwOAQa6+/pyi1BERESkglTEnLPpQGsza2lmNYFBwKuJFczsMDPLj+1m4KlyjlFERESkQpR7cubuO4CrgTeBL4EX3X2OmY00s3Oiaj2Br8xsPtAYuKe84xQRERGpCBUxrIm7vw68XqjstoTll4GXyzsuERERkYqmNwSIiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxEhKyZmZXW1mDcoqGBEREZGqLtWesyOA6Wb2opn1MzMri6BEREREqqqUkjN3vwVoDTwJXAJ8bWb3mtkPyyA2ERERkSon5Tln7u7AsuizA2gAvGxmo4raJ+pl+8rMFpjZ8CTbjzKzKWY2w8z+Y2ZnphqniIiIyIEg1Tln15rZv4FRwAdAB3e/CugKDCxinzTgUeAMoC0w2MzaFqp2C/Ciu3cGBgF/TiVOERERkQNF9RT3Pww4z90XJxa6+04zO6uIfY4HFrj7QgAzmwD0B+YmHgKoGy3XA5akGKeIiIjIAcHCqGQpdzY7AZjj7huj9UOBtu7+STH7nA/0c/fLo/WLgG7ufnVCnSbAW4Qh0oOBU93930UcbxgwDKBx48ZdJ0yYUOrvU9Xk5ORwyCGHVHQYkkBtEk9ql/hRm8SP2mTf9erV69/unlm4PNWes8eALgnrm5KUFZbsjs7CGeJg4Gl3f8DMTgSeNbP27r5zjx3dxwBjADIzM71nz577EH7VlpWVha5XvKhN4kntEj9qk/hRm5SdVG8IME/oeouSp70lfNlAs4T1dPYctrwMeDE65kdALcIQqoiIiEillmpytjC6KaBG9LkOWLiXfaYDrc2spZnVJEz4f7VQnW+BPgBmdiwhOVuZYqwiIiIisZdqcvYL4CTge0KPWDei+V9FcfcdwNXAm8CXhLsy55jZSDM7J6r2a+AKM5sFjAcu8VQmx4mIiIgcIFKac+buKwg9X/u63+vA64XKbktYngv8OJXYRERERA5EKSVnZlaLMD+sHWHoEQB3/3mKcYmIiIhUSakOaz5LeL/m6cC/CJP7N6YalIiIiEhVlWpy1srdbwU2uftY4CdAh9TDEhEREamaUk3Otkc/15lZe8LT/FukeEwRERGRKivVh9COMbMGhHdhvgocAtyaclQiIiIiVVSpkzMzqwZscPe1wFTg6DKLSkRERKSKKvWwZvQ2gKv3WlFERERESizVOWdvm9mNZtbMzBrmf8okMhEREZEqKNU5Z/nPM/t/CWWOhjhFRERESiXVNwS0LKtARERERCT1NwRcnKzc3Z9J5bgiIiIiVVWqw5rHJSzXAvoAnwNKzkRERERKIdVhzWsS182sHuGVTiIiIiJSCqnerVnYZqB1GR9TREREpMpIdc7ZPwh3Z0JI9NoCL6YalIiIiEhVleqcs/sTlncAi909O8VjioiIiFRZqSZn3wJL3T0XwMxqm1kLd1+UcmQiIiIiVVCqc85eAnYmrOdFZSIiIiJSCqkmZ9XdfVv+SrRcM8VjioiIiFRZqSZnK83snPwVM+sPrErxmCIiIiJVVqpzzn4BjDOzP0Xr2UDStwaIiIiIyN6l+hDab4ATzOwQwNx9Y9mEJSIiIlI1pTSsaWb3mll9d89x941m1sDM7i6r4ERERESqmlTnnJ3h7uvyV9x9LXBmiscUERERqbJSTc7SzOyg/BUzqw0cVEx9ERERESlGqjcEPAe8a2Z/i9YvBcameEwRERGRKivVGwJGmdl/gFMBA94AmpdFYCIiIiJVUarDmgDLCG8JGAj0Ab7c2w5m1s/MvjKzBWY2PMn2h8xsZvSZb2brkh1HREREpLIpVc+ZmR0DDAIGA6uBFwiP0uhVgn3TgEeB0wjPRZtuZq+6+9z8Ou5+Q0L9a4DOpYlTRERE5EBT2p6zeYResrPdvbu7/5HwXs2SOB5Y4O4Lo9c9TQD6F1N/MDC+lHGKiIiIHFBKO+dsIKHnbIqZvUFIsKyE+x4JfJewng10S1bRzJoDLYH3ijqYmQ0DhgE0btyYrKysEoYhOTk5ul4xozaJJ7VL/KhN4kdtUnZKlZy5+0RgopkdDJwL3AA0NrPHgInu/lYxuydL4ryIuoOAl929yF45dx8DjAHIzMz0nj17luAbCEBWVha6XvGiNokntUv8qE3iR21SdlK6IcDdN7n7OHc/C0gHZgJ7TPAvJBtolrCeDiwpou4gNKQpIiIiVUhZ3K0JgLuvcfe/uHvvvVSdDrQ2s5ZmVpOQgL1auJKZ/QhoAHxUVjGKiIiIxF2ZJWcl5e47gKuBNwmP3XjR3eeY2UgzOyeh6mBggrsXNeQpIiIiUumk+oaAUnH314HXC5XdVmj9jvKMSURERCQOyr3nTERERESKpuRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIiIhIjCg5ExEREYkRJWciIiIiMaLkTERERCRGlJyJiIiIxIiSMxEREZEYUXImIiIiEiNKzkRERERiRMmZiIiISIwoORMRERGJESVnIiIiIjGi5ExEREQkRpSciYiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRipkOTMzPqZ2VdmtsDMhhdR5wIzm2tmc8zs+fKOUURERKQiVC/vE5pZGvAocBqQDUw3s1fdfW5CndbAzcCP3X2tmR1e3nGKiIiIVISK6Dk7Hljg7gvdfRswAehfqM4VwKPuvhbA3VeUc4wiIiIiFaLce86AI4HvEtazgW6F6hwDYGYfAGnAHe7+RrKDmdkwYBhA48aNycrKKut4K62cnBxdr5hRm8ST2iV+1CbxozYpOxWRnFmSMi+0Xh1oDfQE0oH3zay9u6/bY0f3McAYgMzMTO/Zs2eZBluZZWVloesVL2qTeFK7xI/aJH7UJmWnIoY1s4FmCevpwJIkdf7u7tvd/b/AV4RkTURERKRSq4jkbDrQ2sxamllNYBDwaqE6k4BeAGZ2GGGYc2G5RikiIiJSAco9OXP3HcDVwJvAl8CL7j7HzEaa2TlRtTeB1WY2F5gC3OTuq8s7VhEREZHyVhFzznD314HXC5XdlrDswK+ij4iIiEiVoTcEiIiIiMSIkjMRERGRGFFyJiIiIhIjSs5EREREYqRCbggQEYm7m266iQ0bNhSsjx8/HoC6devyhz/8oaLCEpEqQD1nIiJJJCZmJSkXESkrSs5EREREYkTJmYiIiEiMKDkTERERiRElZyIiIiIxouRMRCSJunXr7lO5iEhZ0aM0RESSSHxcRlZWFj179qy4YESkSlHPmYiIiEiMKDkTERERiRElZyIiIiIxouRMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI3rOmYhICZhBy5awcGFYz82FOnXgBz+A5cvh1Vdh7lwYPrx0x//vf2HQIFizBpo0gWXLYOdOuPzyPY/57bfQti3ccQe0aAH/8z/w3nvw6KMwezZs3QoHHwzusG0bNGwImzdDzZphm3v4LtnZ4Xjffw+rVsExx4RtK1bAunWlvVIikiolZyIiJbRsWUieGjaEUaPgoIN2bTvnnPAprd/8Bm64ISRaDRvCjTeGsuOOC8dt23ZX3RtugDPOCAni6NHQrRvcfz8MGAATJkDnzvDnP8PJJ4fl++6DjIyQfA0dCg8+GH7ecAP06QNffAH9+8NXX8Ef/wgzZpT+e1Rm27dvJzs7m9zc3IoOJZbq1avHl19+WdFhxFKtWrVIT0+nRo0aJaqv5ExEpIQ6dYJ77oEHHoCxY6FnT/j887DtttvgiSfgiCNCT9Tpp8M334SEJyMj9Gx98QUMHgyffhp63fK5h+3PPx+2tWkD06bBrbeG3rS//31XcjZpEhx9dOgZe/PNkMD9/vchofrnP2Hy5HC+k08O9b/5Bnr1gs8+g2OPhTfeCMnZqafC//1fSM7Gjw9xQVi+885yu6QHlOzsbA499FBatGiBmVV0OLGzceNGDj300IoOI3bcndWrV5OdnU3Lli1LtI/mnImIlNCvfgUvvBCG/JYsCQlYviZNQs/VjBkhiZsyJSRY//oXfPQRTJwIl14ahhXr1AnJ0uWXh31Xr4b69aF69ZDYHX10+AmQnr5redOmkIjdfjssXRriOOss2LIFGjQIx7/iCvj445B8dekSyl99FVq1CkOeixfDjh0hyfvuu3DcF14IydnixWF4tXfv8rumB5Lc3FwaNWqkxEz2iZnRqFGjfepxVXImIlJC558Pa9eGJK1z5923bd4ceqXat4fp08N8sWrVwnq1anDRRdCjB3z9daifmQl//WtYdt91nPzlxP//5y/ffnsYiqxTJ/SanX32rn3mzYOrrgrbV64MvWfTpoXkbOTIkKydeWZIDk8+OcxVq14dPvkkHK99+zAkev75kJZW5peu0lBiJqWxr783GtYUESmho48O87uefhrGjYOf/nRX4vTXv4aEbPbskOBMnbprv7y80DO2ZEny4x52WOgJa9UqDEM2bAgbN4aEacgQaNo01PvkkzB0OmRI6P26774w923nzpCgXXRR6F3buDHMRXv33RDPwQeHGwD+8Y8QS05OGJbduhXefz8kj926wX/+E3rqsrLCkK2IVAz1nImIlNCyZfDb38K554YesOoJf97WqRPmdAEsWLCrfP36cMfk1Klh+PLll/c8rhmceCJcdx00bx564X7+85BwTZiw60aD4cPh+OPDMGa3blC3brhBITMTDjkkJF+zZ4e7Mnv3DslctWrw8MNhblzHjmE+29SpIaGrWTN8rroqDGc2bBh6237967Bd4mnixImYGfPmzavoUEply5Yt9OjRg1mzZtGpUyc6depEw4YNadmyJZ06deLUU0/dp+OdfvrpbNy4sdg6I0aMYMqUKaWKNzc3lx49epCXl1eq/UtDyZmISAl16hQm3L/ySuh5atVq17batcNQ4yGHhARpwwY46SQ4/PCw/Zhj4Kab4IILYNGi3eecAYwZA88+G+aXdewYhki//ho6dAhzwgAeeggaNw5J4Zo1oUftjTdCYlinTjh29+6hV23hQvjRj0L5ffeFpGz27DBP7aSTYPv2cOyjjgqP0WjUKMTYuHHo5fvss3K7rJXWuHHjaNGiBdWqVaNFixaMGzeuTI47fvx4unfvzoQJE8rkeEXZX8nIU089xXnnnUdGRgYzZ85k5syZnHPOOfzhD39g5syZvPPOO7vV37FjR7HHe/PNN/d6I8I999xDr169ShVvrVq1OOWUU3g52V9W+0mFJGdm1s/MvjKzBWa2x1OBzOwSM1tpZjOjz+XJjiMiUp4K3xBw5ZXhOWcQhhp/8YswZNi7dxiqnDYN/v3v8MiNiRPh5pvhhz8M870KO/rocKfmkUfCa6+FocVjjoHnngtzxgD+939DD9fmzWH+2JIlYVL/qFGhV+6pp8Ldm/k9e717w913w/z5MHNmmEs2Zw785S+hh27JknCuY48NzzZr2zYc/9//3nWzgJTOuHHjGDZsGIsXL8bdWbx4McOGDUs5QcvJyeGDDz7gySef3CM5GzVqFB06dCAjI4Ph0cPxFixYwKmnnkpGRgZdunThm2++ISsri7POOqtgv6uvvpqnn34agBYtWjBy5Ei6d+/OSy+9xBNPPMFxxx1HRkYGAwcOZPPmzQAsX76cAQMGkJGRQUZGBh9++CF33XUXjzzySMFxR4wYwejRo5Nem/79+xf7Pd955x1OPfVUBg0aROdogufZZ59N165dadeuHX/Nn7AJpKens27dOhYsWED79u257LLLaNeuHWeccUbBJPyf/exnTJo0qaD+HXfcQefOnenYsSPz588HYMWKFfTp04cuXbrwy1/+kiOPPJJ10QP/zj333DJLrkui3OecmVka8ChwGpANTDezV919bqGqL7j71eUdn4hIUc4/PzwfbG83BHz/fehJK3xDwJVXhsdwwO43BJRU377hZoOTTgoT/Q87LDwWY+bMEE/16uFRGJdfHoZSX3019JpBGPL88Y/D0OX48eHOzvr14bTTwpy0I4+EF18MD6Y96aTdh2xlT9dffz0zZ84scvvHH3/M1q1bdyvbvHkzl112GU888UTSfTp16sTDDz9c7HknTZpEv379OOaYY2jYsCGff/45Xbp0YfLkyUyaNIlPPvmEOnXqsGbNGgCGDBnC8OHDGTBgALm5uezcuZPv9pJ516pVi2nTpgGwevVqrrjiCgBuueUWnnzySa655hquvfZaevTowcSJE8nLyyMnJ4e6dety8cUXc91117Fz504mTJjAp59+utuxt23bxsKFC2mR7C+UQj7++GPmzp3LUUcdBcDYsWNp2LAhmzdvJjMzk00kV7sAACAASURBVIEDB9KgQYPd9vnqq68YP348HTp04LzzzmPSpEkMGjRoj2M3btyYGTNmMHr0aB588EEef/xxbrvtNvr168dNN93EP//5Tx577LGC+hkZGXz88cd7jbmsVETP2fHAAndf6O7bgAlA8Sm0iEhM5N8Q8Otf714+cWJIcGbPDs8OSxyJycsLw51F3RCwL0aMCMnYlCkhlieegGeeCXdo1qsXet9+8pMw/NqlSximhDB3bfDgENcrr8CFF8Jll4W5aO+/Hx7Jceed4Zlq69ZB69apx1qVFU7M9lZeUuPHjy9INgYNGsT48eOB0NN06aWXUid6gF7Dhg3ZuHEj33//PQMGDABC0lUn8QF7RbjwwgsLlmfPns3JJ59Mhw4dGDduHHPmzAHgvffe46qrrgIgLS2NevXq0bx5cxo1asSMGTN466236Ny5M40aNdrt2KtWraJ+/fol+q4nnnhiQWIG8NBDD5GRkcGJJ55IdnY233zzzR77tGrVig4dOgDQtWtXFi1alPTY55133h51pk2bVnBtzzrrrN2GSqtXr46ZsWXLlhLFnqqK+NvoSCAxbc8GuiWpN9DMTgHmAze4e9JU38yGAcMgZMJZWVllG20llpOTo+sVM2qTeMrJyQGcrKx/ccEFh7J9+1E0ajSHKVPS2bbtKLKyPmTFiq7Ur7+NrKwvmDkzk23bapKV9SE5OWls29adP/zhU0aPbs2ddy6lR4+VRZ4rN/cEPvjg32zZksamTR3IyppesC3caVmdevV28MUXdfnwwx9x2WWfcdNNR3DMMfUYPXoDmZl1+fjjeTzySFuOO24NWVnLyMlJ4913T2DYsI948MH6HHHEUSxYMJPp02vQoMF2vv22Ns8804HHHvuc++8/lE2bmrNixUxWrCiHi1tKFfFvpV69egUTz++6665i67Zr1y5pD1WzZs34xz/+UeR+xU1sX716Ne+99x5ffPEFZkZeXh5mxq233srWrVvZunXrbvtv2LABd9/jmNu2bWPbtm0F5Rs3biQ3N5eNGzfi7rvtM3ToUJ5//vmC5Oz9998vqLdx40a2bdtWcNy8vDyGDBnCmDFjWLFiBYMHD97j3Dt27GDLli17lG/fvn238s2bN3PQQQcVrE+ZMoUpU6bw9ttvU7t2bfr27cuaNWt2iyUnJ4caNWoU7LNjx46C75V4fHcv+P65ubkF1y2/BzAxto0bN5IWPVsm/7rtbQ5cUXJzc0v+O5vfEOX1Af4H+GvC+kXAHwvVaQQcFC3/AnivJMfu2rWrS8lNmTKlokOQQtQm8TRlyhQH91q1dn1OP939oYfcDz881Bkxwr1ePfeTTnLv2NH9Bz8I5Zde6l6zZlj+9tuwvHy5+/Tp7pddFsrff989Lc39yCPDz1q13GvXdq9ePRy/e/dQb/Ro9yOOcD/2WPe2bUM9cDcL5+/Rw33yZPdNm9wbNnRfty7s97e/uV94YVgeOtT9scfCcvfu4Vht2rinp4efffq4L1q0ny9oGaiIfytz584tcd3nnnvO69Sp40DBp06dOv7cc8+V+vyPP/64Dxs2bLeyU045xadOneqTJ0/2E0880Tdt2uTu7qtXr3Z3927duvnEiRPd3T03N9c3bdrk3377rTdv3txzc3N93bp13qJFC//b3/7m7u7Nmzf3lStXFhy/UaNGvnz5ct+2bZufeuqpPnToUHd3v/DCC/2hhx5yd/cdO3b4+vXrfcOGDb5161Y/5phjvGXLlr5jx46k3yM9Pd23bNmyW9nQoUP9pZdeKlh/++23vX///gXrL7/8sp977rnu7j579myvWbOmv//+++7ufuSRR/ratWv966+/9oyMjIJ97rvvPr/rrrvc3X3IkCEF1yG/vrv7Rx995H369HF392HDhvn999/v7u6vvfaaAwX1li1b5u3atUv6fUoq2e8P8JknyWcqoucsG2iWsJ4O7NbZ7+6rE1afAH5fDnGJiBQr2YjG9deHn3ffHT6FPfVU+AA0axaeLQbhLs7EOWfVq+96EXlRrrkmfCDMJ7vwwvCojpkzw1Dqpk3h0RgQbhDId8kl4QNhSDbf++8Xfz4pvSFDhgBhUvy3337LUUcdxT333FNQXhrjx48vmOifb+DAgTz//PM89thjzJw5k8zMTGrWrMmZZ57Jvffey7PPPsuVV17JbbfdRo0aNXjppZc4+uijueCCC+jYsSOtW7cumHCfzF133UW3bt1o3rw5HTp0KOhVeuSRRxg2bBhPPvkkaWlpPPbYY7Rv356aNWvSq1cv6tevX9DjVFjfvn2ZNm3aPj0y4yc/+QljxowhIyODNm3a0K1bsgG31Nx555389Kc/Zdy4cfTu3ZvGjRtz8MEHA6Hn7ic/+UmZn7NIyTK2/fkhDKUuBFoCNYFZQLtCdZokLA8APi7JsdVztm/USxM/apN4yu85SyYtzf34490PPjh83nknlE+b5t6gQejdatDA/cMPQ/kXX7g3abKrB+7xx0PPWc2a7j/6kftBB4Ver6jjo0h9+4ZPvkaN3P/yl9S/64Ei7j1nVdGGDRs8Ly/PMzIyfP78+UXW+/zzz/1nP/tZOUZWMlu2bPHt27e7u/v777/viTnFOeec419//XVKx9+XnrNyvyHA3XcAVwNvAl8CL7r7HDMbaWbRoxa51szmmNks4FrgkvKOU0SksNq1d32uu25Xed264REaP/1p+ACcdx4MHBh62/r3D+sQHoHRrVsoX78ezjgjlG/bFibk5+aGZ5ONGBHKE4+ZaOnS8PiNfA0bhpesi1SUefPm0apVK/r06UPrYu4o6dy5M7169SrXh7qWxKJFizjuuOPo2LEjN9xwA3/5y1+AcBPH+eefT6vEBxvuZxVys7S7vw68XqjstoTlm4GbyzsuEZHiFHWjVn4i9fDDu4YqV66E/Ec+PfpouFsTwtDl7NlhuWbN8BDYb7+FGjXCMCWEx2/kv4Pz+eeTnzPxfZz5qumx4lKB2rRpw8KFC0tU9+c///l+jmbftWnThhkzZuxRftBBB3HRRReVayz6pywiUkKJPWf9+u0qTzUpmjUrPLE/X1ra7o/iSKZp0/Bcsho1QpKXna3HX4hUFkrORERKaMuWXZ833thVfu+94eevf73rjQGHHx4eVgtw7bVwxBFhOT09PMgWwlDm3m4CKEpmZniF03//Gx5su2VL8uFPETnwKDkTESlC/rsRe/fuDTg1a+4o6Dk74YRQJy8PPvwwvLz88cfhxhtD+e23h/dlmoW7Ne+4I5Q//HC409IsvNYp8ea7Nm2gVi14662QuEHRc84++yzMM2vZMiSFtWsXPQQqIgcWJWciIkkUfjcihAdl5uWF8cbEO/k7dAhzwK64Au6/P5TdcUd4Ar976Cm7/fZQfv31cM45oXzr1tDrlpERtuXfENC48a6yoixdGl50vn17SOTS03VDgEhloeRMRCSJESNGFLzkeZc6NG3aii1bdk32h/DCcwi9Yiujh/8XviFg+fKwnJ0NY8eG5fwbAqD4GwKS9YjphoCqa+LEiZgZ8+bNq+hQSmXLli306NGDvLw8WrZsyVeF/qq4/vrrGTVqVJH7L1q0iPbt2wPw2Wefce211yat16JFC1atWlVsLPfmz0mInHTSSSX5CkkNGjSIr/P/4aZIr7YVEUni22+/LVF5Who0b576+RITq7S0ou8Mzde0KSTeGLdmjW4IiJObbrqJDRs27FFet25d/vCHP6R07PHjx9O9e3cmTJjAHfnj5ftBXl5ekQ+STcVTTz3FeeedR1paGoMGDWLChAncHnUt79y5k5dffpkPPvigRMfKzMwkMzOz1LHce++9/Pa3vy1Y//DDD0t9rKuuuopRo0YV+WL7faG/s0REkkh84fLeyivihoBf/hKmToUNG8LP9et3vQVAKl6yxKy48pLKycnhgw8+4Mknn2TChAm7bRs1ahQdOnQgIyOj4E0CCxYs4NRTTyUjI4MuXbrwzTffkJWVxVlnnVWw39VXX83T0asjWrRowciRI+nevTsvvfQSTzzxBMcddxwZGRkMHDiwoDd5+fLlDBgwgIyMDDIyMvjwww+56667eCShS3nEiBGMHj16j+8wbtw4+vfvD8DgwYN3+x5Tp06lRYsWNG/enEWLFnHyySfTpUsXunTpkjRxSvwuq1evpm/fvnTu3Jkrr7yyYDoCwLnnnkvXrl1p164dY8aMAWD48OFs2bKFTp06Fby54ZDomTfuzk033UT79u3p0KEDL7zwQsH5evbsyfnnn0+bNm0YMmRIwXlOPvlk3nnnnVK/ezORes5ERJK45557GDZsWKGhzc0sWVKD2rXDnLCPPw6lW7eG55i5h8n+AC+/DGefHYYwa9eG114L5a+/DqedFib+m4Whz7Zti44j/2aAwkOb/ftD9+7QqFE4zq237np1k5SPBx54YI+yrl270rNnz73um5OTU/CQ03y//vWv97rfpEmT6NevH8cccwwNGzbk888/p0uXLkyePJlJkybxySefUKdOHdasWQOE10gNHz6cAQMGkJuby86dO5O+kD1RrVq1mDZtGhASniuuuAKAW265hSeffJJrrrmGa6+9lh49ejBx4sSCF4bXrVuXiy++mOuuu46dO3cyYcIEPv30092OvW3bNhYuXEiLFi0A6NixI9WqVWPWrFlkZGQwYcIEBg8eDMDhhx/O22+/Ta1atfj6668ZPHgwn332WZFx33nnnXTv3p3bbruN1157rSAJg9Bb17BhQ7Zs2cJxxx3HwIED+d3vfsef/vQnZs6cucexXnnlFWbOnMmsWbNYtWoVxx13HKeccgoAM2bMYM6cOTRt2pQf//jHfPDBB3Tv3p1q1arRqlUrZs2aRdeuXYu9xnuj5ExEJIk9343Yssh3Iz7+OPzoR7uXde8Oa9fuedz27cNk/sJyc3ct//Ofu5aLuwPz7beL+wZSGY0fP57roxe6Dho0iPHjx9OlSxfeeecdLr30UurUqQNAw4YN2bhxI99//z0DBgwAQtJVEhfmT34EZs+ezS233MK6devIycnh9NNPB+C9997jmWeeASAtLY169epRrVo1GjVqxIwZM1i+fDmdO3emUaNGux171apV1K9ff7ey/N6zdu3a8fe//52RI0cC4Qacq6++mpkzZ5KWlsb8+fOLjXvq1Km88sorQHgXZ4MGDQq2jR49mokTJwLw3Xff8fXXX+8RW6Jp06YxePBg0tLSaNy4MT169GD69OnUrVuX448/nvT0dAA6derEokWL6N69OxASyiVLlig5ExHZX4YMGcKQIUMKhjJEEpWkp6sohxxyyD7vv3r1at577z1mz56NmZGXl4eZMWrUKNwdM9utvie7awSoXr06O3fuLFjPTfzLAApe9g1wySWXMGnSJDIyMnj66afJysoqNsbLL7+cp59+mmXLliV9C0Dt2rX3ON/gwYPp27cvPXr0oGPHjhx++OEAPPTQQzRu3JhZs2axc+fOEiWXha8BhKHId955h48++og6derQs2fPPWIorKhrB+GNAfnS0tJ2G8bMzc2ldu3ae41zbzTnTEQkBTt27NlrJrI/vPzyy1x88cUsXryYRYsW8d1339GyZUumTZtG3759eeqppwqG4desWUPdunVJT09n0qRJQHhH5ObNm2nevDlz585l69atrF+/nnfffbfIc27cuJEmTZqwfft2xo0bV1Dep08fHnvsMSDcOJA/l27AgAG88cYbTJ8+vaCXLVGDBg3Iy8vbLTn64Q9/SKNGjRg+fHjBkCbA+vXradKkCdWqVePZZ5/d67s4TznllIIYJ0+ezNqo63r9+vU0aNCAOnXqMG/ePD7On48A1KhRg+2Jr+dIONYLL7xAXl4eK1euZOrUqRx//PHFnh9g/vz5tGvXbq/19kbJmYiISBmrW7fuPpWXxPjx4wuGKPMNHDiQ559/nn79+nHOOeeQmZlJp06duD964N6zzz7L6NGj6dixIyeddBLLli2jWbNmXHDBBXTs2JEhQ4bQuXPnIs9511130a1bN0477TTatGlTUP7II48wZcoUOnToQNeuXZkzZw4ANWvWpFevXlxwwQVF3unZt2/fgjlt+QYPHsy8efN2+36//OUvGTt2LCeccALz58/frUcvmdtvv52pU6fSpUsX3nrrrYKbd/r168eOHTvo2LEjt956KyfkP0EaGDZsWMF1SDRgwAA6duxIRkYGvXv3ZtSoURyRf1dPEZYvX07t2rVp0qRJsfVKworrujvQZGZmenGTBWV3GqqJH7VJPKld4qci2uTLL7/k2GOPLddzHkg2btzIwQcfTJcuXXjppZdoXcSzXWbMmMGDDz7Is88+W84R7l8PPfQQdevW5bLLLku6Pdnvj5n92933eBaIes5EREQkZfPmzaNVq1b06dOnyMQMoHPnzvTq1Wuvw5QHmvr16zM0/zk5KdINASIiIpKyNm3asDDxycjFSHazwIHu0ksvLbNjqedMRESkhCrTVCApP/v6e6PkTEREpARq1arF6tWrlaDJPnF3Vq9eXeLnzIGGNUVEREokPT2d7OxsVua/3V52k5ubu08JSFVSq1atggfXloSSMxERkRKoUaMGLVu2rOgwYisrK6vYx3JIyWlYU0RERCRGlJyJiIiIxIiSMxEREZEYqVRvCDCzlcDiio7jAHIYsKqig5DdqE3iSe0SP2qT+FGb7Lvm7v6DwoWVKjmTfWNmnyV7bYRUHLVJPKld4kdtEj9qk7KjYU0RERGRGFFyJiIiIhIjSs6qtjEVHYDsQW0ST2qX+FGbxI/apIxozpmIiIhIjKjnTERERCRGlJyJiIiIxIiSs0rOzBqa2dtm9nX0s0ER9YZGdb42s6FJtr9qZrP3f8SVXyptYmZ1zOw1M5tnZnPM7HflG33lYmb9zOwrM1tgZsOTbD/IzF6Itn9iZi0Stt0clX9lZqeXZ9yVXWnbxcxOM7N/m9kX0c/e5R17ZZXKv5Vo+1FmlmNmN5ZXzAcyJWeV33DgXXdvDbwbre/GzBoCtwPdgOOB2xMTBjM7D8gpn3CrhFTb5H53bwN0Bn5sZmeUT9iVi5mlAY8CZwBtgcFm1rZQtcuAte7eCngI+H20b1tgENAO6Af8OTqepCiVdiE8APVsd+8ADAWeLZ+oK7cU2yTfQ8Dk/R1rZaHkrPLrD4yNlscC5yapczrwtruvcfe1wNuE/+FgZocAvwLuLodYq4pSt4m7b3b3KQDuvg34HEgvh5gro+OBBe6+MLqWEwhtkyixrV4G+piZReUT3H2ru/8XWBAdT1JX6nZx9xnuviQqnwPUMrODyiXqyi2VfyuY2bnAQkKbSAkoOav8Grv7UoDo5+FJ6hwJfJewnh2VAdwFPABs3p9BVjGptgkAZlYfOJvQ+yb7bq/XOLGOu+8A1gONSrivlE4q7ZJoIDDD3bfupzirklK3iZkdDPwGuLMc4qw0qld0AJI6M3sHOCLJphElPUSSMjezTkArd7+h8PwBKd7+apOE41cHxgOj3X3hvkco7OUa76VOSfaV0kmlXcJGs3aEYbW+ZRhXVZZKm9wJPOTuOVFHmpSAkrNKwN1PLWqbmS03sybuvtTMmgArklTLBnomrKcDWcCJQFczW0T4XTnczLLcvSdSrP3YJvnGAF+7+8NlEG5VlQ00S1hPB5YUUSc7SojrAWtKuK+UTirtgpmlAxOBi939m/0fbpWQSpt0A843s1FAfWCnmeW6+5/2f9gHLg1rVn6vEibGEv38e5I6bwJ9zaxBNOm8L/Cmuz/m7k3dvQXQHZivxKxMlLpNAMzsbsJ/+K4vh1grs+lAazNraWY1CRP8Xy1UJ7Gtzgfe8/Dk7leBQdEdai2B1sCn5RR3ZVfqdomG+l8Dbnb3D8ot4sqv1G3i7ie7e4vo/yMPA/cqMds7JWeV3++A08zsa+C0aB0zyzSzvwK4+xrC3LLp0WdkVCb7R6nbJOoVGEG4Y+pzM5tpZpdXxJc40EXzYq4mJL1fAi+6+xwzG2lm50TVniTMm1lAuDFmeLTvHOBFYC7wBvD/3D2vvL9DZZRKu0T7tQJujf5tzDSzZHM6ZR+k2CZSCnp9k4iIiEiMqOdMREREJEaUnImIiIjEiJIzERERkRhRciYiIiISI0rORERERGJEyZmIVAlmlpfweIWZZlZmt/qbWQszm11WxxORqk1vCBCRqmKLu3eq6CBERPZGPWciUqWZ2SIz+72ZfRp9WkXlzc3sXTP7T/TzqKi8sZlNNLNZ0eek6FBpZvaEmc0xs7fMrHaFfSkROaApORORqqJ2oWHNCxO2bXD344E/EV4xQ7T8jLt3BMYBo6Py0cC/3D0D6ALMicpbA4+6eztgHTBwP38fEamk9IYAEakSzCzH3Q9JUr4I6O3uC82sBrDM3RuZ2Sqgibtvj8qXuvthZrYSSHf3rQnHaAG87e6to/XfADXc/e79/81EpLJRz5mICHgRy0XVSWZrwnIemtMrIqWk5ExEBC5M+PlRtPwhMChaHgJMi5bfBa4CMLM0M6tbXkGKSNWgv+xEpKqobWYzE9bfcPf8x2kcZGafEP5gHRyVXQs8ZWY3ASuBS6Py64AxZnYZoYfsKmDpfo9eRKoMzTkTkSotmnOW6e6rKjoWERHQsKaIiIhIrKjnTERERCRG1HMmIiIiEiNKzkRERERiRMmZiOyz6EXfbmZ7vePbzC4xs2l7q5dkvyFm9tY+7vNbM/vrvp6rrJhZjpkdXVHnF5HKQcmZSCUXvTtym5kdVqh8ZpRgtaiYyHZL8nISPrMA3H2cu/dNqOv5772M1nuaWXbi8dz9Xne/fD/Futv5o7I7zOy5hPMf4u4L93KcPeKuaMm+m4hUHCVnIlXDf9n1/C7MrAMQpxdz148Sm0Oid1ZKEUrSWykiBzYlZyJVw7PAxQnrQ4FnEiuYWT0ze8bMVprZYjO7xcyqRdvSzOx+M1tlZguBnyTZ90kzW2pm35vZ3WaWlkrAicOhZjY1Kp4V9a4NBSYDTRN63Jom9mQl9MoNNbNvo9hHJBy/tpmNNbO1Zvalmf1vqj1aiT1QZnammc01s43RNbnRzA4uIu6DzOxhM1sSfR42s4Oi4/Q0s2wz+42ZLQP+ZmazzezshPPWiL5fp1TiT/J9qkW/B4vNbEX0+1Ev2lbLzJ4zs9Vmts7MpptZ42jbJWa2MPru/zWzIWUZl0hlp+RMpGr4GKhrZsdGSdOFwHOF6vwRqAccDfQgJHP5T8W/AjgL6AxkAucX2ncssANoFdXpC5TZ8KK7nxItZkS9a2OBM4AlCT1uS4rYvTvwI6APcJuZHRuV3w60IHzf04CflVW8kSeBK939UKA98J67byoi7hHACUAnIAM4Hrgl4VhHAA2B5sAwQmKdGO+ZhBezJ74BoSxcEn16Ea7TIcCfom1DCb8vzYBGwC+ALVECOho4I/ruJwFlHZdIpabkTKTqyO89Ow2YB3yfvyEhYbvZ3Te6+yLgAeCiqMoFwMPu/p27rwHuS9i3MSHhuN7dN7n7CuAhdr2XsiRWRb0v68zsxlJ/w+TudPct7j4LmEVIfiB8p3vdfa27ZxMSir35PCHOdcDwYupuB9qaWd3oHJ8XU3cIMNLdV7j7SuBOdl17gJ3A7e6+1d23EBLrM23Xez0vIrRvWRsCPOjuC909B7gZGBQNrW4nJGWt3D3P3f/t7hsS4m1vZrXdfam7z9kPsYlUWkrORKqOZ4GfEnpCnim07TCgJrA4oWwxcGS03BT4rtC2fM2BGsDShKTlL8Dh+xDbYe5eP/rcvw/7lcSyhOXNhN4f2PM7JS4XpUtCnPWB3xVTdyChR2uxmf3LzE4spm5T9rz2TRPWV7p7bv5K1Nv2ATDQzOoTkuNxyQ5sZpMThlD3dXgxWVzVgcaE36c3gQnRUOwoM6sR9Q5eSOhJW2pmr5lZm308r0iVpuRMpIpw98WEGwPOBF4ptHkVoSekeULZUezqXVtKGL5K3JbvO2AruydYdd29XVnGn0SqrzdZCqQnrDcrqmJpuPt0d+9PSFInAS/mb0pSfQl7XvvEYdpk+4wlDG3+D/CRu3+fpA7ufkbCEGrSBK4YyeLaASx39+3ufqe7tyUMXZ5FNK/R3d9099OAJoRe2if28bwiVZqSM5Gq5TKgd9S7UcDd8wjJwz1mdqiZNQd+xa55aS8C15pZupk1IGE4z92XAm8BD5hZ3WgS+Q/NrEcZx76cMO8pcb1R/gT1UngRuNnMGpjZkcDVqQaYz8xqWnhOWz133w5sAPKizcniHg/cYmY/sPDIk9vYc05gYZOALsB17NkTWho1o0n++Z+0KK4bzKylmR0C3Au84O47zKyXmXWI6m0gJPd5ZtbYzM6J5p5tBXISvruIlICSM5EqxN2/cffPith8DbAJWAhMA54Hnoq2PUEYwpoFfM6ePW8XE4ZF5wJrgZcJvSZl6Q5gbDR0eoG7zyMkDwujsqbF776HkUA2oTfxHULMW8sw3ouARWa2gTDE9zOAIuK+G/gM+A/wBeEa313cwaO5Z/8HtGTP9iiNOfz/9u4+zqr6vPf+5xJQQB60QdFIYDR6FCQIOtUqJBnlbtTYqKFWgyRU1NBYE7QmJjbmxErk3J5j0qqNp4Yk6smRW4+PDY0ay1EnRGM0IA8KJIUgKEJ8wCiMgDpy3X/sLR1khhmY2bMXw+f9es2L2eu31trX7IuRr7/1BBubfE2i1P//Dcym9DltovT3BEoXKdxDKZgtAX5BKVDuAXyV0qzb65QuLvnbDqhP2m344HNJAiLiIuBzmdnRM34VExHfBv5LZnb0laaSqsiZM0m7pYg4MCJGlw/DHk5ptuf+atfVVhHxJ5QOU0+vdi2SOpbhTNLuak9KV5WuBx4Ffgr8z6pW1EYR8UVKF2I8lJmzW1tf0q7Fw5qSJEkF4syZJElSgXSpB+gOGDAga2pqql3GLuOtt95i7733rnYZasKe5fr3tAAAIABJREFUFJN9KR57Ujz2ZMfNnTv3tczc74PLu1Q4q6mpYc6clu4SoA+qr6+nrq6u2mWoCXtSTPaleOxJ8diTHRcRK5tb7mFNSZKkAjGcSZIkFUjFwllE3BIRr0TEcy2M7xsR90fEwoh4OiKGNxn7u4hYFBHPRcQdEdGzUnVKkiQVSSXPObsN+D4tP/Ptm8D8zPxsRBwB3ASMLT/jbgowLDM3RsRdwOfK+5MkqUt69913WbVqFZs2bap2KTulf//+LFmypNplFFLPnj0ZNGgQPXr0aNP6FQtnmTk7Imq2s8ow4P8tr/vbiKiJiIFN6uoVEe8CvSk9o02SpC5r1apV9O3bl5qaGiKi2uXssPXr19O3b99ql1E4mcnatWtZtWoVBx98cJu2qebVmguAccDjEXEsMAQYlJlzI+K7wAuUHr7775n57y3tJCImA5MBBg4cSH19fcUL7yoaGhr8vArGnhSTfSmertiT/v3786EPfYiGhoZql7JT3nvvPdavX1/tMgppzz335I033mjz39lqhrNrgRsiYj7wLDAPaIyIfYEzgIOBN4C7I+LzmXl7czvJzOmUny1XW1ubXsbbdl72XDz2pFhmzJjBlVdeyQsvvMDgwYOZNm0aEyZMqHZZomv+rixZsoR+/fpVu4yd5szZ9vXs2ZNRo0a1ad2qhbPMXAdMAojS/O3z5a+Tgecz89Xy2H3ACUCz4UySKmHGjBlMnjyZDRs2ALBy5UomT54MYECTVFFVu5VGROwTEXuWX14IzC4HtheAP4uI3uXQNhbwDENJnerKK6/cEszet2HDBq688soqVSRtbcaMGdTU1LDHHntQU1PDjBkz2r3PPn36dEBl25eZnHTSSaxcuZKRI0cycuRIDjjgAA466KAtr995550272/SpEn87ne/2+46N910U7s+n7Fjx/Lmm2/u9PY7qmIzZxFxB1AHDIiIVcBVQA+AzLwZGAr8JCLeAxYDF5THnoqIe4BngEZKhzunV6pOSWrOCy+8sEPLpc60K8/sPvjggxx11FEMGTKE+fPnA/AP//AP9OnTh6997WvbrJ+ZZCZ77NH8fNKtt97a6ntefPHF7ar53HPP5eabb+Yb3/hGu/bTVpW8WnN8K+NPAoe1MHYVpTAnSVUxePBgVq7c9skqgwcPrkI12t1ceumlW4JLc37961/z9ttvb7Vsw4YNXHDBBfzwhz9sdpuRI0dy/fXX73AtK1eu5Pzzz+fVV19lv/3249Zbb2Xw4MHcfffdXH311XTr1o3+/fvzwAMPsGjRIiZNmsQ777zD5s2buffeeznssK3/qX8/WG7PsmXLOPPMMxkzZgxPPfUUP/vZz7j66qt55pln2LhxI+eccw7f/va3ARgzZgzf//73GT58OAMGDOBLX/oSDz30EL179+anP/0p+++/P9/61rcYMGAAl156KWPGjGHMmDE8+uijvPnmm9x6662ccMIJvPXWW0ycOJFly5YxbNgwli5dyo9+9CNGjhzJGWecwdixYzstnPmEAElqxrRp0+jdu/dWy3r37s20adOqVJH0nz4YzFpb3h5f/vKXmThxIgsXLmTChAlMmTIFgKlTp/Lwww+zYMECZs6cCcDNN9/MJZdcwvz585kzZw6DBg3aZn9PPPEExxxzTKvvu3jxYi644ALmzZvHQQcdxLXXXsucOXNYsGABs2bNYvHixdts8+abb/LJT36SBQsWcPzxx3PLLbc0u+/M5Omnn+a6665j6tSpAPzzP/8zBxxwAAsWLOCKK65g3rx5W9YfMGAA69ev54033mj9A+sAXerB55LUUd4/NOTVmqqG1ma4ampqmp3ZHTJkSIffYuTJJ5/kvvvuA+ALX/gCX//61wEYPXo05513HmeffTbjxo2jW7duHH/88UybNo1Vq1Yxbty4bWbNAF5//fU2XdX50Y9+lD/90z/d8vqOO+7gxz/+MY2NjaxevZrFixczbNiwrbbp1asXp556KgDHHHMMv/zlL5vd97hx47ass2LFCgAef/zxLTNjRx11FEceeeRW2+y3336sWbOGffbZp9Xa28uZM0lqwYQJE1ixYgWPPvooK1asMJipMKo5s/v+DXJvvvlmrrnmGl588UVGjhzJ2rVrOffcc5k5cya9evXi5JNP5tFHH91m++7du7N58+ZW32fvvffe8v3SpUu54YYbePTRR1m4cCGnnHJKs09S2HPPPbd8361bNxobG5vd91577bXNOpm53Xo2bdpEr169Wq27IxjOJEnaxUyYMIHp06czZMgQIoIhQ4Ywffr0ivwPxAknnMCdd94JlM4XGzNmDAC///3vOe6445g6dSoDBgzgpZdeYvny5RxyyCFMmTKF008/nYULF26zv8MPP5zly5fvUA3r1q2jb9++9OvXjzVr1vDwww+3/wf7gDFjxnDXXXcB8Oyzz2512HTz5s289tprfOQjH+nw922OhzUlSdoFTZgwocPD2IYNG7Y6T+yyyy7jxhtv5Pzzz+e6667bckEAwOWXX87SpUvJTMaOHcvHPvYxbrrpJm6//XZ69OjBAQccsOWk/aZOO+006uvrOfTQQ9tc19FHH82wYcMYPnw4hxxyCKNHj27/D/sBX/nKV5g4cSIjRozg6KOPZvjw4fTv3x+Ap59+mjFjxtCtW7cOf9/mRGvTeLuS2tranDNnTrXL2GV0xTts7+rsSTHZl+Lpij1ZsmQJQ4cOrXYZO62tTwhYs2YNEydOZNasWZ1QVds1NjbS2NhIz549Wbp0KZ/61KdYunQp3bt35+KLL+bss8/mk5/85E7vv7n+RsTczKz94LrOnEmSpE5z4IEH8sUvfpF169YV6nFVDQ0NjB07lsbGRjKTH/zgB3TvXopJo0aNalcw21GGM0mS1KnOPvvsapewjX322Ye5c+c2O3bhhRd2ai1eECBJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiTtYi6//HL+5m/+Zpuvyy+/vF377dOnTwdV2LLM5KSTTmLdunXU1dVtc0PZ66+/nr/927/d7j7er3P16tWcddZZza5TV1dHa7fXuv7669mwYcOW15/+9Kd3+vmZX/va15p9IsLOMJxJkrSLWbdu3Q4tL5IHH3yQo446in79+jF+/PgtTx9435133sn48ePbtK8Pf/jD3HPPPTtdywfD2YMPPrjTz878yle+wrXXXrvTtTTlrTQkSSqg733ve9ssO+aYY9p0892GhgZ+8IMfbLXsq1/96k7VsXLlSs4//3xeffXVLU8IGDx4MHfffTdXX3013bp1o3///jzwwAMsWrSISZMm8c4777B582buvffebR5+PmPGDCZPngzAWWedxbe+9S3efvtt9tprL1asWMHq1asZM2YMDQ0NnHHGGfzxj3/k3Xff5ZprruGMM87Yal8rVqzgL/7iL3juuefYuHEjkyZNYvHixQwdOpSNGzduWe+iiy7iN7/5DRs3buSss87i6quv5sYbb2T16tWceOKJDBgwgMcee4yamhrmzJnDgAED+Md//EduueUWoHQrjUsvvZQVK1Zw6qmnMmbMGH71q19x0EEH8dOf/pRevXoxZMgQ1q5dyx/+8AcOOOCAnfqs3+fMmSRJatGXv/xlJk6cyMKFC5kwYQJTpkwBYOrUqTz88MMsWLCAmTNnAqWHoV9yySXMnz+fOXPmbPUoqPc98cQTHHPMMQB86EMf4thjj+XnP/85UJo1O+ecc4gIevbsyf33388zzzzDY489xle/+tXtPpz8X/7lX+jduzcLFy7kyiuv3OqeZdOmTWPOnDksXLiQX/ziFyxcuJApU6bw4Q9/mMcee4zHHntsq33NnTuXW2+9laeeeopf//rX/PCHP2TevHlA6SHsF198MYsWLWKfffbh3nvv3bLd0UcfzRNPPLEzH/NWnDmTJKmAdnamC0rnZLVn+6aefPJJ7rvvPgC+8IUv8PWvfx2A0aNHc95553H22Wczbtw4unXrxvHHH8+0adNYtWoV48aN22bWDOD111/f6jFP7x/aPOOMM7jzzju3zFZlJt/85jeZPXs2e+yxBy+99BIvv/xyi7NSs2fP3hIcR4wYwYgRI7aM3XXXXUyfPp3GxkbWrFnD4sWLtxr/oMcff5zPfvaz7L333gCMGzeOX/7yl5x++ukcfPDBjBw5EijNZK5YsWLLdvvvvz+rV69u9TNtjTNnkiSpzSICKM2SXXPNNbz44ouMHDmStWvXcu655zJz5kx69erFySef3OwJ8t27d2fz5s1bXp955pk88sgjPPPMM2zcuJGjjz4aKB3+fPXVV5k7dy7z589n4MCBbNq0qU21NfX888/z3e9+l0ceeYSFCxdy2mmntbqf7c3Q7bXXXlu+79atG42NjVteb9q0iV69em13321hOJMkaRfT0jMpK/GsyhNOOGHLSfszZsxgzJgxAPz+97/nuOOOY+rUqQwYMICXXnqJ5cuXc8ghhzBlyhROP/10Fi5cuM3+Dj/8cJYvX77ldZ8+fairq+P888/f6kKAN998k/33358ePXrw2GOPsXLlyu3W+YlPfIIZM2YA8Nxzz21573Xr1rH33nvTv39/Xn75ZR566KEt2/Tt25f169c3u69//dd/ZcOGDbz11lvcf//9fPzjH2/1s/qP//gPhg8f3up6rfGwpiRJu5jrrruuIvvdsGHDVueJXXbZZdx4442cf/75XHfddVsuCIDS7TyWLl1KZjJ27Fg+9rGPcdNNN3H77bfTo0cPDjjgAL797W9v8x6nnXYa9fX1HHrooVuWjR8/nnHjxm115eaECRP4zGc+Q21tLSNHjuSII47Ybu0XXXQRkyZNYsSIEYwcOZJjjz0WgKOOOopRo0Zx5JFHcsghhzB69Ogt20yePJlTTz2VAw88cKvzzo4++mjOO++8Lfu48MILGTVq1FaHMD/o3XffZdmyZdTW1m63zraI7U3d7Wpqa2uztXua6D/V19e36aofdR57Ukz2pXi6Yk+WLFnC0KFDq13GTlu/fv1W55K1ZM2aNUycOJFZs2Z1QlWd5/2LF77zne80O95cfyNibmZuk+Y8rClJkjrNgQceyBe/+MVd4p5sO6KxsbHDLsLwsKYkSepUZ599drVL6HB/9Vd/1WH7cuZMkqSC6EqnGuk/7WhfDWeSJBVAz549Wbt2rQGti8lM1q5dS8+ePdu8jYc1JUkqgEGDBrFq1SpeffXVapeyUzZt2rRDAWR30rNnz2afltASw5kkSQXQo0cPDj744GqXsdPq6+sZNWpUtcvoEjysKUmSVCCGM0mSpAKpWDiLiFsi4pWIeK6F8X0j4v6IWBgRT0fE8CZj+0TEPRHx24hYEhHHV6pOSZKkIqnkzNltwCnbGf8mMD8zRwATgRuajN0A/DwzjwCOApZUqkhJkqQiqVg4y8zZwOvbWWUY8Eh53d8CNRExMCL6AZ8Aflweeycz36hUnZIkSUVS0WdrRkQN8LPM3OYR7RHx34CemXlZRBwL/Ao4DngPmA4spjRrNhe4JDPfauE9JgOTAQYOHHhM04emavsaGhro06dPtctQE/akmOxL8diT4rEnO+7EE09s9tma1Qxn/SgdvhwFPAscAVwI9AB+DYzOzKci4gZgXWb+19bezwef75iu+ODgXZ09KSb7Ujz2pHjsyY5r6cHnVbvPWWauAyYBREQAz5e/egOrMvOp8qr3AFdUpUhJkqROVrVbaZSvyNyz/PJCYHZmrsvMPwAvRsTh5bGxlA5xSpIkdXkVmzmLiDuAOmBARKwCrqJ0yJLMvBkYCvwkIt6jFL4uaLL5V4AZ5fC2nPIMmyRJUldXsXCWmeNbGX8SOKyFsfnANsdgJUmSujqfECBJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKpCKhbOIuCUiXomI51oY3zci7o+IhRHxdEQM/8B4t4iYFxE/q1SNkiRJRVPJmbPbgFO2M/5NYH5mjgAmAjd8YPwSYEllSpMkSSqmioWzzJwNvL6dVYYBj5TX/S1QExEDASJiEHAa8KNK1SdJklRE3av43guAccDjEXEsMAQYBLwMXA98Hejb2k4iYjIwGWDgwIHU19dXqt4up6Ghwc+rYOxJMdmX4rEnxWNPOk41w9m1wA0RMR94FpgHNEbEXwCvZObciKhrbSeZOR2YDlBbW5t1da1uorL6+nr8vIrFnhSTfSkee1I89qTjVC2cZeY6YBJARATwfPnrc8DpEfFpoCfQLyJuz8zPV6tWSZKkzlK1W2lExD4RsWf55YXA7Mxcl5l/n5mDMrOGUlB71GAmSZJ2FxWbOYuIO4A6YEBErAKuAnoAZObNwFDgJxHxHrAYuKBStUiSJO0qKhbOMnN8K+NPAoe1sk49UN9xVUmSJBWbTwiQJEkqEMOZJElSgRjOJEmSCsRwJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTJEkqEMOZJElSgRjOJEmSCsRwJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTJEkqEMOZJElSgRjOJEmSCsRwJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTJEkqkDaFs4j4aETsVf6+LiKmRMQ+lS1NkiRp99PWmbN7gfci4lDgx8DBwP9XsaokSZJ2U20NZ5szsxH4LHB9Zv4dcGDlypIkSdo9tTWcvRsR44G/Bn5WXtajMiVJkiTtvtoaziYBxwPTMvP5iDgYuH17G0TELRHxSkQ818L4vhFxf0QsjIinI2J4eflHIuKxiFgSEYsi4pId+YEkSZJ2ZW0KZ5m5ODOnZOYdEbEv0Dczr21ls9uAU7Yz/k1gfmaOACYCN5SXNwJfzcyhwJ8BF0fEsLbUKUmStKtr69Wa9RHRLyL+BFgA3BoR/7i9bTJzNvD6dlYZBjxSXve3QE1EDMzMNZn5THn5emAJcFBb6pQkSdrVdW/jev0zc11EXAjcmplXRcTCdr73AmAc8HhEHAsMAQYBL7+/QkTUAKOAp1raSURMBiYDDBw4kPr6+naWtftoaGjw8yoYe1JM9qV47Enx2JOO09Zw1j0iDgTOBq7soPe+FrghIuYDzwLzKB3SBCAi+lC6hcelmbmupZ1k5nRgOkBtbW3W1dV1UHldX319PX5exWJPism+FI89KR570nHaGs6mAg8DT2TmbyLiEGBpe964HLgmAUREAM+Xv4iIHpSC2YzMvK897yNJkrQraVM4y8y7gbubvF4O/GV73rj8hIENmfkOcCEwu3zoNCjd6HZJZm73vDZJkqSupq0XBAwq3/bilYh4OSLujYhBrWxzB/AkcHhErIqICyLiSxHxpfIqQ4FFEfFb4FTg/VtmjAa+AJwUEfPLX5/eqZ9OkiRpF9PWw5q3Unpc01+VX3++vOzPW9ogM8dvb4eZ+SRwWDPLHweijXVJkiR1KW29Ce1+mXlrZjaWv24D9qtgXZIkSbultoaz1yLi8xHRrfz1eWBtJQuTJEnaHbU1nJ1P6TYafwDWAGdRvtJSkiRJHaetj296ITNPz8z9MnP/zDyT0g1kJUmS1IHaOnPWnMs6rApJkiQB7QtnXlEpSZLUwdoTzrLDqpAkSRLQyn3OImI9zYewAHpVpCJJkqTd2HbDWWb27axCJEmS1L7DmpIkSepghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBVKxcBYRt0TEKxHxXAvj+0bE/RGxMCKejojhTcZOiYjfRcSyiLiiUjVKkiQVTSVnzm4DTtnO+DeB+Zk5ApgI3AAQEd2Am4BTgWHA+IgYVsE6JUmSCqNi4SwzZwOvb2eVYcAj5XV/C9RExEDgWGBZZi7PzHeAO4EzKlWnJElSkXSv4nsvAMYBj0fEscAQYBBwEPBik/VWAce1tJOImAxMBhg4cCD19fWVqrfLaWho8PMqGHtSTPaleOxJ8diTjlPNcHYtcENEzAeeBeYBjUA0s262tJPMnA5MB6itrc26urqOr7SLqq+vx8+rWOxJMdmX4rEnxWNPOk7VwllmrgMmAUREAM+Xv3oDH2my6iBgdacXKEmSVAVVu5VGROwTEXuWX14IzC4Htt8Ah0XEweXxzwEzq1WnJElSZ6rYzFlE3AHUAQMiYhVwFdADIDNvBoYCP4mI94DFwAXlscaI+DLwMNANuCUzF1WqTkmSpCKpWDjLzPGtjD8JHNbC2IPAg5WoS5Ikqch8QoAkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSoQw5kkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSoQw5kkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSoQw5kkSVKBGM4kSZIKxHAmSZJUIIYzSZKkAjGcSZIkFYjhTJIkqUAqGs4i4paIeCUinmthvH9E/FtELIiIRRExqcnY/ygvWxIRN0ZEVLJWSZKkIqj0zNltwCnbGb8YWJyZRwF1wPciYs+IOAEYDYwAhgN/CnyysqVKkiRVX0XDWWbOBl7f3ipA3/KsWJ/yuo3l5T2BPYG9gB7Ay5WsVZIkqQgiMyv7BhE1wM8yc3gzY32BmcARQF/gnMx8oDz2XeBCIIDvZ+aVLex/MjAZYODAgcfceeedFfgpuqaGhgb69OlT7TLUhD0pJvtSPPakeOzJjjvxxBPnZmbtB5d3r0YxTZwMzAdOAj4KzIqIXwL7A0OBQeX1ZkXEJ8ozcVvJzOnAdIDa2tqsq6vrjLq7hPr6evy8isWeFJN9KR57Ujz2pONU+2rNScB9WbIMeJ7SLNpngV9nZkNmNgAPAX9WxTolSZI6RbXD2QvAWICIGAgcDiwvL/9kRHSPiB6ULgZYUrUqJUmSOklFD2tGxB2UrsIcEBGrgKsondxPZt4MfAe4LSKepXRu2Tcy87WIuIfSoc5nKV0c8PPM/LdK1ipJklQEFQ1nmTm+lfHVwKeaWf4e8DeVqkuSJKmoqn1YU5IkSU0YziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQghjNJkqQCMZxJkiQViOFMkiSpQAxnkiRJBWI4kyRJKhDDmSRJUoEYziRJkgrEcCZJklQg3atdgCQV0eWXX866deu2vL7jjjsA6NevH9ddd121ypK0G3DmTJKa0TSYtWW5JHWUioWziLglIl6JiOdaGO8fEf8WEQsiYlFETGoyNjgi/j0ilkTE4oioqVSdkiRJRVLJmbPbgFO2M34xsDgzjwLqgO9FxJ7lsZ8A12XmUOBY4JUK1ilJklQYFQtnmTkbeH17qwB9IyKAPuV1GyNiGNA9M2eV99OQmRsqVackSVKRVPOCgO8DM4HVQF/gnMzcHBH/BXgjIu4DDgb+L3BFZr7X3E4iYjIwGWDgwIHU19d3Ru1dQkNDg59XwdiTXYM9qj5/V4rHnnScaoazk4H5wEnAR4FZEfHLck0fB0YBLwD/BzgP+HFzO8nM6cB0gNra2qyrq6t03V1GfX09fl7FYk+K44EHHmj25P9+/frZowLwd6V47EnHqWY4mwRcm5kJLIuI54EjgFXAvMxcDhAR/wr8GS2EM0mqhKa3y/AfHUmdqZq30ngBGAsQEQOBw4HlwG+AfSNiv/J6JwGLq1KhJElSJ6vYzFlE3EHpKswBEbEKuAroAZCZNwPfAW6LiGeBAL6Rma+Vt/0a8Ej5YoG5wA8rVackSVKRVCycZeb4VsZXA59qYWwWMKISdUmSJBWZTwiQJEkqEMOZJElSgRjOJEmSCiRKd7LoGiLiVWBltevYhQwAXqt2EdqKPSkm+1I89qR47MmOG5KZ+31wYZcKZ9oxETEnM2urXYf+kz0pJvtSPPakeOxJx/GwpiRJUoEYziRJkgrEcLZ7m17tArQNe1JM9qV47Enx2JMO4jlnkiRJBeLMmSRJUoEYziRJkgrEcNbFRcSfRMSsiFha/nPfFtb76/I6SyPir5sZnxkRz1W+4q6vPT2JiN4R8UBE/DYiFkXEtZ1bfdcSEadExO8iYllEXNHM+F4R8X/K409FRE2Tsb8vL/9dRJzcmXV3dTvbl4j484iYGxHPlv88qbNr76ra87tSHh8cEQ0R8bXOqnlXZjjr+q4AHsnMw4BHyq+3EhF/AlwFHAccC1zVNDBExDigoXPK3S20tyffzcwjgFHA6Ig4tXPK7loiohtwE3AqMAwYHxHDPrDaBcAfM/NQ4J+A/17edhjwOeBI4BTgf5b3p3ZqT18o3QD1M5n5MeCvgf/dOVV3be3syfv+CXio0rV2FYazru8M4H+Vv/9fwJnNrHMyMCszX8/MPwKzKP2DQ0T0AS4DrumEWncXO92TzNyQmY8BZOY7wDPAoE6ouSs6FliWmcvLn+WdlHrTVNNe3QOMjYgoL78zM9/OzOeBZeX9qf12ui+ZOS8zV5dHot9qAAAEJ0lEQVSXLwJ6RsRenVJ119ae3xUi4kxgOaWeqA0MZ13fwMxcA1D+c/9m1jkIeLHJ61XlZQDfAb4HbKhkkbuZ9vYEgIjYB/gMpdk37bhWP+Om62RmI/Am8KE2bqud056+NPWXwLzMfLtCde5OdronEbE38A3g6k6os8voXu0C1H4R8X+BA5oZurKtu2hmWUbESODQzPy7D54/oO2rVE+a7L87cAdwY2Yu3/EKRSufcSvrtGVb7Zz29KU0GHEkpcNqn+rAunZn7enJ1cA/ZWZDeSJNbWA46wIy8/9paSwiXo6IAzNzTUQcCLzSzGqrgLomrwcB9cDxwDERsYLS35X9I6I+M+vQdlWwJ++bDizNzOs7oNzd1SrgI01eDwJWt7DOqnIg7g+83sZttXPa0xciYhBwPzAxM39f+XJ3C+3pyXHAWRHxP4B9gM0RsSkzv1/5snddHtbs+mZSOjGW8p8/bWadh4FPRcS+5ZPOPwU8nJn/kpkfzswaYAzwHwazDrHTPQGIiGso/Yfv0k6otSv7DXBYRBwcEXtSOsF/5gfWadqrs4BHs3Tn7pnA58pXqB0MHAY83Ul1d3U73Zfyof4HgL/PzCc6reKub6d7kpkfz8ya8r8j1wP/zWDWOsNZ13ct8OcRsRT48/JrIqI2In4EkJmvUzq37Dflr6nlZaqMne5JeVbgSkpXTD0TEfMj4sJq/BC7uvJ5MV+mFHqXAHdl5qKImBoRp5dX+zGl82aWUbow5orytouAu4DFwM+BizPzvc7+Gbqi9vSlvN2hwH8t/27Mj4jmzunUDmhnT7QTfHyTJElSgThzJkmSVCCGM0mSpAIxnEmSJBWI4UySJKlADGeSJEkFYjiTtFuIiPea3F5hfkR02KX+EVETEc911P4k7d58QoCk3cXGzBxZ7SIkqTXOnEnarUXEioj47xHxdPnr0PLyIRHxSEQsLP85uLx8YETcHxELyl8nlHfVLSJ+GBGLIuLfI6JX1X4oSbs0w5mk3UWvDxzWPKfJ2LrMPBb4PqVHzFD+/ieZOQKYAdxYXn4j8IvMPAo4GlhUXn4YcFNmHgm8AfxlhX8eSV2UTwiQtFuIiIbM7NPM8hXASZm5PCJ6AH/IzA9FxGvAgZn5bnn5mswcEBGvAoMy8+0m+6gBZmXmYeXX3wB6ZOY1lf/JJHU1zpxJEmQL37e0TnPebvL9e3hOr6SdZDiTJDinyZ9Plr//FfC58vcTgMfL3z8CXAQQEd0iol9nFSlp9+D/2UnaXfSKiPlNXv88M9+/ncZeEfEUpf9hHV9eNgW4JSIuB14FJpWXXwJMj4gLKM2QXQSsqXj1knYbnnMmabdWPuesNjNfq3YtkgQe1pQkSSoUZ84kSZIKxJkzSZKkAjGcSZIkFYjhTJIkqUAMZ5IkSQViOJMkSSqQ/x/MyqlXQRsfdgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot Epoch history\n",
    "fig1 = plt.figure(figsize=(10,10))\n",
    "\n",
    "# Create a single plot of all results\n",
    "ax1 = fig1.add_subplot( 2,1,1 )\n",
    "ax2 = fig1.add_subplot( 2,1,2 )\n",
    "\n",
    "# X-axis\n",
    "x_vals = range(len(hist.history['acc']))\n",
    "\n",
    "# Plots\n",
    "ax1.set_ylim( ymin=0.9*min(hist.history['acc'] + hist.history['val_acc']), ymax=1.1)\n",
    "\n",
    "ax1.plot( x_vals, hist.history['acc'], label='Accuracy (Training)', c='k', marker='o', linestyle='-')\n",
    "ax1.plot( x_vals, hist.history['val_acc'], label='Accuracy (Validation)', c='0.4', marker='s', linestyle='--')\n",
    "\n",
    "ax2.plot( x_vals, hist.history['loss'], label='Loss (Training)', c='k', marker='o', linestyle='-')\n",
    "ax2.plot( x_vals, hist.history['val_loss'], label='Loss (Validation)', c='0.4', marker='s', linestyle='--')\n",
    "\n",
    "# Add text note on the max and min accuracy and loss points\n",
    "acc_min_idx = np.argmin(hist.history['acc'])\n",
    "ax1.text( x=acc_min_idx, y=hist.history['acc'][acc_min_idx]*1.02, c='b',\n",
    "          s=f\"Min: {hist.history['acc'][acc_min_idx]:.4f}\\nEpoch: {acc_min_idx}\" )\n",
    "\n",
    "acc_max_idx = np.argmax(hist.history['acc'])\n",
    "ax1.text( x=acc_max_idx, y=hist.history['acc'][acc_max_idx]*1.02, c='b',\n",
    "          s=f\"Max: {hist.history['acc'][acc_max_idx]:.4f}\\nEpoch: {acc_max_idx}\" )\n",
    "\n",
    "loss_min_idx = np.argmin(hist.history['loss'])\n",
    "ax2.text( x=loss_min_idx, y=hist.history['loss'][loss_min_idx]*1.02, c='b',\n",
    "          s=f\"Min: {hist.history['loss'][loss_min_idx]:.4f}\\nEpoch: {loss_min_idx}\" )\n",
    "         \n",
    "loss_max_idx = np.argmax(hist.history['loss'])\n",
    "ax2.text( x=loss_max_idx, y=hist.history['loss'][loss_max_idx]*1.02, c='b',\n",
    "          s=f\"Max: {hist.history['loss'][loss_max_idx]:.4f}\\nEpoch: {loss_max_idx}\" )\n",
    "\n",
    "# If populated, plot the loss and accuracy from the test samples\n",
    "try:\n",
    "    loss_val = m_score_list[0]\n",
    "    acc_val = m_score_list[1]\n",
    "\n",
    "    ax1.text( x=min(x_vals), y=acc_val*1.02, c='darkgreen', s=f\"{acc_val:.4f}\")\n",
    "    ax1.hlines(y=acc_val, xmin=min(x_vals), xmax=max(x_vals),\n",
    "              label='Accuracy (Testing)', color='g', linewidth=1, linestyle=':')    \n",
    "    \n",
    "    ax2.text( x=min(x_vals), y=loss_val*0.97, c='darkgreen', s=f\"{loss_val:.4f}\")\n",
    "    ax2.hlines(y=loss_val, xmin=min(x_vals), xmax=max(x_vals),\n",
    "              label='Loss (Testing)', color='g', linewidth=1, linestyle=':')    \n",
    "except:\n",
    "    pass\n",
    "    \n",
    "ax1.legend(loc='lower right')\n",
    "ax1.set_xlabel(\"Epoch\")\n",
    "ax1.set_ylabel(\"Accuracy\")\n",
    "ax1.set_title(\"Model Fitting History - Accuracy\")\n",
    "ax1.grid(True)\n",
    "\n",
    "ax2.legend(loc='upper right')\n",
    "ax2.set_xlabel(\"Epoch\")\n",
    "ax2.set_ylabel(\"Loss\")\n",
    "ax2.set_title(\"Model Fitting History - Loss\")\n",
    "ax2.grid(True)\n",
    "         \n",
    "# Save the image\n",
    "fig1.savefig(\"docs/Figure-DPA_Fit_History.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configure the Image Generator for the Official Test Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using Testing Folders and Files:\n",
      "\tTesting Labels File: './data/test.csv'\n",
      "\tTesting Image Directory: './data/test_images/'\n",
      "\tTemporary Augmented Image Directory: './data/temp_aug_images/'\n",
      "\tFlag to Save Augmented Images: False\n"
     ]
    }
   ],
   "source": [
    "# Summary\n",
    "print(f\"Using Testing Folders and Files:\")\n",
    "print(f\"\\tTesting Labels File: '{test_labels_file}'\")\n",
    "print(f\"\\tTesting Image Directory: '{dir_test_src}'\")\n",
    "print(f\"\\tTemporary Augmented Image Directory: '{dir_temp_aug_images}'\")\n",
    "print(f\"\\tFlag to Save Augmented Images: {flag_save_aug}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1928"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load test counts into a dataframe\n",
    "test_image_list_df = pd.read_csv(test_labels_file)\n",
    "test_image_list_df['id_code_fn'] = test_image_list_df['id_code'] + \".png\"\n",
    "len(test_image_list_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 1928 validated image filenames.\n"
     ]
    }
   ],
   "source": [
    "# Create a Image Data Generator for the official testing data\n",
    "official_test_datagen = ImageDataGenerator()\n",
    "\n",
    "# Define generator batch size and step size\n",
    "official_test_n_images = len(test_tally_df)\n",
    "official_test_n_batch_size = 1\n",
    "official_test_n_step_size = official_test_n_images // official_test_n_batch_size\n",
    "\n",
    "# Associate the generator with the training images\n",
    "official_test_generator = official_test_datagen.flow_from_dataframe(\n",
    "    dataframe=test_image_list_df,                    # Dataframe with filenames of images and column for labels\n",
    "    directory=dir_test_src,                     # Directory containing images used for training, validation, and testing\n",
    "    x_col='id_code_fn',                         # Column containing filenames of images in directory dir_train_src\n",
    "    target_size=(image_dims[1], image_dims[0]), # The (height, width) to which each image will be resized\n",
    "    color_mode='rgb',                           # Images will be color ('rgb') (3 color channels) vs. 'grayscale'\n",
    "    batch_size=official_test_n_batch_size,      # Batch size\n",
    "    class_mode=None,                            # Performing prediction only, no labels provided\n",
    "    shuffle=False,                              # Don't shuffle the input samples\n",
    "    seed=1,                                     # Random number seed - only needed if shuffling\n",
    "    save_to_dir=dir_temp_aug_images if flag_save_aug else None, # Directory to save aug images if flag is True\n",
    "    save_prefix='official_test_aug_',           # Prefix to apply to any saved images\n",
    "    save_format='png',                          # File format to appy to any saved images\n",
    "    interpolation='bicubic'                     # Interpolation method used to resize samples\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict the Classifications based upon the Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1928/1928 [==============================] - 132s 68ms/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([2, 0, 0, ..., 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict the output (using the test_generator)\n",
    "official_test_generator.reset()\n",
    "official_pred_onehot = model.predict_generator(official_test_generator,\n",
    "                             steps=len(official_test_generator),\n",
    "                             verbose=1)\n",
    "\n",
    "official_predicted_class = np.argmax(official_pred_onehot, axis=1)\n",
    "official_predicted_class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the Predictions from the Official Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id_code</th>\n",
       "      <th>diagnosis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0005cfc8afb6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>003f0afdcd15</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>006efc72b638</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>00836aaacf06</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>009245722fa4</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1923</td>\n",
       "      <td>ff2fd94448de</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1924</td>\n",
       "      <td>ff4c945d9b17</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1925</td>\n",
       "      <td>ff64897ac0d8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1926</td>\n",
       "      <td>ffa73465b705</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1927</td>\n",
       "      <td>ffdc2152d455</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1928 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           id_code  diagnosis\n",
       "0     0005cfc8afb6          2\n",
       "1     003f0afdcd15          0\n",
       "2     006efc72b638          0\n",
       "3     00836aaacf06          2\n",
       "4     009245722fa4          0\n",
       "...            ...        ...\n",
       "1923  ff2fd94448de          0\n",
       "1924  ff4c945d9b17          0\n",
       "1925  ff64897ac0d8          0\n",
       "1926  ffa73465b705          0\n",
       "1927  ffdc2152d455          0\n",
       "\n",
       "[1928 rows x 2 columns]"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save the prediction results to a file\n",
    "\n",
    "# Get the list of test image id codes used by the test generator to make the predictions\n",
    "official_test_id_code_list = [ x.replace(\".png\",\"\") for x in official_test_generator.filenames ]\n",
    "\n",
    "# Associate the test image id codes and the preoductions in a dataframe\n",
    "official_results_df = pd.DataFrame({\"id_code\": official_test_id_code_list, \"diagnosis\": official_predicted_class})\n",
    "official_results_df.to_csv(\"./submission.csv\",index=False)\n",
    "official_results_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "nbpresent": {
   "slides": {
    "03210a56-863e-4749-b7ba-ed75bfceceee": {
     "id": "03210a56-863e-4749-b7ba-ed75bfceceee",
     "prev": "86b3b05f-6e9a-49dc-8a83-97f72c348c5f",
     "regions": {
      "2be9481c-cff7-4f32-b835-90f2a2cb989a": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "46550b81-7dd8-4efc-b710-0f10002c9f2b",
        "part": "whole"
       },
       "id": "2be9481c-cff7-4f32-b835-90f2a2cb989a"
      },
      "3f484569-30ec-4529-8d3c-ac88b8c6dbfb": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "2b83c32a-d7e2-4db6-b45f-afbbf028fe90",
        "part": "whole"
       },
       "id": "3f484569-30ec-4529-8d3c-ac88b8c6dbfb"
      }
     }
    },
    "22531930-3fc8-45ff-a4f1-32e94c1d1455": {
     "id": "22531930-3fc8-45ff-a4f1-32e94c1d1455",
     "prev": "8eb735f6-11af-4f77-a4ed-b637fb18ac08",
     "regions": {
      "1407c818-c9fd-470e-8260-6b6904d888de": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "664c64ef-5517-4725-95cb-874bbc7711c7",
        "part": "whole"
       },
       "id": "1407c818-c9fd-470e-8260-6b6904d888de"
      }
     }
    },
    "29c55765-6994-4dbf-b82f-117b4f1e0cee": {
     "id": "29c55765-6994-4dbf-b82f-117b4f1e0cee",
     "prev": "b7c25bc1-4dd7-421b-9951-0c0c403c176d",
     "regions": {
      "44583f15-ec4f-4aec-8e7c-f01652e954fc": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "7c23961d-86fe-4d07-bc58-6d61f03c3731",
        "part": "whole"
       },
       "id": "44583f15-ec4f-4aec-8e7c-f01652e954fc"
      }
     }
    },
    "39c5764b-ab2c-4aae-8500-dc39319e4d38": {
     "id": "39c5764b-ab2c-4aae-8500-dc39319e4d38",
     "prev": "22531930-3fc8-45ff-a4f1-32e94c1d1455",
     "regions": {
      "d0237596-8095-44dc-81c6-6d9c6b204684": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "348e164e-a000-4def-92c0-db93f9a617a1",
        "part": "whole"
       },
       "id": "d0237596-8095-44dc-81c6-6d9c6b204684"
      }
     }
    },
    "39e950f1-f2c4-422a-8f45-835c5c61e7e0": {
     "id": "39e950f1-f2c4-422a-8f45-835c5c61e7e0",
     "prev": "4b5f1824-db4a-4ef3-9515-a88d3f2276c1",
     "regions": {
      "0a72cc48-c38f-4e8c-94d3-3198eccde68a": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "c3820b22-bf55-4dca-ba8b-f08060567b6e",
        "part": "whole"
       },
       "id": "0a72cc48-c38f-4e8c-94d3-3198eccde68a"
      },
      "b49c6eca-318f-41b8-a7a5-a620d18f639c": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "aeeeb44d-7825-4050-b7ba-70813bd3396b",
        "part": "whole"
       },
       "id": "b49c6eca-318f-41b8-a7a5-a620d18f639c"
      }
     }
    },
    "4792c0cc-ccd4-48bf-9b0e-a3c1f0f3008d": {
     "id": "4792c0cc-ccd4-48bf-9b0e-a3c1f0f3008d",
     "prev": "39c5764b-ab2c-4aae-8500-dc39319e4d38",
     "regions": {
      "d4adae26-ff2f-4a81-ad91-fd3a9bfd965c": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "4d6b8af6-438e-4034-bda1-1f977cf12441",
        "part": "whole"
       },
       "id": "d4adae26-ff2f-4a81-ad91-fd3a9bfd965c"
      },
      "d75a8261-ee89-4f4d-b388-2f429e112f44": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "a3b6e99c-2c85-47da-b29b-3b53c3c309b8",
        "part": "whole"
       },
       "id": "d75a8261-ee89-4f4d-b388-2f429e112f44"
      }
     }
    },
    "4b5f1824-db4a-4ef3-9515-a88d3f2276c1": {
     "id": "4b5f1824-db4a-4ef3-9515-a88d3f2276c1",
     "prev": "e92a9020-6631-400a-aa50-46c1fa0a5f0c",
     "regions": {
      "2bd81133-11e3-48df-bf12-2d0a57949bca": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "69fab88f-0864-47cc-933b-eaffb638cbf3",
        "part": "whole"
       },
       "id": "2bd81133-11e3-48df-bf12-2d0a57949bca"
      },
      "d3322861-28ab-4f1e-8a1f-3a7f4d3da8c1": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "3810c2af-fdbe-4f01-bc9f-2bb1a8ae70c6",
        "part": "whole"
       },
       "id": "d3322861-28ab-4f1e-8a1f-3a7f4d3da8c1"
      }
     }
    },
    "57992fb9-2f74-4e8e-8772-7b9f04606a56": {
     "id": "57992fb9-2f74-4e8e-8772-7b9f04606a56",
     "prev": "595093b4-3f86-406c-abb5-aea78e1241b3",
     "regions": {
      "3b3c8191-9f7e-4ab2-afac-b2b47008484a": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "a6d0bdff-9bba-472e-b354-da47b8761dbe",
        "part": "whole"
       },
       "id": "3b3c8191-9f7e-4ab2-afac-b2b47008484a"
      }
     }
    },
    "5907f515-df93-4e86-82c6-b8721b6e6346": {
     "id": "5907f515-df93-4e86-82c6-b8721b6e6346",
     "prev": "39e950f1-f2c4-422a-8f45-835c5c61e7e0",
     "regions": {
      "22f9005a-ed7a-4399-b501-497565ada2da": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "2e255efc-6ae1-4554-b32f-9400b47007c9",
        "part": "whole"
       },
       "id": "22f9005a-ed7a-4399-b501-497565ada2da"
      },
      "82d64cef-48f7-413c-ad01-e7fa2921f7c0": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "711a40c8-c479-448b-a3bc-5c7ff8369838",
        "part": "whole"
       },
       "id": "82d64cef-48f7-413c-ad01-e7fa2921f7c0"
      },
      "d943d3ea-6cd3-4d4a-8914-7fea0231efda": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "9bfbb27b-cae0-4feb-809a-77718bb2e6a3",
        "part": "whole"
       },
       "id": "d943d3ea-6cd3-4d4a-8914-7fea0231efda"
      }
     }
    },
    "595093b4-3f86-406c-abb5-aea78e1241b3": {
     "id": "595093b4-3f86-406c-abb5-aea78e1241b3",
     "prev": "5907f515-df93-4e86-82c6-b8721b6e6346",
     "regions": {
      "3366a1dc-6c33-45f0-a97f-a2744ed0587f": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "e54b79a1-316d-4fd1-ad5f-6cd580b51f99",
        "part": "whole"
       },
       "id": "3366a1dc-6c33-45f0-a97f-a2744ed0587f"
      },
      "40b6dba8-5994-4fd0-9ce5-9d36f3bb9f93": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "3d860568-b048-40f2-a64d-f585211d6c1c",
        "part": "whole"
       },
       "id": "40b6dba8-5994-4fd0-9ce5-9d36f3bb9f93"
      }
     }
    },
    "76838eab-c2c3-4ff3-b09f-854a7c74c6d4": {
     "id": "76838eab-c2c3-4ff3-b09f-854a7c74c6d4",
     "prev": null,
     "regions": {
      "b0032c00-1d89-4c94-a7a0-488f243c381e": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "0dc0cdcb-d8df-4a29-a0d2-a3cf6cc889ce",
        "part": "whole"
       },
       "id": "b0032c00-1d89-4c94-a7a0-488f243c381e"
      }
     }
    },
    "7a25817a-e5bc-4021-9c6b-61bc74dbdb92": {
     "id": "7a25817a-e5bc-4021-9c6b-61bc74dbdb92",
     "prev": "b04719fc-934a-404b-a642-dc732a3d1589",
     "regions": {
      "38d56409-500a-45cf-8e79-722d68d8fc38": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "335f050d-99af-43b3-a5ec-69fe896e48e2",
        "part": "whole"
       },
       "id": "38d56409-500a-45cf-8e79-722d68d8fc38"
      },
      "c838cb6d-409c-4c17-a1eb-3d7dc6aab32b": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "19ca6a7e-4b5d-4b0f-88c5-e68634abb65d",
        "part": "whole"
       },
       "id": "c838cb6d-409c-4c17-a1eb-3d7dc6aab32b"
      }
     }
    },
    "86b3b05f-6e9a-49dc-8a83-97f72c348c5f": {
     "id": "86b3b05f-6e9a-49dc-8a83-97f72c348c5f",
     "prev": "57992fb9-2f74-4e8e-8772-7b9f04606a56",
     "regions": {
      "49a41ef6-224d-46c1-bb81-254b8a4cee88": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "d7df6fa0-ce96-4073-b95d-ce78b3f4fdf1",
        "part": "whole"
       },
       "id": "49a41ef6-224d-46c1-bb81-254b8a4cee88"
      },
      "7ce316ab-4799-4e15-b095-374da21bc1aa": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "4a5226b5-d431-4cbe-86a6-291952a45a37",
        "part": "whole"
       },
       "id": "7ce316ab-4799-4e15-b095-374da21bc1aa"
      },
      "a7d701c6-61e3-48e3-8e0d-a4b318aef7d8": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "5e425283-82de-4c48-b2ae-31a1d6983849",
        "part": "whole"
       },
       "id": "a7d701c6-61e3-48e3-8e0d-a4b318aef7d8"
      },
      "e180cf23-71a7-46a8-8598-86a079bd7d78": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "28aaab45-1e14-4db5-9d2c-63b239cafe13",
        "part": "whole"
       },
       "id": "e180cf23-71a7-46a8-8598-86a079bd7d78"
      }
     }
    },
    "8eb735f6-11af-4f77-a4ed-b637fb18ac08": {
     "id": "8eb735f6-11af-4f77-a4ed-b637fb18ac08",
     "prev": "7a25817a-e5bc-4021-9c6b-61bc74dbdb92",
     "regions": {
      "ca38ef02-01f9-42cb-9e35-d67f8d665597": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "69b638ed-ed32-4824-bb81-df958c215a33",
        "part": "whole"
       },
       "id": "ca38ef02-01f9-42cb-9e35-d67f8d665597"
      },
      "fc52547b-a8f8-40e2-9397-5b18ca2b71b3": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "7317e4f2-150e-4336-ac48-d82641441aa8",
        "part": "whole"
       },
       "id": "fc52547b-a8f8-40e2-9397-5b18ca2b71b3"
      }
     }
    },
    "af2f5933-b31d-4d30-b08a-e4c09432b9bb": {
     "id": "af2f5933-b31d-4d30-b08a-e4c09432b9bb",
     "prev": "29c55765-6994-4dbf-b82f-117b4f1e0cee",
     "regions": {
      "479b436f-50bb-4c3a-9a4e-748e85eacde6": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "6d53481c-69ff-41a0-937e-4e8a39e2b2b5",
        "part": "whole"
       },
       "id": "479b436f-50bb-4c3a-9a4e-748e85eacde6"
      },
      "a02a5bdc-cab5-4ddc-aa9d-8ee5e0db6bd4": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "b0f8400f-71e1-4892-9012-88e54c053a36",
        "part": "whole"
       },
       "id": "a02a5bdc-cab5-4ddc-aa9d-8ee5e0db6bd4"
      }
     }
    },
    "b04719fc-934a-404b-a642-dc732a3d1589": {
     "id": "b04719fc-934a-404b-a642-dc732a3d1589",
     "prev": "af2f5933-b31d-4d30-b08a-e4c09432b9bb",
     "regions": {
      "701cdc94-3399-40dc-8e45-10d6edbf0959": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "fa78fa75-54cf-40d3-b52c-4b51c5070d9a",
        "part": "whole"
       },
       "id": "701cdc94-3399-40dc-8e45-10d6edbf0959"
      },
      "81f01069-a0c6-4ab1-98d1-d465e96aac6d": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "fb58a52c-71e9-402d-af86-eab194cf8050",
        "part": "whole"
       },
       "id": "81f01069-a0c6-4ab1-98d1-d465e96aac6d"
      },
      "ae73fc9b-89cf-4405-ab54-34e6c8b73a19": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "241f356d-567a-421d-b27c-7ce2d146d1bb",
        "part": "whole"
       },
       "id": "ae73fc9b-89cf-4405-ab54-34e6c8b73a19"
      }
     }
    },
    "b7c25bc1-4dd7-421b-9951-0c0c403c176d": {
     "id": "b7c25bc1-4dd7-421b-9951-0c0c403c176d",
     "prev": "76838eab-c2c3-4ff3-b09f-854a7c74c6d4",
     "regions": {
      "47f524f5-68f8-49df-9a61-61eab6cab03c": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "966e050d-5739-4ef9-a2ae-9a4957e14f3e",
        "part": "whole"
       },
       "id": "47f524f5-68f8-49df-9a61-61eab6cab03c"
      },
      "4b3ba01a-29e7-4e67-ac78-988162f7dde9": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "91401f60-44eb-42b2-86ea-c82b5e955ace",
        "part": "whole"
       },
       "id": "4b3ba01a-29e7-4e67-ac78-988162f7dde9"
      },
      "a47d9662-0a49-4b84-aae0-908067552ebd": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "6de4dae8-c949-4927-89a3-9741aba83e8d",
        "part": "whole"
       },
       "id": "a47d9662-0a49-4b84-aae0-908067552ebd"
      },
      "e64668fe-aafe-4e24-a46d-6a6266eeee77": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "7068c96d-f8f0-42bb-bdfd-dcbf9335323c",
        "part": "whole"
       },
       "id": "e64668fe-aafe-4e24-a46d-6a6266eeee77"
      }
     }
    },
    "d3ecec0a-67fe-4558-a268-50695fd7fe1c": {
     "id": "d3ecec0a-67fe-4558-a268-50695fd7fe1c",
     "prev": "fcda7de6-cb93-4b63-9dea-d5f23fecd968",
     "regions": {
      "b268bb57-2543-4b91-b9c0-84aa3cf09d5c": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "67aab8fa-45cd-4ede-8285-e3dad22d8b16",
        "part": "whole"
       },
       "id": "b268bb57-2543-4b91-b9c0-84aa3cf09d5c"
      }
     }
    },
    "e92a9020-6631-400a-aa50-46c1fa0a5f0c": {
     "id": "e92a9020-6631-400a-aa50-46c1fa0a5f0c",
     "prev": "ee94ae9b-9e13-414c-a017-db1d8913aaa8",
     "regions": {
      "bb15b4ea-132f-4146-ab77-93a4969c2904": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "7d5d2002-5cae-4d5f-8e2e-d5eaf5cd17da",
        "part": "whole"
       },
       "id": "bb15b4ea-132f-4146-ab77-93a4969c2904"
      }
     }
    },
    "ee94ae9b-9e13-414c-a017-db1d8913aaa8": {
     "id": "ee94ae9b-9e13-414c-a017-db1d8913aaa8",
     "prev": "f8fce519-927e-4fd0-88d8-23ece4403794",
     "regions": {
      "0b578bb1-2cd1-480b-8d65-dde0861851cb": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "f1c699bc-6717-4eb9-8e6d-66d433ba4618",
        "part": "whole"
       },
       "id": "0b578bb1-2cd1-480b-8d65-dde0861851cb"
      }
     }
    },
    "f8fce519-927e-4fd0-88d8-23ece4403794": {
     "id": "f8fce519-927e-4fd0-88d8-23ece4403794",
     "prev": "4792c0cc-ccd4-48bf-9b0e-a3c1f0f3008d",
     "regions": {
      "7510ae01-fe22-48b0-92cb-29b880cc85fc": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "e61ba538-5828-4a5a-a2c5-4fcceac35889",
        "part": "whole"
       },
       "id": "7510ae01-fe22-48b0-92cb-29b880cc85fc"
      }
     }
    },
    "fcda7de6-cb93-4b63-9dea-d5f23fecd968": {
     "id": "fcda7de6-cb93-4b63-9dea-d5f23fecd968",
     "prev": "03210a56-863e-4749-b7ba-ed75bfceceee",
     "regions": {
      "43f5b3bd-9e0c-4a98-a18c-1b9f7beaa3b8": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "318099ba-93dd-4c2c-a3ee-fc6622c472c8",
        "part": "whole"
       },
       "id": "43f5b3bd-9e0c-4a98-a18c-1b9f7beaa3b8"
      },
      "dbab9920-f08a-457a-949d-2c8289707798": {
       "attrs": {
        "height": 0.4,
        "width": 0.8,
        "x": 0.1,
        "y": 0.5
       },
       "content": {
        "cell": "8bb35ef7-00cc-4ad9-9dd4-6fac480df38e",
        "part": "whole"
       },
       "id": "dbab9920-f08a-457a-949d-2c8289707798"
      }
     }
    }
   },
   "themes": {}
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}