mnist_1201.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实验目的：\n",
    "使用一站式在线平台搭建神经网络，实现深度学习算法的网络组建、训练、预测过程\n",
    "### 实验要求：\n",
    "完成给定数据集上的分类任务，定义全连接前馈神经网络，softmax输出层以及损失函数 ，训练网路，使用给定的数据集中的测试数据验证精度。\n",
    "### 基本的实验数据集\n",
    "MNIST数据集 Cifar10数据集等"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load in \n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "\n",
    "# Input data files are available in the \"../input/\" directory.\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n",
    "\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))\n",
    "\n",
    "# Any results you write to the current directory are saved as output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import platform\n",
    "import tensorflow as tf\n",
    "print('python_version',platform.python_version())\n",
    "print('tensorflow',tf.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "mnist=keras.datasets.mnist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LossHistory类\n",
    "保存loss（err）和acc（score）并创建实例,在训练的过程中用实例的来画loss和acc图\n",
    "### 类中的方法\n",
    " 1. on_train_begin 方法：初始化 losses，accuracy，val_loss，val_acc\n",
    " 2. on_batch_end：获取每一次训练的的losses，accuracy，val_loss，val_acc的值\n",
    " 3. on_epoch_end和on_batch_end作用相同\n",
    " 4. loss_plot把每一次的losses，accuracy，val_loss，val_acc值的绘画\n",
    " \n",
    "    4.1 画图用的是：matplotlib.pyplot包的 plot方法\n",
    "        iters x轴，val_acc[loss_type] y轴 ， 'b' 颜色 label 标签的名字\n",
    "        plt.plot(iters, self.val_acc[loss_type], 'b', label='val acc') \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LossHistory(keras.callbacks.Callback):\n",
    "    def on_train_begin(self, logs={}):\n",
    "        self.losses = {'batch': [], 'epoch': []}\n",
    "        self.accuracy = {'batch': [], 'epoch': []}\n",
    "        self.val_loss = {'batch': [], 'epoch': []}\n",
    "        self.val_acc = {'batch': [], 'epoch': []}\n",
    "\n",
    "    def on_batch_end(self, batch, logs={}):\n",
    "        self.losses['batch'].append(logs.get('loss'))\n",
    "        self.accuracy['batch'].append(logs.get('acc'))\n",
    "        self.val_loss['batch'].append(logs.get('val_loss'))\n",
    "        self.val_acc['batch'].append(logs.get('val_acc'))\n",
    "\n",
    "    def on_epoch_end(self, batch, logs={}):\n",
    "        self.losses['epoch'].append(logs.get('loss'))\n",
    "        self.accuracy['epoch'].append(logs.get('acc'))\n",
    "        self.val_loss['epoch'].append(logs.get('val_loss'))\n",
    "        self.val_acc['epoch'].append(logs.get('val_acc'))\n",
    "\n",
    "    def loss_plot(self, loss_type):\n",
    "        iters = range(len(self.losses[loss_type]))\n",
    "        #创建一个图\n",
    "        plt.figure()\n",
    "        # acc\n",
    "        plt.plot(iters, self.accuracy[loss_type], 'r', label='train acc')#plt.plot(x,y)，这个将数据画成曲线\n",
    "        # loss\n",
    "        plt.plot(iters, self.losses[loss_type], 'g', label='train loss')\n",
    "        if loss_type == 'epoch':\n",
    "            # val_acc\n",
    "            plt.plot(iters, self.val_acc[loss_type], 'b', label='val acc')\n",
    "            # val_loss\n",
    "            plt.plot(iters, self.val_loss[loss_type], 'k', label='val loss')\n",
    "        plt.grid(True)#设置网格形式\n",
    "        plt.xlabel(loss_type)\n",
    "        plt.ylabel('acc-loss')#给x，y轴加注释\n",
    "        plt.legend(loc=\"upper right\")#设置图例显示位置\n",
    "        plt.show()\n",
    "\n",
    "#创建一个实例LossHistory\n",
    "history = LossHistory()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ### 搭建神经网络，识别手写数字\n",
    "（1）加载数据，引入相关的包\n",
    "\n",
    "（2）根据网络结构，定义各参数、变量，这是最重要的一步，搭建好网络结构就只需要喂数据\n",
    "\n",
    "（3）搭建好网络结构，提供数据进行训练，和保存训练模型\n",
    "\n",
    "（4）用数据训练好模型，\n",
    "\n",
    "（5）模型评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 获取 数据 \n",
    "def get_train_val(mnist_path):\n",
    "    # mnist下载地址：https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
    "    (train_images, train_labels), (test_images, test_labels) = mnist.load_data(mnist_path)\n",
    "    # 打印 训练数据和测试数据的长度\n",
    "    print(\"train_images nums:{}\".format(len(train_images)))\n",
    "    print(\"test_images nums:{}\".format(len(test_images)))\n",
    "    return train_images, train_labels, test_images, test_labels\n",
    " \n",
    "# 展示 图片 数据\n",
    "def show_mnist(images,labels):\n",
    "    for i in range(25):\n",
    "        plt.subplot(5,5,i+1)\n",
    "        plt.xticks([])\n",
    "        plt.yticks([ ])\n",
    "        plt.grid(False)\n",
    "        plt.imshow(images[i],cmap=plt.cm.gray)\n",
    "        plt.xlabel(str(labels[i]))\n",
    "    plt.show()\n",
    "\n",
    "# one_hot 编码，对标签进行 one_hot\n",
    "def one_hot(labels):\n",
    "    onehot_labels=np.zeros(shape=[len(labels),10])\n",
    "    for i in range(len(labels)):\n",
    "        index=labels[i]\n",
    "        onehot_labels[i][index]=1\n",
    "    return onehot_labels\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 这里开始搭建网络:一个简单的全连接层网络\n",
    "1. keras.Sequential() 函数：创建顺序模型\n",
    "2. model.add函数：来添加网络结构的层数，简单网络一般是输入层，隐含层，输出层三层结构 \n",
    "3. keras.layers.Dense 函数：创建全连接的网络，有两个关键的参数，一个是神经元的个数units,一个是激活函数的类型activation（注意:每一层神经元的个数是上一层的输出个数）\n",
    "4. 搭建完输入层，隐含层和输出层，一个简单的神经网络就搭建完成了\n",
    "5. 最后训练模型\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 搭建 简单 网络 \n",
    "def mnist_net(input_shape):\n",
    "    '''\n",
    "    构建一个简单的全连接层网络模型：\n",
    "    输入层为28x28=784个输入节点\n",
    "    隐藏层120个节点\n",
    "    输出层10个节点\n",
    "    :param input_shape: 指定输入维度\n",
    "    :return:\n",
    "    '''\n",
    "    # 使用 顺序模型\n",
    "    model = keras.Sequential()\n",
    "    # 输入层  input_shape 输入的 大小\n",
    "    model.add(keras.layers.Flatten(input_shape=input_shape))\n",
    "    #隐含层  units 神经元的个数120，activation 激活函数 位 rule\n",
    "    model.add(keras.layers.Dense(units=120, activation=tf.nn.relu)) \n",
    "    #输出层 units 神经元的个数10（10个数字），activation 激活函数 位 softmax\n",
    "    model.add(keras.layers.Dense(units=10, activation=tf.nn.softmax))\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### CNN网络与前面的相比,多了卷积层和池化层\n",
    " 1. keras.Sequential() 函数：创建顺序模型\n",
    " 2. model.add函数：来添加网络结构的层数，简单网络一般是输入层，隐含层，输出层三层结构\n",
    " 3. keras.layers.Dense函数：创建全连接的网络，有俩个关键的参数，一个是神经元的个数units,一个是激活函数的类型activation 注意:每一层神经元的个数是上一层的输出个数\n",
    " 4. keras.layers.Conv2D：创建卷积层，过滤器filters,卷积核的大小kernel_size,边缘处理padding,激活函数activation\n",
    " 5. keras.layers.MaxPool2D：创建完卷积层，然后就是用池化层，这里用最大池化和平均池化，pool_size池化的大小\n",
    " 6. 最后还是一样，一个全连接的输出层\n",
    " 7. 训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 搭建 CNN \n",
    "def mnist_cnn(input_shape):\n",
    "    '''\n",
    "    构建一个CNN网络模型\n",
    "    :param input_shape: 指定输入维度\n",
    "    :return:\n",
    "    '''\n",
    "    # 使用 顺序模型\n",
    "    model=keras.Sequential()\n",
    "    # 卷积层 ，kernel_size卷积核大小，activation ，激活函数 ，input_shape 输入大小\n",
    "    model.add(keras.layers.Conv2D(filters=32,kernel_size = 5,strides = (1,1),\n",
    "                                  padding = 'same',activation = tf.nn.relu,input_shape = input_shape))\n",
    "    # 池化层 pool_size 池化大小\n",
    "    model.add(keras.layers.MaxPool2D(pool_size=(2,2), strides = (2,2), padding = 'valid'))\n",
    "     # 卷积层 \n",
    "    model.add(keras.layers.Conv2D(filters=64,kernel_size = 3,strides = (1,1),padding = 'same',activation = tf.nn.relu))\n",
    "    # 池化层\n",
    "    model.add(keras.layers.MaxPool2D(pool_size=(2,2), strides = (2,2), padding = 'valid'))\n",
    "    # 需要丢弃的输入比例。可以防止过拟合\n",
    "    model.add(keras.layers.Dropout(0.25))\n",
    "    # 展平一个张量\n",
    "    model.add(keras.layers.Flatten())\n",
    "    #全连接层\n",
    "    model.add(keras.layers.Dense(units=128,activation = tf.nn.relu))\n",
    "    model.add(keras.layers.Dropout(0.5))\n",
    "    # 全连接层 输出\n",
    "    model.add(keras.layers.Dense(units=10,activation = tf.nn.softmax))\n",
    "    return model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 网络结构搭建完后 ，开始训练模型\n",
    " 1. 在这之前可以对数据做一些处理，正则化等操作，这里用了标准化数据\n",
    " 2. np.expand_dims 看成增加维度，变化数据的维度\n",
    " 3. 标签需要one_hot，使用onehot的直接原因是现在多分类cnn网络的输出通常softmax层，而它的输出是一个概率分布，从而要求输入的标签也以概率分布的形式出现，进而算交叉熵之类。\n",
    " 4. 配置训练模型。optimizer优化器，loss损失函数，学习率，迭代次数等参数\n",
    " 5. 最后开始训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练 模型\n",
    "def trian_model(train_images,train_labels,test_images,test_labels):\n",
    "    # re-scale to 0~1.0之间，把数据 压缩到 0-1 之间\n",
    "    train_images=train_images/255.0\n",
    "    test_images=test_images/255.0\n",
    "    # mnist数据转换为四维\n",
    "    train_images=np.expand_dims(train_images,axis = 3)\n",
    "    test_images=np.expand_dims(test_images,axis = 3)\n",
    "    print(\"train_images :{}\".format(train_images.shape))\n",
    "    print(\"test_images :{}\".format(test_images.shape))\n",
    "    # 标签 one—hot \n",
    "    train_labels=one_hot(train_labels)\n",
    "    test_labels=one_hot(test_labels)\n",
    " \n",
    "    # 建立模型\n",
    "    model = mnist_net(input_shape=(28,28,1)) # 简单 网络\n",
    "    #model=mnist_cnn(input_shape=(28,28,1))  # CNN 网络\n",
    "    #配置训练模型。optimizer优化器，loss损失函数\n",
    "    model.compile(optimizer=tf.optimizers.Adam(),loss=\"categorical_crossentropy\",metrics=['accuracy'])\n",
    "    # 开始 训练 模型 ，callbacks 加入了回调函数，中间获取每次训练的acc和loss \n",
    "    model.fit(x=train_images,y=train_labels,epochs=5,validation_data=(test_images,test_labels),callbacks=[history])\n",
    "    \n",
    "    # 测试模式 返回 损失值和准确率。\n",
    "    test_loss,test_acc=model.evaluate(x=test_images,y=test_labels)\n",
    "    print(\"Test Accuracy %.2f\"%test_acc)\n",
    " \n",
    "    # 开始预测\n",
    "    cnt=0\n",
    "    predictions=model.predict(test_images)\n",
    "    for i in range(len(test_images)):\n",
    "        target=np.argmax(predictions[i])\n",
    "        label=np.argmax(test_labels[i])\n",
    "        if target==label:\n",
    "            cnt +=1\n",
    "    # 准确率\n",
    "    print(\"True prediction of total : %.2f\"%(cnt/len(test_images)))\n",
    "    # 保存 模型\n",
    "    model.save('mnist-model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 性能评估 \n",
    "\n",
    "用开始创建的类history绘图的acc和loss变化情况，可以看每一层迭代时的变化情况\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**参数对性能的影响:**\n",
    "\n",
    "可参考 ：https://blog.csdn.net/program_developer/article/details/78597738\n",
    "\n",
    "这里是举几个参数说明:\n",
    "\n",
    "激活函数： \n",
    "\n",
    "Sigmoid函数：常用的非线性的激活函数，它能够把输入的连续实值变换为0和1之间的输出，特别的，如果是非常大的负数，那么输出就是0；如果是非常大的正数，输出就是1.\n",
    "\n",
    "tanh函数：解决了Sigmoid函数的不是zero-centered输出问题\n",
    "\n",
    "Relu函数：Relu目前仍是最常用的activation function，在搭建人工神经网络的时候推荐优先尝试！\n",
    "\n",
    "迭代次数：epochs（epochs指的就是训练过程中数据将被“轮”多少次）\n",
    "\n",
    "训练的数量: Batch Size（batch size将决定我们一次训练的样本数目）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnist_path = '/kaggle/input/dataet/mnist.npz'\n",
    "# 获取 数据 \n",
    "train_images, train_labels, test_images, test_labels=get_train_val(mnist_path)\n",
    "# 展示 数据 \n",
    "show_mnist(train_images, train_labels)\n",
    "# 训练 和 预测\n",
    "trian_model(train_images, train_labels, test_images, test_labels)\n",
    "# 绘制acc-loss曲线\n",
    "history.loss_plot('epoch')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 要求\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "参考处理mnist数据集的网络结构，对cifar10数据集进行操作。完成cifar10数据集上的分类任务，定义全连接前馈神经网络，softmax输出层以及损失函数 ，训练网路，使用cifar10的数据集中的测试数据验证精度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 参考操作步骤\n",
    "\n",
    "（1）加载数据，引入相关的包\n",
    "\n",
    "（2）根据网络结构，定义各参数、变量，这是最重要的一步，搭建好网络结构就只需要喂数据\n",
    "\n",
    "（3）搭建好网络结构，提供数据进行训练并保存训练模型\n",
    "\n",
    "（4）用数据训练好模型\n",
    "\n",
    "（5）模型评价\n",
    "\n",
    "#### 搭建网络结构\n",
    "\n",
    "1. keras.Sequential() 函数创建顺序模型\n",
    "2. model.add函数来添加网络结构的层数，简单网络一般是输入层，隐含层，输出层三层结构\n",
    "3. keras.layers.Dense 函数创建全连接的网络，有两个关键的参数，一个是神经元的个数units,一个是激活函数的类型activation，注意:每一层神经元的个数是上一层的输出个数\n",
    "4. keras.layers.Conv2D 创建卷积层，过滤器filters,卷积核的大小kernel_size,边缘处理padding,激活函数activation\n",
    "5. keras.layers.MaxPool2D 创建完卷积层，然后就是用池化层，这里用最大池化和平均池化，pool_size池化的大小\n",
    "6. 最后还是一样，一个全连接的输出层\n",
    "* 训练模型\n",
    "\n",
    "#### 网络结构搭建完后 ，开始训练模型\n",
    "1. 在这之前可以对数据做一些处理，正则化等操作这里用了标准化数据\n",
    "2. np.expand_dims 看成 增加维度，变化数据的维度\n",
    "3. 标签需要one_hot，使用onehot的直接原因是现在多分类cnn网络的输出通常是softmax层，而它的输出是一个概率分布，从而要求输入的标签也以概率分布的形式出现，进而算交叉熵之类。\n",
    "4. 配置训练模型。optimizer优化器，loss损失函数，学习率，迭代次数等参数\n",
    "* 最后开始训练\n",
    "\n",
    "#### 性能评估 \n",
    "\n",
    "用开始创建的类history绘图的acc和loss变化情况，可以看每一层迭代时的变化情况\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**参数对性能的影响:**\n",
    "\n",
    "可参考 ：https://blog.csdn.net/program_developer/article/details/78597738\n",
    "\n",
    "这里是举几个参数说明:\n",
    "\n",
    "激活函数： \n",
    "\n",
    "Sigmoid函数：常用的非线性的激活函数，它能够把输入的连续实值变换为0和1之间的输出，特别的，如果是非常大的负数，那么输出就是0；如果是非常大的正数，输出就是1.\n",
    "\n",
    "tanh函数：解决了Sigmoid函数的不是zero-centered输出问题\n",
    "\n",
    "Relu函数：ReLU目前仍是最常用的activation function，在搭建人工神经网络的时候推荐优先尝试\n",
    "\n",
    "迭代次数：epochs（epochs指的就是训练过程中数据将被“轮”多少次）\n",
    "\n",
    "训练的数量: Batch Size（batch size将决定我们一次训练的样本数目）\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里使用ModelCheckpoint 记录中间训练的过程\n",
    "\n",
    "最后绘制 acc 和 loss 的变化情况"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}