Skip to content

Commit

Permalink
match chapter sections (rasbt#6)
Browse files Browse the repository at this point in the history 
* match chapter sections

* Added the note for tfds.Split issues
@rasbt @vmirly
rasbt authored and vmirly committed Oct 13, 2019
1 parent f9b35c1 commit d0553b1
Showing 1 changed file with 169 additions and 63 deletions.
232 changes: 169 additions & 63 deletions ch13/ch13-notebook.ipynb
View file
Original file line number Diff line number Diff line change
@@ -11,19 +11,69 @@
"Code License: [MIT License](https://github.com/rasbt/python-machine-learning-book-3rd-edition/blob/master/LICENSE.txt)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Chapter 13: Parallelizing Neural Network Training with TensorFlow\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## TensorFlow and training performance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Chapter 13: Parallelizing Neural Network Training with TensorFlow\n",
"===="
"### Performance challenges"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Section 1: Working with Tensors"
"### What is TensorFlow?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### How we will learn TensorFlow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## First steps with TensorFlow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Installing TensorFlow"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#! pip install tensorflow"
]
},
{
@@ -56,7 +106,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"2.0.0\r\n"
"2.0.0\n"
]
}
],
@@ -68,7 +118,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating tensors"
"### Creating tensors in TensorFlow"
]
},
{
@@ -182,7 +232,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Manipulating dtype and shape of a tensor: cast, reshape, transpose, and squeeze"
"### Manipulating the data type and shape of a tensor"
]
},
{
@@ -270,7 +320,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Applying math and linear-algebra operations on tensors"
"### Applying mathematical operations to tensors"
]
},
{
@@ -415,7 +465,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Split, stack and concatenate"
"### Split, stack, and concatenate tensors"
]
},
{
@@ -538,7 +588,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Section 2: Working with TensorFlow Dataset"
"## Building input pipelines using tf.data: The TensorFlow Dataset API"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating a TensorFlow Dataset from existing tensors "
]
},
{
@@ -610,7 +667,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Combining two datasets: ds_x and ds_y"
"### Combining two tensors into a joint dataset"
]
},
{
@@ -677,13 +734,6 @@
" ' y: ', example[1].numpy())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Transformation"
]
},
{
"cell_type": "code",
"execution_count": 29,
@@ -712,7 +762,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Shuffle, batch and repeat"
"### Shuffle, batch, and repeat"
]
},
{
@@ -941,7 +991,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create a Dataset from files on disk"
"### Creating a dataset from files on your local storage disk"
]
},
{
@@ -1016,13 +1066,6 @@
"plt.show()\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 37,
@@ -1129,7 +1172,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Fetching available TensorFlow datasets (tensorflow_datasets)"
"### Fetching available datasets from the tensorflow_datasets library"
]
},
{
@@ -1339,7 +1382,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Fetching CelebA dataset"
"Fetching CelebA dataset"
]
},
{
@@ -1581,7 +1624,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Alternative way of loading a dataste"
"Alternative ways for loading a dataset"
]
},
{
@@ -1684,7 +1727,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Section 3: Building NN Models in TensorFlow "
"## Building a neural network model in TensorFlow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### The TensorFlow Keras API (tf.keras)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Building a linear regression model"
]
},
{
@@ -1952,7 +2009,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Alternative way of Training using `.compile()` and `.fit()` methods"
"### Model training via the .compile() and .fit() methods"
]
},
{
@@ -2452,7 +2509,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Build a Classificaiton Model"
"## Building a multilayer perceptron for classifying flowers in the Iris dataset"
]
},
{
@@ -2724,6 +2781,13 @@
"! echo $PWD/ch13-cls-learning-curve.pdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Evaluating the trained model on the test dataset"
]
},
{
"cell_type": "code",
"execution_count": 13,
@@ -2742,6 +2806,13 @@
"print('Test loss: {:.4f} Test Acc.: {:.4f}'.format(*results))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Saving and reloading the trained model"
]
},
{
"cell_type": "code",
"execution_count": 14,
@@ -2850,7 +2921,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Choosing Activation Function\n"
"## Choosing activation functions for multilayer neural networks\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Logistic function recap"
]
},
{
@@ -2938,6 +3016,13 @@
"print('Predicted class label: %d' % y_class) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Estimating class probabilities in multiclass classification via the softmax function"
]
},
{
"cell_type": "code",
"execution_count": 4,
@@ -2972,6 +3057,13 @@
"np.sum(y_probas)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Broadening the output spectrum using a hyperbolic tangent"
]
},
{
"cell_type": "code",
"execution_count": 9,
@@ -3052,13 +3144,27 @@
"log_act = expit(z)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Rectified linear unit activation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Appendix\n",
"\n",
"## Splitting a dataset: danger of mixing train/test example"
"## Splitting a dataset: danger of mixing train/test examples"
]
},
{
@@ -3144,17 +3250,17 @@
},
{
"cell_type": "code",
"execution_count": 83,
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'features': <tf.Tensor: id=140312, shape=(4,), dtype=float32, numpy=array([6.1, 2.8, 4.7, 1.2], dtype=float32)>, 'label': <tf.Tensor: id=140313, shape=(), dtype=int64, numpy=1>}\n",
"{'features': <tf.Tensor: id=1881, shape=(4,), dtype=float32, numpy=array([6.1, 2.8, 4.7, 1.2], dtype=float32)>, 'label': <tf.Tensor: id=1882, shape=(), dtype=int64, numpy=1>}\n",
"\n",
"{'features': <tf.Tensor: id=140319, shape=(4,), dtype=float32, numpy=array([5.7, 3. , 4.2, 1.2], dtype=float32)>, 'label': <tf.Tensor: id=140320, shape=(), dtype=int64, numpy=1>}\n",
"(<tf.Tensor: id=140364, shape=(4,), dtype=float32, numpy=array([6.8, 2.8, 4.8, 1.4], dtype=float32)>, <tf.Tensor: id=140365, shape=(), dtype=int64, numpy=1>)\n",
"{'features': <tf.Tensor: id=1888, shape=(4,), dtype=float32, numpy=array([5.7, 3. , 4.2, 1.2], dtype=float32)>, 'label': <tf.Tensor: id=1889, shape=(), dtype=int64, numpy=1>}\n",
"(<tf.Tensor: id=1919, shape=(4,), dtype=float32, numpy=array([6.9, 3.1, 5.1, 2.3], dtype=float32)>, <tf.Tensor: id=1920, shape=(), dtype=int64, numpy=2>)\n",
"Training Set: 116 Test Set: 34\n",
"10 12 12\n",
"Training Set: 116 Test Set: 34\n",
@@ -3169,14 +3275,33 @@
}
],
"source": [
"#first_67_percent = tfds.Split.TRAIN.subsplit(tfds.percent[:67])\n",
"#last_33_percent = tfds.Split.TRAIN.subsplit(tfds.percent[-33:])\n",
"\n",
"split_train, split_test = tfds.Split.TRAIN.subsplit([2, 1])\n",
"##--------------------------- Attention ------------------------##\n",
"## ##\n",
"## Note: currently, tfds.Split has a bug in TF 2.0.0 ##\n",
"## ##\n",
"## I.e., splitting [2, 1] is expected to result in ##\n",
"## 100 train and 50 test examples ##\n",
"## ##\n",
"## but instead, it results in 116 train and 34 test examples ##\n",
"## ##\n",
"##--------------------------------------------------------------##\n",
"\n",
"import tensorflow as tf\n",
"import tensorflow_datasets as tfds\n",
"import numpy as np\n",
"\n",
"## method 1: specifying percentage:\n",
"#first_67_percent = tfds.Split.TRAIN.subsplit(tfds.percent[:67])\n",
"#last_33_percent = tfds.Split.TRAIN.subsplit(tfds.percent[-33:])\n",
"\n",
"#ds_train_orig = tfds.load('iris', split=first_67_percent)\n",
"#ds_test = tfds.load('iris', split=last_33_percent)\n",
"\n",
"\n",
"## method 2: specifying the weights\n",
"split_train, split_test = tfds.Split.TRAIN.subsplit([2, 1])\n",
"\n",
"ds_train_orig = tfds.load('iris', split=split_train)\n",
"ds_test = tfds.load('iris', split=split_test)\n",
"\n",
@@ -3221,27 +3346,6 @@
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"..."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Summary"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"..."
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -3287,7 +3391,9 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"toc-showmarkdowntxt": false,
"toc-showtags": false
},
"nbformat": 4,
"nbformat_minor": 4

0 comments on commit d0553b1

Please sign in to comment.