Ch16 titles (rasbt#28)

* Added titles to ch16-part1 * fixing the levels of titles in ch16-part1 * added the ending cells * Added titles to ch16-part2 * Added python files
nanw01 · Nov 3, 2019 · b337a20 · b337a20
1 parent b6e6054
commit b337a20
Show file tree

Hide file tree

Showing 4 changed files with 1,152 additions and 37 deletions.
diff --git a/ch16/ch16_part1.ipynb b/ch16/ch16_part1.ipynb
@@ -4,15 +4,45 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Chapter 16\n",
+    "*Python Machine Learning 3rd Edition* by [Sebastian Raschka](https://sebastianraschka.com) & [Vahid Mirjalili](http://vahidmirjalili.com), Packt Publishing Ltd. 2019\n",
+    "\n",
+    "Code Repository: https://github.com/rasbt/python-machine-learning-book-3rd-edition\n",
+    "\n",
+    "Code License: [MIT License](https://github.com/rasbt/python-machine-learning-book-3rd-edition/blob/master/LICENSE.txt)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Chapter 16: Modeling Sequential Data Using Recurrent Neural Networks (part 1/2)\n",
     "========\n"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Introduction"
+    "# Introducing sequential data\n",
+    "\n",
+    "## Modeling sequential data⁠—order matters\n",
+    "\n",
+    "## Representing sequences\n",
+    "\n",
+    "## The different categories of sequence modeling"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# RNNs for modeling sequences\n",
+    "\n",
+    "## Understanding the RNN looping mechanism\n",
+    "\n",
+    "## Computing activations in an RNN\n",
+    "\n",
+    "## Hidden-recurrence vs. output-recurrence\n"
    ]
   },
   {
@@ -112,8 +142,21 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "## The challenges of learning long-range interactions\n",
     "\n",
-    "# Project 1: Sentiment Analysis"
+    "## Long Short-Term Memory cells "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Implementing RNNs for sequence modeling in TensorFlow\n",
+    "\n",
+    "## Project one: predicting the sentiment of IMDb movie reviews\n",
+    "\n",
+    "### Preparing the movie review data\n",
+    "\n"
    ]
   },
   {
@@ -240,7 +283,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Train/validaiton/test splits"
+    " * **Train/validaiton/test splits**"
    ]
   },
   {
@@ -264,17 +307,16 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Tokenizer and Encoder\n",
-    "\n",
-    " * `tfds.features.text.Tokenizer`: https://www.tensorflow.org/datasets/api_docs/python/tfds/features/text/Tokenizer\n",
-    " * `tfds.features.text.TokenTextEncoder`: https://www.tensorflow.org/datasets/api_docs/python/tfds/features/text/TokenTextEncoder"
+    " * **Tokenizer and Encoder**\n",
+    "   * `tfds.features.text.Tokenizer`: https://www.tensorflow.org/datasets/api_docs/python/tfds/features/text/Tokenizer\n",
+    "   * `tfds.features.text.TokenTextEncoder`: https://www.tensorflow.org/datasets/api_docs/python/tfds/features/text/TokenTextEncoder"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Encoding sequences: keeping the last 100 items in each sequence"
+    " * **Encoding sequences: keeping the last 100 items in each sequence**"
    ]
   },
   {
@@ -322,7 +364,7 @@
     }
    ],
    "source": [
-    "## Step 3: endoding each unique token into an integer\n",
+    "## Step 3: endoding each unique token into integers\n",
     "\n",
     "encoder = tfds.features.text.TokenTextEncoder(token_counts)\n",
     "\n",
@@ -408,7 +450,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Batch"
+    " * **batch() vs. padded_batch()**"
    ]
   },
   {
@@ -500,9 +542,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Building an RNN layer:\n",
-    "\n",
-    "### Embedding layer\n",
+    "#### Embedding layers for sentence encoding\n",
     "\n",
     "\n",
     " * `input_dim`: number of words, i.e. maximum integer index + 1.\n",
@@ -558,19 +598,20 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Different RNN Layers:\n",
+    "### Building an RNN model\n",
     "\n",
-    " * `tf.keras.layers.SimpleRNN(units, return_sequences=False)`\n",
-    " * `tf.keras.layers.LSTM(..)`\n",
-    " * `tf.keras.layers.GRU(..)`\n",
-    " * `tf.keras.layers.Bidirectional()`\n",
+    "* **Keras RNN layers:**\n",
+    "  * `tf.keras.layers.SimpleRNN(units, return_sequences=False)`\n",
+    "  * `tf.keras.layers.LSTM(..)`\n",
+    "  * `tf.keras.layers.GRU(..)`\n",
+    "  * `tf.keras.layers.Bidirectional()`\n",
     " \n",
-    "**Determine `return_sequenes=?`**\n",
-    " * In a multi-layer RNN, all RNN layers except the last one should have `return_sequenes=True`\n",
-    " * For the last RNN layer, decide based on the type of problem: \n",
-    "   * many-to-many: -> `return_sequences=True`\n",
-    "   * many-to-one : -> `return_sequenes=False`\n",
-    "   * ..\n",
+    "* **Determine `return_sequenes=?`**\n",
+    "  * In a multi-layer RNN, all RNN layers except the last one should have `return_sequenes=True`\n",
+    "  * For the last RNN layer, decide based on the type of problem: \n",
+    "     * many-to-many: -> `return_sequences=True`\n",
+    "     * many-to-one : -> `return_sequenes=False`\n",
+    "     * ..\n",
     "    "
    ]
   },
@@ -710,7 +751,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Bidirectional LSTM using full-length sequences"
+    "### Building an RNN model for the sentiment analysis task"
    ]
   },
   {
@@ -817,7 +858,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## SimpleRNN with short sequences"
+    " * **Trying SimpleRNN with short sequences**"
    ]
   },
   {
@@ -1206,7 +1247,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Appendix\n"
+    "# Appendix\n"
    ]
   },
   {
@@ -1364,6 +1405,33 @@
     "tf.print(embed(np.array([1])))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "\n",
+    "Readers may ignore the next cell.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[NbConvertApp] Converting notebook ch16_part1.ipynb to script\n",
+      "[NbConvertApp] Writing 16880 bytes to ch16_part1.py\n"
+     ]
+    }
+   ],
+   "source": [
+    "! python ../.convert_notebook_to_script.py --input ch16_part1.ipynb --output ch16_part1.py"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,