12. Appendix
search
Quick search
code
Show Source
Berkeley Course 2019
PDF
Jupyter Notebooks
Discuss
GitHub
中文版
Dive into Deep Learning
Table Of Contents
Preface
Installation
1. Introduction
2. Crashcourse in the Preliminaries
2.1. Data Manipulation
2.2. Linear Algebra
2.3. Automatic Differentiation
2.4. Probability and Statistics
2.5. Naive Bayes Classification
2.6. Sampling
2.7. Documentation
3. Linear Neural Networks
3.1. Linear Regression
3.2. Linear Regression Implementation from Scratch
3.3. Concise Implementation of Linear Regression
3.4. Softmax Regression
3.5. Image Classification Data (Fashion-MNIST)
3.6. Implementation of Softmax Regression from Scratch
3.7. Concise Implementation of Softmax Regression
4. Multilayer Perceptrons
4.1. Multilayer Perceptron
4.2. Implementation of Multilayer Perceptron from Scratch
4.3. Concise Implementation of Multilayer Perceptron
4.4. Model Selection, Underfitting and Overfitting
4.5. Weight Decay
4.6. Dropout
4.7. Forward Propagation, Back Propagation, and Computational Graphs
4.8. Numerical Stability and Initialization
4.9. Environment
4.10. Predicting House Prices on Kaggle
5. Deep Learning Computation
5.1. Layers and Blocks
5.2. Parameter Management
5.3. Deferred Initialization
5.4. Custom Layers
5.5. File I/O
5.6. GPUs
6. Convolutional Neural Networks
6.1. From Dense Layers to Convolutions
6.2. Convolutions for Images
6.3. Padding and Stride
6.4. Multiple Input and Output Channels
6.5. Pooling
6.6. Convolutional Neural Networks (LeNet)
6.7. Deep Convolutional Neural Networks (AlexNet)
6.8. Networks Using Blocks (VGG)
6.9. Network in Network (NiN)
6.10. Networks with Parallel Concatenations (GoogLeNet)
6.11. Batch Normalization
6.12. Residual Networks (ResNet)
6.13. Densely Connected Networks (DenseNet)
7. Recurrent Neural Networks
7.1. Sequence Models
7.2. Language Models
7.3. Recurrent Neural Networks
7.4. Text Preprocessing
7.5. Implementation of Recurrent Neural Networks from Scratch
7.6. Concise Implementation of Recurrent Neural Networks
7.7. Back-propagation Through Time
7.8. Gated Recurrent Unit (GRU)
7.9. Long Short-term Memory (LSTM)
7.10. Deep Recurrent Neural Networks
7.11. Bidirectional Recurrent Neural Networks
8. Optimization Algorithms
8.1. Optimization and Deep Learning
8.2. Gradient Descent and Stochastic Gradient Descent
8.3. Mini-batch Stochastic Gradient Descent
8.4. Momentum
8.5. Adagrad
8.6. RMSProp
8.7. Adadelta
8.8. Adam
9. Computational Performance
9.1. A Hybrid of Imperative and Symbolic Programming
9.2. Asynchronous Computing
9.3. Automatic Parallelism
9.4. Multi-GPU Computation Implementation from Scratch
9.5. Concise Implementation of Multi-GPU Computation
10. Computer Vision
10.1. Image Augmentation
10.2. Fine Tuning
10.3. Object Detection and Bounding Boxes
10.4. Anchor Boxes
10.5. Multiscale Object Detection
10.6. Object Detection Data Set (Pikachu)
10.7. Single Shot Multibox Detection (SSD)
10.8. Region-based CNNs (R-CNNs)
10.9. Semantic Segmentation and Data Sets
10.10. Fully Convolutional Networks (FCN)
10.11. Neural Style Transfer
10.12. Image Classification (CIFAR-10) on Kaggle
10.13. Dog Breed Identification (ImageNet Dogs) on Kaggle
11. Natural Language Processing
11.1. Word Embedding (word2vec)
11.2. Approximate Training
11.3. Implementation of Word2vec
11.4. Subword Embedding (fastText)
11.5. Word Embedding with Global Vectors (GloVe)
11.6. Finding Synonyms and Analogies
11.7. Text Sentiment Classification: Using Recurrent Neural Networks
11.8. Text Sentiment Classification: Using Convolutional Neural Networks (textCNN)
11.9. Encoder-Decoder (seq2seq)
11.10. Beam Search
11.11. Attention Mechanism
11.12. Machine Translation
12. Appendix
12.1. List of Main Symbols
12.2. Mathematical Basis
12.3. Using Jupyter Notebook
12.4. Using AWS to Run Code
12.5. GPU Purchase Guide
12.6. How to Contribute to This Book
12.7.
d2l
Package Index
Dive into Deep Learning
Table Of Contents
Preface
Installation
1. Introduction
2. Crashcourse in the Preliminaries
2.1. Data Manipulation
2.2. Linear Algebra
2.3. Automatic Differentiation
2.4. Probability and Statistics
2.5. Naive Bayes Classification
2.6. Sampling
2.7. Documentation
3. Linear Neural Networks
3.1. Linear Regression
3.2. Linear Regression Implementation from Scratch
3.3. Concise Implementation of Linear Regression
3.4. Softmax Regression
3.5. Image Classification Data (Fashion-MNIST)
3.6. Implementation of Softmax Regression from Scratch
3.7. Concise Implementation of Softmax Regression
4. Multilayer Perceptrons
4.1. Multilayer Perceptron
4.2. Implementation of Multilayer Perceptron from Scratch
4.3. Concise Implementation of Multilayer Perceptron
4.4. Model Selection, Underfitting and Overfitting
4.5. Weight Decay
4.6. Dropout
4.7. Forward Propagation, Back Propagation, and Computational Graphs
4.8. Numerical Stability and Initialization
4.9. Environment
4.10. Predicting House Prices on Kaggle
5. Deep Learning Computation
5.1. Layers and Blocks
5.2. Parameter Management
5.3. Deferred Initialization
5.4. Custom Layers
5.5. File I/O
5.6. GPUs
6. Convolutional Neural Networks
6.1. From Dense Layers to Convolutions
6.2. Convolutions for Images
6.3. Padding and Stride
6.4. Multiple Input and Output Channels
6.5. Pooling
6.6. Convolutional Neural Networks (LeNet)
6.7. Deep Convolutional Neural Networks (AlexNet)
6.8. Networks Using Blocks (VGG)
6.9. Network in Network (NiN)
6.10. Networks with Parallel Concatenations (GoogLeNet)
6.11. Batch Normalization
6.12. Residual Networks (ResNet)
6.13. Densely Connected Networks (DenseNet)
7. Recurrent Neural Networks
7.1. Sequence Models
7.2. Language Models
7.3. Recurrent Neural Networks
7.4. Text Preprocessing
7.5. Implementation of Recurrent Neural Networks from Scratch
7.6. Concise Implementation of Recurrent Neural Networks
7.7. Back-propagation Through Time
7.8. Gated Recurrent Unit (GRU)
7.9. Long Short-term Memory (LSTM)
7.10. Deep Recurrent Neural Networks
7.11. Bidirectional Recurrent Neural Networks
8. Optimization Algorithms
8.1. Optimization and Deep Learning
8.2. Gradient Descent and Stochastic Gradient Descent
8.3. Mini-batch Stochastic Gradient Descent
8.4. Momentum
8.5. Adagrad
8.6. RMSProp
8.7. Adadelta
8.8. Adam
9. Computational Performance
9.1. A Hybrid of Imperative and Symbolic Programming
9.2. Asynchronous Computing
9.3. Automatic Parallelism
9.4. Multi-GPU Computation Implementation from Scratch
9.5. Concise Implementation of Multi-GPU Computation
10. Computer Vision
10.1. Image Augmentation
10.2. Fine Tuning
10.3. Object Detection and Bounding Boxes
10.4. Anchor Boxes
10.5. Multiscale Object Detection
10.6. Object Detection Data Set (Pikachu)
10.7. Single Shot Multibox Detection (SSD)
10.8. Region-based CNNs (R-CNNs)
10.9. Semantic Segmentation and Data Sets
10.10. Fully Convolutional Networks (FCN)
10.11. Neural Style Transfer
10.12. Image Classification (CIFAR-10) on Kaggle
10.13. Dog Breed Identification (ImageNet Dogs) on Kaggle
11. Natural Language Processing
11.1. Word Embedding (word2vec)
11.2. Approximate Training
11.3. Implementation of Word2vec
11.4. Subword Embedding (fastText)
11.5. Word Embedding with Global Vectors (GloVe)
11.6. Finding Synonyms and Analogies
11.7. Text Sentiment Classification: Using Recurrent Neural Networks
11.8. Text Sentiment Classification: Using Convolutional Neural Networks (textCNN)
11.9. Encoder-Decoder (seq2seq)
11.10. Beam Search
11.11. Attention Mechanism
11.12. Machine Translation
12. Appendix
12.1. List of Main Symbols
12.2. Mathematical Basis
12.3. Using Jupyter Notebook
12.4. Using AWS to Run Code
12.5. GPU Purchase Guide
12.6. How to Contribute to This Book
12.7.
d2l
Package Index
12. Appendix
¶
12.1. List of Main Symbols
12.1.1. Numbers
12.1.2. Sets
12.1.3. Operators
12.1.4. Functions
12.1.5. Derivatives and Gradients
12.1.6. Probability and Statistics
12.1.7. Complexity
12.2. Mathematical Basis
12.2.1. Linear Algebra
12.2.2. Differentials
12.2.3. Probability
12.2.4. Summary
12.2.5. Exercise
12.2.6. Scan the QR Code to Discuss
12.3. Using Jupyter Notebook
12.3.1. Edit and Run the Code in This Book Locally
12.3.2. Advanced Options
12.3.3. Summary
12.3.4. Exercise
12.3.5. Scan the QR Code to Discuss
12.4. Using AWS to Run Code
12.4.1. Apply for an Account and Log In
12.4.2. Create and Run an EC2 Instance
12.4.3. Install CUDA
12.4.4. Acquire the Code for this Book and Install MXNet GPU Version
12.4.5. Run Jupyter Notebook
12.4.6. Close Unused Instances
12.4.7. Summary
12.4.8. Exercise
12.4.9. Scan the QR Code to Discuss
12.5. GPU Purchase Guide
12.5.1. Selecting a GPU
12.5.2. Machine Configuration
12.5.3. Summary
12.5.4. Exercise
12.5.5. Scan the QR Code to Discuss
12.6. How to Contribute to This Book
12.6.1. Summary
12.6.2. Exercise
12.6.3. References
12.6.4. Scan the QR Code to Discuss
12.7.
d2l
Package Index
Previous
11.12. Machine Translation
Next
12.1. List of Main Symbols