|
Course Content
|
|
Topics
|
|
1
|
Introduction
to neural networks
Biological
neural networks, artificial neuron, activation functions, history of neural
networks
|
|
2
|
Regression and
classification
Gradient
descent learning, stochastic gradient descent learning, linear neuron,
linear regression, perceptron, perceptron learning algorithm, pattern
recognition, discrete perceptron, discrete perceptron learning, logistic
regression, learning a logistic neuron
|
|
3
|
Neuron layers
Softmax layer, softmax learning
algorithm, perceptron layers, learning algorithms
|
|
4
|
Feedforward
networks
Feedforward
neural networks, multilayer perceptron, backpropagation algorithm, deep
feedforward neural networks
|
|
5
|
Model selection
and overfitting
Holdout
method, K-split resampling technique, K-fold cross-validation, three-way
data splits, overfitting and underfitting of neural networks, early
stopping, regularization and weight decay, drop-outs
|
|
6
|
Convolution
neural networks (CNN)
Feature
extraction by convolution, pooling of feature maps, convolutional neural
networks, deep CNN, learning with momentum
|
|
7
|
CNN
architectures
AlexNet,
VGG, ResNet, batch normalization, grouped and separable convolutions,
pre-training and fine-tuning, data augmentation
|
|
8
|
Recurrent
neural networks (RNN)
Recurrent
neural networks with hidden recurrence (Elman type) and with output
recurrence (Jordan type), learning deep RNN, vanishing and exploding
gradient problem in RNN, memory cells and gating, long short-term memory
(LSTM) units, sequence-to-sequence models
|
|
9
|
Attention
Multimodal
attention, self-attention, Transformers
|
|
10
|
Autoencoders
Learning of
autoencoders, denoising autoencoders, sparse autoencoders, building stacked
and deep autoencoders
|
|
11
|
Generative
adversarial networks (GAN)
Generative models, discriminator and
generator minmax game, training GAN, DCGAN, cGAN, CycleGAN, StyleGAN
|
|
12
|
Revision
|
|
|
|
|