diff --git a/README.md b/README.md index a3e0272..b6c4ca1 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,20 @@ # TensorFlow Tutorial - used by Nvidia + Learn TensorFlow from scratch by examples and visualizations with interactive jupyter notebooks. Learn to compete in the [Kaggle leaf detection challenge](https://www.kaggle.com/c/leaf-classification)! All exercises are designed to be run from a CPU on a laptop, but can be accelerated with GPU resources. Lab 1-4 was used in the [Deep Learning using TensorFlow](http://www.eventbrite.com/e/deep-learning-using-tensorflow-tickets-27071720244#) in London by Nvidia and Persontyle +## Credits + +Labs 1, 2, 3 and 5 have been translated from Theano/Lasagne with minor modifications from the following repositories: [Nvidia Summer Camp](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016) and [02456 deep learning](https://github.com/DeepLearningDTU/02456-deep-learning). Original authors: [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee). + +Thanks to professor [Ole Winther](http://cogsys.imm.dtu.dk/staff/winther/) for supervision and sponsoring the labs. + ## Setup and Installation -Guides for downloading and installing TensorFlow on Linux, OSX and Windows using Docker. +Guides for downloading and installing TensorFlow on Linux, OSX and Windows using Docker can be found [here](https://github.com/alrojo/tensorflow-tutorial/tree/master/download-and-setup). ## Material @@ -17,19 +24,19 @@ The material consists of 5 labs. Logistic regression, feed forward neural network (FFN) on the (in)famous MNIST! -Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) Chapters 1-4 (Do 3-5 of the optional exercises). +Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) chapters 1-4 (Do 3-5 of the optional exercises). ### [Lab2 - CNN](https://github.com/alrojo/tensorflow-tutorial/tree/master/lab2_CNN) Convolutional Neural Network (CNN) and Spatial Transformer on images. -Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) Chapter 6 (stop when reaching section called Other approaches to deep neural nets). +Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) chapter 6 (stop when reaching section called Other approaches to deep neural nets). ### [Lab3 - RNN](https://github.com/alrojo/tensorflow-tutorial/tree/master/lab3_RNN) Recurrent Neural Network (RNN) on Translation using Encoder-Decoder model and Encoder-Decoder with attention. -Optional reading material from [Alex Graves](https://www.cs.toronto.edu/~graves/preprint.pdf) Chapters 3.1, 3.2 and 4, +Optional reading material from [Alex Graves](https://www.cs.toronto.edu/~graves/preprint.pdf) chapters 3.1, 3.2 and 4, ### [Lab4 - Kaggle](https://github.com/alrojo/tensorflow-tutorial/tree/master/lab4_Kaggle) @@ -39,10 +46,4 @@ Compete in the kaggle competition [Leaf Classification](https://www.kaggle.com/c Unsupervised learning with autoencoder (AE) reconstructing the MNIST from only two latent variables. -Optional reading material from [Hinton and Salakhutdinov](https://www.cs.toronto.edu/~hinton/science.pdf). - -## Credits - -Labs 1, 2, 3 and 5 have been translated from Theano/Lasagne with minor modifications from the following repositories: [Nvidia Summer Camp](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016) and [02456 deep learning](https://github.com/DeepLearningDTU/02456-deep-learning). Original authors: [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee). - -Thanks to professor [Ole Winther](http://cogsys.imm.dtu.dk/staff/winther/) for supervision and sponsoring the labs. +Optional reading material from [deeplearningbook.org](http://www.deeplearningbook.org/contents/autoencoders.html) chapter 14. diff --git a/download-and-setup/README.md b/download-and-setup/README.md index 80742eb..52c9b66 100644 --- a/download-and-setup/README.md +++ b/download-and-setup/README.md @@ -71,8 +71,4 @@ From within the notebook, click on `/mnt`, click on `myproject`, now you can sta ## Installation of nvidia-docker for GPU -NOTICE: For the Nvidia deep learning camp we have a setup with Boston. This will be available later today at lab 3 - -To run neural nets on GPU accelerated hardware we use a slight modification of docker called [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) (reason is that docker does not yet support the use-case of the specialised hardware and drivers we need). - -Not yet supported %(need sudo access to a GPU server to test this) +Use the following [guide](http://cs224d.stanford.edu/) for AWS setup. diff --git a/lab1_FFN/lab1_FFN.ipynb b/lab1_FFN/lab1_FFN.ipynb index 71040d1..6b9957b 100644 --- a/lab1_FFN/lab1_FFN.ipynb +++ b/lab1_FFN/lab1_FFN.ipynb @@ -1,5 +1,13 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Credits\n", + "TensorFlow translation of [Lasagne tutorial](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016/blob/master/lab1/lab1_FFN.ipynb). Thanks to [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee)." + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -57,7 +65,7 @@ "metadata": {}, "source": [ "# Neural networks 101\n", - "In this notebook you will implement a simple neural network in TensorFlow utilizing the graph building and automatic differentiation engine of TensorFlow. We assume that you are already familiar with backpropation (if not please see [Andrej Karpathy](http://cs.stanford.edu/people/karpathy/) or [Michal Nielsen](http://neuralnetworksanddeeplearning.com/chap2.html).\n", + "In this notebook you will implement a simple neural network in TensorFlow utilizing the graph building and automatic differentiation engine of TensorFlow. We assume that you are already familiar with backpropagation (if not please see [Andrej Karpathy](http://cs.stanford.edu/people/karpathy/) or [Michal Nielsen](http://neuralnetworksanddeeplearning.com/chap2.html).\n", "We'll not spend much time on how TensorFlow works, but you can refer to [this short tutorial](https://www.tensorflow.org/versions/r0.10/get_started/basic_usage.html) if you are interested, or [the python documentation](https://www.tensorflow.org/versions/r0.10/api_docs/index.html).\n", "\n", "(Additionally, for the ambitious people we have previously made an assignment where you will implement both the forward and backpropagation in a neural network by hand, https://github.com/DTU-deeplearning/day1-NN/blob/master/exercises_1.ipynb)(Ole, skal jeg også implementere det?)\n", @@ -267,7 +275,7 @@ "# variables are accessed through tensorflow\n", "print(\"variables\")\n", "variables = [var.name for var in tf.all_variables()]\n", - "print(operations)" + "print(variables)" ] }, { @@ -276,7 +284,7 @@ "source": [ "To train our neural network we need to update the parameters in direction of the negative gradient w.r.t the cost function we defined earlier.\n", "We can use `tf.train.Optimizer` to get the gradients (using `compute_gradients`) for all parameters in the network w.r.t ``cost_train``.\n", - "Imaggine that `cost_train` is a function and we want to go downhill. We go downhill by changing the value of the paramters in direction of the negative gradient. \n", + "Imagine that `cost_train` is a function and we want to go downhill. We go downhill by changing the value of the paramters in direction of the negative gradient. \n", "\n", "Finally we can use the built-in `minimize` to calculate the stochastic gradient descent (SGD) update rule for each paramter in the network.\n", "\n", diff --git a/lab2_CNN/lab2_CNN.ipynb b/lab2_CNN/lab2_CNN.ipynb index 6f07258..a6f7a1b 100644 --- a/lab2_CNN/lab2_CNN.ipynb +++ b/lab2_CNN/lab2_CNN.ipynb @@ -1,5 +1,21 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Credits\n", + "TensorFlow translation of [Lasagne tutorial](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016/blob/master/lab2/lab2_CNN.ipynb). Thanks to [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Dependancies and supporting functions\n", + "Loading dependancies and supporting functions by running the code block below." + ] + }, { "cell_type": "code", "execution_count": null, diff --git a/lab4_Kaggle/lab4_Kaggle.ipynb b/lab4_Kaggle/lab4_Kaggle.ipynb index d94b814..aff874d 100755 --- a/lab4_Kaggle/lab4_Kaggle.ipynb +++ b/lab4_Kaggle/lab4_Kaggle.ipynb @@ -1405,7 +1405,7 @@ "# wrapping conv with batch_norm\n", "def conv(l_in, num_outputs, kernel_size, scope, stride=1):\n", " return convolution2d(l_in, num_outputs=num_outputs, kernel_size=kernel_size,\n", - " stride=stride, normalize_fn=batch_norm, scope=scope)\n", + " stride=stride, normalizer_fn=batch_norm, scope=scope)\n", "\n", "# pre-activation: http://arxiv.org/abs/1603.05027\n", "# wrapping convolutions and batch_norm\n", diff --git a/lab5_AE/lab5_AE.ipynb b/lab5_AE/lab5_AE.ipynb index 276135d..3a46e38 100644 --- a/lab5_AE/lab5_AE.ipynb +++ b/lab5_AE/lab5_AE.ipynb @@ -1,5 +1,21 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Credits\n", + "TensorFlow translation of [Lasagne tutorial](https://github.com/DeepLearningDTU/02456-deep-learning/blob/master/week5/lab51_AE.ipynb). Thanks to [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Dependancies and supporting functions\n", + "Loading dependancies and supporting functions by running the code block below." + ] + }, { "cell_type": "code", "execution_count": null,