From ee3e34cab3cb7132e28a4970c2f959a198152536 Mon Sep 17 00:00:00 2001 From: Alexander Rosenberg Johansen Date: Sat, 24 Sep 2016 22:19:08 +0200 Subject: [PATCH 01/10] Update README.md --- download-and-setup/README.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/download-and-setup/README.md b/download-and-setup/README.md index 80742eb..52c9b66 100644 --- a/download-and-setup/README.md +++ b/download-and-setup/README.md @@ -71,8 +71,4 @@ From within the notebook, click on `/mnt`, click on `myproject`, now you can sta ## Installation of nvidia-docker for GPU -NOTICE: For the Nvidia deep learning camp we have a setup with Boston. This will be available later today at lab 3 - -To run neural nets on GPU accelerated hardware we use a slight modification of docker called [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) (reason is that docker does not yet support the use-case of the specialised hardware and drivers we need). - -Not yet supported %(need sudo access to a GPU server to test this) +Use the following [guide](http://cs224d.stanford.edu/) for AWS setup. From 63fd9182627bc6e6bcffe015701ec9cc6eab4482 Mon Sep 17 00:00:00 2001 From: Alexander Rosenberg Johansen Date: Sun, 25 Sep 2016 12:47:18 +0200 Subject: [PATCH 02/10] Update README.md --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index a3e0272..de3e543 100644 --- a/README.md +++ b/README.md @@ -17,19 +17,19 @@ The material consists of 5 labs. Logistic regression, feed forward neural network (FFN) on the (in)famous MNIST! -Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) Chapters 1-4 (Do 3-5 of the optional exercises). +Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) chapters 1-4 (Do 3-5 of the optional exercises). ### [Lab2 - CNN](https://github.com/alrojo/tensorflow-tutorial/tree/master/lab2_CNN) Convolutional Neural Network (CNN) and Spatial Transformer on images. -Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) Chapter 6 (stop when reaching section called Other approaches to deep neural nets). +Optional reading material from [Michael Nielsen](http://neuralnetworksanddeeplearning.com/) chapter 6 (stop when reaching section called Other approaches to deep neural nets). ### [Lab3 - RNN](https://github.com/alrojo/tensorflow-tutorial/tree/master/lab3_RNN) Recurrent Neural Network (RNN) on Translation using Encoder-Decoder model and Encoder-Decoder with attention. -Optional reading material from [Alex Graves](https://www.cs.toronto.edu/~graves/preprint.pdf) Chapters 3.1, 3.2 and 4, +Optional reading material from [Alex Graves](https://www.cs.toronto.edu/~graves/preprint.pdf) chapters 3.1, 3.2 and 4, ### [Lab4 - Kaggle](https://github.com/alrojo/tensorflow-tutorial/tree/master/lab4_Kaggle) @@ -39,7 +39,7 @@ Compete in the kaggle competition [Leaf Classification](https://www.kaggle.com/c Unsupervised learning with autoencoder (AE) reconstructing the MNIST from only two latent variables. -Optional reading material from [Hinton and Salakhutdinov](https://www.cs.toronto.edu/~hinton/science.pdf). +Optional reading material from [deeplearningbook.org](http://www.deeplearningbook.org/contents/autoencoders.html) chapter 14. ## Credits From dd7342a2ceff298985d36bac547b985f116aebe2 Mon Sep 17 00:00:00 2001 From: "A. Rosenberg Johansen" Date: Wed, 28 Sep 2016 00:12:14 +0200 Subject: [PATCH 03/10] Issue #2 typo fix --- lab4_Kaggle/lab4_Kaggle.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lab4_Kaggle/lab4_Kaggle.ipynb b/lab4_Kaggle/lab4_Kaggle.ipynb index d94b814..aff874d 100755 --- a/lab4_Kaggle/lab4_Kaggle.ipynb +++ b/lab4_Kaggle/lab4_Kaggle.ipynb @@ -1405,7 +1405,7 @@ "# wrapping conv with batch_norm\n", "def conv(l_in, num_outputs, kernel_size, scope, stride=1):\n", " return convolution2d(l_in, num_outputs=num_outputs, kernel_size=kernel_size,\n", - " stride=stride, normalize_fn=batch_norm, scope=scope)\n", + " stride=stride, normalizer_fn=batch_norm, scope=scope)\n", "\n", "# pre-activation: http://arxiv.org/abs/1603.05027\n", "# wrapping convolutions and batch_norm\n", From 9c7303848dd31c6e1a69ae660c77ebf05f83ecd1 Mon Sep 17 00:00:00 2001 From: "A. Rosenberg Johansen" Date: Wed, 28 Sep 2016 10:37:46 +0200 Subject: [PATCH 04/10] Issue #3 operations -> variables --- lab1_FFN/lab1_FFN.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lab1_FFN/lab1_FFN.ipynb b/lab1_FFN/lab1_FFN.ipynb index 71040d1..bc85811 100644 --- a/lab1_FFN/lab1_FFN.ipynb +++ b/lab1_FFN/lab1_FFN.ipynb @@ -267,7 +267,7 @@ "# variables are accessed through tensorflow\n", "print(\"variables\")\n", "variables = [var.name for var in tf.all_variables()]\n", - "print(operations)" + "print(variables)" ] }, { From aca0b72154cbcfd17df179239cef33a6308e1533 Mon Sep 17 00:00:00 2001 From: Alexander Rosenberg Johansen Date: Wed, 28 Sep 2016 22:25:52 +0200 Subject: [PATCH 05/10] setup and installation link in README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index de3e543..15402d7 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Lab 1-4 was used in the [Deep Learning using TensorFlow](http://www.eventbrite.c ## Setup and Installation -Guides for downloading and installing TensorFlow on Linux, OSX and Windows using Docker. +Guides for downloading and installing TensorFlow on Linux, OSX and Windows using Docker can be found [here](https://github.com/alrojo/tensorflow-tutorial/tree/master/download-and-setup). ## Material From 3ca08c45216c57f3a886f63fb631fa716357a162 Mon Sep 17 00:00:00 2001 From: "A. Rosenberg Johansen" Date: Fri, 30 Sep 2016 11:35:16 +0200 Subject: [PATCH 06/10] credits for lab 1, 2, 5 --- lab1_FFN/lab1_FFN.ipynb | 8 ++++++++ lab2_CNN/lab2_CNN.ipynb | 16 ++++++++++++++++ lab5_AE/lab5_AE.ipynb | 16 ++++++++++++++++ 3 files changed, 40 insertions(+) diff --git a/lab1_FFN/lab1_FFN.ipynb b/lab1_FFN/lab1_FFN.ipynb index bc85811..e270350 100644 --- a/lab1_FFN/lab1_FFN.ipynb +++ b/lab1_FFN/lab1_FFN.ipynb @@ -1,5 +1,13 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Credits\n", + "TensorFlow translation of [Lasagne tutorial](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016/blob/master/lab1/lab1_FFN.ipynb). Thanks to [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee)." + ] + }, { "cell_type": "markdown", "metadata": {}, diff --git a/lab2_CNN/lab2_CNN.ipynb b/lab2_CNN/lab2_CNN.ipynb index 6f07258..a6f7a1b 100644 --- a/lab2_CNN/lab2_CNN.ipynb +++ b/lab2_CNN/lab2_CNN.ipynb @@ -1,5 +1,21 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Credits\n", + "TensorFlow translation of [Lasagne tutorial](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016/blob/master/lab2/lab2_CNN.ipynb). Thanks to [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Dependancies and supporting functions\n", + "Loading dependancies and supporting functions by running the code block below." + ] + }, { "cell_type": "code", "execution_count": null, diff --git a/lab5_AE/lab5_AE.ipynb b/lab5_AE/lab5_AE.ipynb index 276135d..3a46e38 100644 --- a/lab5_AE/lab5_AE.ipynb +++ b/lab5_AE/lab5_AE.ipynb @@ -1,5 +1,21 @@ { "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Credits\n", + "TensorFlow translation of [Lasagne tutorial](https://github.com/DeepLearningDTU/02456-deep-learning/blob/master/week5/lab51_AE.ipynb). Thanks to [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Dependancies and supporting functions\n", + "Loading dependancies and supporting functions by running the code block below." + ] + }, { "cell_type": "code", "execution_count": null, From 06ffacad26ae4dbadbc9b1a683c64b3275be6c2c Mon Sep 17 00:00:00 2001 From: Alexander Rosenberg Johansen Date: Mon, 3 Oct 2016 16:41:06 +0200 Subject: [PATCH 07/10] Update README.md --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 15402d7..a024a89 100644 --- a/README.md +++ b/README.md @@ -5,6 +5,12 @@ All exercises are designed to be run from a CPU on a laptop, but can be accelera Lab 1-4 was used in the [Deep Learning using TensorFlow](http://www.eventbrite.com/e/deep-learning-using-tensorflow-tickets-27071720244#) in London by Nvidia and Persontyle +## Credits + +Labs 1, 2, 3 and 5 have been translated from Theano/Lasagne with minor modifications from the following repositories: [Nvidia Summer Camp](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016) and [02456 deep learning](https://github.com/DeepLearningDTU/02456-deep-learning). Original authors: [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee). + +Thanks to professor [Ole Winther](http://cogsys.imm.dtu.dk/staff/winther/) for supervision and sponsoring the labs. + ## Setup and Installation Guides for downloading and installing TensorFlow on Linux, OSX and Windows using Docker can be found [here](https://github.com/alrojo/tensorflow-tutorial/tree/master/download-and-setup). @@ -40,9 +46,3 @@ Compete in the kaggle competition [Leaf Classification](https://www.kaggle.com/c Unsupervised learning with autoencoder (AE) reconstructing the MNIST from only two latent variables. Optional reading material from [deeplearningbook.org](http://www.deeplearningbook.org/contents/autoencoders.html) chapter 14. - -## Credits - -Labs 1, 2, 3 and 5 have been translated from Theano/Lasagne with minor modifications from the following repositories: [Nvidia Summer Camp](https://github.com/DeepLearningDTU/nvidia_deep_learning_summercamp_2016) and [02456 deep learning](https://github.com/DeepLearningDTU/02456-deep-learning). Original authors: [skaae](https://github.com/skaae), [casperkaae](https://github.com/casperkaae) and [larsmaaloee](https://github.com/larsmaaloee). - -Thanks to professor [Ole Winther](http://cogsys.imm.dtu.dk/staff/winther/) for supervision and sponsoring the labs. From 52aa6924b4dcd6e75474e4a38002f3c884406ede Mon Sep 17 00:00:00 2001 From: Ahmed El-Hinidy Date: Fri, 14 Oct 2016 03:07:46 +0200 Subject: [PATCH 08/10] Fixed typo in lab 1 --- lab1_FFN/lab1_FFN.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lab1_FFN/lab1_FFN.ipynb b/lab1_FFN/lab1_FFN.ipynb index e270350..9ec96d3 100644 --- a/lab1_FFN/lab1_FFN.ipynb +++ b/lab1_FFN/lab1_FFN.ipynb @@ -65,7 +65,7 @@ "metadata": {}, "source": [ "# Neural networks 101\n", - "In this notebook you will implement a simple neural network in TensorFlow utilizing the graph building and automatic differentiation engine of TensorFlow. We assume that you are already familiar with backpropation (if not please see [Andrej Karpathy](http://cs.stanford.edu/people/karpathy/) or [Michal Nielsen](http://neuralnetworksanddeeplearning.com/chap2.html).\n", + "In this notebook you will implement a simple neural network in TensorFlow utilizing the graph building and automatic differentiation engine of TensorFlow. We assume that you are already familiar with backpropagation (if not please see [Andrej Karpathy](http://cs.stanford.edu/people/karpathy/) or [Michal Nielsen](http://neuralnetworksanddeeplearning.com/chap2.html).\n", "We'll not spend much time on how TensorFlow works, but you can refer to [this short tutorial](https://www.tensorflow.org/versions/r0.10/get_started/basic_usage.html) if you are interested, or [the python documentation](https://www.tensorflow.org/versions/r0.10/api_docs/index.html).\n", "\n", "(Additionally, for the ambitious people we have previously made an assignment where you will implement both the forward and backpropagation in a neural network by hand, https://github.com/DTU-deeplearning/day1-NN/blob/master/exercises_1.ipynb)(Ole, skal jeg også implementere det?)\n", From 8efd0d1a4beff8bccdf2a85122cdb835c21c2a75 Mon Sep 17 00:00:00 2001 From: Ahmed El-Hinidy Date: Fri, 14 Oct 2016 05:36:47 +0200 Subject: [PATCH 09/10] Fix typo in lab 1 --- lab1_FFN/lab1_FFN.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lab1_FFN/lab1_FFN.ipynb b/lab1_FFN/lab1_FFN.ipynb index 9ec96d3..6b9957b 100644 --- a/lab1_FFN/lab1_FFN.ipynb +++ b/lab1_FFN/lab1_FFN.ipynb @@ -284,7 +284,7 @@ "source": [ "To train our neural network we need to update the parameters in direction of the negative gradient w.r.t the cost function we defined earlier.\n", "We can use `tf.train.Optimizer` to get the gradients (using `compute_gradients`) for all parameters in the network w.r.t ``cost_train``.\n", - "Imaggine that `cost_train` is a function and we want to go downhill. We go downhill by changing the value of the paramters in direction of the negative gradient. \n", + "Imagine that `cost_train` is a function and we want to go downhill. We go downhill by changing the value of the paramters in direction of the negative gradient. \n", "\n", "Finally we can use the built-in `minimize` to calculate the stochastic gradient descent (SGD) update rule for each paramter in the network.\n", "\n", From 6216daba0f1293cc3799ffe641d611ea0f3d3622 Mon Sep 17 00:00:00 2001 From: Parampreet Singh Date: Mon, 31 Oct 2016 22:12:04 +0530 Subject: [PATCH 10/10] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index de3e543..7c88f02 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ # TensorFlow Tutorial - used by Nvidia + Learn TensorFlow from scratch by examples and visualizations with interactive jupyter notebooks. Learn to compete in the [Kaggle leaf detection challenge](https://www.kaggle.com/c/leaf-classification)! All exercises are designed to be run from a CPU on a laptop, but can be accelerated with GPU resources.