TF Strata

Getting Started With TensorFlow
Strata
March 14, 2017
Amy Unruh and Yufeng Guo

Your guides
These slides:
http://bit.ly/tf-strata
Amy Yufeng
[email protected] [email protected]
@amygdala @YufengG
http://bit.ly/tf-strata bit.ly/tensorflow-workshop
Welcome and Logistics
Google Cloud Platform 3

Overview of the Tutorial
Intro and Demos
Getting started with TensorFlow coding: basic concepts
Linear Classification in TensorFlow (MNIST)
Building a DNN Classifier using higher-level libs:
a more accurate MNIST model
Wide & Deep learning with TensorFlow: census data
Wrapup
Slides: http://bit.ly/tf-strata
GitHub:
https://github.com/amygdala/tensorflow-workshop
Optional docker image for style transfer demos:
https://github.com/random-forests/WTM
Brief intro to some NN concepts

What is Machine Learning?
data algorithm insight
Field of study that gives computers the ability to learn without being
explicitly programmed".
You can think of ML as programming with data instead of instructions.
The system learns from the data so it can react correctly to new data.".
But: http://research.google.com/pubs/pub43146.html
("Machine Learning: The High Interest Credit Card of Technical Debt")
Deep Learning
Current state of the art in many tasks:
Image: classification, captioning, colorization
Text: translation, parsing, summarization
Speech: recognition, generation
Games: AlphaGo, Atari, Mario Kart
Many more.
Why now?
Performance improvements: GPUs & Cloud computing
Algorithmic improvements
Larger datasets
Excellent tools
Open Source Models
github.com/tensorflow/models
Show and Tell
https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html
Parsey McParseface
Google
https://research.googleblog.com/2016/05/announcing-syntaxnet-worlds-most.html
Parseys Cousins, in 40 languages
https://research.googleblog.com/2016/08/meet-parseys-cousins-syntax-for-40.html
Translation
https://research.googleblog.com/2016/09/a-neural-network-for-machine.html
Summarization
Original text
Alice and Bob took the train to visit the zoo. They saw a baby giraffe, a
lion, and a flock of colorful tropical birds.
Abstractive summary
Alice and Bob visited the zoo and saw animals and birds.
https://research.googleblog.com/2016/08/text-summarization-with-tensorflow.html
Inception: Image classification
An Alaskan Malamute (left) and a Siberian Husky (right). Images from Wikipedia.
https://research.googleblog.com/2016/08/improving-inception-and-image.html
Transfer Learning
Most of the network is unchanged Only the last layer of weights

and the outputs are updated
Image: neuralnetworksanddeeplearning.com
Bootstrapping with the Inception model
22
Detection of Diabetic Eye Disease
https://research.googleblog.com/2016/11/deep-learning-for-detection-of-diabetic.html
Skin Cancer Image Classification
Benign
Malignant
http://www.nature.com/nature/journal/v542/n7639/full/nature21056.html
Hug?
Dont Hug?
https://goo.gl/zB8TrG
25
TensorFlow for Poets
Codelab goo.gl/xGsB9d
Video goo.gl/KewA03
Shared Research in TensorFlow
Inception https://research.googleblog.com/2016/08/improving-inception-and-image.html
Show and Tell https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html
Parsey McParseface https://research.googleblog.com/2016/05/announcing-syntaxnet-worlds-most.html
Translation https://research.googleblog.com/2016/09/a-neural-network-for-machine.html
Summarization https://research.googleblog.com/2016/08/text-summarization-with-tensorflow.html
Pathology https://research.googleblog.com/2017/03/assisting-pathologists-in-detecting.html
Many more: https://github.com/tensorflow/models/

Style Transfer
goo.gl/WPJtVe
Image source - Wikipedia
Fast Style Transfer
From https://github.com/lengstrom/fast-style-transfer/
From https://github.com/lengstrom/fast-style-transfer/
From: A Matlab Plugin to Visualize Neurons from Deep Models, Donglai Wei et. al.
https://github.com/lengstrom/fast-style-transfer/
https://research.googleblog.com/2016/10/supercharging-style-transfer.html
Multistyle Transfer with
Project Magenta
https://en.wikipedia.org/
0 1 2 3 4 5
6 7 8 9 10 11
12 13 14 15 16 17
18 19 20 21 22 23
24 25 26 27 28 29
Drive folder
goo.gl/1UDjBE
30 31
Blending styles
Style 0 Style 31 Blended

Project Magenta ht
Can ML generate compelling media? A

Music
Images and Video X
Text (Jokes, Stories) tt
Interestingness/Surprise
Structure learning
Measure success
magenta.tensorflow.org
Check out this great talk
goo.gl/ZFzpzu
Learn more
Google Research Blog

goo.gl/T3iD6q
Paper
goo.gl/6fS16m
Whats TensorFlow?
(and why is it so great for ML?)

A multidimensional array.
A graph of operations.
A graph of
A multidimensional array.
operations.
Operates over tensors: n-dimensional arrays

Using a flow graph: data flow computation framework
Flexible, intuitive construction

automatic differentiation
Support for threads, queues, and asynchronous
computation; distributed runtime
Train on CPUs, GPUs, ...and coming soon, TPUS...
Run wherever you like
https://cloudplatform.googleblog.com/2016
/05/Google-supercharges-machine-learnin
g-tasks-with-custom-chip.html
48
Tensors - generalized matrices
Tensors have a Shape thats described with a vector.
[ 10000, 256, 256, 3 ]
10000 Images
Each Image has 256 Rows
Each Row has 256 Pixels
Each Pixel has 3 channels (RGB)
http://bit.ly/tf-strata Google Cloud Platform 49

Computation is a dataflow graph
biases Graph of Nodes, also called Operations or ops.
weights Add Relu
MatMul Xent
examples
labels
s o rs
Computation is a dataflow graph ten
with
biases Edges are N-dimensional arrays: Tensors
weights Add Relu
MatMul Xent
examples
labels
t a t e
Computation is a dataflow graph
w ith s
'Biases' is a variable Some ops compute gradients = updates biases
biases
... Add ... Mul =
learning rate
Build a graph; then run it.
From: http://googleresearch.blogspot.com/2016_03_01_archive.html

Python Frontend C++ Frontend ...
TensorFlow Distributed Execution Engine
CPU GPU Android iOS ...

Build models
Layers

Train and evaluate
Keras models
Estimator
Model
Build models
Layers

Models in a box
Canned Estimators
Train and evaluate

Keras models
Estimator
Model
Build models
Layers

Tensorboard: Graph Visualization

TensorBoard
https://www.tensorflow.org/versions/r0.12/how_tos/embedding_viz/index.html
playground.tensorflow.org
TensorFlow API Documentation:
https://www.tensorflow.org/api_docs/

A first look at some code:
Creating and running
a TensorFlow graph

Define the TensorFlow graph

Create a TensorFlow graph
Follow along at:
https://github.com/amygdala/tensorflow-workshop/tree/master/workshop_sections/starter_tf_graph
import numpy as np
import tensorflow as tf
# a constant
m1 = np.array([[1.,2.], [3.,4.], [5.,6.], [7., 8.]], dtype=np.float32)
# Input data placeholder

m1_input = tf.placeholder(tf.int32, shape=[4,2])

Create a TensorFlow graph, cont.
Follow along at:
m2 = tf.Variable(tf.random_uniform([2,3], -1.0, 1.0))

m3 = tf.matmul(m1_input, m2)
# Add variable initializer op.

init = tf.initialize_all_variables()

Running the graph
(deferred execution)

Running a TensorFlow graph in a session
Follow along at:
with tf.Session() as session:

# We must initialize all variables before we use them.
init.run()
feed_dict = {m1_input: m1}
result = session.run([m3], feed_dict=feed_dict)

print("\nresult: {}\n".format(result))

We just used core TensorFlow
concepts...

Core TensorFlow data structures and concepts...
- Graph: A TensorFlow computation, represented as a

dataflow graph.
- collection of ops that may be executed together as a
group
- Operation: a graph node that performs computation on
tensors
- Tensor: a handle to one of the outputs of an Operation
- provides a means of computing the value in a
TensorFlow Session.
Core TensorFlow data structures and concepts
- Constants
- Placeholders: must be fed with data on execution
- Variables: a modifiable tensor that lives in TensorFlow's
graph of interacting operations.
- Session: encapsulates the environment in which Operation
objects are executed, and Tensor objects are evaluated.
Lets add more nodes to this graph..
Follow along at:
Add m3 to itself
Store the result in m4
Return the results for both m3 and m4
Useful link: https://www.tensorflow.org/api_guides/python/math_ops
MNIST: The Hello World Of ML

Computer Vision -- MNIST
What we see What the computer sees
Training data Testing data
training_data = mnist.train.images
training_labels = mnist.train.labels
testing_data = mnist.test.images
testing_labels = mnist.test.labels
A simple version of MNIST

f(x)= mx + b
Linear Regression
Linear Classification: Apply a Logistic Function
f(x)= mx + b * L
f(x) = predicted value
y = 0 if f(x) < 0
y = f(x) otherwise
Minimize loss: optimizers
error
tf.train.GradientDescentOptimizer
function minimum
parameters (weights, biases)

Training = computing values for the variables
W and b
Because TensorFlow knows the entire graph of your computations, it can
automatically use the backpropagation algorithm to efficiently determine
how your variables affect the cost (loss function) you ask it to minimize.
Then it can apply your choice of optimization algorithm to modify the

variables and reduce the cost.
one-hot decoding
Common TF NN pattern:
- Create the model (inference) graph

- Define the loss function
- specify the optimizer and learning rate
training step op
- In a training loop, call
sess.run([train_step,..], feed_dict={....})
where feed_dict maps inputs to placeholder values
Lab: A simple version of MNIST
Workshop section:
mnist_series/
01_README_mnist_simple.md

Lets make our MNIST model more accurate...
well add some hidden layers
(and take a look at TensorBoard)

...
Hidden layers
{
...
0 1 2 9
One fully connected NN layer typically looks like...
Implementation as Matrix Multiplication
ReLu
(Rectified
Linear unit)
Models in a box
Canned Estimators
Train and evaluate

Keras models
Estimator
Model
Build models
Layers

Lab: hidden-layers version of MNIST
with TensorFlows Estimator APIs
Workshop section:
mnist_series/mnist_tflearn.md

Tensorboard: Graph Visualization

Comparing two different optimizers for DNNClassifier

Wide & Deep:
Using the TensorFlow High-level APIs

Models in a box
Canned Estimators
Train and evaluate

Keras models
Estimator
Model
Build models
Layers

Many levels of abstraction
Choose the right one for you:
Layers, losses, metrics
Training/Eval loop functions
Estimator (BaseEstimator)
Any model you want, but must separate input from the rest of the model
Predefined estimators

,
Limited configuration options: feature columns, metrics.

Typical structure
Load data
Set up feature columns
Create your model
Run the training loop (fit the model)
Evaluate your model's accuracy (and other metrics)
(optional) Predict new examples

tf.learn high-level structure

Motivation - a "magical" food app

Launch and Iterate!
Naive character matching
Say "Fried chicken"
Get "Chicken Fried Rice"
Oops. Now what?
Machine learning to the rescue!
v2.0: memorize all the things
Train a linear TF model
Your app is gaining traction!

Problem: Your users are bored!
Too many & waffles
Show me similar, but different food
Your users are picky

v3.0: More generalized recommendations for all

No good deed goes unpunished
Some recommendations are "too general"

Irrelevant dishes are being sent
Your users are still picky

No good deed goes unpunished
2 types of requests: specific and general
"iced decaf latte with nonfat milk" != "hot latte with
whole milk"
seafood or italian food or fast food
How to balance this?

v4.0: Why not both?

Lab: Wide and Deep:
Using TensorFlows high-level APIs
Workshop section:
wide_n_deep

Meet our dataset: US Census Data
Just as exciting as chicken and waffles
Task: predict the probability that the individual has
an annual income of over 50,000 dollars
Over 32k training examples
Was extracted from the 1994 US Census by Barry
Becker.
Column Name Type Description
age Continuous The age of the individual
workclass Categorical The type of employer the individual has (government,

military, private, etc.).
fnlwgt Continuous The number of people the census takers believe that
observation represents (sample weight). This variable will
not be used.
education Categorical The highest level of education achieved for that individual.
education_num Continuous The highest level of education in numerical form.
marital_status Categorical Marital status of the individual.

occupation Categorical The occupation of the individual.
relationship Categorical Wife, Own-child, Husband, Not-in-family, Other-relative,

Unmarried.
race Categorical White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other,

Black.
gender Categorical Female, Male.
capital_gain Continuous Capital gains recorded.
capital_loss Continuous Capital Losses recorded.

hours_per_week Continuous Hours worked per week.
native_country Categorical Country of origin of the individual.
income_bracket Categorical ">50K" or "<=50K", meaning whether the person

makes more than $50,000 annually.

Typical structure
Load data
Create your model

(File) Queues In TensorFlow
http://bit.ly/tf-workshop-slides bit.ly/tensorflow-workshop
116
File Queues

Input data format

Input data format

Typical structure
Load data
Create your model

Feature columns

Feature columns continued

Typical structure
Load data
Create your model

Make the model (Estimator)

Typical structure
Load data
Create your model

Fit and Evaluate

Checkpointing and reloading a trained model
Everything is stored in the folder
If you run multiple operations on the same and

supply the same directory, training will resume where it left off.
To the code!

Time for a vision test

Wrapup
Thank you!
And next steps.
Learning more
Tutorials and code
tensorflow.org
Totally new to ML? These slides + exercises

Recipes goo.gl/KewA03 goo.gl/WPJtVe
Intro to Deep Learning with TensorFlow TensorFlow Playground

Udacity class goo.gl/iHssII goo.gl/mXhncM
Stanfords CS231n Chris Olahs blog

cs231n.github.io colah.github.io
Udacitys Machine Learning Nanodegree Michael Nielsens book

goo.gl/ODpXj4 neuralnetworksanddeeplearning.com
The End

TF Strata

Uploaded by

Copyright:

Available Formats

TF Strata

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TF Strata

Uploaded by

Copyright:

Available Formats

Getting Started With TensorFlow

Amy Unruh and Yufeng Guo

Google Cloud Platform 3

Optional docker image for style transfer demos:

Google Cloud Platform 6

data algorithm insight

data algorithm insight

data algorithm insight

data algorithm insight

Image: classification, captioning, colorization

Text: translation, parsing, summarization

Speech: recognition, generation

Games: AlphaGo, Atari, Mario Kart

Most of the network is unchanged Only the last layer of weights

Show and Tell https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html

Parsey McParseface https://research.googleblog.com/2016/05/announcing-syntaxnet-worlds-most.html

Many more: https://github.com/tensorflow/models/

Style 0 Style 31 Blended

Can ML generate compelling media? A

Google Research Blog

Google Cloud Platform 46

Operates over tensors: n-dimensional arrays

Flexible, intuitive construction

[ 10000, 256, 256, 3 ]

http://bit.ly/tf-strata Google Cloud Platform 49

biases Graph of Nodes, also called Operations or ops.

weights Add Relu

biases Edges are N-dimensional arrays: Tensors

weights Add Relu

'Biases' is a variable Some ops compute gradients = updates biases

... Add ... Mul =

Google Cloud Platform 54

TensorFlow Distributed Execution Engine

CPU GPU Android iOS ...

Python Frontend C++ Frontend ...

TensorFlow Distributed Execution Engine

CPU GPU Android iOS ...

Python Frontend C++ Frontend ...

TensorFlow Distributed Execution Engine

CPU GPU Android iOS ...

Train and evaluate

Python Frontend C++ Frontend ...

TensorFlow Distributed Execution Engine

CPU GPU Android iOS ...

http://bit.ly/tf-strata Google Cloud Platform 59

Google Cloud Platform 62

Google Cloud Platform 63

Google Cloud Platform 64

# Input data placeholder

http://bit.ly/tf-strata Google Cloud Platform 65

m2 = tf.Variable(tf.random_uniform([2,3], -1.0, 1.0))

# Add variable initializer op.

http://bit.ly/tf-strata Google Cloud Platform 66

Google Cloud Platform 67

with tf.Session() as session:

feed_dict = {m1_input: m1}

result = session.run([m3], feed_dict=feed_dict)

http://bit.ly/tf-strata Google Cloud Platform 68