UNIVERSITY OF CENTRAL FLORIDA
by Astrid Jackson
UNIVERSITY OF CENTRAL FLORIDA
Tensors: n-dimensional arrays
Vector: 1-D tensor Matrix: 2-D tensor
Flow: data flow computation framework
A sequence of tensor operations
2 2
3 3
UNIVERSITY OF CENTRAL FLORIDA
,
+
, , , ,
+
, ,
s o f t m a x
,
inputs
,
weights
+
biases
outputs
4 4
UNIVERSITY OF CENTRAL FLORIDA
= softmax
,
,
,
,
,
,
,
,
,
∙
+
5 5
UNIVERSITY OF CENTRAL FLORIDA
= softmax
,
,
,
,
,
,
,
,
,
∙
+
import tensorflow as tf logits = tf.matmul(x, w) + b y = tf.nn.softmax(logits)
6 6
UNIVERSITY OF CENTRAL FLORIDA
,
,
,
,
,
,
,
,
,
w
b
Variable( initial−value , name= optional−name ) import tensorflow as tf w = tf.Variable( tf.random_normal([3, 3]), name='w' ) b = tf.Variable( tf.zeros([3]), name='b' ) y = tf.nn.softmax( tf.matmul( x, w ) + b )
7 7
UNIVERSITY OF CENTRAL FLORIDA
Softmax
Code defines data flow graph Each variable corresponds to a node in the graph, not the result
Variable
Add
Variable
MatMul
import tensorflow as tf w = tf.Variable( tf.random_normal([3, 3]), name='w' ) b = tf.Variable( tf.zeros([3]), name='b' ) y = tf.nn.softmax( tf.matmul( x, w ) + b )
8 8
UNIVERSITY OF CENTRAL FLORIDA
biases
Add
Softmax
weights MatMul
Xent
inputs
targets
Graph of Nodes, also called operations (ops)
9 9
UNIVERSITY OF CENTRAL FLORIDA
biases
Add
Softmax
weights MatMul
Xent
inputs
targets
Edges are N-dimensional arrays: Tensors
10 10
UNIVERSITY OF CENTRAL FLORIDA
Backward graph and update are added automatically to graph 'Biases' are variable Some ops compute gradients
-= updates biases
biases
…
learning rate
Add
…
Mul
-=
11 11
UNIVERSITY OF CENTRAL FLORIDA
Fetch Graph
Softmax
Code defines data flow graph •
How to execute the graph?
Variable
Softmax
Variable
MatMul
Session •
Manage resource for graph execution
import tensorflow as tf sess = tf.Session() w = tf.Variable( tf.random_normal([3, 3]), name='w' ) b = tf.Variable( tf.zeros([3]), name='b' ) y = tf.nn.softmax( tf.matmul( x, w ) + b ) print sess.run(y)
12 12
UNIVERSITY OF CENTRAL FLORIDA
Fetch Graph
Softmax
Variable is an empty node •
Fill in the content of Variable node
Variable
Softmax
Variable
MatMul
import tensorflow as tf sess = tf.Session() w = tf.Variable( tf.random_normal([3, 3]), name='w' ) b = tf.Variable( tf.zeros([3]), name='b' ) y = tf.nn.softmax( tf.matmul( x, w ) + b ) sess.run( tf.initialize_all_variables() ) print sess.run(y)
13 13
UNIVERSITY OF CENTRAL FLORIDA
Fetch Graph
Softmax
How about x? placeholder( data_type , shape= optional_shape , name= optional_name ) import tensorflow as tf
Variable
Softmax
Variable
MatMul
sess = tf.Session() x = tf.placeholder( "float", [1, 3] ) w = tf.Variable( tf.random_normal([3, 3]), name='w' ) b = tf.Variable( tf.zeros([3]), name='b' ) y = tf.nn.softmax( tf.matmul( x, w ) + b ) sess.run( tf.initialize_all_variables() ) print sess.run(y feed_dict={x: np.array([[1., 2., 3.]])})
Feed
UNIVERSITY OF CENTRAL FLORIDA
Needs to release resource after use sess.close()
Using context manager with tf.Session() as sess: …
Interactive session sess = InteractiveSession()
14 14
UNIVERSITY OF CENTRAL FLORIDA
15 15
Select loss function Loss function for softmax softmax_cross_entropy_with_logits(logits, labels, name= optional−name ) labels = tf.placeholder("float", [1, 3]) logits = tf.matmul( x, w ) + b cross_entropy = tf.nn.softmax_cross_entropy_with_logits ( logits, labels, name='xentropy')
UNIVERSITY OF CENTRAL FLORIDA
Gradient descent class GradientDescentOptimizer(learning_rate, use_locking=False, name='GradientDescent') labels = tf.placeholder("float", [1, 3]) cross_entropy = tf.nn.softmax_cross_entropy_with_logits( tf.matmul( x, w ) + b, labels, name='xentropy') optimizer = tf.train.GradientDescentOptimizer(0.1) train_op = optimizer.minimize(cross_entropy) sess.run(train_op, feed_dict={x:np.array([[1., 2., 3.]]), labels:np.aray([[0., 1., 0.]])})
16 16
17 17
UNIVERSITY OF CENTRAL FLORIDA
Gradient descent requires multiple steps to converge
labels = tf.placeholder("float", [1, 3]) cross_entropy = tf.nn.softmax_cross_entropy_with_logits( tf.matmul( x, w ) + b, labels, name='xentropy') optimizer = tf.train.GradientDescentOptimizer(0.1) train_op = optimizer.minimize(cross_entropy) for step in range(10): sess.run(train_op, feed_dict={x:np.array([[1., 2., 3.]]), labels:np.array([[0., 1., 0.]])})
18 18
UNIVERSITY OF CENTRAL FLORIDA
How well does model do?
correct_prediction = tf.equal( tf.argmax(logits, 1), tf.argmax(labels, 1) ) accuracy = tf.reduce_mean( tf.cast(correct_prediction, tf.float32) ) print sess.run(accuracy, feed_dict={x:np.array([[1., 2., 3.]]), labels:np.array([[0., 1., 0.]])})
UNIVERSITY OF CENTRAL FLORIDA
x = tf.placeholder("float", [1, 3]) out = x num_layers = 2 for layer in range(num_layers): w = tf.Variable( tf.random_normal([3, 3]) ) b = tf.Variable( tf.zeros([1, 3]) ) out = tf.nn.relu( tf.matmul(out, w) + b ) …
19 19
UNIVERSITY OF CENTRAL FLORIDA
TensorBoard writer = tf.train.SummaryWriter( '/tmp/tf_logs', sess.graph)
Launch TensorBoard
tensorboard --logdir=/tmp/tf_logs Navigate web browser to: localhost:6006
20 20
21 21
UNIVERSITY OF CENTRAL FLORIDA
Add summaries scalar_summary
histogram_summary
summary_op = tf.merge_all_summaries() for step in range(10): _, summary = sess.run([train_op, summary_op], feed_dict=…) writer.add_summary(summary, step)
UNIVERSITY OF CENTRAL FLORIDA
tf.train.Saver(…) Default will associate all variables all_variables()
save(sess, save_path, …) restore(sess, save_path, …) Replace initialization
22 22
UNIVERSITY OF CENTRAL FLORIDA
Familiarity with C++ required
Build TensorFlow from source
23 23
Eigen Tensors https://bitbucket.org/eigen/eigen/src/default/unsu pported/Eigen/CXX11/src/Tensor/README.md?file viewer=file-view-default
UNIVERSITY OF CENTRAL FLORIDA
24 24
http://tensorflow.org https://github.com/tensorflow/tensorflow
TensorFlow wrappers Keras: https://keras.io/
Eigen Tensor library https://bitbucket.org/eigen/eigen/src/default/unsu pported/Eigen/CXX11/src/Tensor/README.md?file viewer=file-view-default
