Simple Basics of TensorFlow

TensorFlow is a library for numerical calculation where information moves through the chart. Information in TensorFlow is spoken to by n-dimensional clusters called Tensors. Chart is made of data(Tensors) and scientific tasks.

• Nodes on the graph: represent mathematical operations. 

• Edges on the graph: represent the Tensors that flow between operations. 

There is one more aspect in which TensorFlow is very different from any other programming language. In TensorFlow, you first need to create a blueprint of whatever you want to create. While you are creating the graph, variables don’t have any value. Later when you have created the complete graph, you have to run it inside a session, only then the variables have any values. More on this later. 

How about we begin learning by doing. Run python and import tensorflow:

1 2 3 4 5 6 7

sankit@sankit:~$ python Python 2.7.6 (default, Oct 26 2016, 20:30:19)  [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import tensorflow as tf

(i) Graph in TensorFlow:

Chart is the foundation of TensorFlow and each calculation/task/factors live on the diagram. Everything that occurs in the code, dwells on a default diagram gave by TensorFlow. You can access this graph by:

1 2	graph = tf.get_default_graph()

You can get the list of all the operations by typing this:

graph.get_operations()
Currently, the output is empty as shown by [], as there is nothing in the graph.
If you want to print to name of the operations in the graph, do this:

1 2 3

for op in graph.get_operations():     print(op.name)

This will again be empty. We shall use this to print names of operations after we have added ops to the graph.

Also, It’s possible to create multiple graphs. But let’s worry about that later. 

(ii) TensorFlow Session:

A graph is used to define operations, but the operations are only run within a session. Graphs and sessions are created independently of each other. You can imagine graph to be similar to a blueprint, and a session to be similar to a construction site.

Graph only defines the computations or builds the blueprint. However, there are no variables, no values unless we run the graph or part of the graph within a session.

You can create a session like this:

1 2 3 4 5

sess=tf.Session() ... your code ... ... your code ... sess.close()

Whenever, you open a session, you need to remember to close it. Or you can use ‘with block’ like this.

1 2 3

with tf.Session() as sess:     sess.run(f)

Advantage of with block is: session closes automatically at the end of the with block. We use with block in most of our code and recommend you to do so too.

iii). Tensors in TensorFlow:

TF holds data in Tensors which are similar to numPy multi-dimensional arrays(although they are different from numPy Arrays):

a) Constants:

are constants whose value can’t be changed. You can declare a constant like this: 

1 2 3 4 5 6

a=tf.constant(1.0) a <tf.Tensor'Const:0' shape=() dtype=float32> print(a) Tensor("Const:0", shape=(), dtype=float32)

As you can see, this is different from other programming languages like python, you can’t print/access constant a unless you run it inside a session. Let’s do it: 

1 2 3

with tf.Session() as sess:     print(sess.run(a))

This will produce 1.0 as output.

b) Variables:

are again Tensors which are like variables in any other language. 

1 2 3 4

>>>b = tf.Variable(2.0,name="test_var") >>>b <tensorflow.python.ops.variables.Variable object at 0x7f37ebda1990>

Variables(as you can guess by the name) can hold different values as opposed to constants. However, they need to be separately initialized by an init op. It could be taxing to initialize all the variables individually. However, TensorFlow provides a mechanism to initialize all the variables in one go. Let’s see how to do that:

For tf version 0.11 and earlier, use initialize_all_variables()  

>>>init_op = tf.initialize_all_variables()

Use global_variables_initializer() for tf version 0.12 and later.

>>>init_op = tf.global_variables_initializer()

This will add init_op to our tensorflow default graph.

Now run this init_op before you try to access your variable:

1 2 3 4

with tf.Session() as sess:     sess.run(init_op)     print(sess.run(b))

This will output 2.0

Now, try to print the operations on the graph: 

1 2 3 4

graph = tf.get_default_graph() for op in graph.get_operations():     print(op.name)

This will now output:

Const
test_var/initial_value
test_var
test_var/Assign
test_var/read
init

As you can see, we have declared ‘a’ as Const so this has been added to the graph. Similarly, for the variable b, many ‘test_var’ states have been added to the TensorFlow graph like test_var/initial_value, test_var/read etc. You can visualize the complete network using TensorBoard, which is a tool to visualize a TensorFlow graph and training process.

c)Placeholders: 

are tensors which are waiting to be initialized/fed. Placeholders are used for training data which is only fed when the code is actually run inside a session. What is fed to Placeholder is called feed_dict. Feed_dict are key value pairs for holding data:

1 2 3 4 5 6 7 8 9 10 11 12

>>>a = tf.placeholder("float") >>>b = tf.placeholder("float") >>>y = tf.multiply(a, b) // Earlier this used to be tf.mul which has changed with Tensorflow 1.0 //Typically we load feed_dict from somewhere else, //may be reading from a training data folder. etc //For simplicity, we have put values in feed_dict here >>>feed_dict ={a:2,b:3} >>>with tf.Session() as sess:        print(sess.run(y,feed_dict))

Output will be 6.

iv) Device in TensorFlow:

TensorFlow has very strong in-built capabilites to run your code on a gpu or a cpu or a cluster of gpu etc. It provides you options to select the device you want to run your code. However, this is not something that you need to worry about when you are just getting started. We shall write a separate tutorial on this later. So, here is the complete picture: