TensorFlow Example#

Photo credit: TensorFlow

Intro to Machine Learning
This tutorial is based on Lawrence Moroney’s excellent video tutorial Intro to Machine Learning.

  • First of all, we create some data in Python.

  • In particular, we create two variables x and y with 6 values respectively.

A Python library called NumPy provides lots of array type data structures to do this.

  1. Import the library numpy as np and

  2. specify the values for x and y as an array in NumPy with np.array[]. We use datatype float.

# Create some data
import numpy as np

x = np.array([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
y = np.array([-2.0, 1.0, 4.0, 7.0, 10.0, 13.0], dtype=float)

Now let’s take a look at the numbers.
Can you see the relationship between them?

Variable

Value 1

Value 2

Value 3

Value 4

Value 5

Value 6

x

-1

0

1

2

3

4

y

-2

1

4

7

10

13

As you look at them, you might notice that the value of x is increasing by 1 as you read left to right and the corresponding value of y is increasing by 3. You probably think that y equals 3 times x plus or minus something. Then, you’d probably look at the 0 on x and see that y is 1, and you’d come up with the relationship:

\(y = 3x + 1.\)

That’s almost exactly how you would use code to train a model to spot the patterns in the data! How would you train a neural network to do the equivalent task? By feeding it with your set of x’s and y’s, it should be able to figure out the relationship between them.

But before we train our model, let`s first use the Python library seaborn to visualize the realtionship between x and y.

# Plot relationship with scatterplot
import pandas as pd
import altair as alt

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

df = pd.DataFrame({"x":x, "y":y})

alt.Chart(df).mark_point().encode(
    x="x",
    y="y"
)

Neural Network#

  • We import TensorFlow and calling it tf for ease of use.

  • The framework for defining a neural network as a set of sequential layers is called keras, so import that, too.

# Import TensorFlow and Keras 
import tensorflow as tf
from tensorflow import keras

print(f"TensorFlow version {tf.__version__}")

2023-01-12 14:33:57.109574: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

TensorFlow version 2.11.0

Define and compile the neural network#

First, we create the simplest possible neural network:

  • it has one layer (keras.layers.Dense()),

  • that layer has one neuron (units=1),

  • and the input shape to it is only one value (input_shape=[1]).

# Define neural network

model = tf.keras.Sequential([
    keras.layers.Dense(units=1, input_shape=[1])])

2023-01-12 14:34:44.630059: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

  • Next, we write the code to compile your neural network.

  • When you do so, you need to specify two functions:

    • an optimizer and

    • a loss function.

  • We use mean_squared_error for the loss and stochastic gradient descent (sgd) for the optimizer.

  • You don’t need to understand the math for those yet, but you can see that they work!

# Compile neural network

model.compile(optimizer='sgd', 
             loss='mean_squared_error')

In this example, you know that the relationship between the numbers is \(y=3x+1\).

  1. When the computer is trying to learn that, it makes a guess, maybe \(y=10x+10\).

  2. The loss function measures the guessed answers against the known correct answers and measures how well or badly it did.

  3. Next, the model uses the optimizer function to make another guess.

  4. Based on the loss function’s result, it tries to minimize the loss. At this point, maybe it will come up with something like \(y=5x+5\). While that’s still pretty bad, it’s closer to the correct result (the loss is lower).

  5. The model repeats that for the number of epochs, which you’ll see shortly.

Now you have all the code you need to define the neural network. The next step is to train it to see if it can infer the patterns between those numbers and use them to create a model.

Train the neural network#

The process of training the neural network, where it learns the relationship between the x’s and y’s, is in the model.fit call. That’s where it will go through the loop before making a guess, measuring how good or bad it is (the loss), or using the optimizer to make another guess. It will do that for the number of epochs that you specify.

When you run that code, you’ll see the loss will be printed out for each epoch.

For example, you can see that for the first few epochs, the loss value is quite large, but it’s getting smaller with each step. As the training progresses, the loss soon gets very small. By the time the training is done, the loss is extremely small, showing that our model is doing a great job of inferring the relationship between the numbers.

# Train neural network

model.fit(x, y, epochs=50)

Epoch 1/50
1/1 [==============================] - 0s 258ms/step - loss: 72.6197
Epoch 2/50
1/1 [==============================] - 0s 3ms/step - loss: 57.1348
Epoch 3/50
1/1 [==============================] - 0s 3ms/step - loss: 44.9520
Epoch 4/50
1/1 [==============================] - 0s 4ms/step - loss: 35.3671
Epoch 5/50
1/1 [==============================] - 0s 4ms/step - loss: 27.8262
Epoch 6/50
1/1 [==============================] - 0s 4ms/step - loss: 21.8933
Epoch 7/50
1/1 [==============================] - 0s 4ms/step - loss: 17.2256
Epoch 8/50
1/1 [==============================] - 0s 7ms/step - loss: 13.5532
Epoch 9/50
1/1 [==============================] - 0s 5ms/step - loss: 10.6640
Epoch 10/50
1/1 [==============================] - 0s 4ms/step - loss: 8.3909
Epoch 11/50
1/1 [==============================] - 0s 4ms/step - loss: 6.6024
Epoch 12/50
1/1 [==============================] - 0s 4ms/step - loss: 5.1954
Epoch 13/50
1/1 [==============================] - 0s 3ms/step - loss: 4.0884
Epoch 14/50
1/1 [==============================] - 0s 3ms/step - loss: 3.2174
Epoch 15/50
1/1 [==============================] - 0s 4ms/step - loss: 2.5321
Epoch 16/50
1/1 [==============================] - 0s 3ms/step - loss: 1.9930
Epoch 17/50
1/1 [==============================] - 0s 4ms/step - loss: 1.5688
Epoch 18/50
1/1 [==============================] - 0s 4ms/step - loss: 1.2350
Epoch 19/50
1/1 [==============================] - 0s 3ms/step - loss: 0.9725
Epoch 20/50
1/1 [==============================] - 0s 3ms/step - loss: 0.7658
Epoch 21/50
1/1 [==============================] - 0s 4ms/step - loss: 0.6033
Epoch 22/50
1/1 [==============================] - 0s 5ms/step - loss: 0.4754
Epoch 23/50
1/1 [==============================] - 0s 4ms/step - loss: 0.3747
Epoch 24/50
1/1 [==============================] - 0s 4ms/step - loss: 0.2955
Epoch 25/50
1/1 [==============================] - 0s 4ms/step - loss: 0.2332
Epoch 26/50
1/1 [==============================] - 0s 3ms/step - loss: 0.1841
Epoch 27/50
1/1 [==============================] - 0s 3ms/step - loss: 0.1455
Epoch 28/50
1/1 [==============================] - 0s 3ms/step - loss: 0.1151
Epoch 29/50
1/1 [==============================] - 0s 3ms/step - loss: 0.0912
Epoch 30/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0724
Epoch 31/50
1/1 [==============================] - 0s 3ms/step - loss: 0.0576
Epoch 32/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0459
Epoch 33/50
1/1 [==============================] - 0s 3ms/step - loss: 0.0367
Epoch 34/50
1/1 [==============================] - 0s 3ms/step - loss: 0.0294
Epoch 35/50
1/1 [==============================] - 0s 3ms/step - loss: 0.0237
Epoch 36/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0192
Epoch 37/50
1/1 [==============================] - 0s 5ms/step - loss: 0.0156
Epoch 38/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0128
Epoch 39/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0106
Epoch 40/50
1/1 [==============================] - 0s 5ms/step - loss: 0.0088
Epoch 41/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0075
Epoch 42/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0063
Epoch 43/50
1/1 [==============================] - 0s 5ms/step - loss: 0.0055
Epoch 44/50
1/1 [==============================] - 0s 7ms/step - loss: 0.0048
Epoch 45/50
1/1 [==============================] - 0s 5ms/step - loss: 0.0042
Epoch 46/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0038
Epoch 47/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0034
Epoch 48/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0031
Epoch 49/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0028
Epoch 50/50
1/1 [==============================] - 0s 4ms/step - loss: 0.0027

<keras.callbacks.History at 0x15f801db0>

Use the model#

You now have a model that has been trained to learn the relationship between x and y. You can use the model.predict method to have it figure out the y for a previously unknown x.

For example, if x is 10, what do you think y will be? Take a guess before you run the following code:

# Make a prediction

print(model.predict([10.0]))

1/1 [==============================] - 0s 70ms/step
[[30.779314]]

You might have thought 31, but it ended up not exactly there. Why do you think that is?

Neural networks deal with probabilities, so it calculated that there is a very high probability that the relationship between x and y is \(y=3x+1\), but it can’t know for sure with only six data points. Therefore, the result is very close to 31, but not necessarily 31.

As you work with neural networks, you’ll see that pattern recurring. You will almost always deal with probabilities, not certainties, and will do a little bit of coding to figure out what the result is based on the probabilities.

Summary#

# Import modules
import numpy as np
import tensorflow as tf
from tensorflow import keras

# Create some data
x = np.array([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
y = np.array([-2.0, 1.0, 4.0, 7.0, 10.0, 13.0], dtype=float)

# Model definition
model = tf.keras.Sequential([
    keras.layers.Dense(units=1, input_shape=[1])])

# Compile model
model.compile(optimizer='sgd', 
              loss='mean_squared_error')

# Model fitting
model.fit(x, y, epochs=50, verbose=0)

# Model prediction
print(model.predict([10.0]))

1/1 [==============================] - 0s 49ms/step
[[31.521452]]