Learn Linear Regression in Tensorflow in Python with me
Hello there, We are gonna learn to make a simple Linear Regression Model in python using tensorflow, it more simple than you think, we'll understand each and every step, it's pretty fun, so be with me.
What will you need before going ahead :
let's breakdown what we are doing here :
Now let's look how our predictor is doing and what's the cost of it, so let's plug-in the calculated W and b into the line formula:
here only one value of weight and bias is there but if there are more dependent variable x1,x2,... etc we would have multiple weight and bias values. we'll see that in other tutorials,I'll link them in future when I get to learn them.
doesn't make much sense, let's plot the original data ,and the line which our predictor makes.
Output :
this fitted line approximates our data set, and we can use the weights and biases using the line formula. we will model the linear regression to neural networks in future posts.
Stay tuned, Thank You ;)
What will you need before going ahead :
- What is Linear Regression ?
- Introduction to Tensorflow
- and as always patience
Linear Regression as I understand :
We know the equation of line goes something like this y = Wx + b ,
where W = slope
b = y-intercept
so if we have some x's and y's known as the data, and we want to predict what will be the next y given any x, it's called linear regression. here we do that by initially assigning random values for m and b, so that formula will give you some random values initially.
using this random m,b formula we can calculate our guess-y.
now question is how to make this formula accurate as per our data, such that it will predict the next y according to what our dataset did for some values.
For this we have to calculate the error or specifically mean square error, that's basically taking the difference of the guess-y and y from the data we have (guess-y - y) and squaring this,that is the square error for one point. now taking the mean of this is to divide by n(number of points) we have. But we usually divide with 2n. Reason being when this will be derivated 2 of the power will cancel with this. just for simplicity. you can try only by n.
So the formula for cost is :
cost = (guess-y - y)2 / (2*n)
we have to minimize this cost this is done by gradient descent, that's the algorithm to minimize cost by changing m and b.
we have to minimize this cost this is done by gradient descent, that's the algorithm to minimize cost by changing m and b.
Let's get started :
first of all you must have tensorflow, matplotlib and numpy installed. Import these libraries :
Now in order for the random numbers to be predictable we'll fix the seed :
fixing seed will basically won't change the output when the program is run, as the random number generate takes this seed as a base to generate random numbers.
Now let's generate our dataset x's and y's
the linspace will create uniformly distributed 50 floating point numbers for x and y
then random noise is added to the data using random.uniform
n will hold the number of data points we have in our data set
Now lets plot the points we created randomly using scatter plot(just the dots) :
That should look something like this :
Now we are gonna create placeholders in tensorflow, placeholders are similar to variables in programming whose values can be changed but these are not initialized these are just to allocate space sort of thing and also using these we can pass our data into the optimizer such that it can calculate cost and minimize it by changing the W and b.
Now let's create Variables in tensorflow , Variables in tensorflow are similar to placeholders except they are initialized and they are modified by the optimizer during training hence we need to create variables of w and b because they are gonna change.
Now we will define hyperparameters for our model , these are learning rate and epochs , learning rate is with how much difference is the w and b gonna change, lower the rate accurate the model. Epochs are the number of passes we go through data.
Now let's make tensorflow formulas for y-guess and cost and optimizer,these will be executed when the tf session will be called using graph based computation.
the initialiser will initialise all the variables to the random value we assigned to it.
let's break down the code :
now
first of all you must have tensorflow, matplotlib and numpy installed. Import these libraries :
import numpy as np import tensorflow as tf import matplotlib.pyplot as plt
Now in order for the random numbers to be predictable we'll fix the seed :
np.random.seed(101) tf.set_random_seed(101)
fixing seed will basically won't change the output when the program is run, as the random number generate takes this seed as a base to generate random numbers.
Now let's generate our dataset x's and y's
# Genrating linear data # There will be 50 data points ranging from 0 to 50 x = np.linspace(0, 50, 50) y = np.linspace(0, 50, 50) # Adding noise to the random linear data x += np.random.uniform(-4, 4, 50) y += np.random.uniform(-4, 4, 50) n = len(x) # Number of data points
the linspace will create uniformly distributed 50 floating point numbers for x and y
then random noise is added to the data using random.uniform
n will hold the number of data points we have in our data set
Now lets plot the points we created randomly using scatter plot(just the dots) :
pp.scatter(x,y) pp.xlabel('x') pp.ylabel('y') pp.title('training dataset') pp.show()
That should look something like this :
Now we are gonna create placeholders in tensorflow, placeholders are similar to variables in programming whose values can be changed but these are not initialized these are just to allocate space sort of thing and also using these we can pass our data into the optimizer such that it can calculate cost and minimize it by changing the W and b.
Placeholders are for passing training data
X = tf.placeholder("float") Y = tf.placeholder("float")
Now let's create Variables in tensorflow , Variables in tensorflow are similar to placeholders except they are initialized and they are modified by the optimizer during training hence we need to create variables of w and b because they are gonna change.
W = tf.Variable(np.random.randn(), name = "W") b = tf.Variable(np.random.randn(), name = "b")
Variable are modified during training
Now we will define hyperparameters for our model , these are learning rate and epochs , learning rate is with how much difference is the w and b gonna change, lower the rate accurate the model. Epochs are the number of passes we go through data.
learning_rate = 0.01 training_epochs = 1000
Now let's make tensorflow formulas for y-guess and cost and optimizer,these will be executed when the tf session will be called using graph based computation.
# formula for y-guess y_pred = tf.add(tf.multiply(X, W), b) # Mean Squared Error Cost Function cost = tf.reduce_sum(tf.pow(y_pred-Y, 2)) / (2 * n) # Gradient Descent Optimizer optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) # Global Variables Initializer init = tf.global_variables_initializer()
the initialiser will initialise all the variables to the random value we assigned to it.
let's break down the code :
this is basically creating the formula which tensorflow can understand
y = Wx+bnow
cost = tf.reduce_sum(tf.pow(y_pred-Y, 2)) / (2 * n)
calculates the cost using summing all the values in the array using reduce_sum function, the sum is for (y-pred-y)^2, then this sum is divided by 2n
then
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
this creates a optimizer taking the learning rate and what to minimize, in this case the cost.
Now let's create a session of tensorflow which will execute all the formulas in graph based computations.
# Starting the Tensorflow Session with tf.Session() as sess: # Initializing the Variables sess.run(init) # Iterating through all the epochs for epoch in range(training_epochs): # Feeding each data point into the optimizer using Feed Dictionary for (_x, _y) in zip(x, y): sess.run(optimizer, feed_dict = {X : _x, Y : _y}) # Displaying the result after every 50 epochs if (epoch + 1) % 50 == 0: # Calculating the cost a every epoch c = sess.run(cost, feed_dict = {X : x, Y : y}) print("Epoch", (epoch + 1), ": cost =", c, "W =", sess.run(W), "b =", sess.run(b)) # Storing necessary values to be used outside the Session training_cost = sess.run(cost, feed_dict ={X: x, Y: y}) weight = sess.run(W) bias = sess.run(b)
let's breakdown what we are doing here :
- first with sess.run(init) will run the initializer that will initialize all the global variables that we declared before.
- we are running the for loops for the number of times we sat the training_epoch.
- then we are passing the x's and y's one by one to the optimizer using feed_dict parameter, this will pass each value into the optimizer and it will process it and will change the W,b accordingly so as to minimize the cost. and this optimizer is executed by using sess.run
- then we are printing the value of cost using the if loop.
- notice every formula we created before are being executed in the tensorflow session hence, we only declare the things before , and run everything when our model is completed such that things can be executed parallely.
- then finally we are storing the values of training_cost, weight and bias to use outside the session
Output: Epoch: 50 cost = 5.8868036 W = 0.9951241 b = 1.2381054 Epoch: 100 cost = 5.7912707 W = 0.99812365 b = 1.0914398 Epoch: 150 cost = 5.7119675 W = 1.0008028 b = 0.96044314 Epoch: 200 cost = 5.6459413 W = 1.0031956 b = 0.8434396 Epoch: 250 cost = 5.590799 W = 1.0053328 b = 0.7389357 Epoch: 300 cost = 5.544608 W = 1.007242 b = 0.6455922 Epoch: 350 cost = 5.5057883 W = 1.008947 b = 0.56222 Epoch: 400 cost = 5.473066 W = 1.01047 b = 0.48775345 Epoch: 450 cost = 5.4453845 W = 1.0118302 b = 0.42124167 Epoch: 500 cost = 5.421903 W = 1.0130452 b = 0.36183488 Epoch: 550 cost = 5.4019217 W = 1.0141305 b = 0.30877414 Epoch: 600 cost = 5.3848577 W = 1.0150996 b = 0.26138115 Epoch: 650 cost = 5.370246 W = 1.0159653 b = 0.21905091 Epoch: 700 cost = 5.3576994 W = 1.0167387 b = 0.18124212 Epoch: 750 cost = 5.3468933 W = 1.0174294 b = 0.14747244 Epoch: 800 cost = 5.3375573 W = 1.0180461 b = 0.11730931 Epoch: 850 cost = 5.3294764 W = 1.0185971 b = 0.090368524 Epoch: 900 cost = 5.322459 W = 1.0190892 b = 0.0663058 Epoch: 950 cost = 5.3163586 W = 1.0195289 b = 0.044813324 Epoch: 1000 cost = 5.3110332 W = 1.0199214 b = 0.02561663
Now let's look how our predictor is doing and what's the cost of it, so let's plug-in the calculated W and b into the line formula:
# Calculating the predictions predictions = weight * x + bias print("Training cost =", training_cost, "Weight =", weight, "bias =", bias, '\n')
Output: Training cost = 5.3110332 Weight = 1.0199214 bias = 0.02561663
here only one value of weight and bias is there but if there are more dependent variable x1,x2,... etc we would have multiple weight and bias values. we'll see that in other tutorials,I'll link them in future when I get to learn them.
doesn't make much sense, let's plot the original data ,and the line which our predictor makes.
# Plotting the Results plt.plot(x, y, 'ro', label ='Original data') plt.plot(x, predictions, label ='Fitted line') plt.title('Linear Regression Result') plt.legend() plt.show()
Output :
this fitted line approximates our data set, and we can use the weights and biases using the line formula. we will model the linear regression to neural networks in future posts.
Stay tuned, Thank You ;)
Comments
Post a Comment