As the name implies, linear regression is an approach to modeling the relationship between a dependent variable $y$ and one or more independent variables denoted as $x$ in a linear form. Linear means that the dependent variable is directly proportional to the independent variables. Keeping other things constant if $x$ is increased/decreased then $y$ also changes linearly. Mathematically the relationship is expressed in the simplest form as: $$y=Ax+B$$

Here $A$ and $B$ are constant factors. The goal in supervised learning using linear regression is finding the value of constants  $A$ and $B$ using the data sets. Then we can use it to predict the values of $y$ in the future for any values of  $x$. Now, the cases where we have a single independent variable is called simple linear regression, while if there is more than one independent variable, then the process is called multiple linear regression.

Quick facts about Linear Regression

Linear regression was the first type of regression analysis to be studied rigorously, and to be used extensively in practical applications. This is because models which depend linearly on their unknown parameters are easier to fit than models which are non-linearly related to their parameters and because the statistical properties of the resulting estimators are easier to determine. Linear regression can be applied to many situations. Most of the applications fall into one of the following two broad categories:

• Prediction: In prediction or forecasting, linear regression can be first used to fit a predictive model to an observed data set of $y$ and $x$ values. After developing such a model, the fitted model can be used to make a prediction of the value of $y$ for an additional value of $x$.
  For example: If we have a dataset of rainfall amounts and corresponding temperatures, then we can fit a linear model and use it to predict the amount of rainfall for a temperature value whose rainfall amount is not known beforehand.
• Finding strength of relationship: Given a variable $y$ and a number of independent variables $x_1, …, x_p$ that may be related to $y$, linear regression analysis can be used to quantify the strength of the relationship between $y$ and the $x_j$, to assess which $x_j$ may have no relationship with $y$ at all, and to identify which subsets of the $x_j$ contain redundant information about $y$.
  For example: If we have a dataset of rainfall amounts and corresponding humidity and temperatures, then we can use regression analysis to find out how strongly does the amount of rainfall depends upon each of these factors.

To estimate the parameters of the linear regression model various techniques can be used. The most common ones are Least Squares (LS) method and maximum-likelihood estimation methods. Let’s discuss here an example of simple linear regression using ordinary least squares method.

How do we go about picking or finding the parameters of the model? One way is to make the predicted value of $y$ as close to the actual value of the training set. For example: Suppose we have a training data $(x_{data}, y_{data})$. Then the reasonable thing to do would be to make the predicted value $y$ as close to $y_data$ as possible. Therefore we try to minimize the sum of the square of the error i.e $$S = S_j (y_{{data}_j} – y_­­­­j)^2$$

For the simple case of a single independent variable after solving we obtain the following formulas for $A_0$ and $A_1$. $$A_{0}=\frac{n \sum_{i=1}^{n} y_{i} * x_{i}-\sum_{i=1}^{n} y_{i} * \sum_{i=1}^{n} x_{i}}{\sum_{i=1}^{n}\left(x_{i}-\text {mean}\right)^{2}}$$

$$A_{1}=y-A_{0} * \text {mean}
$$

3. Linear regression machine learning with Excel

Let’s look at an example: The data in the table below shows the temperature during the race and the corresponding average finish time in minutes of a marathon.