What is machine learning? Explain different types of regression analysis.

9 A] What is machine learning? Explain different types of regression analysis.

Machine Learning (ML) is a subset of artificial intelligence (AI) that involves developing algorithms and statistical models that enable computers to perform tasks without being explicitly programmed. These systems learn from and make predictions or decisions based on data. ML is widely used in fields such as natural language processing, computer vision, healthcare, finance, and more.

1 Simple Linear Regression

Simple Linear Regression is a foundational regression technique used to model the relationship between one independent variable (predictor) and one dependent variable (response). The equation for a simple linear regression line is:

Where:

• y = Dependent variable (response)
• x = Independent variable (predictor)
• n₀ = Intercept (value of y when x=0)
• n₁ = Slope or linearity coefficient, representing the change in y for a unit change in x

The goal is to find the best-fitting line (regression line) that minimizes the deviations (errors) between observed values (y) and predicted values (y^).

2 Least Square Estimation

The Least Squares Estimation method determines the coefficients b0b_0 (intercept) and b1b_1 (slope) that minimize the sum of squared errors between observed values and predicted values.

Objective function:

Where:

• Q = Sum of squared errors
• y_i= Observed value for the i^th data point
• y^_i = Predicted value for the i^th data point

The best-fit line coefficients are computed by taking the derivative of Q with respect to b₀ and b₁, setting them to zero, and solving for the coefficients.

3 Multiple Regressions

Multiple Regression extends simple linear regression to include two or more independent variables. The model predicts the dependent variable as a linear combination of the independent variables: y=n0+n1x1+n2x2+…+nkxky = n_0 + n_1x_1 + n_2x_2 + \ldots + n_kx_k

Where:

• y = Dependent variable
• x₁,x₂,…, x_k = Independent variables
• n₀,n₁,n₂,…, n_k = Coefficients representing the relationship strength

Multiple Regression is commonly used when analyzing datasets with multiple influencing factors.

4 Modelling Possibilities using Regression

Regression analysis offers flexibility for modeling complex relationships. Some possibilities include:

• Linear Regression Models: Ideal for data with linear relationships.
• Polynomial Regression Models: Fit data with curved trends by including higher-degree terms.
• Logistic Regression: Used for binary or categorical dependent variables.
• Interaction Effects: Capture interactions between variables by including interaction terms.
• Regularization Techniques: Prevent overfitting using Ridge or Lasso regression.

5 Predictions using Regression Analysis

Regression models are used to predict the values of dependent variables based on the values of independent variables. The process involves:

1. Training the model using historical data to determine coefficients.
2. Applying the trained model to new data for prediction.

For instance, predicting a student’s college GPA based on high school performance involves:

• Using historical GPA and high school data to train the model.
• Using the regression equation to predict new students’ GPAs.

6 K-Nearest-Neighbour Regression Analysis

K-Nearest-Neighbour (KNN) Regression is a non-parametric method that predicts the dependent variable based on the kk-nearest data points in the feature space. The prediction is the average of the kk-nearest neighbors’ dependent variable values.

Key Steps:

1. Determine the distance (e.g., Euclidean) between a data point and all other points.
2. Select the k closest points.
3. Compute the average of their yy-values for prediction.

KNN is particularly effective for non-linear relationships and datasets with local patterns.