Data deviations from predicted values are Gaussian.
B/c above, MSE cost function is appropriate.
What are other cost functions we can use?
Loss function:
Mean squared error cost function.
Regression problems are classic way to introduce the concepts of overfitting & generalization in ML.
What happens when a model is under fitted? or over fitted?
Under fitting:
Model is too simple understand our problem & capture the underlying patterns in data.
High bias, low variance.
Signs: High error on training data, poor performance on new data.
Over fitting:
Model is too complex & learns the noise in training data.
Performs very well on training data but poorly on new unseen data
Low bias, high variance.
Very low error on training data, but high error on test data.
Both under & over fitting lead to poor generalization & reduced model performance on new data. The goal is find the right balance, often referred as sweet spot of model complexity.
What is Generalization?
Model ability to perform well on new, unseen data after being trained on specific dataset. It's the capacity of model to apply what it has learned from training data to make accurate predictions on new data or previously unseen data.
Importance:
The goal of machine learning models is to generalize well, as this allows them to be useful in real world applications where they will encounter new data.
A good generalization occurs when a model is neither under fitted or over fitted.
How to improve generalization?
Use regularization methods i.e L1, L2 regularization or dropout.
Increase training data quantity & diversity.
A small generalization error indicates good generalization.
WHat should we use absolute value or squared value?
A example of linear regression:
1.Statistical assumptions:
What are other cost functions we can use?
Loss function:
Mean squared error cost function.
Regression problems are classic way to introduce the concepts of overfitting & generalization in ML.
What happens when a model is under fitted? or over fitted?
Under fitting:
Over fitting:
Both under & over fitting lead to poor generalization & reduced model performance on new data. The goal is find the right balance, often referred as sweet spot of model complexity.
What is Generalization?
Model ability to perform well on new, unseen data after being trained on specific dataset. It's the capacity of model to apply what it has learned from training data to make accurate predictions on new data or previously unseen data.
Importance:
The goal of machine learning models is to generalize well, as this allows them to be useful in real world applications where they will encounter new data.
A good generalization occurs when a model is neither under fitted or over fitted.
How to improve generalization?
WHat should we use absolute value or squared value?
Bia varaiance trade off?