The Machine Learning Landscape

Machine learning is the science (and art) of programming computers so they can learn from data. It is used for the following:

• Problems for which existing solutions require a lot of fine-tuning or long lists of rules (a machine learning model can often simplify code and perform better than the traditional approach)
• Complex problems for which using a traditional approach yields no good solution (the best machine learning techniques can perhaps find a solution
• Fluctuating environments (a machine learning system can easily be retrained on new data, always keeping it up to date)
• Getting insights about complex problems and large amounts of data

Training Supervision

Supervised, unsupervised, self-supervised, semi-supervised, and reinforcement learning.

1. Supervised learning - in supervised learning, the training set you feed to the algorithm includes the desired solutions, called labels
2. Unsupervised learning - in unsupervised learning, as you might guess, the training data is unlabeled. The system tries to learn without a teacher. a. Visualization algorithms are also good examples of unsupervised learning: you feed them a lot of complex and unlabeled data, and they output a 2D or 3D representation of your data that can easily be plotted b. A related task is dimensionality reduction, in which the goal is to simplify the data without losing too much information.
3. Semi-supervised learning - Since labeling data is usually time-consuming and costly, you will often have plenty of unlabeled instances, and few labeled instances. Some algorithms can deal with data that’s partially labeled.
4. Self-supervised learning - Another approach to machine learning involves actually generating a fully labeled dataset from a fully unlabeled one. Again, once the whole dataset is labeled, any supervised learning algorithm can be used. This approach is called self-supervised learning.
5. Reinforcement learning - The learning system, called an agent in this context, can observe the environment, select and perform actions, and get rewards or penalties in return

The words target and label are generally treated as synonyms in supervised learning, but target is more common in regression tasks and label is more common in classification tasks. Moreover, features are sometimes called predictors or attributes.

Transferring knowledge from one task to another is called transfer learning, and it’s one of the most important techniques in machine learning today, especially when using deep neural networks

Batch vs. Online Learning

Another criterion used to classify machine learning systems is whether or not the system can learn incrementally from a stream of incoming data.

1. Batch learning - In batch learning, the system is incapable of learning incrementally: it must be trained using all the available data. This will generally take a lot of time and computing resources, so it is typically done offline.
2. Online learning - In online learning, you train the system incrementally by feeding it data instances sequentially, either individually or in small groups called mini-batches. Online learning is useful for systems that need to adapt to change extremely rapidly (e.g., to detect new patterns in the stock market).
   1. Out-of-core learning is usually done offline (i.e., not on the live system), so online learning can be a confusing name. Think of it as incremental learning.

Instance-Based Versus Model-Based Learning

One more way to categorize machine learning systems is by how they generalize. Having a good performance measure on the training data is good, but insufficient; the true goal is to perform well on new instances.

1. Instance-based learning - the system learns the examples by heart, then generalizes to new cases by using a similarity measure to compare them to the learned examples (or a subset of them).
2. Model-based learning - another way to generalize from a set of examples is to build a model of these examples and then use that model to make predictions

Model selection consists in choosing the type of model and fully specifying its architecture. Training a model means running an algorithm to find the model parameters that will make it best fit the training data, and hopefully make good predictions on new data.

...

Main Challenges of Machine Learning

1. Insufficient Quantity of Training Data
2. Nonrepresentative Training Data
3. Poor-Quality Data
4. Irrelevant Features Feature engineering steps: 1. Feature selection - select the most useful features to train 2. Feature extraction - combine existing features to produce a more useful one 3. Creating new features by gathering new data
5. Overfitting the Training Data - overfitting means that the model performs well on the training data, but it does not generalize well. Happens when the model is too complex relative to the amount and noisiness of the training data. Regularization - is the process of constraining a model to make it simple and reduce the risk of overfitting. The amount of regularization to apply during learning can be controlled by a hyperparameter. It is a parameter of a learning algorithm. Regularization hyperparameter high -> flat model (slope almost zero)
6. Underfitting the Training Data - it is the opposite of overfitting where the model is too simple to learn the underlying structure of the data. Here are the main options for fixing this problem:
   1. Select a more powerful model, with more parameters.
   2. Feed better features to the learning algorithm (feature engineering).
   3. Reduce the constraints on the model (for example by reducing the regularization hyperparameter).

Machine learning is about making machines get better at some task by learning from data, instead of having to explicitly code rules.

The system will not perform well if your training set is too small, or if the data is not representative, is noisy, or is polluted with irrelevant features (garbage in, garbage out). Lastly, your model needs to be neither too simple (in which case it will underfit) nor too complex (in which case it will overfit).

Testing and Validating

The only way to know how well a model will generalize to new cases is to actually try it out on new cases. A better option is to split your data into two sets: the training set and the test set. As these names imply, you train your model using the training set, and you test it using the test set.

Generalization Error- sometimes called out-of-sample error, it gives the error rate on new cases.

• If the training error is low but the generalization error is high, it means that your model is overfitting the training data.

If you are hesitant between two types of model, you can train both and compare how well they generalize using the test set.

Holdout Validation

The Problem: If you repeatedly tune your model's hyperparameters by measuring performance on the test set, you will inadvertently overfit your model to that specific test set. This gives you an overly optimistic performance estimate, and the model will likely perform poorly on new, unseen data (i.e., it will have a high generalization error).

The Solution: Split your data into three sets:

• Training Set: Used to train the models.
• Validation Set (or Dev Set): A portion of the training data held out to select the best model and tune hyperparameters.
• Test Set: Used only once at the very end to evaluate the final model's performance.

The Process:

1. Train: Train multiple candidate models with different hyperparameters on the training set.
2. Select: Evaluate these models on the validation set and select the best-performing one.
3. Retrain: Retrain the best model on the combined training set + validation set. This creates the final model.
4. Evaluate: Evaluate the final model on the test set to get an unbiased estimate of its generalization performance.

Key Trade-off (Validation Set Size):

• If too small: The evaluation is imprecise, and you might not select the truly best model.
• If too large: The remaining training set is much smaller. Candidate models are then trained on significantly less data than the final model, making the comparison unreliable (likened to "selecting a sprinter for a marathon").

Data Mismatch

The most important rule to remember is that both the validation set and the test set must be as representative as possible of the data you expect to use in production.

The Problem: When Training Data != Production Data Sometimes, you have abundant training data that is not representative of what you'll see in production (e.g., training a flower identifier on millions of high-quality web images, but the app will use phone camera pictures).

This creates a dilemma: If your model performs poorly on the dev set (phone pictures), is it because of:

1. Overfitting: A standard model training error.
2. Data Mismatch: The model learned features for web images that don't work for phone pictures.

The Solution: A Diagnostic train-dev Set To isolate the problem, you create a fourth data split:

• train set: Large but mismatched data (e.g., web images).
• train-dev set: A small set taken from the same source as the training set (more web images).
• dev set: Data from the real-world distribution (e.g., phone pictures).
• test set: More data from the real-world distribution (more phone pictures).

Diagnosing the Error Source: After training on the train set, analyze the errors in this order:

1. Look at train-dev set error.
   • If error is high, the problem is Overfitting.
   • Why? The model has "memorized" the train set instead of learning its general patterns. It can't even perform well on unseen data of the same type.
   • Solution: Use regularization, get more training data, try a different model architecture.
2. If train-dev error is low, look at dev set error.
   • If error is now high, the problem is Data Mismatch.
   • Why? The model learned the training distribution well, but that knowledge doesn't transfer to the real-world data distribution.
   • Solution: Try to make the training data more like the dev set (e.g., data augmentation like adding blur or noise to web images) or gather more data that is representative of the production environment.
3. If dev set error is low, look at test set error.
   • If error is now high, you may have overfit the dev set. It also suggests the dev and test sets might not be from the same distribution, which should be fixed.

By separating the analysis of overfitting from the analysis of data mismatch, you can choose the correct strategy to improve your model.

Exercises

1. How would you define machine learning?
   • A machine learning, for me, is a model that is a representative of a set of data that is able to learn through different methods.
2. Can you name four types of applications where it shines?
   • Fraud detection, classification tasks, forecasting, customer segmentation
3. What is a labeled training set?
   • A training set that is already labeled used for supervised machine learning.
4. What are the two most common supervised tasks?
   • Classification and regression
5. Can you name four common unsupervised tasks?
   • Clustering, anomaly detection, novelty detection, associate learning
6. What type of algorithm would you use to allow a robot to walk in various unknown terrains?
   • Reinforcement learning
7. What type of algorithm would you use to segment your customers into multiple groups?
   • Clustering
8. Would you frame the problem of spam detection as a supervised learning problem or an unsupervised learning problem?
   • Supervised learning
9. What is an online learning system?
   • This is where you train the system incrementally.
10. What is out-of-core learning?
    • Same in the process of online learning system but offline.
11. What type of algorithm relies on a similarity measure to make predictions?
    • An instance-based learning model
12. What is the difference between a model parameter and a model hyperparameter?
    • Model parameter - is a parameter of the model used in prediction (a, b in ax+b (linear model))
    • Model hyperparameter - is a parameter of the learning model/algorithm itself.
13. What do model-based algorithms search for? What is the most common strategy they use to succeed? How do they make predictions?
    • sd
14. Can you name four of the main challenges in machine learning?
    • Overfitting
    • Underfitting
    • Nonrepresentative data
    • Poor quality of the data
15. If your model performs great on the training data but generalizes poorly to new instances, what is happening? Can you name three possible solutions?
    • It is possible that the data overfit.
    • Possible solutions are: get more data, simplify the model (parameter reduction, regularization), and noise reduction.
16. What is a test set, and why would you want to use it?
    • A test set is a set of data used for evaluating the model. We want to use it to determine how our model performs upon using the training data.
17. What is the purpose of a validation set?
    • A portion of the training data held out to select the best model and tune hyperparameters.
18. What is the train-dev set, when do you need it, and how do you use it?
    • The train-dev set is used when there is a risk of mismatch between the training data and the data used in the validation and test datasets (which should always be as close as possible to the data used once the model is in production). The train-dev set is a part of the training set that's held out (the model is not trained on it). The model is trained on the rest of the training set, and evaluated on both the train-dev set and the validation set. If the model performs well on the training set but not on the train-dev set, then the model is likely overfitting the training set. If it performs well on both the training set and the train-dev set, but not on the validation set, then there is probably a significant data mismatch between the training data and the validation + test data, and you should try to improve the training data to make it look more like the validation + test data.
19. What can go wrong if you tune hyperparameters using the test set?
    • It will not perform very well in production data.