Choosing ML Algorithms: Build a Baseline Before Trying Six

You can spend an afternoon comparing random forests, support vector machines, neural networks, and nearest neighbors without learning whether the task needed machine learning at all.

A baseline makes the comparison less comfortable and more useful. It asks how much of the result can be achieved with a deliberately simple system.

Consider a hypothetical support queue. The task is to route new tickets to one of several teams. The inputs are ticket text and information available when a ticket arrives. The model must respond quickly, and a wrong route costs someone time.

Start with an embarrassing competitor

The simplest baseline sends every ticket to the most common team. It will be useless for many tickets, but it establishes a floor that a headline accuracy number might otherwise disguise.

A second baseline uses a short keyword rule for clearly identifiable requests. A third could use word counts with logistic regression. Despite its name, logistic regression is commonly used for classification. It can combine many small signals and remains a useful reference point for text tasks.

These baselines answer different questions. The majority rule measures how much the class distribution gives away. The keyword rule tests existing domain knowledge. The trained linear model tests whether a relatively simple learned boundary is sufficient.

Use the same held-out data for the comparison, with related tickets kept together where necessary. Otherwise changes in the exam can masquerade as changes in the model.

Choose the next experiment from the failures

If the linear model misses interactions in structured inputs, a decision tree may be worth testing. Trees express combinations of conditions without requiring you to specify each interaction manually. An unrestricted tree can also fit quirks of the training set.

A random forest combines many trees and can reduce the instability of an individual tree. That benefit comes with a larger model and less direct interpretability. Gradient-boosted trees build an ensemble sequentially to improve an objective; they are another strong candidate for many tabular problems, but still need tuning and evaluation.

Nearest-neighbor methods make predictions using similar stored examples. They offer an intuitive reference point, while making distance definitions, feature scaling, memory use, and prediction cost particularly visible. A support vector machine provides another family of decision boundaries, with kernel choices that can add flexibility and computational cost.

A neural network becomes a candidate when its representation and capacity address a real limitation of simpler methods. Merely occupying the final row of an algorithm comparison does not make it the inevitable destination.

Clustering belongs to a different experiment. Grouping similar tickets without route labels can help explore the queue; it does not directly evaluate how well we predict the required team.

Keep a comparison sheet that includes the work

For each candidate, record routing errors by team, time to produce a prediction, training and maintenance requirements, and a few representative mistakes. Leave numerical cells blank until measured. Do not borrow performance percentages from another dataset.

The winning candidate may depend on the operating constraint. A small reduction in routing errors might justify a more complex service in a large queue. In a small organization, a transparent rule plus a manual fallback could be easier to maintain and sufficient for the job.

The baseline is not something to discard once the interesting models arrive. Keep it in the evaluation. If a future retraining makes the sophisticated system worse than the simple one, you will want an unmistakable signal.