Nobody Warns You About Class Imbalance

Real datasets are lopsided. You collect 200 healthy participants and 30 patients, because that is who showed up, and now your model has decided that the safest strategy in life is to call everyone healthy.

There are three broad ways out, and they are not equally good.

1. Change the data

Undersampling throws away majority-class samples until the classes balance. It is simple, and it is painful, because you are deleting real data you worked hard to collect. On small datasets this usually costs more than it gains.

Oversampling duplicates minority samples. Better than nothing, but exact copies encourage the model to memorise those specific points.

SMOTE is the popular middle path. Instead of copying a minority sample, it draws a line between that sample and one of its nearest minority neighbours, then invents a new point somewhere along that line. You get new samples that are plausible rather than identical.

Borderline-SMOTE refines this further by concentrating on the minority samples that sit near the decision boundary, which is where the model is actually confused. Those are the ones worth reinforcing.

2. Change the loss

Often the simplest and most underrated option: tell the model that mistakes on the minority class hurt more. Most libraries support this directly with a class weight parameter, and it costs you nothing in data.

model = LogisticRegression(class_weight="balanced")

Try this before you reach for anything clever. It frequently gets you most of the way there.

3. Change the threshold

Leave the data and the loss alone, and simply move the cutoff. Cheap, transparent, and easy to justify in a paper.

The trap that catches everyone

Here is the mistake I want you to remember, because I have seen it in submitted manuscripts more than once.

Never apply SMOTE before splitting your data. Never apply it to the test set at all.

If you generate synthetic samples across the whole dataset and then split, synthetic points derived from test samples end up in training. Your model has effectively seen interpolated versions of the very data you are about to evaluate on. Your scores go up. Your credibility goes down.

Resampling belongs inside the cross-validation loop, applied to the training fold only. The test fold stays exactly as messy and imbalanced as reality is, because that is the point of a test.

for train_idx, test_idx in cv.split(X, y): X_tr, y_tr = smote.fit_resample(X[train_idx], y[train_idx]) model.fit(X_tr, y_tr) score(model, X[test_idx], y[test_idx]) # untouched, still imbalanced

Use a pipeline and this becomes hard to get wrong, which is the highest praise I can give any tool.