Feature Scaling, and How to Not Shoot Yourself in the Foot

Scaling is the least glamorous step in the pipeline and one of the easiest to get wrong in a way that silently costs you performance.

The reason it matters: many models care about distance or magnitude. If one feature ranges from 0 to 1 and another ranges from 0 to 50000, the second one will dominate simply because its numbers are bigger, not because it is more informative. Distance-based models (KNN, SVM) and anything using gradient descent are all sensitive to this. Tree-based models mostly are not, which is one reason they are so forgiving.

The three you will actually use

StandardScaler subtracts the mean and divides by the standard deviation, giving you roughly zero mean and unit variance. It is the sensible default when your data is not too far from normally distributed. Its weakness is that both the mean and the standard deviation are dragged around by outliers.

MinMaxScaler squashes everything into a fixed range, usually 0 to 1. Useful when a model or an activation function expects bounded input. Its weakness is severe: a single extreme outlier compresses every other sample into a tiny corner of the range.

RobustScaler uses the median and the interquartile range instead of the mean and standard deviation. Because medians and quartiles barely notice outliers, this one holds its shape when the data gets ugly.

Biomedical data is ugly

This is where it stops being academic. Physiological measurements are full of legitimate extremes. A participant coughs mid-recording. A sensor slips. Someone's value really is unusual, and it is real, and you cannot just delete it.

So on messy signal data I tend to reach for RobustScaler first, because it does not let a handful of extreme values decide the scale for everyone else.

The rule you cannot break

Fit on training data only. Then apply that same fitted scaler to validation and test. If you fit on everything, you have leaked test statistics into training and your evaluation is no longer measuring what you think it is.

Put the scaler in a pipeline. Then cross-validation refits it correctly inside every fold and you physically cannot make this mistake, which is the ideal relationship to have with a mistake.