JEPA Representation Learning for Intrusion Detection on Real-World Drifting Network Traffic
Supervisors: Prof. Guido Marchetto, Prof. Alessio Sacco, Cristiano Serra, MSc
Does a representation learned from unlabeled network traffic keep an intrusion classifier more accurate than raw flow features on traffic recorded years later?
Built a self-supervised JEPA model in PyTorch with a context encoder, an EMA target encoder, a predictor and grouped masking of correlated features. Pre-trained one encoder per year on unlabeled flows of real Internet backbone traffic from 2011, 2016 and 2021 (MAWIFlow benchmark).
Evaluated it against raw flow features by training four classifiers (logistic regression, decision tree, random forest, XGBoost) on each year and testing them on every year (ROC-AUC, PR-AUC). Replaced the benchmark’s scaling, which lost most of the resolution in the features.
Found that the representation makes the classes more linearly separable. In domain, logistic regression on it closes most of its ROC-AUC gap to XGBoost on raw features, but the representation does not reduce the drop across years, and the 2011 labels mean something different from the later ones. Also implemented VICReg and SIGReg regularizers, which did not improve the classifiers.
- Python
- PyTorch (CUDA)
- scikit-learn
- XGBoost
- Polars
- NumPy
- Matplotlib