AI-Based Network Attack Classifier

Project Overview

Traditional network security systems rely on rule-based signature matching — a brittle approach that requires constant manual updates and fails to detect novel attack variants. This project replaces that approach with a machine learning classifier trained on raw HTTP payloads from network security appliance logs.

The model classifies each payload into one of five categories: Brute Force, Cross-Site Scripting, SQL Injection, System Command Execution, or Normal. The system was developed as part of the AI Security Operations (AI-SOC) specialist training programme in South Korea.

Dataset

Label	Attack Type	Count	% of Total
Brute_Force	Credential stuffing via automated tools (Hydra); 100% POST requests	135,037	61.4%
normal	Legitimate traffic baseline	44,000	20.0%
Cross_Site_Scripting	XSS via scanner tools (Arachni, Nikto); 97% GET requests	16,542	7.5%
SQL_Injection	SQLmap-driven injection attempts; HTTP/1.1 only	12,525	5.7%
System_Cmd_Execution	OS command injection; identified via Connector user-agent	11,896	5.4%
Total		220,000	100%

Feature Engineering

Rather than vectorising raw text (computationally expensive at this scale), we performed statistical frequency analysis across label-payload pairs to identify keywords that appeared with high discriminative frequency for specific attack classes. 9 binary/categorical features were extracted from each payload string.

method

HTTP method (GET=1, POST=0)

Signal: Brute Force = 100% POST

ver

HTTP version (1.0=1, 1.1=0)

Signal: Brute Force skews HTTP/1.0

word

OS/tool keyword: hydra(1), linux(2), win(3), mac(4), nikto(5)

Signal: hydra → Brute Force exclusively

sql

Presence of "sqlmap" in payload (binary)

Signal: Strong indicator of SQL_Injection

ar

Presence of "arachni" scanner (binary)

Signal: XSS + SysCmd attacks

uid

Presence of "userid" parameter (binary)

Signal: Brute Force login targeting

zap

Presence of OWASP ZAP scanner (binary)

Signal: Active scanning / Brute Force

connector

Presence of "connector" user-agent (binary)

Signal: System_Cmd_Execution only

apple

Presence of "applewebkit" user-agent (binary)

Signal: Normal traffic distinguisher

Preprocessing Pipeline

# Feature extraction and encoding pipeline
def prepro(data):
    # Structural features from payload token positions
    data['method'] = data['payload'].str.split().str[0]  # GET / POST
    data['ver']    = data['payload'].str.split().str[1]  # HTTP version

    # Keyword-based binary feature extraction
    data['sql']       = data['payload'].apply(add_sql)       # sqlmap presence
    data['word']      = data['payload'].apply(add_word)      # OS/tool keyword
    data['ar']        = data['payload'].apply(add_ar)        # arachni scanner
    data['uid']       = data['payload'].apply(add_uid)       # userid param
    data['connector'] = data['payload'].apply(add_con)       # connector UA
    data['zap']       = data['payload'].apply(add_zap)       # ZAP scanner
    data['apple']     = data['payload'].apply(add_apple)     # AppleWebKit UA

    # Label encoding
    data['ver']    = data['ver'].map({'HTTP/1.1':0, 'HTTP/1.0':1})
    data['method'] = data['method'].map({'POST':0, 'GET':1})

    return data[['method','ver','word','uid','ar','sql','connector','zap','apple']], tg

Model Training

Training Setup

Parameter	Value
Train / Test Split	80% / 20% (stratified by label)
Random Seed	42
Test Set Size	44,000 samples
Stratification	Preserved class distribution across splits
Hardware	Intel i9-13900, 64GB RAM, RTX 4070
Environment	Python 3.12.7 · Anaconda · Jupyter Lab

Algorithm Selection — Why Random Forest

Three classifiers were evaluated. Random Forest (bagging ensemble) was selected as the primary model because it outperformed both XGBoost and Logistic Regression in overall accuracy, while training significantly faster. The bagging approach — randomly sampling subsets of data and averaging results across parallel trees — proved more stable than voting-based ensembles for this imbalanced dataset.

Results

Model Accuracy Comparison

Random Forest

92.6%

XGBoost

92.6%

Logistic Regression

91.7%

Random Forest — Per-Class Classification Report (Test Set: 44,000 samples)

Class	Precision	Recall	F1-Score	Support
Brute_Force	0.99	1.00	0.99	27,008
normal	0.93	0.99	0.96	8,800
SQL_Injection	1.00	0.87	0.93	2,505
Cross_Site_Scripting	0.55	0.87	0.67	3,308
System_Cmd_Execution	1.00	0.02	0.04	2,379
Weighted Avg	0.95	0.93	0.91	44,000

Known Limitations

System_Cmd_Execution — Recall: 0.02

The model almost completely missed System Command Execution attacks (recall = 0.02, F1 = 0.04). This is a consequence of the keyword-based feature engineering approach — the connector feature, while unique to this class, was insufficient as a sole discriminator. The class is also the smallest in the dataset (5.4%), amplifying the imbalance effect.

Class Imbalance

Brute Force accounts for 61.4% of samples, pulling overall accuracy upward. The weighted average F1 of 0.91 is flattered by Brute Force's F1 of 0.99. The macro average F1 of 0.72 is a more accurate representation of true cross-class performance.

Future Improvements

01Replace keyword-based feature extraction with TF-IDF or embedding vectorisation of full payload text — eliminates the need for manual keyword curation and captures richer signal.
02Apply SMOTE or class-weight balancing to address the Brute Force dominance and improve System_Cmd_Execution recall.
03Hyperparameter tuning via GridSearchCV or Bayesian Optimisation (Optuna) to improve XSS and SysCmd performance.
04Evaluate on external datasets (UNSW-NB15, KISTI Network Intrusion) to test generalisation beyond the training domain.
05Explore deep learning approaches (LSTM or transformer-based) for sequence-aware payload classification.