Engineering AI for Security & Health.

A Real-Time Channel-Level Intrusion Detection System Based on Multimodal Learning

Reproduction of "A Real-Time Channel-Level Intrusion Detection System Based on Multimodal Learning" (RCML-IDS), with a training notebook and a Streamlit demo app.1. The problem this paper solvesThe paper targets intrusion detection (IDS) for IoT networks, and frames the motivation around two challenges that, together, expose a real gap in how IDS systems are currently built and deployed.Challenge 1 - encrypted traffic. The overwhelming majority of IoT traffic today is encrypted end-to-end, using TLS, VPN tunnels, or anonymizing overlays like Tor. Historically, IDS approaches leaned on two families of techniques that both assume some visibility into the payload: port-based classification (mapping well-known port numbers to expected application protocols) and deep packet inspection (DPI), which parses payload content against signatures or protocol grammars. Encryption breaks both assumptions at once, the port can be multiplexed or tunneled, and the payload bytes are, from the detector's perspective, statistically close to random noise. This is not a hypothetical problem for IoT specifically: constrained IoT devices increasingly ship with TLS stacks by default (partly because of past IDS/DPI-driven privacy and security incidents), which means the population of traffic an IDS can no longer "read" keeps growing every year.Challenge 2 - hard real-time constraints. In domains the paper explicitly calls out, vehicular networks, industrial control systems, and other latency- sensitive IoT deployments, detection is only useful if it happens before an attacker can complete their objective. A system that correctly classifies an attack five minutes after it started has, in practical terms, failed: by then the attacker may have already exfiltrated data, pivoted to another host, or completed a denial-of-service. This reframes IDS from a pure classification accuracy problem into a latency-bounded classification problem, where the metric that matters isn't just "did we get the label right" but "how long did it take us to commit to that label."The paper backs the urgency of solving both problems simultaneously with a concrete trend: it cites IoT attack frequency growing roughly 82% between 2023 and 2024, attributing part of that growth to attackers using larger, more automated botnets and AI-assisted tooling to scale reconnaissance and exploitation. Put differently, the threat side of the equation is scaling faster than manual, signature-based defenses can keep up with, which is the implicit argument for why a learned, multimodal, real-time approach is worth the added engineering complexity.2. Why I think traditional approaches fall shortThe paper breaks the existing landscape down into four method families, and for each one it identifies a structural limitation, not just a performance gap, but a reason the approach cannot close that gap without a fundamentally different design.Statistical-feature methods. These are the classic machine-learning IDS pipelines: hand-craft features like flow duration, byte/packet counts, inter-arrival-time statistics, then feed them to a shallow classifier (SVM, random forest, gradient boosting, etc.). Two problems compound here. First, feature engineering is itself a bottleneck, it requires domain expertise, is brittle to new attack variants, and doesn't transfer well across datasets or protocols. Second, and more fundamentally for the real-time argument, most of these statistics (mean/std of packet length, flow duration, byte ratios) are only well-defined once a flow has finished. Computing them online, on a still-in-progress flow, gives biased, incomplete estimates. So these methods are architecturally coupled to waiting for flow completion, which directly conflicts with the real-time requirement from Section 1.Single-modality learned methods. These use a single feature type, either raw bytes or packet-length sequences, fed into a neural network. The paper argues packet-length alone is too coarse-grained to disambiguate certain attack classes, and backs this empirically in its Table III: on the CICIoMT2024 dataset, up to 39% of samples across different attack classes share an identical packet-length sequence. That means a length-only classifier is information-theoretically capped, no amount of additional training data or model capacity can separate two classes whose input features are literally identical. There's also a security argument: a detector that keys off a single, low-dimensional signal (like packet length) is easier for an adversary to evade, since they only need to manipulate one property (e.g. pad packets to a benign-looking length) to slip past detection.Flow-level aggregation (five-tuple based). Most conventional flow definitions key on the classic five-tuple (source IP, destination IP, source port, destination port, protocol). This granularity is too narrow for attacks that unfold across many short-lived flows between the same pair of hosts, port scanning is the textbook example, where each individual probe is its own tiny flow, but the pattern of many sequential probes against different ports from the same source is what signals the attack. A five-tuple flow view sees each probe in isolation and misses the aggregate behavior; the paper argues this pattern is much easier to catch when traffic is grouped at the channel level (a persistent, bidirectional IP pair), which naturally accumulates the sequence of probe packets into one observable unit.Existing "real-time" models. A number of prior systems claim real-time capability, but the paper's reading is that they mostly achieve this by shrinking the model (fewer parameters, lighter architectures) to cut inference time, without changing the data pipeline around it. If the preprocessing step still waits for a complete flow (or a fixed number of packets ω regardless of elapsed time) before handing data to the model, the end-to-end latency is dominated by that wait, not by inference, so a faster model doesn't actually bound worst-case detection latency. This is the gap the paper's time-window design (Section 3 below) is built to close directly, and it's also the basis for the later comparison against iDetector in Section 7.3. Key contributionsI'd summarize the paper's three main contributions as follows, each addressing one of the structural gaps identified in Section 2.Online time-window preprocessing (Algorithm 1). Instead of waiting for a flow to finish, the system samples traffic in fixed time windows (e.g. t = 1 second) and analyzes whatever arrived in that window immediately once it closes, it does not wait for a "logical" flow boundary like a TCP FIN/RST or an idle timeout. Concretely, the algorithm tracks a window start timestamp t0; every incoming packet whose timestamp falls within [t0, t0 + t] is accumulated, and the instant a packet arrives after t0 + t, the current window is closed, dispatched for feature extraction, and a new window begins. Because the wait is bounded by a constant (t) rather than by flow behavior (which is attacker-controlled and unbounded), detection latency becomes predictable and dominated by the window size, not by how long the underlying connection happens to stay open. This one design choice is what makes the rest of the pipeline genuinely "online" rather than just "fast."Multimodal learning - raw bytes + packet lengths, fused with a multi-loss objective. Rather than picking one modality and hoping it's discriminative enough (the single-modality problem from Section 2), the model learns separate representations from packet content (bytes) and packet timing/size patterns (lengths), then combines them. Critically, the fusion isn't just concatenation followed by a single classification loss, the model computes three separate cross-entropy losses (byte-only, length-only, and fused) and sums them during training, which forces each individual modality encoder to remain independently discriminative rather than letting one modality's gradient dominate and the other atrophy into a near-constant, unhelpful representation. The paper's ablation study quantifies this: dropping down to a length-only variant costs roughly 14 percentage points of accuracy compared to the full multimodal model, a large gap for what looks, on paper, like a "minor" architectural choice.Channel-level aggregation with a hierarchical Transformer byte encoder. Traffic is grouped by channel (bidirectional IP pair) rather than by flow, which, as argued in Section 2, captures cross-flow behavioral patterns like scanning. On top of that grouping, the byte modality is processed by two stacked Transformer encoders rather than one: a Packet-level Byte Encoder (PBE) that first learns a representation of the bytes within a single packet (and is self-supervised pre-trained BERT-style, i.e. trained to reconstruct randomly masked bytes before ever seeing a label), followed by a Channel-level Byte Encoder (CBE) that takes the sequence of per-packet embeddings produced by the PBE and models relationships between packets in the same channel. In parallel, the length modality is handled by a Bi-LSTM over the packet-length sequence, since length is inherently a temporal/ordered signal rather than a "bag of bytes."Put together, the authors claim this is the first multimodal IDS capable of true real-time online processing, as opposed to a model that is merely small and fast but still gated on flow completion. They also ship a lightweight variant, RCML-light, which drops the PBE pre-training stage and reduces Transformer/LSTM depth, running at roughly 20ms/sample, light enough to run on a resource-constrained edge device like a Raspberry Pi 4B, which the paper actually benchmarks (see Section 8 below).4. Architecture flowchart5. Datasets and why each was pickedDatasetWhy it's in the paperMQTT-IoT-IDS2020MQTT is a core IoT protocol; simulated network with 4 attack types. Tests adaptability in a simulated environment.Ton-IoTReal-world IoT traffic with 9 attack types across varied devices. Tests generalization.CICIoMT2024Internet of Medical Things: 5 attack types, 40 medical devices, multiple protocols. Tests a specialized, complex domain.Self-collectedSmall IoT network the authors built themselves (Raspberry Pi, smart lights, sensors, camera), 4 attack types. Tests real-world, low-sample-count conditions.I read the choice of four datasets as a deliberate progression rather than just "more benchmarks are better." MQTT-IoT-IDS2020 is a controlled, simulated network, so it isolates whether the architecture can learn the intended signal at all, without the confound of messy real-world noise. Ton-IoT moves to genuinely captured real-world traffic across a broader device population and nine distinct attack types, which stresses generalization, does the model's representation hold up outside the clean, simulated distribution it might have overfit to? CICIoMT2024 narrows back down to a single, high-stakes vertical (medical IoT: infusion pumps, monitors, and similar devices) with 40 physical devices and several coexisting protocols (Wi-Fi, MQTT, Bluetooth), testing whether the model transfers to a domain-specific, multi-protocol setting rather than the single-protocol MQTT case. Finally, the self-collected dataset, built by the authors from a small home-lab-scale network of a Raspberry Pi, smart lights, temperature/humidity sensors, and a camera, stress-tests the model under realistic low-sample-count conditions, which is arguably the most honest test of the four, since most real deployments will not have the luxury of large labeled datasets per attack class.All four datasets are split 9:1 train:test, with the validation set further carved out of the training portion at the same 9:1 ratio (so the full split is roughly 81% train / 9% validation / 10% test), and splits are stratified to preserve class balance. Across the three public datasets, RCML-IDS lands around ~98% accuracy; on the harder, low-sample self-collected set it still holds ~93%, a meaningful but expected drop that the paper attributes to limited per-class examples rather than a fundamental architectural weakness (see the ablation and limitations discussion below).6. Model design and hyperparametersWhy Transformers for the byte modality. The Transformer encoder is used twice in the pipeline (PBE and CBE), and I think it's the single most consequential architectural decision in the paper. The core mechanism is multi-head self-attention: for a sequence of p byte embeddings, every position computes an attention-weighted combination of every other position, so a byte near the end of the packet can directly influence, and be influenced by, a byte near the beginning, with no decay over distance the way a recurrent or convolutional receptive field would impose. This matters because protocol structure (header fields, length-prefixed fields, checksums, repeated magic bytes) often correlates bytes that are far apart in the raw byte stream. A 1D-CNN, the choice made by the AppNet baseline, only sees a local receptive field per layer and has to stack many layers to approximate long-range dependencies, which the paper implicitly argues is a weaker inductive bias for this task. Self-attention is also fully parallelizable across the sequence dimension (unlike an RNN, which is inherently sequential), which keeps inference latency reasonable even though the byte path is the most computationally expensive part of the model.Why a hierarchical PBE -> CBE split instead of one flat encoder. Rather than throwing every byte from every packet in a channel into one giant Transformer, the model factors the problem into two stages that mirror the natural structure of the data: first learn what a single packet's bytes mean (PBE), then learn how a sequence of packets within the same channel relate to each other (CBE), operating on the PBE's per-packet embeddings rather than raw bytes. This hierarchical decomposition is directly analogous to how a document-level Transformer might first encode each sentence and then encode relationships between sentence embeddings, instead of flattening the entire document into one token stream, it keeps each stage's sequence length (and therefore attention cost, which scales quadratically with sequence length) much smaller than a flat approach would require.Why BERT-style pre-training for the PBE. Before the PBE ever sees a label, it is trained as a masked byte-prediction task: each packet's byte sequence is wrapped with [PKT]/[SEP] boundary tokens, a random 15% of byte positions are replaced with a [MASK] token, and the encoder is trained to reconstruct the original byte value at each masked position from cross-entropy loss over the remaining, unmasked context, exactly BERT's masked language modeling objective, just applied to byte vocabulary instead of word/subword vocabulary. This gives the PBE a useful representation of "what byte patterns are plausible here" purely from unlabeled traffic, before any attack/benign labels are involved, and the paper's ablation shows this pre-training step alone is worth roughly 4 percentage points of accuracy, a meaningful chunk of the model's total performance for a stage that requires no labeled data at all.Why Bi-LSTM for the length modality. Packet length over time is a genuinely temporal, ordered signal, the n-th packet's length is naturally "before" the (n+1)-th, so a recurrent model that explicitly encodes order via its hidden state is a more direct fit than a Transformer would be here (especially given the length sequence is much shorter and lower-dimensional than the byte sequence, so the quadratic attention cost isn't buying much). Bidirectionality lets the representation of each position depend on both the packets that came before and after it within the window, and the final hidden state (concatenated across both directions) becomes the channel-level length embedding v_l.Hyperparameter sensitivity - n and p. Two hyperparameters received a dedicated sensitivity analysis in the paper, and both results are more interesting than "bigger is better":n = 30 packets per channel. Sweeping n from 5 to 50, accuracy rises then falls, peaking at 98.76% exactly at n = 30. Past that point, more packets don't just fail to help, they hurt, because a longer sequence adds more opportunities for irrelevant or noisy packets to dilute the channel-level representation, on top of the added computational and memory cost from a longer CBE input sequence.p = 100 bytes per packet. Sweeping p over {50, 100, 150, 200, 300}, 100 bytes gives the clearest best result. The reasoning given is specific and, I think, the more insightful of the two findings: bytes further into a packet's payload are increasingly likely to be encrypted ciphertext, which is statistically close to uniform random noise from the model's point of view. Feeding more of that noise into the PBE doesn't add signal, it dilutes the useful header/early-payload structure that is learnable, while linearly increasing training and inference time. p = 100 sits at the point where most of the learnable structure has already been captured and additional bytes are mostly cost with no return.Other architectural hyperparameters (Transformer: 8 attention heads, 2 layers, 128-dim byte embeddings; Bi-LSTM: 32-dim length embedding, 128 hidden units, bidirectional) are fixed to fairly standard mid-size Transformer/RNN defaults rather than being separately swept, the paper's tuning effort is concentrated on n and p, which is reasonable since those two control the shape of the input the whole model has to work with.7. Comparison against baselinesModelFeaturesArchitectureACC (MQTT)ERNNlengthLSTM67.10Fs-NetlengthBi-GRU + AutoEncoder73.791D-CNNbyte1D-CNN83.74YaTCbyteTransformer (MAE)94.50AppNetbyte + length1D-CNN + LSTM93.92iDetectorbyte + length + timeEdgeNet92.85RCML-IDSbyte + lengthTransformer + LSTM98.76Reading down this table by feature type tells its own story. ERNN and Fs-Net, the two length-only baselines, trail everything else by a wide margin (67–74% accuracy), direct empirical confirmation of the "packet length alone is too coarse" argument from Section 2. 1D-CNN, byte-only, does noticeably better (83.74%) simply by having access to content, but its convolutional receptive field caps how much long-range structure it can capture. YaTC, also byte-only but Transformer-based (using a masked autoencoder pre-training scheme), jumps to 94.50%, actually beating both of the other multimodal baselines (AppNet at 93.92%, iDetector at 92.85%), which is a useful sanity check that the Transformer-over-bytes idea itself, even without a second modality, is already strong. That makes the comparison against YaTC the most important one in the table: it isolates the effect of RCML-IDS's hierarchical PBE→CBE design (packet-level encoding, then channel-level encoding over packet embeddings) versus YaTC's approach of stacking raw bytes from multiple packets into one flat matrix and encoding the whole thing as a single block. RCML-IDS's additional ~4-point gain over YaTC is attributable to that hierarchical structure plus the added length modality, not to "using a Transformer" in general, since YaTC already does that.AppNet is the most directly comparable baseline architecturally, it's also byte + length multimodal, but its byte path is a 1D-CNN rather than a Transformer, and it under-performs RCML-IDS by about 5 points, which is consistent with the long-range-dependency argument from Section 6. iDetector adds a third modality (inter-packet time interval) on top of byte and length, but discretizes both length and time-interval values into a 0–255 range and reshapes them into a 2D image-like representation before encoding, the paper argues this discretization step throws away precision on what are inherently continuous-valued features, which plausibly explains why iDetector, despite using more modalities than AppNet, doesn't actually outperform it by much.The comparison I find most decisive, though, isn't accuracy at all, it's detection latency (Fig. 11 in the paper), because it's the one dimension where RCML-IDS and iDetector are trying to solve the exact same problem (online, real-time-capable multimodal detection) with fundamentally different mechanisms. iDetector is online in the sense that it doesn't wait for a complete flow, but instead of a time-window it waits for a fixed packet count ω before dispatching a sample. Because packet arrival rate is attacker- and network-controlled and highly variable, this makes iDetector's detection latency (DLT) wildly inconsistent: at its default ω = 224, the paper reports a maximum DLT of 1851 seconds and an average of 924 seconds; even at a much smaller ω = 30, the average is still around 111 seconds. RCML-IDS, by contrast, keeps DLT tightly clustered around ~1.10 seconds across conditions, because latency is bounded by the fixed time-window size t rather than by how fast packets happen to arrive. This is, to me, the paper's single most defensible and distinctive empirical result, it's not just "our model is a bit more accurate," it's "our model is the only one in this comparison whose worst-case latency is actually bounded and predictable."There's also a speed/accuracy trade-off worth calling out: RCML-IDS (full) has the longest training time of all compared methods, a direct cost of learning from raw byte content through two stacked Transformers plus a pre-training phase. RCML-light removes the pre-training stage and reduces model depth to claw back roughly an 8x inference speedup (~20ms/sample vs ~160ms/sample for the full model), at the cost of the ~4% accuracy the pre-training stage was shown to contribute, a trade the authors position as appropriate for latency-critical or resource-constrained deployments like smart-home hubs or environmental monitoring nodes, where "good enough, consistently fast" beats "best possible, occasionally slow."8. Limitations I noticedTraining-time / inference-time trade-off. Full-model training is the slowest of all compared methods, a direct consequence of the two-stage Transformer byte path plus BERT-style pre-training. RCML-light claws back roughly an 8x inference speedup by dropping pre-training and shrinking model depth, but that speed comes directly out of the ~4% accuracy the pre-training stage was shown (via ablation) to contribute, there's no free lunch here, and the paper is upfront that this is an "acceptable performance degradation" rather than a strictly better configuration.Degraded performance under low-sample-count conditions. On the self-collected dataset, accuracy drops from the ~98% seen on the three public datasets down to ~93%, which the paper attributes to a limited number of examples for some attack classes rather than a fundamental architectural weakness. This is a meaningful caveat for anyone considering deploying the approach in a genuinely new environment: performance is likely to regress until enough labeled traffic accumulates for each attack type of interest, which is exactly the situation most real, newly-deployed IoT networks start in.Narrow coverage in the real-time experiments. Because the real-time latency benchmark is built on unidirectional traffic replay (packets are replayed from a capture rather than generated by a live, interactive attacker), only attacks that don't require a response from the victim could be tested end-to-end, concretely, the two scanning attacks (scan_A, scan_sU). Attacks that inherently need two-way interaction, like SSH brute-force (Sparta) or MQTT brute-force, could not be fully simulated under this replay methodology, so the headline ~1.10s DLT figure is technically only validated for scanning-style attacks, not for the full attack taxonomy the offline accuracy numbers cover.Edge-hardware constraints force a materially different configuration. Getting the model to run on a Raspberry Pi 4B required stripping the PBE pre-training stage entirely, reducing Transformer and LSTM depth from 2 layers to 1, and cutting n from 30 down to 10 packets per channel. This is a substantial departure from the "full" architecture evaluated for the headline accuracy numbers, it implicitly concedes that the full model, as specified, does not run as-is on constrained edge hardware, and that edge deployment requires its own separately-tuned, lower-capacity configuration rather than just running the same model slower.The adversarial-robustness argument is asserted, not tested. The paper argues, as part of its motivation, that multimodal fusion should be harder to evade than single-modality detection (an attacker manipulating packet length alone, for instance, would still be caught by the byte-content signal, and vice versa). This is a reasonable hypothesis, but the paper doesn't actually construct adversarial traffic and measure whether the fused model resists evasion better than its single-modality ablations, it's listed as future work rather than demonstrated, which leaves a gap between the paper's own stated motivation and what it empirically shows.