Part 1 of this series introduced subslice and worked through the basics — what it does and a handful of detection examples. This post continues the discussion up from there: how subslice compares with other transforms, how transforms are handled during Suricata’s prefilter process, and future directions for the keyword.
You can find part 1 here.
Chaining subslice with other transforms
Part 1 showed a two-step chain without explanation: subslice: 0, 2; to_lowercase on the PE magic bytes. Transforms compose, and several can be used to get the behavior you want (there is a practical limit: 15).. The rule below trims the User-Agent to its first 4 bytes, makes it lower case, and matches on that:
alert http any any -> any any (
msg:"curl variant User-Agent";
http.user_agent; subslice: 0, 4, truncate; to_lowercase;
content:"curl"; bsize:4;
sid:2001; rev:1;)subslice trims the buffer to 4 bytes, to_lowercase normalizes the case, and content makes the actual comparison. The bsize:4 at the end isn’t strictly required, it can reduce false positives by limiting the buffer to exactly 4 bytes.
That ability to chain is what sets subslice apart from the plain content modifiers. depth:2 can tell content where to look, but it can’t hand a bounded window off to to_sha256 or from_base64 like subslice can.
How transforms interact with the prefilter
Before Suricata fully evaluates rules, it runs everything through a multi-pattern matcher (MPM) as a prefilter. During rule load time, rules with content prime the MPM with a single content match — its fast_pattern. The MPM looks for all of those patterns at the same time and returns rules that can match and thus, require complete evaluation. Most of Suricata’s speed comes from this step.
Transforms are part of this pipeline too. If the fast_pattern follows a transform chain, Suricata registers the transformed content with the MPM and runs the transform at prefilter time. It treats each distinct chain as its own buffer variant, so two rules on the same sticky buffer with different chains land in separate MPM entries. Content that follows subslice is fully eligible to serve as the fast_pattern.
Running subslice at prefilter time costs little as it only changes the offset and byte count— i.e., where the buffer starts and it’s size — so there’s nothing to copy and nothing to hash. to_sha256 is different as it has to hashes every candidate packet, using more CPU time. subslice is far simpler..
Rule evaluation with transforms
When a packet arrives, the MPM returns the rules on which fast_pattern matched, and those become the candidate rules. Every candidate rule then goes through full evaluation, which applies the transform chain to the sticky buffer and checks the content matches against transform output.
The transform runs only once per packet, and the result is cached. If the transformed content was the fast_pattern, the result from prefilter is reused. If a different content triggered the hit, the transform runs during inspection, and the cache still holds it for any other rule on the same buffer and chain.
The benefit is that the cost doesn’t multiply. Multiple rules that all have subslice: 0, 4 on http.user_agent share a single transform per packet, not multiple; if more rules are added on the same chain, there is no additional cost.
Future directions: reading the offset from the buffer
Currently, offset and nbytes have to be constants which is suitable for fixed-format protocols, known header sizes, and prefix or suffix matching, but it falls short if a field’s position depends on variable-length data ahead of it. Part 1’s third example — skipping a fixed-length header — was able to use subslice: 20 because that routing header was a fixed 20 bytes; a protocol that writes its own header length into the header is still out of reach.
Redmine issue 8234 tracks the work. There are 2 candidate syntaxes, and neither loads yet. I put “not supported yet” in each msg so a copy-paste can’t be mistaken for a working rule.
The first borrows a from xor where extract <nbytes> <offset> is inline, and the transform pulls the value straight out of the buffer when it runs with out needing a separate keyword:
alert http any any -> any any (
msg:"Custom protocol payload match (not supported yet)";
file_data; subslice: extract 2 0; content:"EXEC";
sid:2002; rev:1;)The second uses byte_extract, which already knows how to read a length into a named variable. subslice would just reference that variable, the same way plenty of other keywords already accept a byte_extract name wherever they’d otherwise take a constant.
alert http any any -> any any (
msg:"Custom protocol payload match (not supported yet)";
file_data; byte_extract:2,0,header_len; subslice: header_len; content:"EXEC";
sid:2003; rev:1;)Both use the same two bytes at offset 0 and jump past the header to the payload. The only difference is bookkeeping: the inline form keeps the length private to the transform, while the byte_extract form turns it into a variable that a later byte_test, or a content-position modifier like offset or depth, can reuse. Both still operate on file_data — the sticky-buffer requirement from Part 1 doesn’t change, so this only reaches length-prefixed data that Suricata has already parsed into an application-layer buffer, never raw TCP payload. And chaining still works: you can still add to_lowercase, from_base64, or any other transform after a dynamically-positioned subslice.
Try it out!
The examples from this post are in the following rule file along with the pcap and the script used to create the pcap. Remember, this transform is only in Suricata’s main branch
- Subslice rule file (add -S /path/to/these.rules): subslice part 2 rules
- Pcap file to use with the subslice rule file (add -r /path/to/this.pcap): subslice part 2 pcap file
- Python program that created the pcap file: subslice part 2 pcap generator script
Wrapping up
subslice closes a real gap in Suricata’s detection language. Part 1 covered the mechanics; this post filled in the rest — how it compares to and works with other transforms, prefilter behavior, and why the cost is constant no matter how many rules share a chain. This is helpful when developing rules or debugging rules.
Once reading the offset straight from the buffer is available, subslice turns into a more useful building block for binary protocol detection.