Flow state and reassembly (single pass explanation)#

Each flow is keyed by (src IP, dst IP, src port, dst port, protocol) and hashed to a worker shard. A worker owns the flow table and reassembly state for its shard, ensuring ordered handling without locks.

Collection begins on the first payload packet. The reassembler only needs a contiguous prefix from the base sequence number, so it:

• Accepts out-of-order fragments and overlaps
• Merges contiguous/overlapping ranges
• Tracks contiguous length (contigLen)
• Caps total buffered bytes (max-buffer)

Flow state progression:

• NEW → COLLECTING: first payload observed
• COLLECTING → SPLIT_READY: TLS record complete
• COLLECTING → PASS_THROUGH: timeout/parse/buffer pressure
• SPLIT_READY → INJECTED → PASS_THROUGH: split segments injected, then bypass
• CLOSED: FIN/RST or idle timeout GC

Complexity notes:

• In-order packets: O(1) amortized per packet (append)
• Out-of-order: O(log n) search + O(n) merge/shift in worst case
• Memory strictly bounded by max-buffer

TLS ClientHello detection#

Detection checks only the TLS record header and handshake type:

• contentType == 0x16
• version in 0x0301..0x0304
• handshakeType == 0x01

Once these bytes are contiguous, gov-pass reads recordLen and waits until the full record (5 + recordLen) is buffered.

Split + reinjection and defaults#

The split plan is deterministic:

Key defaults:

Tuning guidance:

• Increase max-buffer only if you see partial ClientHello records
• Decrease collect-timeout if latency spikes are unacceptable
• Reduce max-seg-payload if you need smaller injected segments

Windows (WinDivert) driver behavior and queue tuning#

On Windows, the adapter uses WinDivert at the network layer with a filter such as: outbound and ip and tcp.DstPort == 443.

Key runtime knobs map directly to WinDivert queue parameters:

• QueueLen: maximum packets held by the driver. If this is too small under bursty load, packets will be dropped and the flow will fail open.
• QueueTime: maximum time (ms) a packet can be queued before being delivered to user-space. Lower values reduce latency but increase the chance of pressure under load.
• QueueSize: total bytes allowed in the queue. This is the last line of defense against large bursts of oversized packets.

Tuning guidance:

• Start with higher QueueLen and QueueSize if you expect high fan-out (many concurrent HTTPS flows).
• Keep QueueTime low enough to avoid adding jitter to ClientHello timing, but not so low that the queue constantly flushes under load.
• If you see frequent fail-open on Windows, check queue pressure first before adjusting reassembly parameters.

Symptom to action (Windows):

WinDivert’s helper routine is used to recalculate IP/TCP checksums after packet rebuild, which keeps the injection path simple and avoids platform-specific checksum logic.

Linux (NFQUEUE + raw socket): edge cases and performance#

The Linux adapter uses NFQUEUE to receive packets and a raw socket with IP_HDRINCL for reinjection. A few details matter in practice:

• Queue bypass and loop prevention: reinjected packets must be marked (SO_MARK) and bypass NFQUEUE rules, or the flow will loop indefinitely.
• Queue pressure: queue-maxlen and kernel queue capacity determine how much backpressure is tolerated before packet loss or bypass occurs. Using --queue-bypass keeps the system fail-open during pressure.
• Copy range: larger copy-range increases user-space work but is necessary to reconstruct the full TLS record. Default is full packet.
• Offload effects: GRO/GSO/TSO can create large aggregated packets that hide true segment boundaries. Disabling offload on the egress interface stabilizes observed packet sizes during testing and validation.
• NFQUEUE errors: netlink ENOBUFS indicates queue overflow; the adapter enables netlink.NoENOBUFS to avoid noisy error storms.

Performance considerations:

• The system is latency-sensitive only during the collection window. After injection, flows are pass-through and no longer incur reassembly costs.
• Worker sharding keeps flow-local state hot in CPU cache and reduces contention.

Symptom to action (Linux):

PCAP-based validation example (illustrative)#

You can validate the split by capturing a single HTTPS request and checking that the first TLS record is segmented into multiple TCP packets.

Capture (example):

Inspect for the ClientHello record and TCP segment sizes (example):

Sample output (illustrative):

Expected pattern:

• A very small first segment (length ~5)
• One or more follow-up segments that complete the ClientHello record
• Sequence numbers increment by exactly the segment lengths

If you disable splitting (or force fail-open), you should see the ClientHello carried in a single larger TCP segment instead of the split pattern above.

Code appendix#

Detection (ClientHello)#

Role: Used by the TLS inspector before a split decision is made.

Reassembly (merge/compact)#

Role: Merges overlapping segments and compacts when the front becomes contiguous.

Injection (sendSegments + checksum)#

Role: Builds split segments, recalculates checksums, and injects packets.

Platform (WinDivert)#

Role: Opens the WinDivert handle and receives raw packets for the engine.

Test/validation snippet (PCAP helper)#

Role: Captures HTTPS traffic and prints the first data segments.

Source code#

For the full implementation and platform-specific details, see the GitLab repository: homelabird-group/gov-pass.

Primary sources: README.md, docs/DESIGN.md, docs/DESIGN_LINUX.md, docs/DESIGN_BSD.md, docs/DESIGN_ANDROID.md.