｢‍｣ Lingenic

HSV Streaming v1.5

Framing, flow control, and device switching

HSV Streaming defines how to send HSV over connections: WebSocket, TCP, serial. Framing with STX/ETX, flow control with XON/XOFF, chunked transfers with ETB/EM, device switching with DC2/DC4. HSV Streaming assumes ordered delivery. Conforms to RFC 20.

Message Types

Three forms:

[SOH] [header] [STX] [content] [ETX]    Header + content
[STX] [content] [ETX]                   Content only
[SOH] [header] [STX] [ETX]              Header only (empty content)

SOH starts a header. STX terminates the header and starts content. ETX ends the block.

This matches the original ASCII design: SOH begins routing/control info, STX marks where content begins, ETX marks the end.

Content Blocks

STX/ETX frames content. Any payload format works:

# Plain text
[STX] Hello World [ETX]

# JSON
[STX] {"user": "alice", "action": "login"} [ETX]

# HSV records
[STX] name [US] Alice [RS] age [US] 30 [ETX]

# XML
[STX] <message>Hello</message> [ETX]

# Binary (shift-encoded — see Binary Transfer)
[STX] [SO] K [SI] PNG [CR] [LF] [SO] ; [SI] [LF] ... [ETX]

# Multi-line content (newlines are literal)
[STX] Line one
Line two
Line three [ETX]

# Code with special characters (no escaping needed)
[STX] if (x < 10 && y > 5) { return "yes"; } [ETX]

# Unicode, emoji, RTL, CJK (all valid)
[STX] Hello 世界! Привет мир! مرحبا 🎉🚀✨ [ETX]

# Math and symbols
[STX] ∀x ∈ ℝ: x² ≥ 0 ∧ √(x²) = |x| [ETX]

# Mixed scripts
[STX] English · Ελληνικά · עברית · 日本語 · 한국어 [ETX]

Any data format inside. Seven control characters pass through as literal data:

These can be used for terminal animation or any application needing whitespace/formatting.

Headers

SOH starts a header, STX ends it:

[SOH] hsv [US] 1.5 [RS] type [US] request [STX] [content] [ETX]    Header + content
[SOH] route [US] chat [RS] user [US] alice [STX] [content] [ETX]   Routing + content
[SOH] ack [US] msg-123 [STX] [ETX]                                 Header only (control message)

Headers use HSV structure ([US] for key-value, [RS] for properties). Applications define their own header schemas.

Session

Server sends header on connect:

[SOH] hsv [US] 1.5 [RS] protocol [US] stream [RS] maxlen [US] 65536 [STX] [ETX]

maxlen advertises the maximum content block size in bytes that the sender will accept. If omitted, the limit is implementation-defined.

End stream:

␄

Flow Control

␓ (0x13)    XOFF - pause processing
␑ (0x11)    XON - resume processing
␖ (0x16)    Sync/keepalive
␆ (0x06)    Acknowledged
␕ (0x15)    Error/resend
␘ (0x18)    Cancel operation
␅ (0x05)    Enquiry (request status)

XOFF/XON are application-level backpressure: the receiver is saying "I can't process more right now" / "ready again." This is distinct from transport-level flow control (e.g., TCP window sizing), which operates independently. Over transports with built-in flow control, XOFF/XON signal application readiness, not byte-level throttling.

ENQ requests a status response from the peer — the expected reply is ACK (operational) or NAK (error). Use it for health checks or to verify a connection is still live.

Acknowledgment

Bare ␆ and ␕ acknowledge the last received message — sufficient for serial or simple request/response. For concurrent messages, use HSV headers to correlate by message ID or sequence number:

# Sender tags a message with an ID
[SOH] id [US] msg-42 [STX] Hello everyone! [ETX]

# Receiver acknowledges by ID
[SOH] ack [US] msg-42 [STX] [ETX]

# Receiver rejects by ID
[SOH] nak [US] msg-42 [STX] [ETX]

# Sequence numbers (integer)
[SOH] seq [US] 1 [STX] First message [ETX]
[SOH] seq [US] 2 [STX] Second message [ETX]
[SOH] ack [US] 2 [STX] [ETX]                   Acknowledges up to seq 2

The correlation scheme (message IDs, sequence numbers, or both) is application-defined. HSV provides the framing; the application chooses its acknowledgment model.

Chunked Transfer

␗ (0x17)    End of chunk, more coming
␙ (0x19)    End of medium, data exhausted

Content exceeding the receiver's maxlen MUST be sent as ETB-delimited chunks, each within the limit, with ETX on the final chunk:

# Sending a large payload (maxlen = 65536)
[STX] [first 65536 bytes] ␗
[second 65536 bytes] ␗
[remaining bytes] [ETX]

The receiver concatenates chunks until ETX. ␙ signals "no more data" (distinct from ␄ which ends the stream) — use it when the total length is unknown and the source is exhausted.

Error Recovery

Either side can send CAN (␘) to abort the current block. The sender sends CAN to say "ignore what I was sending." The receiver sends CAN to say "stop, I can't process this." Both sides discard any partial content in progress.

Three conditions trigger recovery:

• Timeout — STX received but ETX never arrives (sender crashed, connection dropped). Receiver discards the partial block and sends NAK.
• Oversize — content exceeds maxlen without ETB chunking. Receiver discards and sends NAK.
• Cancel — CAN received mid-block. Receiver discards the partial block.

Resync: after discarding, the receiver scans forward for the next SOH (0x01) or STX (0x02) to find the start of the next valid message. This is reliable because reserved bytes never appear in content — unlike text-delimited protocols, HSV resync is deterministic.

Binary Transfer

Binary data contains reserved bytes that would break HSV framing. Shift encoding remaps control codes to printable ASCII using [SO]/[SI], preserving every streaming property: deterministic resync, parallel safety, clean ETB chunking. No escaping, no base64.

File Transfer

Header carries metadata, content carries shift-encoded bytes:

# Send a file (small, fits in one block)
[SOH] type [US] file [RS] name [US] photo.png [RS] size [US] 48230 [RS] encoding [US] shift [STX] [SO] K [SI] PNG ... [SO] ; [SI] [LF] [shift-encoded bytes] [ETX]

# Acknowledge receipt
[SOH] ack [US] msg-1 [STX] [ETX]

Chunked File Transfer

Large files chunk via ETB, each chunk within maxlen:

# Header declares the transfer
[SOH] type [US] file [RS] name [US] backup.tar [RS] size [US] 5242880 [RS] encoding [US] shift [RS] id [US] xfer-7 [STX]
[shift-encoded chunk 1] ␗
[shift-encoded chunk 2] ␗
...
[shift-encoded final chunk] [ETX]

# Receiver acknowledges the complete transfer
[SOH] ack [US] xfer-7 [STX] [ETX]

Each chunk boundary falls on a shift-safe byte — ETB (0x17) cannot appear in shift-encoded output, so boundaries are unambiguous. The receiver concatenates and shift-decodes after reassembly.

Why Not DLE Transparency?

DLE wrapping ([DLE][SPA]…[DLE][EPA]) passes raw bytes through but sacrifices streaming guarantees: resync fails because any byte can appear inside the block, ETB chunking is ambiguous, and parsing is sequential. Shift encoding costs ~15% overhead on dense binary (vs. base64's 33%) and preserves the protocol's structural properties. DLE transparency is available for local/embedded use where both ends are tightly coupled and streaming properties aren't needed.

Backend Switching

Device Control characters for distributed architecture:

␒ (0x12)    Connect to backend
␔ (0x14)    Disconnect (return to lobby)

DC2/DC4 prefix a standard HSV message carrying the device name. The lobby responds with ACK on successful connection or NAK (with the device name) on failure:

# Connect to chat service
␒ [SOH] dc [US] chat [STX] [ETX]

# Lobby responds: success
␆

# Lobby responds: backend not found
[SOH] nak [US] chat [STX] [ETX]

# Disconnect from chat (return to lobby)
␔ [SOH] dc [US] chat [STX] [ETX]

Reserved vs Allowed vs Forbidden

Why Forbidden?

NUL (0x00) — C string terminator. Functions like strlen(), strcpy(), and most string APIs stop at NUL. Data gets silently truncated. Database fields, log entries, filenames — anything touching C code risks corruption. Even "safe" languages often use C libraries underneath.

SUB (0x1A) — Ctrl+Z, the DOS/Windows end-of-file marker. Open a file in text mode on Windows, and reading stops at 0x1A. Streams get truncated mid-transfer. Copy a file with SUB in it, get half a file. This isn't legacy — it's still the default behavior in 2026.

ESC (0x1B) — The escape character that starts terminal control sequences. An attacker who can inject ESC into your stream can: change terminal colors, move the cursor, clear the screen, redefine keys, and in some terminals execute arbitrary commands. Decades of CVEs. HSV forbids ESC entirely — if you need terminal control, use HSV's structured operations instead.

If your data contains these bytes, use binary mode (DLE transparency).

Comparison

vs JSON Streaming (NDJSON)

{"msg":"line 1\nline 2"}
{"code":"if (x < 3 \u0026\u0026 y > 5)"}
{"html":"\u003cdiv\u003eHello\u003c/div\u003e"}

[STX] {"msg":"line 1
line 2"} [ETX]
[STX] {"code":"if (x < 3 && y > 5)"} [ETX]
[STX] {"html":"<div>Hello</div>"} [ETX]

NDJSON uses newline as its record delimiter, so newlines in values must be escaped. Many JSON serializers also escape <, >, & for HTML-embedding safety (not required by RFC 8259, but common in practice). HSV frames the JSON—content passes through unchanged.

vs Length-Prefixed

0x00 0x00 0x00 0x0C Hello World!

Must buffer entire message to know length.
Can't stream partial content.
Binary header not human-readable.

[STX] Hello World! [ETX]

Stream bytes as they arrive.
Send ␗ for partial chunks.
Inspect with any text tool.

vs WebSocket

Message framing only.
No routing metadata standard.
No acknowledgment protocol.
No backpressure signaling.
No chunked transfer.

[SOH] route [US] chat [STX] Hello! [ETX]
Routing in header, content in body.
␆ acknowledge, ␕ request resend.
␓ pause, ␑ resume.
␗ partial, ␙ complete.

WebSocket provides reliable transport. HSV adds application-level semantics: routing, acknowledgment, flow control, chunking.

Multi-format Example

[SOH] type [US] json [STX] {"x":1} [ETX]
[SOH] type [US] xml [STX] <x>1</x> [ETX]
[SOH] type [US] text [STX] Hello 🌍 [ETX]
[SOH] type [US] hsv [STX] x [US] 1 [ETX]

Framing is consistent
Payload format varies
Routing via headers
No escaping in text mode*

HSV Streaming adds framing to any payload format. Content can contain newlines, quotes, emoji, any bytes except the 26 reserved.

Transport

Where It's Useful

Serial and Embedded

STX/ETX framing was designed for serial lines. HSV Streaming gives UART, RS-232, and IoT devices a structured protocol—headers, types, acknowledgment, flow control—without HTTP overhead. The maxlen negotiation matters here: a microcontroller with 4 KB of RAM can advertise maxlen [US] 2048 and the sender will chunk accordingly.

WebSocket Application Layer

WebSocket gives you a reliable bidirectional pipe but no application semantics. Today that gap is filled by ad-hoc JSON envelope protocols ({"type":"msg","payload":...}). HSV Streaming fills it with a standard framing layer: message IDs, ACK/NAK, backpressure, backend switching, chunked transfer. The transport is already in every browser.

Server-to-Server

HSV over raw TCP is an alternative to gRPC or custom binary protocols for internal services. The stream is inspectable with text tools—cat, hexdump, pipe through grep—because the framing is visible ASCII delimiters, not length-prefixed binary. Useful for debugging and for environments where adding a Protobuf dependency is unwelcome.

Terminal Security

ESC (0x1B) is forbidden in HSV. This eliminates escape sequence injection—an entire class of terminal attack where untrusted data containing VT100 sequences can rewrite the screen, change window titles, or trigger terminal emulator vulnerabilities. HSV Streaming separates data from control messages structurally, not by parsing escape sequences out of a mixed byte stream.

Example: Chat Protocol

# Connect (header only)
[SOH] hsv [US] 1.5 [RS] protocol [US] chat [RS] user [US] alice [STX] [ETX]

# Send message with ID
[SOH] id [US] 1 [STX] Hello everyone! [ETX]

# Receive message with ID
[SOH] from [US] bob [RS] id [US] 7 [STX] Hi Alice! [ETX]

# Acknowledge by ID
[SOH] ack [US] 7 [STX] [ETX]

# Disconnect
␄

Simple request/response, pub/sub, or bidirectional — HSV Streaming provides the framing and flow control.