Encryption & Key Exchange

Encryption requirements

Changing an encoding does not provide confidentiality by itself. FTExfil protects each chunk before it is transformed into a DNS name. The encryption layer must accept varying chunk lengths with little overhead, and it must detect modifications to ciphertext.

The design follows the AES-CTR and HMAC combination used by the original FTE construction. Encryption and authentication use separate keys.

AES in counter mode

AES is a block cipher, but counter mode turns it into a stream cipher. Successive counter values are encrypted to form a keystream, which is XORed with the plaintext. Decryption regenerates the same keystream and applies the XOR operation again.

CTR mode does not require the input to be padded to a whole AES block. This is useful for DNS tunneling because each chunk is protected separately, the final chunk may be shorter, and padding would consume scarce space in the query name.

Why authentication is necessary

Encryption alone does not prevent changes to the message. In CTR mode, flipping a ciphertext bit causes a corresponding predictable bit flip in the decrypted plaintext. A modified chunk could therefore be accepted unless the receiver also checks its integrity.

FTExfil appends a Hash-based Message Authentication Code computed with a key separate from the encryption key. After decoding an incoming name, the receiver recomputes the tag and compares it with the received one. A mismatch causes rejection.

The implementation uses HMAC-SHA512, truncated to its first 16 bytes to limit transmission overhead. The authentication tag covers the encrypted header and ciphertext. A constant-time comparison is supplied by the subtle crate.

Encrypted chunk structure

The encryption envelope has three parts:

The envelope adds 32 bytes to the encrypted data length. The transfer layer’s 16-byte metadata section is appended to the file fragment before encryption and is separate from this envelope overhead.

The session key is divided into two 16-byte keys: $K_1$ for AES and $K_2$ for HMAC. The cryptographic primitives are supplied by Rust crates rather than newly implemented cipher or hash functions.

Establishing a shared key

The sender and receiver can use either a fixed development key or an ECDH exchange. The none option is intended for debugging and does not provide meaningful secrecy: it uses a hard-coded key rather than a freshly established session key.

Finite-field Diffie–Hellman

In traditional Diffie–Hellman, the parties agree on public parameters $p$ and $g$, choose private exponents $a$ and $b$, and exchange

Both obtain the same shared secret:

A 2048-bit public value is already 256 bytes before encoding. Such a value cannot fit in a single DNS query name alongside labels and the controlled domain. Larger parameter sizes make the problem more pronounced.

Elliptic-curve Diffie–Hellman

Elliptic-curve variants achieve comparable security with much smaller public keys. A private scalar $d$ produces a public point $Q=dG$, where $G$ is a fixed public curve point. Combining a private scalar with the other party’s public value gives the common shared secret.

FTExfil supports the following curves:

The small keys allow an exchange through DNS TXT messages without the multi-query fragmentation needed for a large finite-field public value.

DNS handshake

The client generates an ephemeral private key, its corresponding public key, and a random 64-bit transfer identifier. It initiates the exchange with a TXT query of the form

k1.<transfer_id_hex>.<curve>.<base32_public_key>.<domain>

The server validates the structure and requested mode, decodes the client’s public key, creates an ephemeral key pair of its own, and computes the shared secret. Its public key is returned in a TXT response:

k1:<curve>:<base32_public_key>

The client then computes the same shared secret using its private key and the received server public key.

Session-key derivation

The raw ECDH shared secret is not used directly as an encryption key. It is passed through HKDF-SHA256 with the transfer identifier as salt and a curve-specific context string:

salt = transfer_id
info = "ftexfil-kex-v1-<curve>"
output length = 32 bytes

The 32-byte output is split into $K_1$ and $K_2$, separating encryption and message authentication. Key exchange affects the initial messages of a transfer rather than every individual chunk, so its cost is small relative to the thousands of DNS packets generated by a large file.

Cryptographic components