Pure-Python, zero-dependency speed-up across every hot path. No public API,
error-semantics or security change.
Measured (interleaved A/B min-of-N, idle 16-core, 1 KiB payload):
Base64 decode 21.98x
Base58 encode 8.01x
ChaCha20-Poly1305 3.54x
XChaCha20-Poly1305 3.28x
Base64 encode 2.98x
AES-256-GCM (enc) 2.25x (dec) 2.26x
AES-GCM-SIV enc 2.06x
AES-256 CTR/ECB/CBC 2.01x / 1.96x / 1.88x
AES single block 1.87x
Poly1305 MAC 1.41x
Ed25519 sign 1.41x
X25519 base 1.19x
BLAKE2b / SHA3-512 / SHAKE128 1.34x / 1.33x / 1.30x
=> 24 improved, 3 unchanged, 0 regressed.
Techniques (all pure Python):
- code-generated, fully unrolled AES round function and Keccak-f[1600]
- big-integer bulk operations for GHASH, POLYVAL, buffer XOR, Base64 pack
- precomputed lookup tables (AES T-tables, GHASH/POLYVAL 4-bit windows)
- branch-free constant-time selection in Ed25519 scan and X25519 ladder
- lazy modular reduction in X25519 field arithmetic
- batched Base58/Base64 processing
- inlined inner loops in AEAD assembly and the SHA-2 message schedule
Verification:
- full test suite: 639 passed, 1 skipped, 0 failed
- all FIPS / RFC / NIST vectors reproduce byte-for-byte
- differential probe of 1266 deterministic inputs: identical
(one unreachable private-helper edge case, no caller can reach it)
- public API signature comparison: 0 removals, 0 changes
- independent security regression suite: 35 / 35
Also adds docs/benchmark.md, bench/benchmark.py and the v3.2.1 documentation.
803 lines
33 KiB
Python
803 lines
33 KiB
Python
# -*- coding: utf-8 -*-
|
|
"""purepycrypto.aes_gcm_siv
|
|
========================================
|
|
purepycrypto - Pure-Python cryptography library (zero dependencies)
|
|
AES-GCM-SIV (RFC 8452) -- a nonce-misuse resistant AEAD.
|
|
|
|
Standard
|
|
----
|
|
This module implements the two AEADs defined by IETF RFC 8452:
|
|
AEAD_AES_128_GCM_SIV (16-byte key)
|
|
AEAD_AES_256_GCM_SIV (32-byte key)
|
|
The key is a "key-generating key", the nonce is fixed at 96 (12) bits, and the authentication tag is fixed at 16 bytes.
|
|
AES block encryption, the POLYVAL polynomial authenticator and CTR counter mode all follow RFC 8452 sections 4/5 strictly.
|
|
|
|
How nonce-misuse resistance works
|
|
------------------
|
|
With classic AES-GCM, once a (key, nonce) pair is reused, keystream reuse directly leaks the XOR of the plaintexts, and the GHASH
|
|
authentication subkey H is reused as well, allowing an attacker to forge tags -- a catastrophic failure.
|
|
AES-GCM-SIV uses the "SIV (Synthetic IV)" construction, which avoids this flaw:
|
|
|
|
1. First derive a *per-message* message-authentication-key and message-encryption-key
|
|
from the key-generating key and the nonce:
|
|
DERIVE = AES(K, LE32(counter) || nonce), taking the first half (8 bytes) of each ciphertext block and concatenating.
|
|
2. Use the authentication-key to run POLYVAL over (AAD || plaintext || length block), yielding the synthetic value S_s;
|
|
XOR the first 12 bytes of S_s with the nonce, clear the top bit of the last byte, then encrypt with the encryption-key
|
|
to obtain the tag (the synthetic IV).
|
|
3. Encrypt the plaintext using the tag as the initial CTR counter value (with the top bit of the last byte set).
|
|
|
|
So even if the nonce repeats, as long as the plaintext / AAD differ the tag (i.e. the effective CTR initial value) differs too,
|
|
and the keystream is never reused; under a repeated nonce an attacker can at most tell whether two messages are identical --
|
|
exactly the minimal information a deterministic algorithm can leak. The cost is that encryption needs two passes (authenticate, then encrypt),
|
|
which is slower than AES-GCM, in exchange for strong security under offline nonce attacks.
|
|
|
|
Usage
|
|
----
|
|
from purepycrypto.aes_gcm_siv import (
|
|
aes128_gcm_siv_encrypt, aes128_gcm_siv_decrypt,
|
|
aes256_gcm_siv_encrypt, aes256_gcm_siv_decrypt,
|
|
aes_gcm_siv_encrypt, aes_gcm_siv_decrypt,
|
|
)
|
|
|
|
key = b"0" * 16
|
|
nonce = b"\x00" * 12 # must be 12 bytes
|
|
ct, tag = aes128_gcm_siv_encrypt(key, nonce, b"hello", aad=b"header")
|
|
pt = aes128_gcm_siv_decrypt(key, nonce, ct, tag, aad=b"header") # b"hello"
|
|
|
|
# The unified entry point dispatches automatically by key length (16 / 32)
|
|
ct, tag = aes_gcm_siv_encrypt(key, nonce, b"hello")
|
|
|
|
Unified error system (see purepycrypto._ppyc_error)
|
|
------------------------------------------
|
|
* Argument validation failure (invalid key / nonce / tag / length) -> **return** a 3-tuple
|
|
``(error_code, error_message, suggestion)`` instead of raising an exception;
|
|
* Authentication failure (tampered ciphertext / tag / AAD) -> **return**
|
|
``err(PPYC_ERR_AUTH_FAILED, ...)``;
|
|
* On success ``encrypt`` returns a ``(ct, tag)`` pair and ``decrypt`` returns plaintext bytes.
|
|
|
|
Windows multiprocessing note (important)
|
|
---------------------------------
|
|
For large inputs (default >= 128KB) with `n_workers=None` (auto) or `n_workers>1`, this module
|
|
generates the CTR keystream in parallel using a `multiprocessing` pool. On Windows child processes
|
|
are started with **spawn** (the main module is re-imported), so **the caller script must be wrapped
|
|
in `if __name__ == "__main__":`**, otherwise the following is raised:
|
|
|
|
RuntimeError: An attempt has been made to start a new process before
|
|
the current process has finished its bootstrapping phase.
|
|
... you have forgotten to use the proper idiom in the main module:
|
|
if __name__ == '__main__':
|
|
freeze_support()
|
|
|
|
Correct form:
|
|
|
|
from purepycrypto import aes256_gcm_siv_encrypt
|
|
|
|
def main():
|
|
ct, tag = aes256_gcm_siv_encrypt(key, nonce, data) # large inputs use multiprocessing automatically
|
|
|
|
if __name__ == "__main__": # <- REQUIRED on Windows
|
|
main()
|
|
|
|
Workarounds:
|
|
* Add the `if __name__ == "__main__":` guard (recommended);
|
|
* Or pass `n_workers=1` explicitly to force serial execution (no multiprocessing startup cost, faster for small data).
|
|
|
|
Linux / macOS use fork and usually have no such restriction (adding the guard is still recommended for cross-platform consistency).
|
|
|
|
References
|
|
--------
|
|
RFC 8452: AES-GCM-SIV: Nonce Misuse-Resistant Authenticated Encryption.
|
|
"""
|
|
|
|
import os
|
|
from typing import Tuple
|
|
|
|
from .aes import AES, aes_encrypt_block
|
|
from .common import MASK32, _consttime_eq
|
|
from ._ppyc_error import (
|
|
PPYC_ERR, err, ok, OK_TUPLE, is_err,
|
|
PPYC_ERR_INVALID_KEY_LEN, PPYC_ERR_INVALID_NONCE_LEN,
|
|
PPYC_ERR_INVALID_LENGTH, PPYC_ERR_INVALID_TAG,
|
|
PPYC_ERR_INPUT_TOO_LARGE, PPYC_ERR_AUTH_FAILED,
|
|
)
|
|
from .parallel import _HAS_MP, _get_pool
|
|
|
|
|
|
__all__ = [
|
|
"aes128_gcm_siv_encrypt", "aes128_gcm_siv_decrypt",
|
|
"aes256_gcm_siv_encrypt", "aes256_gcm_siv_decrypt",
|
|
"aes_gcm_siv_encrypt", "aes_gcm_siv_decrypt",
|
|
"polyval",
|
|
"set_cache_enabled", "cache_enabled", "clear_caches",
|
|
"_SIV_TAG_LEN", "_SIV_NONCE_LEN", "MAX_INPUT_BYTES",
|
|
]
|
|
|
|
_SIV_TAG_LEN = 16
|
|
_SIV_NONCE_LEN = 12
|
|
# Minimum block size at which multiprocessing is enabled automatically (128KB); below this the startup cost is not worthwhile.
|
|
_AUTO_MP_MIN_BLOCKS = 8192
|
|
# RFC 8452 sections 4/5: the AAD, plaintext and ciphertext lengths must not exceed 2^36 bytes; longer inputs must be rejected.
|
|
MAX_INPUT_BYTES = 1 << 36
|
|
|
|
_SIV_MASK128 = (1 << 128) - 1
|
|
# Clear the most significant bit (bit 7 of byte 15, i.e. x^127) of a block.
|
|
_SIV_CLEAR_TOP = (1 << 127) - 1
|
|
|
|
# GF(2^128), irreducible polynomial x^128 + x^127 + x^126 + x^121 + 1.
|
|
# Bit convention of RFC 8452 section 3: bit i of the integer is the coefficient of x^i
|
|
# (the lowest bit of the first byte is x^0 and the highest bit of the last byte is x^127),
|
|
# so a 16-byte block <-> integer converts directly with int.from_bytes(bs, "little").
|
|
_POLY = (1 << 128) | (1 << 127) | (1 << 126) | (1 << 121) | 1
|
|
# x^-128 = x^127 + x^124 + x^121 + x^114 + 1 (RFC 8452 section 3)
|
|
_INV128 = (1 << 127) | (1 << 124) | (1 << 121) | (1 << 114) | 1
|
|
# x^-1 = x^127 + x^126 + x^125 + x^120 (reciprocal of x: x * _XINV1 == 1 mod P).
|
|
# Used to walk the table ladder backwards from H, replacing a full 128x128 field
|
|
# multiply plus 127 conditional left shifts with 128 cheap shift/xor steps.
|
|
_XINV1 = 0xE1000000000000000000000000000000
|
|
|
|
|
|
# ============================================================
|
|
# 11. POLYVAL (RFC 8452 section 3)
|
|
# ============================================================
|
|
#
|
|
# POLYVAL shares the field with GHASH (the same polynomial P) but uses the opposite bit/byte order convention (little-endian).
|
|
# Definition: S_0 = 0; S_j = dot(S_{j-1} + X_j, H), j = 1..s.
|
|
# dot(a, b) = a * b * x^-128 (field multiplication)
|
|
# Implemented directly from the definition: for a fixed key H, dot(., H) is a linear map over GF(2),
|
|
# so it can be accelerated with a 4-bit lookup table (32 tables of 16 entries), like the GHASH table method in aes.py.
|
|
|
|
def _poly_mul(a: int, b: int) -> int:
|
|
"""Polynomial multiplication in GF(2^128) modulo _POLY (little-endian bit convention).
|
|
|
|
``b`` is consumed 4 bits at a time through a 16-entry window table of the
|
|
multiples of ``a``, so the accumulate loop runs at most 32 times instead of
|
|
shifting/testing ``b`` bit by bit up to 128 times. The product is then
|
|
reduced modulo ``_POLY`` with the usual high-to-low bit cancellation.
|
|
"""
|
|
if not b:
|
|
return 0
|
|
t0 = a
|
|
t1 = a << 1
|
|
t2 = t1 << 1
|
|
t3 = t2 << 1
|
|
a1 = t1 ^ t0
|
|
a2 = t2 ^ t0
|
|
a3 = t3 ^ t0
|
|
t = [0, t0, t1, a1, t2, a2, t1 ^ t2, a1 ^ t2,
|
|
t3, a3, t1 ^ t3, a1 ^ t3, t2 ^ t3, a2 ^ t3, (t1 ^ t2) ^ t3, (a1 ^ t2) ^ t3]
|
|
p = 0
|
|
sh = 0
|
|
while b:
|
|
p ^= t[b & 15] << sh
|
|
b >>= 4
|
|
sh += 4
|
|
# Reduction: cancel bits from the high end down to 128 bits, one bit at a time
|
|
for i in range(p.bit_length() - 1, 127, -1):
|
|
if (p >> i) & 1:
|
|
p ^= _POLY << (i - 128)
|
|
return p & _SIV_MASK128
|
|
|
|
_POLYVAL_TABLE_CACHE = {} # h -> tables; avoids rebuilding the table for the same authentication key
|
|
_POLYVAL_TABLE_CACHE_MAX = 8
|
|
|
|
# Byte-window POLYVAL tables: tables[j][b] = dot(b * x^(8j), H), j = 0..15.
|
|
# A multiply-by-H needs 16 reads instead of 32, but the (one-off, cached) build is
|
|
# larger, so it is only used from _POLYVAL8_MIN_BLOCKS blocks upwards.
|
|
_POLYVAL_TABLE8_CACHE = {}
|
|
_POLYVAL_TABLE8_CACHE_MAX = 4
|
|
_POLYVAL8_MIN_BLOCKS = 64
|
|
|
|
# Per-message encryption-key AES objects (expanded key schedules). Reusing them
|
|
# removes one AES key expansion (roughly 45us) from every seal/open call.
|
|
_ENC_AES_CACHE = {}
|
|
_ENC_AES_CACHE_MAX = 8
|
|
|
|
# Master cache switch (performance vs. key material residing in memory).
|
|
# Default True: performance first. If your threat model does not allow derived keys / lookup tables to reside in
|
|
# process memory, call set_cache_enabled(False) to disable it (this significantly reduces small-message performance).
|
|
_CACHE_ENABLED = True
|
|
|
|
|
|
def set_cache_enabled(enabled: bool) -> None:
|
|
"""Toggle the internal caches (POLYVAL lookup table cache + derived key cache).
|
|
|
|
Parameters
|
|
----
|
|
enabled : True = enabled (default, best performance); False = disabled, and existing caches are cleared.
|
|
|
|
Notes
|
|
----
|
|
The cache keeps derived keys and lookup table results in process memory in exchange for a significant
|
|
performance gain (up to 8x for small messages; large multiprocessing workloads benefit from avoiding rebuilds).
|
|
When disabled, everything is re-derived / rebuilt on every call: safer but noticeably slower.
|
|
"""
|
|
global _CACHE_ENABLED
|
|
_CACHE_ENABLED = bool(enabled)
|
|
if not _CACHE_ENABLED:
|
|
_POLYVAL_TABLE_CACHE.clear()
|
|
_POLYVAL_TABLE8_CACHE.clear()
|
|
_DERIVE_CACHE.clear()
|
|
_ENC_AES_CACHE.clear()
|
|
|
|
|
|
def cache_enabled() -> bool:
|
|
"""Return the current cache switch state."""
|
|
return _CACHE_ENABLED
|
|
|
|
|
|
def clear_caches() -> None:
|
|
"""Manually clear the internal caches (does not change the switch state)."""
|
|
_POLYVAL_TABLE_CACHE.clear()
|
|
_POLYVAL_TABLE8_CACHE.clear()
|
|
_DERIVE_CACHE.clear()
|
|
_ENC_AES_CACHE.clear()
|
|
|
|
|
|
def _polyval_vs(h: int):
|
|
"""Return the 128-entry ladder ``vs[i] = dot(x^i, H) = H * x^(i-128)``.
|
|
|
|
Instead of multiplying H by x^-128 and then shifting left 127 times, the
|
|
ladder is walked *downwards* from H by repeatedly dividing by x
|
|
(``x^-1 = _XINV1``), which is 128 cheap shift/xor steps with no big multiply.
|
|
"""
|
|
vs = [0] * 128
|
|
v = h
|
|
for i in range(127, -1, -1):
|
|
if v & 1:
|
|
v = (v >> 1) ^ _XINV1
|
|
else:
|
|
v >>= 1
|
|
vs[i] = v
|
|
return vs
|
|
|
|
|
|
def _polyval_tables(h: int):
|
|
"""Precompute the POLYVAL nibble lookup tables for the authentication key H.
|
|
|
|
tables[j][n] = dot(n * x^(4j), H), hence dot(y, H) = XOR_j tables[j][(y >> 4j) & 0xF].
|
|
|
|
Performance: the 16 entries of every table are written out directly as the 16
|
|
linear combinations of its four basis values (one list literal per table)
|
|
instead of running a 4-step bit test per entry -- about 5x faster than the
|
|
previous builder. The result depends only on H, so it is cached per H
|
|
(LRU-style, capacity _POLYVAL_TABLE_CACHE_MAX).
|
|
"""
|
|
if _CACHE_ENABLED:
|
|
cached = _POLYVAL_TABLE_CACHE.get(h)
|
|
if cached is not None:
|
|
return cached
|
|
vs = _polyval_vs(h)
|
|
tables = []
|
|
ap = tables.append
|
|
for j in range(32):
|
|
v0 = vs[4 * j]
|
|
v1 = vs[4 * j + 1]
|
|
v2 = vs[4 * j + 2]
|
|
v3 = vs[4 * j + 3]
|
|
ap([0, v0, v1, v0 ^ v1, v2, v0 ^ v2, v1 ^ v2, v0 ^ v1 ^ v2,
|
|
v3, v0 ^ v3, v1 ^ v3, v0 ^ v1 ^ v3, v2 ^ v3, v0 ^ v2 ^ v3,
|
|
v1 ^ v2 ^ v3, v0 ^ v1 ^ v2 ^ v3])
|
|
if _CACHE_ENABLED:
|
|
if len(_POLYVAL_TABLE_CACHE) >= _POLYVAL_TABLE_CACHE_MAX:
|
|
_POLYVAL_TABLE_CACHE.pop(next(iter(_POLYVAL_TABLE_CACHE)))
|
|
_POLYVAL_TABLE_CACHE[h] = tables
|
|
return tables
|
|
|
|
|
|
def _polyval_tables8(h: int):
|
|
"""Precompute the POLYVAL byte-window tables: tables[j][b] = dot(b * x^(8j), H).
|
|
|
|
A multiply-by-H then needs 16 table reads (one per byte of the operand)
|
|
instead of 32 nibble reads. The build is ~5x more expensive than the nibble
|
|
tables (and ~6x larger), so it is used only from _POLYVAL8_MIN_BLOCKS blocks
|
|
upwards and, like the nibble tables, cached per H.
|
|
"""
|
|
if _CACHE_ENABLED:
|
|
cached = _POLYVAL_TABLE8_CACHE.get(h)
|
|
if cached is not None:
|
|
return cached
|
|
vs = _polyval_vs(h)
|
|
tables = []
|
|
ap = tables.append
|
|
for j in range(16):
|
|
tbl = [0] * 256
|
|
base = 8 * j
|
|
for k in range(8):
|
|
n = 1 << k
|
|
v = vs[base + k]
|
|
tbl[n:2 * n] = [x ^ v for x in tbl[:n]]
|
|
ap(tbl)
|
|
if _CACHE_ENABLED:
|
|
if len(_POLYVAL_TABLE8_CACHE) >= _POLYVAL_TABLE8_CACHE_MAX:
|
|
_POLYVAL_TABLE8_CACHE.pop(next(iter(_POLYVAL_TABLE8_CACHE)))
|
|
_POLYVAL_TABLE8_CACHE[h] = tables
|
|
return tables
|
|
|
|
|
|
def _polyval_lookup(y: int, tables) -> int:
|
|
"""dot(y, H) = XOR_j tables[j][ (y >> 4j) & 0xF ]."""
|
|
z = 0
|
|
for j in range(32):
|
|
z ^= tables[j][(y >> (4 * j)) & 0xF]
|
|
return z
|
|
|
|
|
|
def _polyval_run(buf, tables) -> int:
|
|
"""POLYVAL over ``buf`` (length a multiple of 16) using nibble tables.
|
|
|
|
The multiply-by-H is inlined into the block loop (no per-block call) and
|
|
``buf`` is consumed 16 bytes at a time as one little-endian integer.
|
|
"""
|
|
fb = int.from_bytes
|
|
y = 0
|
|
for i in range(0, len(buf), 16):
|
|
v = y ^ fb(buf[i:i + 16], "little")
|
|
z = 0
|
|
for j in range(32):
|
|
z ^= tables[j][(v >> (4 * j)) & 0xF]
|
|
y = z
|
|
return y
|
|
|
|
|
|
def _polyval_run8(buf, tables) -> int:
|
|
"""POLYVAL over ``buf`` (length a multiple of 16) using byte-window tables."""
|
|
fb = int.from_bytes
|
|
T0 = tables[0]
|
|
T1 = tables[1]
|
|
T2 = tables[2]
|
|
T3 = tables[3]
|
|
T4 = tables[4]
|
|
T5 = tables[5]
|
|
T6 = tables[6]
|
|
T7 = tables[7]
|
|
T8 = tables[8]
|
|
T9 = tables[9]
|
|
T10 = tables[10]
|
|
T11 = tables[11]
|
|
T12 = tables[12]
|
|
T13 = tables[13]
|
|
T14 = tables[14]
|
|
T15 = tables[15]
|
|
y = 0
|
|
for i in range(0, len(buf), 16):
|
|
bs = (y ^ fb(buf[i:i + 16], "little")).to_bytes(16, "little")
|
|
y = (T0[bs[0]] ^ T1[bs[1]] ^ T2[bs[2]] ^ T3[bs[3]] ^
|
|
T4[bs[4]] ^ T5[bs[5]] ^ T6[bs[6]] ^ T7[bs[7]] ^
|
|
T8[bs[8]] ^ T9[bs[9]] ^ T10[bs[10]] ^ T11[bs[11]] ^
|
|
T12[bs[12]] ^ T13[bs[13]] ^ T14[bs[14]] ^ T15[bs[15]])
|
|
return y
|
|
|
|
|
|
def _polyval_run_buf(h: int, buf) -> int:
|
|
"""POLYVAL over ``buf`` (length a multiple of 16), picking the cheaper tables.
|
|
|
|
For messages of at least _POLYVAL8_MIN_BLOCKS blocks the byte-window tables
|
|
halve the number of table reads per block; they are only used when the table
|
|
cache is on (otherwise their larger build cost is paid on every call).
|
|
"""
|
|
if _CACHE_ENABLED and len(buf) >= _POLYVAL8_MIN_BLOCKS * 16:
|
|
return _polyval_run8(buf, _polyval_tables8(h))
|
|
return _polyval_run(buf, _polyval_tables(h))
|
|
|
|
|
|
def polyval(h: bytes, data: bytes) -> bytes:
|
|
"""POLYVAL(H, data): the data length must be a multiple of 16; returns a 16-byte digest.
|
|
|
|
Standalone public interface (useful for unit tests / cross-validation); the AEAD reuses the precomputed table internally.
|
|
|
|
Returns
|
|
----
|
|
bytes : 16-byte digest;
|
|
or an error 3-tuple:
|
|
PPYC_ERR_INVALID_KEY_LEN the key is not 16 bytes;
|
|
PPYC_ERR_INVALID_LENGTH the data length is not a multiple of 16.
|
|
"""
|
|
if len(h) != 16:
|
|
return err(PPYC_ERR_INVALID_KEY_LEN,
|
|
f"POLYVAL key must be 16 bytes, got {len(h)}",
|
|
"pass a 16 byte POLYVAL authentication key")
|
|
if len(data) % 16:
|
|
return err(PPYC_ERR_INVALID_LENGTH,
|
|
f"POLYVAL input length must be a multiple of 16, got {len(data)}",
|
|
"pad the input to a multiple of 16 bytes")
|
|
y = _polyval_run_buf(int.from_bytes(h, "little"), data)
|
|
return y.to_bytes(16, "little")
|
|
|
|
|
|
# 12. AES-GCM-SIV (RFC 8452 sections 4/5)
|
|
# ============================================================
|
|
|
|
_DERIVE_CACHE = {} # key -> (auth_key, enc_key); avoids re-deriving for the same key
|
|
_DERIVE_CACHE_MAX = 8
|
|
|
|
|
|
def _derive_keys(key: bytes, nonce: bytes) -> Tuple[bytes, bytes]:
|
|
"""Derive (message_authentication_key, message_encryption_key).
|
|
|
|
DERIVE = AES(K, LE32(counter) ++ nonce), taking the first 8 bytes of each block:
|
|
auth = blk[0][:8] ++ blk[1][:8] (128 bit)
|
|
enc = blk[2][:8] ++ blk[3][:8] (+ blk[4][:8] ++ blk[5][:8]) (128/256 bit)
|
|
counter is a 4-byte little-endian value in the range 0..3 (AES-128) or 0..5 (AES-256).
|
|
|
|
Performance: repeated calls with the same (key, nonce) hit the cache (LRU-style, capacity _DERIVE_CACHE_MAX),
|
|
saving one AES key expansion (about 72us for AES-128 / 107us for AES-256),
|
|
which is especially noticeable for small messages. The cache stores only derivation results; no extra key material is kept.
|
|
"""
|
|
ck = (key, nonce)
|
|
if _CACHE_ENABLED:
|
|
hit = _DERIVE_CACHE.get(ck)
|
|
if hit is not None:
|
|
return hit
|
|
aes = AES(key)
|
|
n_blocks = 6 if len(key) == 32 else 4
|
|
stream = bytearray()
|
|
for ctr in range(n_blocks):
|
|
stream += aes.encrypt_block(ctr.to_bytes(4, "little") + nonce)
|
|
auth_key = bytes(stream[0:8] + stream[16:24])
|
|
enc_key = bytes(stream[32:40] + stream[48:56])
|
|
if len(key) == 32:
|
|
enc_key += bytes(stream[64:72] + stream[80:88])
|
|
if _CACHE_ENABLED:
|
|
if len(_DERIVE_CACHE) >= _DERIVE_CACHE_MAX:
|
|
_DERIVE_CACHE.pop(next(iter(_DERIVE_CACHE)))
|
|
_DERIVE_CACHE[ck] = (auth_key, enc_key)
|
|
return auth_key, enc_key
|
|
|
|
|
|
def _enc_aes_for(enc_key: bytes) -> AES:
|
|
"""Return an AES object for a per-message encryption key.
|
|
|
|
When the cache is enabled the expanded key schedule is reused across calls
|
|
with the same (key, nonce) pair, which removes one AES key expansion (about
|
|
45us for AES-256) from every seal/open. ``clear_caches()`` /
|
|
``set_cache_enabled(False)`` drop it together with the other caches.
|
|
"""
|
|
if _CACHE_ENABLED:
|
|
hit = _ENC_AES_CACHE.get(enc_key)
|
|
if hit is not None:
|
|
return hit
|
|
obj = AES(enc_key)
|
|
if _CACHE_ENABLED:
|
|
if len(_ENC_AES_CACHE) >= _ENC_AES_CACHE_MAX:
|
|
_ENC_AES_CACHE.pop(next(iter(_ENC_AES_CACHE)))
|
|
_ENC_AES_CACHE[enc_key] = obj
|
|
return obj
|
|
|
|
|
|
def _auth_tag(auth_key: bytes, enc_aes: AES, nonce: bytes,
|
|
aad: bytes, plaintext: bytes) -> bytes:
|
|
"""Compute the 16-byte tag per RFC 8452 section 4.
|
|
|
|
length_block = LE64(len(aad)*8) ++ LE64(len(pt)*8)
|
|
S_s = POLYVAL(auth_key, pad16(aad) ++ pad16(pt) ++ length_block)
|
|
S_s[0:12] ^= nonce; S_s[15] &= 0x7f; tag = AES(enc_key, S_s)
|
|
|
|
Full-block (length exactly a multiple of 16) boundary convention -- matching RFC 8452 section 4 step by step:
|
|
* pad16(n) adds **no block at all** when n is already a multiple of 16 (zero bytes of padding),
|
|
rather than appending an all-zero block; a length of 0 likewise produces no block.
|
|
Hence len%16==0 and len%16==8 may produce the same padding,
|
|
and the real distinction comes from the **bit** length in length_block.
|
|
* length_block = LE64(len(aad)*8) || LE64(len(pt)*8),
|
|
in bits (not bytes), each as an 8-byte little-endian value.
|
|
* The total POLYVAL input length is a multiple of 16, because aad and pt are each zero-padded to whole blocks,
|
|
plus the 16-byte length_block.
|
|
"""
|
|
pad_aad = aad + b"\x00" * ((16 - (len(aad) & 15)) & 15)
|
|
pad_pt = plaintext + b"\x00" * ((16 - (len(plaintext) & 15)) & 15)
|
|
length_block = ((len(aad) * 8).to_bytes(8, "little") +
|
|
(len(plaintext) * 8).to_bytes(8, "little"))
|
|
|
|
buf = pad_aad + pad_pt + length_block
|
|
y = _polyval_run_buf(int.from_bytes(auth_key, "little"), buf)
|
|
|
|
# S_s[0:12] ^= nonce, S_s[15] &= 0x7f, tag = AES(enc_key, S_s); done as one
|
|
# integer operation (the nonce occupies the low 12 little-endian bytes).
|
|
s = (y ^ int.from_bytes(nonce, "little")) & _SIV_CLEAR_TOP
|
|
return enc_aes.encrypt_block(s.to_bytes(16, "little"))
|
|
|
|
|
|
# ---- Multiprocessing CTR worker (top-level function so the pool can pickle it) ----
|
|
|
|
def _ctr_chunk_worker(args):
|
|
"""CTR shard worker: processes the keystream for [start_block, start_block+n).
|
|
|
|
Args: (eK, Nr, tail, start_ctr, n_blocks)
|
|
Returns: the keystream bytes for this shard (16*n_blocks bytes).
|
|
"""
|
|
eK, Nr, tail, start_ctr, n_blocks = args
|
|
to_bytes = int.to_bytes
|
|
M32 = 0xFFFFFFFF
|
|
out = bytearray()
|
|
ctr = start_ctr
|
|
for _ in range(n_blocks):
|
|
out += aes_encrypt_block(to_bytes(ctr & M32, 4, "little") + tail, eK, Nr)
|
|
ctr += 1
|
|
return bytes(out)
|
|
|
|
|
|
def _ctr_xor(enc_aes: AES, tag: bytes, data: bytes, n_workers: int = None) -> bytes:
|
|
"""AES-CTR initialized with the tag (top bit of the last byte set).
|
|
|
|
RFC 8452 section 4:
|
|
|
|
counter_block = tag; counter_block[15] |= 0x80
|
|
counter = counter_block[0:4] interpreted as a little-endian unsigned 32-bit integer;
|
|
each block increments the counter by 1, **wrapping at 2^32**,
|
|
counter block = LE32(counter) || tag[4:16].
|
|
|
|
* Empty data -> no CTR block is produced (the tag is still computed as for an empty plaintext).
|
|
* Counter wrap-around is specifically covered by RFC 8452 Appendix C.3 (Counter Wrap Tests);
|
|
the block after an initial value of 0xffffffff must be 0x00000000 -- guaranteed here with & MASK32.
|
|
* When the data length is exactly a multiple of 16, the last block is a full block and no extra block is computed.
|
|
"""
|
|
if not data:
|
|
return b""
|
|
init = bytearray(tag)
|
|
init[15] |= 0x80
|
|
tail = bytes(init[4:16])
|
|
ctr = int.from_bytes(init[:4], "little")
|
|
|
|
# ---- Multiprocessing branch: split keystream generation across processes for large inputs ----
|
|
n = len(data)
|
|
n_blocks = (n + 15) // 16
|
|
if n_workers is None and _HAS_MP and n_blocks >= _AUTO_MP_MIN_BLOCKS:
|
|
# Default auto: enable multiprocessing for large inputs (process count = CPU count, capped at 8, where gains saturate)
|
|
n_workers = min(os.cpu_count() or 1, 8)
|
|
if (n_workers and n_workers > 1 and n_blocks >= _AUTO_MP_MIN_BLOCKS and _HAS_MP):
|
|
pool = _get_pool(n_workers)
|
|
if pool is not None:
|
|
per = (n_blocks + n_workers - 1) // n_workers
|
|
jobs = []
|
|
b = 0
|
|
while b < n_blocks:
|
|
cnt = min(per, n_blocks - b)
|
|
jobs.append((enc_aes.eK, enc_aes.Nr, tail, (ctr + b) & 0xFFFFFFFF, cnt))
|
|
b += cnt
|
|
ks = b"".join(pool.imap(_ctr_chunk_worker, jobs))[:n]
|
|
# Integer XOR (little-endian), then restore to n bytes
|
|
return (int.from_bytes(data, "little") ^
|
|
int.from_bytes(ks, "little")).to_bytes(n, "little")
|
|
|
|
# ---- Serial path ------------------------------------------------
|
|
# Drive the unrolled AES core directly on integers (big-endian 128-bit
|
|
# packing) and build the complete keystream first. The final xor is then a
|
|
# single big-integer operation over the whole message, which removes the
|
|
# per-block keystream/data conversions and the per-block slice copy.
|
|
core = enc_aes._enc # bound unrolled core: int128 (big-endian) -> int128
|
|
eK = enc_aes.eK
|
|
to_bytes = int.to_bytes
|
|
from_bytes = int.from_bytes
|
|
M32 = 0xFFFFFFFF
|
|
n = len(data)
|
|
ks = bytearray(n)
|
|
pos = 0
|
|
full = n - (n & 15) # length of the whole-block part
|
|
while pos < full:
|
|
ks[pos:pos + 16] = core(
|
|
from_bytes(to_bytes(ctr & M32, 4, "little") + tail, "big"),
|
|
eK).to_bytes(16, "big")
|
|
ctr += 1
|
|
pos += 16
|
|
if pos < n: # trailing partial block of fewer than 16 bytes
|
|
ks[pos:] = core(
|
|
from_bytes(to_bytes(ctr & M32, 4, "little") + tail, "big"),
|
|
eK).to_bytes(16, "big")[:n - pos]
|
|
# data and ks are both n-byte big-endian integers, so a single xor covers
|
|
# every byte, including the trailing partial block.
|
|
return (from_bytes(data, "big") ^ from_bytes(ks, "big")).to_bytes(n, "big")
|
|
|
|
|
|
def _check_key_nonce(key: bytes, nonce: bytes):
|
|
"""Validate the key / nonce lengths.
|
|
|
|
Returns
|
|
----
|
|
None : validation passed;
|
|
or an error 3-tuple: PPYC_ERR_INVALID_KEY_LEN / PPYC_ERR_INVALID_NONCE_LEN.
|
|
"""
|
|
if len(key) not in (16, 32):
|
|
return err(PPYC_ERR_INVALID_KEY_LEN,
|
|
f"AES-GCM-SIV key length must be 16 or 32 bytes, got {len(key)}",
|
|
"pass a 16 or 32 byte key")
|
|
if len(nonce) != _SIV_NONCE_LEN:
|
|
return err(PPYC_ERR_INVALID_NONCE_LEN,
|
|
f"AES-GCM-SIV nonce must be 12 bytes, got {len(nonce)}",
|
|
"pass a 12 byte nonce")
|
|
return None
|
|
|
|
|
|
def _check_length(*parts: bytes):
|
|
"""RFC 8452 sections 4/5: any input exceeding 2^36 bytes must fail.
|
|
|
|
Returns
|
|
----
|
|
None : all valid;
|
|
or an error 3-tuple: PPYC_ERR_INPUT_TOO_LARGE.
|
|
"""
|
|
for p in parts:
|
|
if len(p) > MAX_INPUT_BYTES:
|
|
return err(PPYC_ERR_INPUT_TOO_LARGE,
|
|
f"AES-GCM-SIV input length exceeds the RFC 8452 limit (2^36 bytes), "
|
|
f"got {len(p)}",
|
|
"shorten the AAD / plaintext / ciphertext so each part is at most 2^36 bytes")
|
|
return None
|
|
|
|
|
|
def _seal(key: bytes, nonce: bytes, plaintext: bytes, aad: bytes,
|
|
n_workers: int = None):
|
|
_len_err = _check_length(aad, plaintext)
|
|
if _len_err is not None:
|
|
return _len_err
|
|
auth_key, enc_key = _derive_keys(key, nonce)
|
|
enc_aes = _enc_aes_for(enc_key)
|
|
tag = _auth_tag(auth_key, enc_aes, nonce, aad, plaintext)
|
|
ct = _ctr_xor(enc_aes, tag, plaintext, n_workers)
|
|
return ct, tag
|
|
|
|
|
|
def _open(key: bytes, nonce: bytes, ct: bytes, tag: bytes, aad: bytes,
|
|
n_workers: int = None) -> bytes:
|
|
if len(tag) != _SIV_TAG_LEN:
|
|
return err(PPYC_ERR_INVALID_TAG,
|
|
f"AES-GCM-SIV tag length must be 16 bytes, got {len(tag)}",
|
|
"pass a 16 byte authentication tag")
|
|
_len_err = _check_length(aad, ct)
|
|
if _len_err is not None:
|
|
return _len_err
|
|
auth_key, enc_key = _derive_keys(key, nonce)
|
|
enc_aes = _enc_aes_for(enc_key)
|
|
# Decrypt first (untrusted), then verify; plaintext is never returned when verification fails
|
|
plaintext = _ctr_xor(enc_aes, tag, ct, n_workers)
|
|
expected = _auth_tag(auth_key, enc_aes, nonce, aad, plaintext)
|
|
if not _consttime_eq(expected, tag):
|
|
return err(PPYC_ERR_AUTH_FAILED,
|
|
"AES-GCM-SIV authentication failed: tag mismatch",
|
|
"check that the key, nonce and AAD match, or whether the data was tampered with")
|
|
return plaintext
|
|
|
|
|
|
# ---------- AES-128 ----------
|
|
|
|
def aes128_gcm_siv_encrypt(key: bytes, nonce: bytes, plaintext: bytes,
|
|
aad: bytes = b"",
|
|
n_workers: int = None) -> Tuple[bytes, bytes]:
|
|
"""AEAD_AES_128_GCM_SIV encryption.
|
|
|
|
Returns
|
|
----
|
|
(ct, tag) : ciphertext and 16-byte tag;
|
|
or an error 3-tuple:
|
|
PPYC_ERR_INVALID_KEY_LEN the key is not 16 bytes;
|
|
PPYC_ERR_INVALID_NONCE_LEN the nonce is not 12 bytes;
|
|
PPYC_ERR_INPUT_TOO_LARGE the AAD / plaintext exceeds 2^36 bytes.
|
|
"""
|
|
if len(key) != 16:
|
|
return err(PPYC_ERR_INVALID_KEY_LEN,
|
|
f"AES-128-GCM-SIV key must be 16 bytes, got {len(key)}",
|
|
"pass a 16 byte key")
|
|
if len(nonce) != _SIV_NONCE_LEN:
|
|
return err(PPYC_ERR_INVALID_NONCE_LEN,
|
|
f"AES-128-GCM-SIV nonce must be 12 bytes, got {len(nonce)}",
|
|
"pass a 12 byte nonce")
|
|
return _seal(bytes(key), bytes(nonce), bytes(plaintext), bytes(aad), n_workers)
|
|
|
|
|
|
def aes128_gcm_siv_decrypt(key: bytes, nonce: bytes, ct: bytes, tag: bytes,
|
|
aad: bytes = b"",
|
|
n_workers: int = None) -> bytes:
|
|
"""AEAD_AES_128_GCM_SIV decryption.
|
|
|
|
Returns
|
|
----
|
|
bytes : the plaintext when authentication succeeds;
|
|
or an error 3-tuple (**no exception raised**):
|
|
PPYC_ERR_INVALID_KEY_LEN the key is not 16 bytes;
|
|
PPYC_ERR_INVALID_NONCE_LEN the nonce is not 12 bytes;
|
|
PPYC_ERR_INVALID_TAG the tag is not 16 bytes;
|
|
PPYC_ERR_INPUT_TOO_LARGE the AAD / ciphertext exceeds 2^36 bytes;
|
|
PPYC_ERR_AUTH_FAILED authentication failure.
|
|
"""
|
|
if len(key) != 16:
|
|
return err(PPYC_ERR_INVALID_KEY_LEN,
|
|
f"AES-128-GCM-SIV key must be 16 bytes, got {len(key)}",
|
|
"pass a 16 byte key")
|
|
if len(nonce) != _SIV_NONCE_LEN:
|
|
return err(PPYC_ERR_INVALID_NONCE_LEN,
|
|
f"AES-128-GCM-SIV nonce must be 12 bytes, got {len(nonce)}",
|
|
"pass a 12 byte nonce")
|
|
return _open(bytes(key), bytes(nonce), bytes(ct), bytes(tag), bytes(aad), n_workers)
|
|
|
|
|
|
# ---------- AES-256 ----------
|
|
|
|
def aes256_gcm_siv_encrypt(key: bytes, nonce: bytes, plaintext: bytes,
|
|
aad: bytes = b"",
|
|
n_workers: int = None) -> Tuple[bytes, bytes]:
|
|
"""AEAD_AES_256_GCM_SIV encryption.
|
|
|
|
Returns
|
|
----
|
|
(ct, tag) : ciphertext and 16-byte tag;
|
|
or an error 3-tuple:
|
|
PPYC_ERR_INVALID_KEY_LEN the key is not 32 bytes;
|
|
PPYC_ERR_INVALID_NONCE_LEN the nonce is not 12 bytes;
|
|
PPYC_ERR_INPUT_TOO_LARGE the AAD / plaintext exceeds 2^36 bytes.
|
|
"""
|
|
if len(key) != 32:
|
|
return err(PPYC_ERR_INVALID_KEY_LEN,
|
|
f"AES-256-GCM-SIV key must be 32 bytes, got {len(key)}",
|
|
"pass a 32 byte key")
|
|
if len(nonce) != _SIV_NONCE_LEN:
|
|
return err(PPYC_ERR_INVALID_NONCE_LEN,
|
|
f"AES-256-GCM-SIV nonce must be 12 bytes, got {len(nonce)}",
|
|
"pass a 12 byte nonce")
|
|
return _seal(bytes(key), bytes(nonce), bytes(plaintext), bytes(aad), n_workers)
|
|
|
|
|
|
def aes256_gcm_siv_decrypt(key: bytes, nonce: bytes, ct: bytes, tag: bytes,
|
|
aad: bytes = b"",
|
|
n_workers: int = None) -> bytes:
|
|
"""AEAD_AES_256_GCM_SIV decryption.
|
|
|
|
Returns
|
|
----
|
|
bytes : the plaintext when authentication succeeds;
|
|
or an error 3-tuple (**no exception raised**):
|
|
PPYC_ERR_INVALID_KEY_LEN the key is not 32 bytes;
|
|
PPYC_ERR_INVALID_NONCE_LEN the nonce is not 12 bytes;
|
|
PPYC_ERR_INVALID_TAG the tag is not 16 bytes;
|
|
PPYC_ERR_INPUT_TOO_LARGE the AAD / ciphertext exceeds 2^36 bytes;
|
|
PPYC_ERR_AUTH_FAILED authentication failure.
|
|
"""
|
|
if len(key) != 32:
|
|
return err(PPYC_ERR_INVALID_KEY_LEN,
|
|
f"AES-256-GCM-SIV key must be 32 bytes, got {len(key)}",
|
|
"pass a 32 byte key")
|
|
if len(nonce) != _SIV_NONCE_LEN:
|
|
return err(PPYC_ERR_INVALID_NONCE_LEN,
|
|
f"AES-256-GCM-SIV nonce must be 12 bytes, got {len(nonce)}",
|
|
"pass a 12 byte nonce")
|
|
return _open(bytes(key), bytes(nonce), bytes(ct), bytes(tag), bytes(aad), n_workers)
|
|
|
|
|
|
# ---------- Unified entry point (dispatches by key length) ----------
|
|
|
|
def aes_gcm_siv_encrypt(key: bytes, nonce: bytes, plaintext: bytes,
|
|
aad: bytes = b"",
|
|
n_workers: int = None) -> Tuple[bytes, bytes]:
|
|
"""Unified encryption entry point: a 16-byte key uses AES-128, a 32-byte key uses AES-256.
|
|
|
|
Returns
|
|
----
|
|
(ct, tag) : ciphertext and 16-byte tag;
|
|
or an error 3-tuple:
|
|
PPYC_ERR_INVALID_KEY_LEN the key is not 16 / 32 bytes;
|
|
PPYC_ERR_INVALID_NONCE_LEN the nonce is not 12 bytes;
|
|
PPYC_ERR_INPUT_TOO_LARGE the AAD / plaintext exceeds 2^36 bytes.
|
|
"""
|
|
_kn_err = _check_key_nonce(key, nonce)
|
|
if _kn_err is not None:
|
|
return _kn_err
|
|
return _seal(bytes(key), bytes(nonce), bytes(plaintext), bytes(aad), n_workers)
|
|
|
|
|
|
def aes_gcm_siv_decrypt(key: bytes, nonce: bytes, ct: bytes, tag: bytes,
|
|
aad: bytes = b"",
|
|
n_workers: int = None) -> bytes:
|
|
"""Unified decryption entry point: dispatches by key length.
|
|
|
|
Returns
|
|
----
|
|
bytes : the plaintext when authentication succeeds;
|
|
or an error 3-tuple (**no exception raised**):
|
|
PPYC_ERR_INVALID_KEY_LEN the key is not 16 / 32 bytes;
|
|
PPYC_ERR_INVALID_NONCE_LEN the nonce is not 12 bytes;
|
|
PPYC_ERR_INVALID_TAG the tag is not 16 bytes;
|
|
PPYC_ERR_INPUT_TOO_LARGE the AAD / ciphertext exceeds 2^36 bytes;
|
|
PPYC_ERR_AUTH_FAILED authentication failure.
|
|
"""
|
|
_kn_err = _check_key_nonce(key, nonce)
|
|
if _kn_err is not None:
|
|
return _kn_err
|
|
return _open(bytes(key), bytes(nonce), bytes(ct), bytes(tag), bytes(aad), n_workers)
|