Files
dvs dd49043171 v4.2.6: Self-describing format, byte pipeline, and a performance pass
Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
2026-09-27 22:06:34 +08:00

417 lines
16 KiB
Python

# -*- coding: utf-8 -*-
"""
Performance-mode tests.
Four modes exist:
reference the original algorithm, character pipeline
fast same algorithm, bulk-operation implementation, identical output
turbo SHAKE256 keystream, byte pipeline, own ciphertext format
raw byte pipeline with the long-key layer retained
``reference`` and ``fast`` produce indistinguishable ciphertexts and always use
hex. ``turbo`` and ``raw`` accept either ``hex`` or ``base64`` output, and
either a ``bt`` (bytes) or ``hex`` intermediate representation.
Every artefact carries a self-describing header, so anything that inspects the
ciphertext body strips it first, and anything that feeds headerless input
expects ``E017``.
"""
import os
import pytest
from chaoscrypt import (
ChaosCrypt,
ChaosCryptCore,
ChaosCryptError,
MODES,
MODE_REFERENCE,
MODE_FAST,
MODE_TURBO,
MODE_RAW,
DEFAULT_MODE,
ENCODINGS,
ENC_HEX,
ENC_BASE64,
MIDS,
MID_BYTES,
MID_HEX,
)
from chaoscrypt._header import MAGIC, strip_header, read_header
from ._vectors import load
KEY_PWD = "mode-key-password"
USER_PWD = "mode-user-password"
SAMPLES = [
"hello",
"hello world",
"emoji test",
"a",
"x" * 300,
"Mixed 123!@#$%^&*",
"line1\nline2\ttabbed",
]
def _payload(text: str) -> str:
"""The payload of an artefact, with its self-describing header removed."""
return strip_header(text)
def _core(mode, vectors, enc="hex", mid="bt"):
"""A ChaosCryptCore in ``mode`` carrying the frozen key material."""
km = vectors["key_material"]
c = ChaosCryptCore(mode, "hash", enc, mid)
c.upper_mapping = km["upper_mapping"]
c.lower_mapping = km["lower_mapping"]
c.digit_mapping = km["digit_mapping"]
c.equal_mapping = km["equal_mapping"]
c.flip_pattern = km["flip_pattern"]
c.long_key = km["long_key"]
c.short_key = km["short_key"]
c.digit_reverse = {v: k for k, v in c.digit_mapping.items()}
c.equal_reverse = {v: k for k, v in c.equal_mapping.items()}
c._build_maps()
c.keys_loaded = True
return c
@pytest.fixture(scope="module")
def vectors():
return load("chaoscrypt_reference")
@pytest.fixture(scope="module")
def cores(vectors):
return {m: _core(m, vectors) for m in MODES}
@pytest.fixture
def keyfile(tmp_path):
path = str(tmp_path / "modes.key")
ChaosCrypt().create_key(KEY_PWD, path)
return path
# ---------------------------------------------------------------------------
# Mode surface
# ---------------------------------------------------------------------------
class TestModeSurface:
def test_all_four_modes_are_exported(self):
assert MODES == (MODE_REFERENCE, MODE_FAST, MODE_TURBO, MODE_RAW)
def test_default_mode_is_fast(self):
assert DEFAULT_MODE == MODE_FAST
def test_core_defaults_to_the_default_mode(self):
assert ChaosCryptCore().mode == DEFAULT_MODE
def test_unknown_mode_is_rejected(self):
with pytest.raises(ValueError):
ChaosCryptCore("turbo-charged")
def test_valid_mode_is_accepted(self):
for m in MODES:
assert ChaosCryptCore(m).mode == m
# ---------------------------------------------------------------------------
# reference and fast
# ---------------------------------------------------------------------------
class TestReferenceAndFast:
def test_reference_reproduces_frozen_vectors(self, cores, vectors):
core = cores[MODE_REFERENCE]
pwd = vectors["passwords"]["user_password"]
for case in vectors["cases"]:
if case.get("mode", "fast") not in ("reference", "fast"):
continue
got = _payload(core.encrypt_text(case["plaintext"], pwd))
want = _payload(case["ciphertext"])
assert got == want, f"reference drifted for {case['id']}"
def test_fast_is_indistinguishable_from_reference(self, cores, vectors):
"""The whole point of `fast`: same bytes, less time."""
pwd = vectors["passwords"]["user_password"]
for case in vectors["cases"]:
a = cores[MODE_REFERENCE].encrypt_text(case["plaintext"], pwd)
b = cores[MODE_FAST].encrypt_text(case["plaintext"], pwd)
# payloads identical; only the recorded mode differs
assert _payload(a) == _payload(b), case["id"]
assert read_header(a)["mode"] == MODE_REFERENCE
assert read_header(b)["mode"] == MODE_FAST
@pytest.mark.parametrize("text", SAMPLES)
def test_fast_matches_reference_on_samples(self, cores, text):
a = _payload(cores[MODE_REFERENCE].encrypt_text(text, USER_PWD))
b = _payload(cores[MODE_FAST].encrypt_text(text, USER_PWD))
assert a == b
def test_agree_on_large_input(self, cores):
big = "A" * 40_000
a = _payload(cores[MODE_REFERENCE].encrypt_text(big, USER_PWD))
b = _payload(cores[MODE_FAST].encrypt_text(big, USER_PWD))
assert a == b
@pytest.mark.parametrize("mode", [MODE_REFERENCE, MODE_FAST])
def test_ciphertext_payload_is_hex(self, cores, mode):
int(_payload(cores[mode].encrypt_text("data", USER_PWD)), 16)
# ---------------------------------------------------------------------------
# turbo and raw
# ---------------------------------------------------------------------------
class TestTurboAndRaw:
@pytest.mark.parametrize("mode", [MODE_TURBO, MODE_RAW])
@pytest.mark.parametrize("text", SAMPLES)
def test_roundtrip(self, cores, mode, text):
ct = cores[mode].encrypt_text(text, USER_PWD)
assert cores[mode].decrypt_text(ct, USER_PWD) == text
@pytest.mark.parametrize("mode", [MODE_TURBO, MODE_RAW])
@pytest.mark.parametrize("size", [1, 2, 255, 4096])
def test_binary_roundtrip(self, cores, mode, size):
raw = os.urandom(size)
ct = cores[mode].encrypt_bytes(raw, USER_PWD)
assert cores[mode].decrypt_bytes(ct, USER_PWD) == raw
@pytest.mark.parametrize("mode", MODES)
def test_every_ciphertext_carries_a_header(self, cores, mode):
ct = cores[mode].encrypt_text("data", USER_PWD)
assert ct.startswith(MAGIC.decode())
assert read_header(ct)["kind"] == "ciphertext"
@pytest.mark.parametrize("mode", [MODE_TURBO, MODE_RAW])
def test_wrong_password_fails(self, cores, mode):
ct = cores[mode].encrypt_text("secret", USER_PWD)
with pytest.raises(ChaosCryptError):
cores[mode].decrypt_text(ct, "wrong-password")
# ---------------------------------------------------------------------------
# Intermediates and encodings
# ---------------------------------------------------------------------------
class TestIntermediatesAndEncodings:
@pytest.mark.parametrize("mode", [MODE_TURBO, MODE_RAW])
@pytest.mark.parametrize("mid", MIDS)
@pytest.mark.parametrize("enc", ENCODINGS)
def test_all_combinations_round_trip(self, vectors, mode, mid, enc):
c = _core(mode, vectors, enc, mid)
for text in SAMPLES:
ct = c.encrypt_text(text, USER_PWD)
assert c.decrypt_text(ct, USER_PWD) == text, (mode, mid, enc)
@pytest.mark.parametrize("mode", [MODE_TURBO, MODE_RAW])
@pytest.mark.parametrize("mid", MIDS)
def test_binary_round_trip_all_mids(self, vectors, mode, mid):
c = _core(mode, vectors, "hex", mid)
raw = os.urandom(2_000)
assert c.decrypt_bytes(c.encrypt_bytes(raw, USER_PWD), USER_PWD) == raw
@pytest.mark.parametrize("mid", MIDS)
def test_header_records_the_mid(self, vectors, mid):
for mode in (MODE_TURBO, MODE_RAW):
c = _core(mode, vectors, "hex", mid)
cfg = read_header(c.encrypt_text("x", USER_PWD))
assert cfg["mid"] == mid, mode
@pytest.mark.parametrize("enc", ENCODINGS)
def test_header_records_the_encoding(self, vectors, enc):
for mode in (MODE_TURBO, MODE_RAW):
c = _core(mode, vectors, enc, "bt")
cfg = read_header(c.encrypt_text("x", USER_PWD))
want = "base" if enc == ENC_BASE64 else "hex"
assert cfg["out"] == want, mode
def test_legacy_modes_always_report_hex(self, vectors):
for mode in (MODE_REFERENCE, MODE_FAST):
c = _core(mode, vectors, ENC_BASE64, MID_HEX)
cfg = read_header(c.encrypt_text("x", USER_PWD))
assert cfg["out"] == "hex"
assert cfg["mid"] == "hex"
def test_hex_artefact_is_larger_than_bt(self, vectors):
"""A hex intermediate doubles the payload at each layer."""
big = "A" * 5_000
sizes = {}
for mid in MIDS:
c = _core(MODE_TURBO, vectors, "hex", mid)
sizes[mid] = len(_payload(c.encrypt_text(big, USER_PWD)))
assert sizes[MID_HEX] > sizes[MID_BYTES]
def test_base64_output_is_smaller_than_hex(self, vectors):
big = "A" * 5_000
sizes = {}
for enc in ENCODINGS:
c = _core(MODE_TURBO, vectors, enc, MID_BYTES)
sizes[enc] = len(_payload(c.encrypt_text(big, USER_PWD)))
assert sizes[ENC_BASE64] < sizes[ENC_HEX]
def test_hex_payload_is_hex(self, vectors):
c = _core(MODE_TURBO, vectors, "hex", MID_BYTES)
int(_payload(c.encrypt_text("data", USER_PWD)), 16)
def test_base64_payload_is_base64(self, vectors):
import base64 as b64
c = _core(MODE_TURBO, vectors, ENC_BASE64, MID_BYTES)
b64.b64decode(_payload(c.encrypt_text("data", USER_PWD)).encode(),
validate=True)
def test_unknown_encoding_rejected(self):
with pytest.raises(ValueError):
ChaosCryptCore(MODE_TURBO, encoding="base85")
def test_unknown_mid_rejected(self):
with pytest.raises(ValueError):
ChaosCryptCore(MODE_TURBO, mid="nonsense")
def test_mid_aliases_are_accepted(self):
"""'bytes' is an accepted spelling of 'bt', and 'base' of 'base64'."""
assert ChaosCryptCore(MODE_TURBO, mid="bytes").mid == MID_BYTES
assert ChaosCryptCore(MODE_TURBO, encoding="base").encoding == ENC_BASE64
# ---------------------------------------------------------------------------
# The header decides how an artefact is read
# ---------------------------------------------------------------------------
class TestHeaderDrivesDecryption:
def test_a_misconfigured_reader_still_works(self, vectors):
"""The artefact states its own parameters; the reader need not."""
writer = _core(MODE_TURBO, vectors, ENC_BASE64, MID_HEX)
ct = writer.encrypt_text("header rules", USER_PWD)
reader = _core(MODE_TURBO, vectors, "hex", MID_BYTES) # wrong on both
assert reader.decrypt_text(ct, USER_PWD) == "header rules"
def test_headerless_input_is_rejected(self, cores):
for mode in MODES:
with pytest.raises(ChaosCryptError) as ei:
cores[mode].decrypt_text("72370655cafe", USER_PWD)
assert ei.value.code == "E017", mode
def test_malformed_payload_is_rejected(self, vectors):
from chaoscrypt._header import build_header
c = _core(MODE_TURBO, vectors)
bad = build_header(c.header_config("ciphertext")) + "zz"
with pytest.raises(ChaosCryptError) as ei:
c.decrypt_text(bad, USER_PWD)
assert ei.value.code == "E010"
# ---------------------------------------------------------------------------
# Key material must still contribute
# ---------------------------------------------------------------------------
class TestKeyMaterialContributes:
@pytest.mark.parametrize("mode", MODES)
def test_long_key_contributes(self, vectors, mode):
a = _core(mode, vectors)
b = _core(mode, vectors)
b.long_key = "0" * 4096
b._build_maps()
assert (a.encrypt_text("UPPER and lower", USER_PWD)
!= b.encrypt_text("UPPER and lower", USER_PWD))
@pytest.mark.parametrize("mode", MODES)
def test_short_key_contributes(self, vectors, mode):
a = _core(mode, vectors)
b = _core(mode, vectors)
b.short_key = "0" * 512
b._build_maps()
assert (a.encrypt_text("UPPER and lower", USER_PWD)
!= b.encrypt_text("UPPER and lower", USER_PWD))
@pytest.mark.parametrize("mode", MODES)
def test_password_contributes(self, vectors, mode):
c = _core(mode, vectors)
assert (c.encrypt_text("same", "pw-a") != c.encrypt_text("same", "pw-b"))
@pytest.mark.parametrize("mode", MODES)
def test_substitution_tables_contribute(self, vectors, mode):
a = _core(mode, vectors)
b = _core(mode, vectors)
b.upper_mapping = "ZYXWVUTSRQPONMLKJIHGFEDCBA"
b._build_maps()
# both cases present, so upper_mapping cannot be bypassed
assert (a.encrypt_text("UPPER and lower", USER_PWD)
!= b.encrypt_text("UPPER and lower", USER_PWD))
@pytest.mark.parametrize("mode", MODES)
def test_lower_mapping_contributes(self, vectors, mode):
a = _core(mode, vectors)
b = _core(mode, vectors)
b.lower_mapping = "zyxwvutsrqponmlkjihgfedcba"
b._build_maps()
assert (a.encrypt_text("UPPER and lower", USER_PWD)
!= b.encrypt_text("UPPER and lower", USER_PWD))
@pytest.mark.parametrize("mode", MODES)
def test_flip_pattern_contributes(self, vectors, mode):
a = _core(mode, vectors)
b = _core(mode, vectors)
b.flip_pattern = [0] * 10
sample = "UPPER and lower"
assert (a.encrypt_text(sample, USER_PWD)
!= b.encrypt_text(sample, USER_PWD))
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
class TestPublicAPI:
@pytest.mark.parametrize("mode", MODES)
def test_facade_roundtrip(self, keyfile, mode):
cc = ChaosCrypt(key_file=keyfile, key_password=KEY_PWD,
auto_load=True, mode=mode)
ct = cc.encrypt("facade mode test", USER_PWD)
assert cc.decrypt(ct, USER_PWD) == "facade mode test"
@pytest.mark.parametrize("enc", ENCODINGS)
def test_facade_supports_encoding(self, keyfile, enc):
cc = ChaosCrypt(key_file=keyfile, key_password=KEY_PWD, auto_load=True,
mode=MODE_TURBO, encoding=enc)
ct = cc.encrypt("encoding via facade", USER_PWD)
assert cc.decrypt(ct, USER_PWD) == "encoding via facade"
@pytest.mark.parametrize("mid", MIDS)
def test_facade_supports_mid(self, keyfile, mid):
cc = ChaosCrypt(key_file=keyfile, key_password=KEY_PWD, auto_load=True,
mode=MODE_TURBO, mid=mid)
ct = cc.encrypt("mid via facade", USER_PWD)
assert cc.decrypt(ct, USER_PWD) == "mid via facade"
@pytest.mark.parametrize("mode", MODES)
def test_facade_file_roundtrip(self, keyfile, tmp_path, mode):
cc = ChaosCrypt(key_file=keyfile, key_password=KEY_PWD,
auto_load=True, mode=mode)
payload = os.urandom(1_500)
src = tmp_path / "payload.bin"
src.write_bytes(payload)
enc = cc.encrypt_file(str(src), USER_PWD)
out = cc.decrypt_file(enc, USER_PWD, str(tmp_path / "out.bin"))
assert open(out, "rb").read() == payload
# ---------------------------------------------------------------------------
# Errors, common to every mode
# ---------------------------------------------------------------------------
class TestModeErrors:
@pytest.mark.parametrize("mode", MODES)
def test_empty_plaintext_rejected(self, cores, mode):
with pytest.raises(ChaosCryptError) as ei:
cores[mode].encrypt_text("", USER_PWD)
assert ei.value.code == "E008"
@pytest.mark.parametrize("mode", MODES)
def test_empty_password_rejected(self, cores, mode):
with pytest.raises(ChaosCryptError) as ei:
cores[mode].encrypt_text("data", "")
assert ei.value.code == "E006"
def test_empty_ciphertext_rejected(self, cores):
with pytest.raises(ChaosCryptError) as ei:
cores[MODE_FAST].decrypt_text("", USER_PWD)
assert ei.value.code == "E009"