v4.2.6: Self-describing format, byte pipeline, and a performance pass

Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
This commit is contained in:
dvs
2026-09-27 22:06:34 +08:00
parent 24a36d6fb4
commit dd49043171
25 changed files with 5348 additions and 852 deletions
+25 -75
View File
@@ -1,84 +1,34 @@
# -*- coding: utf-8 -*-
"""
Bidirectional interoperability tests against the original implementation.
Legacy interoperability: deliberately removed.
These tests load ``chaoscrypt/legacy/chaoscrypt_legacy.py`` and prove that:
Until 4.2.6 the library could exchange key files and ciphertexts with the
original single-file script. That interoperability was given up when the
self-describing header was introduced: every artefact now begins with a magic
value, two fixed markers and a configuration record, and a reader rejects
anything without one.
* ciphertext produced by the original script decrypts with the packaged
library, and
* ciphertext produced by the packaged library decrypts with the original
script.
The break is intentional in both directions:
They are the regression guard for the promise that the algorithm itself was
ported without any behavioural change.
* a key file written by this version cannot be read by the original script;
* a ciphertext written by this version cannot be decrypted by the original
script;
* artefacts written before the header existed are rejected with
``E017 MissingMagicHeader``.
Reading old material is still possible through the compatibility API, which
synthesises a minimal header and then uses the ordinary path. That is covered
by ``test_header.py``; there is nothing left to assert here.
The original implementation is kept under ``legacy/`` for reference only.
"""
import importlib.util
import os
import pytest
from chaoscrypt import ChaosCrypt
KEY_PWD = "legacy-key-password"
USER_PWD = "legacy-user-password"
SAMPLES = [
"legacy compatibility",
"兼容性检查",
"x" * 300,
"emoji test 🎉",
"multi\nline\ttabbed",
]
def _load_legacy():
"""Import the legacy single-file implementation, or skip if absent."""
here = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
legacy = os.path.join(here, "legacy", "chaoscrypt_legacy.py")
if not os.path.exists(legacy):
pytest.skip("legacy implementation not present")
spec = importlib.util.spec_from_file_location("cc_legacy", legacy)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
class TestLegacyInterop:
def test_library_decrypts_legacy_ciphertext(self, tmp_path):
legacy_mod = _load_legacy()
kf = str(tmp_path / "legacy.key")
old = legacy_mod.EncryptionSystem()
old.generate_keys(kf, KEY_PWD)
pairs = [(tx, old.encrypt(tx, USER_PWD)) for tx in SAMPLES]
new = ChaosCrypt()
new.load(kf, KEY_PWD)
for tx, ct in pairs:
assert new.decrypt(ct, USER_PWD) == tx
def test_legacy_decrypts_library_ciphertext(self, tmp_path):
legacy_mod = _load_legacy()
kf = str(tmp_path / "legacy2.key")
new = ChaosCrypt()
new.create_key(KEY_PWD, kf)
old = legacy_mod.EncryptionSystem()
old._load_keys(kf, KEY_PWD)
for tx in SAMPLES:
ct = new.encrypt(tx, USER_PWD)
assert old.decrypt(ct, USER_PWD) == tx
def test_key_files_are_identical_in_format(self, tmp_path):
"""A library key file must load in the original implementation."""
legacy_mod = _load_legacy()
kf = str(tmp_path / "fmt.key")
ChaosCrypt().create_key(KEY_PWD, kf)
old = legacy_mod.EncryptionSystem()
old._load_keys(kf, KEY_PWD)
assert old.keys_loaded
@pytest.mark.skip(reason="interoperability with pre-4.2.6 artefacts was "
"intentionally dropped when the header was added; "
"see the compatibility API instead")
def test_legacy_interop_removed():
"""Placeholder so the removal shows up in the test report."""
assert True