v4.2.6: Self-describing format, byte pipeline, and a performance pass

Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
This commit is contained in:
dvs
2026-09-27 22:06:34 +08:00
parent 24a36d6fb4
commit dd49043171
25 changed files with 5348 additions and 852 deletions
+3 -3
View File
@@ -62,7 +62,7 @@ class TestTUI:
seq = (f"5\nt.key\n{KEY_PWD}\n{KEY_PWD}\n\n"
f"1\nsecret payload\n{USER_PWD}\n\nq\n")
r1 = _run_tui(seq, str(tmp_path))
m = re.search(r"Ciphertext:\s*\n([0-9a-f]{16,})", r1.stdout)
m = re.search(r"Ciphertext:\s*\n(69CES[^\s]+)", r1.stdout)
assert m, r1.stdout
ct = m.group(1).strip()
@@ -73,7 +73,7 @@ class TestTUI:
def test_load_remembers_key_file(self, tui_dir):
seq = f"6\nt.key\n{KEY_PWD}\n\n1\nremembers\n{USER_PWD}\n\nq\n"
r = _run_tui(seq, tui_dir)
assert re.search(r"Ciphertext:\s*\n([0-9a-f]{16,})", r.stdout), r.stdout
assert re.search(r"Ciphertext:\s*\n(69CES[^\s]+)", r.stdout), r.stdout
def test_wrong_user_password_shows_error(self, tui_dir):
seq = f"6\nt.key\n{KEY_PWD}\n\n2\n{'a' * 64}\nWRONG-{USER_PWD}\n\nq\n"
@@ -97,7 +97,7 @@ class TestTUI:
seq = (f"6\nt.key\n{KEY_PWD}\n\n9\n2\n\n"
f"1\n中文内容\n{USER_PWD}\n\nq\n")
r = _run_tui(seq, tui_dir)
assert re.search(r"密文:\s*\n([0-9a-f]{16,})", r.stdout), r.stdout
assert re.search(r"密文:\s*\n(69CES[^\s]+)", r.stdout), r.stdout
def test_about_page_has_design_notes(self, tmp_path):
r = _run_tui("a\n\nq\n", str(tmp_path))