v4.2.6: Self-describing format, byte pipeline, and a performance pass

Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
This commit is contained in:
dvs
2026-09-27 22:06:34 +08:00
parent 24a36d6fb4
commit dd49043171
25 changed files with 5348 additions and 852 deletions
+39
View File
@@ -71,6 +71,45 @@ TEXTS: Dict[str, Dict[str, str]] = {
"menu_keyinfo": {"en": "Show key information", "zh": "查看当前密钥信息"},
"menu_chpwd": {"en": "Change key file password", "zh": "修改密钥文件密码"},
"menu_lang": {"en": "Switch language", "zh": "切换语言"},
"menu_mode": {"en": "Performance mode", "zh": "性能模式"},
"menu_encoding": {"en": "Output encoding", "zh": "输出编码"},
"info_encoding": {"en": "Output encoding", "zh": "输出编码"},
"encoding_prompt": {"en": "Select output encoding", "zh": "请选择输出编码"},
"encoding_hex_desc": {
"en": "hex - 2 characters per byte, 2.00x expansion",
"zh": "hex - 每字节 2 字符,膨胀 2.00 倍",
},
"encoding_base64_desc": {
"en": "base64 - 4 characters per 3 bytes, 1.33x expansion",
"zh": "base64 - 每 3 字节 4 字符,膨胀 1.33 倍",
},
"encoding_changed": {"en": "Encoding set to", "zh": "输出编码已设为"},
"encoding_note": {
"en": "(only affects turbo and raw; reference and fast always emit hex)",
"zh": "(仅影响 turbo 与 raw;reference 与 fast 恒为 hex)",
},
"info_mode": {"en": "Performance mode", "zh": "性能模式"},
"mode_prompt": {
"en": "Select performance mode",
"zh": "请选择性能模式",
},
"mode_reference_desc": {
"en": "reference - original algorithm, slowest, ciphertext compatible",
"zh": "reference - 原版算法,最慢,密文兼容",
},
"mode_fast_desc": {
"en": "fast - identical ciphertext, C-level bulk operations",
"zh": "fast - 密文相同,C 级批量运算",
},
"mode_turbo_desc": {
"en": "turbo - SHAKE256 keystream, much faster, own ciphertext format",
"zh": "turbo - SHAKE256 密钥流,极快,独立密文格式",
},
"mode_raw_desc": {
"en": "raw - bytes throughout, fastest, smallest output, weakest",
"zh": "raw - 全程字节,最快,输出最小,强度最低",
},
"mode_changed": {"en": "Mode set to", "zh": "模式已设为"},
"menu_about": {"en": "About / security notice", "zh": "关于 / 安全声明"},
"menu_quit": {"en": "Quit", "zh": "退出"},
"menu_prompt": {"en": "Enter option", "zh": "请输入选项编号"},