v4.2.6: Self-describing format, byte pipeline, and a performance pass

Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
This commit is contained in:
dvs
2026-09-27 22:06:34 +08:00
parent 24a36d6fb4
commit dd49043171
25 changed files with 5348 additions and 852 deletions
+85 -5
View File
@@ -45,7 +45,24 @@ from __future__ import annotations
import os
from typing import Optional
from ._core import ChaosCryptCore, KEYFILE_VERSION
from ._core import (
ChaosCryptCore,
KEYFILE_VERSION,
ENCODINGS,
ENC_HEX,
ENC_BASE64,
DEFAULT_ENCODING,
MIDS,
MID_BYTES,
MID_HEX,
DEFAULT_MID,
MODE_REFERENCE,
MODE_FAST,
MODE_TURBO,
MODE_RAW,
MODES,
DEFAULT_MODE,
)
from .errors import (
ChaosCryptError,
ChaosCrypt_ERR_KeyFileNotFound_E001,
@@ -66,7 +83,7 @@ from .errors import (
ChaosCrypt_ERR_UnsupportedKeyVersion_E016,
)
__version__ = "2.1.6"
__version__ = "4.2.6"
__author__ = "Dvs (DvsXT)"
__license__ = "MIT"
__all__ = [
@@ -75,6 +92,20 @@ __all__ = [
"__license__",
"DEFAULT_KEY_FILE",
"KEYFILE_VERSION",
"MODE_REFERENCE",
"MODE_FAST",
"MODE_TURBO",
"MODE_RAW",
"MODES",
"ENCODINGS",
"ENC_HEX",
"ENC_BASE64",
"DEFAULT_ENCODING",
"MIDS",
"MID_BYTES",
"MID_HEX",
"DEFAULT_MID",
"DEFAULT_MODE",
"ChaosCryptCore",
"ChaosCrypt",
"ChaosCryptError",
@@ -126,8 +157,11 @@ class ChaosCrypt:
*,
auto_load: bool = False,
default_key_file: str = DEFAULT_KEY_FILE,
mode: str = DEFAULT_MODE,
encoding: str = DEFAULT_ENCODING,
mid: str = DEFAULT_MID,
) -> None:
self.core = ChaosCryptCore()
self.core = ChaosCryptCore(mode, encoding=encoding, mid=mid)
self.key_file = key_file
self.key_password = key_password
self.default_key_file = default_key_file
@@ -269,8 +303,12 @@ class ChaosCrypt:
keys_data = self.core.generate_keymaterial()
obfuscated = self.core._obfuscate_keys(keys_data)
text = self.core._xor_encrypt_data(obfuscated, key_password)
# The returned text is exactly what would have been written to a key
# file, so it carries the header too.
document = self.core._emit(text, "key")
self.key_password = key_password
return text
return document
def info(self) -> dict:
"""Return a summary of the loaded key material.
@@ -336,12 +374,54 @@ class ChaosCrypt:
}
obfuscated = self.core._obfuscate_keys(keys_data)
encrypted = self.core._xor_encrypt_data(obfuscated, new_password)
# Every artefact carries a header, key files included. Writing the bare
# payload here would produce a file this library then refuses to load.
document = self.core._emit(encrypted, "key")
with open(target, 'w') as f:
f.write(encrypted)
f.write(document)
self.key_password = new_password
return target
# ==================================================================
# Performance settings
# ==================================================================
@property
def mode(self) -> str:
"""The active performance mode."""
return self.core.mode
@property
def encoding(self) -> str:
"""The active output encoding (used by turbo and raw)."""
return self.core.encoding
@property
def mid(self) -> str:
"""The active intermediate representation (used by turbo and raw)."""
return self.core.mid
def set_mid(self, mid: str) -> "ChaosCrypt":
"""Change the intermediate representation in place.
Only affects ``turbo`` and ``raw``; ``reference`` and ``fast`` are
character pipelines and always report ``hex``.
"""
self.core.mid = mid
return self
def set_encoding(self, encoding: str) -> "ChaosCrypt":
"""Change the output encoding in place.
Only affects ``turbo`` and ``raw``; ``reference`` and ``fast`` always
emit hex. Output decoded with the wrong encoding will fail rather than
silently produce garbage, so a ciphertext and its encoding must be kept
together.
"""
self.core.encoding = encoding
return self
# ==================================================================
# Encryption / decryption
# ==================================================================