v4.2.6: Self-describing format, byte pipeline, and a performance pass

Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
This commit is contained in:
dvs
2026-09-27 22:06:34 +08:00
parent 24a36d6fb4
commit dd49043171
25 changed files with 5348 additions and 852 deletions
+163 -75
View File
@@ -7,93 +7,182 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
---
## [2.1.6] — 2026-09-27
## [4.2.6] — 2026-09-27
Packaging and documentation release. The algorithm is unchanged; this version
restructures the project as a distributable library and brings the
documentation in line with it.
Self-describing artefacts, a byte pipeline worth using, and a large performance
pass. This release changes the on-disk format incompatibly and is the first
version intended to be deployed rather than experimented with.
### Added
- **Bilingual interactive TUI.** A menu-driven interface with English and
Chinese, switchable at runtime. Uses `getpass` for password entry on a
terminal and degrades to plain input when piped, so it remains scriptable.
- **Full command line interface.** Subcommands `keygen`, `encrypt`, `decrypt`,
`file`, `info`, `chpwd` and `tui`, with `--json` output, stdin/stdout
piping, and three ways to supply each password (inline, environment
variable, password file).
- **Unified error hierarchy.** Sixteen named error types, all subclasses of
`ChaosCryptError`, each rendering as `ChaosCrypt_ERR_<Detail>_<Code>`. Every
failure path in the library now raises rather than returning a sentinel.
- **Binary and file support.** `encrypt_bytes` / `decrypt_bytes` for arbitrary
binary data, and `encrypt_file` / `decrypt_file` for whole files.
- **Key management helpers.** `change_key_password` re-wraps a key file under
a new password without altering the key material. `export_key_text` and
`load_from_text` move a key without touching disk.
- **Comprehensive documentation.** `SECURITY.md` presents the design, key
space derivation, threat model and attack analysis in full. `docs/`
contains API, CLI and architecture references.
- **Test suite.** Round-trip integrity, binary safety, key management, error
types, CLI behaviour, TUI interaction, internationalisation, and
bidirectional interoperability with the original implementation.
- **Packaging.** `pyproject.toml`, `MANIFEST.in`, `.gitignore`, `py.typed`,
runnable examples, and MIT licensing.
- **Self-describing header on every artefact.** Each key file and ciphertext now
begins with a magic value, two fixed markers, and a complete JSON
configuration record that states how the payload was produced. A reader never
has to be told the mode, the encoding or the intermediate representation: it
reads them. See [docs/FORMAT.md](docs/FORMAT.md).
```
69CES
385aeff1f4179a744c8fb0c5ae4702d325f6230d059f4251135c2536409dee93
{"alg":"chaoscrypt","kind":"ciphertext","mode":"turbo","out":"hex",...}
2025ea4e4b4b93803f2ccadeaf318d1957ce42ff5409410f660348be96a4c0f2
<payload>
```
- **Two new error codes**, for the two ways a header can be wrong:
- `E017 MissingMagicHeader` — the input is not a CES artefact at all
- `E018 MalformedHeader` — a marker is missing, the JSON is unusable, a
required field is absent, or `kind` does not match
- **Byte pipeline for `turbo` and `raw`.** Between the XOR layers the payload is
now carried as raw bytes rather than hexadecimal text. This removes a
doubling of the payload at each layer: expansion drops from **5.33x to 2.00x**
and throughput rises by roughly **4x**. The old behaviour is still available
as `mid="hex"`.
- **Selectable output encoding** for `turbo` and `raw`: `hex` (2.00x) or
`base64` (1.33x).
- **Legacy compatibility API**, for artefacts written before the header existed:
- `wrap_legacy(text, is_key)` — synthesise a minimal header and return a
well-formed artefact
- `decrypt_legacy(text, password, is_key)` — read an artefact that may predate
the header
- `decrypt_legacy_auto(text, password)` — as above, probing whether the input
is a key file or a ciphertext
- `load_legacy_keys(text, key_password)` — load a pre-header key file
Legacy support is a *wrapper* around the ordinary path, never a lenient branch
inside it: the payload is given a header and then read normally, so the strict
path stays strict.
- **Frozen reference vectors** covering eleven cases across every mode,
encoding and intermediate representation, plus five error cases.
- **Documentation**: `docs/FORMAT.md` as the normative format specification,
`docs/benchmark.md` with measured throughput and the profiling behind it, and
a rewritten `SECURITY.md` covering the four modes.
### Changed
- **Project restructured** into a standard library layout. The importable
package now lives at `chaoscrypt/`, with tests, examples and the legacy
implementation alongside it, and documentation and build artefacts at the
repository root.
- **Code comments unified in English**, and public docstrings expanded to
document parameters, return values and raised errors.
- **English is now the default UI language**, with Chinese available as an
alternative.
- **Packaging metadata** updated for the new layout and SPDX licence
expression.
- **Default mode is `fast`**, which is byte-for-byte compatible with the
original algorithm and roughly 30x quicker.
- **Substitution, case flipping and the printable fold** run through
`bytes.translate` over precomputed 256-entry tables.
- **XOR layers** use a single big-integer operation instead of a per-byte loop
with `%02x` formatting.
- **Keystream derivation** caches the length-independent base material, and the
expansion uses cached translation tables rather than `str.join` over a
generator.
- **`_get_dynamic_key`** repeats the long key by multiplication rather than
`key += long_key` in a loop. The loop version was quadratic: covering a
5.6 MB intermediate took 866 ms and dominated the 1 MiB timings. The same
work now takes about 3 ms, with identical output.
- **`_derive_final_key`** decodes only the prefix of the hex round trip that
the output actually consumes, rather than the whole thing.
- **The decrypt path** no longer copies or scans the full artefact: a redundant
`strip()`, a full-string `int(payload, 16)` validation, and three whole-string
slices in `strip_header` were all removed. Decryption throughput roughly
doubled at 512 KiB, from 34 MB/s to 87 MB/s.
- **Version raised to 4.2.6.**
### Fixed
Six defects found and corrected during the restructure:
1. The `-p` and `-P` option groups both registered `--pwd`, causing an
`argparse` conflict that prevented the CLI from starting at all.
2. An incorrect key file password was reported as a corrupted file (`E002`)
rather than a credential error (`E003`). A structural validation step now
distinguishes the two cases.
3. `export_key_text` produced key material without loading it into the
instance, so the returned key could not be used directly.
4. **Non-hexadecimal input was accepted as ciphertext and silently decrypted
into garbage.** This was the most serious defect: a corrupt or truncated
ciphertext was indistinguishable from a successful decryption. Malformed
input now raises `E010` before any processing occurs.
5. The TUI failed outright under piped input, because `getpass` requires a
TTY. It now detects this and falls back to plain input.
6. After generating a key file, the TUI continued to look at the default file
name rather than the one just created. The key file is now remembered for
the rest of the session.
1. Key material derivation is no longer quadratic. `_get_dynamic_key` was the
worst offender, and `_expand_keystream_slow` shared the pattern.
2. Two hex-intermediate pipelines were asymmetric, so a `turbo` or `raw`
artefact produced with `mid="hex"` could not be read back. Both are now
exact mirrors.
3. `change_key_password` and `export_key_text` wrote artefacts without a header,
producing files this library then refused to load.
4. The constructor accepted `encoding="base64"` while the header records
`out="base"`. Both spellings are now accepted, so a caller can read a header
and pass the value straight back in.
5. A payload that is not valid hexadecimal now reports `E010`
(MalformedCiphertext) rather than `E011` (DecryptionFailed). The distinction
matters: the first means the artefact is malformed, the second means the
password or key is wrong.
6. The TUI's encoding menu referenced a missing translation and displayed the
key name instead of the description.
### Security
- Clearly documented that ChaosCrypt is unaudited and non-standardised, and
that it is intended for obfuscation, personal use, and research or teaching
rather than production deployment.
- Documented the deterministic-output property and its consequences, and the
information disclosed by ciphertext length and Base64 padding class.
- Documented the strength of the construction in full, including the
approximately 10^5621 key space, the absence of algebraic and
number-theoretic structure, and the effect of Shor's and Grover's
algorithms.
- **Format break from pre-4.2.6 artefacts.** Headerless input is rejected with
`E017`; use the compatibility API to read old material. Old scripts cannot
read new artefacts. This is deliberate and documented, not an oversight.
- **`fast` is not a weaker mode**: it performs the same arithmetic and produces
the same bytes. The test suite asserts this against the frozen vectors.
- **`raw` with a hex intermediate is the weakest configuration** and is
documented as such: fewer structural layers stand between an attacker and the
substituted text.
- The header is metadata. It adds no secrecy and is not authenticated; an
application that must detect header tampering should MAC the whole artefact.
### Performance
Measured on the machine described in [docs/benchmark.md](docs/benchmark.md),
hex output, keystream cache warm.
| Mode / mid | 64 KiB | 1 MiB | 16 MiB | 128 MiB | 1024 MiB |
|---|---:|---:|---:|---:|---:|
| `raw` bt enc | 26.9 | 57.3 | 63.4 | 59.4 | 47.9 |
| `raw` bt dec | 60.6 | 61.0 | 69.0 | 64.5 | 31.7 |
| `raw` hex enc | 20.7 | 19.3 | 18.9 | 18.2 | — |
| `raw` hex dec | 20.2 | 21.8 | 22.7 | 18.0 | — |
Throughput in MB/s. It holds steady across four orders of magnitude, which is
what a linear algorithm looks like.
For scale, the original implementation managed **0.08 MB/s** — about twelve
seconds per megabyte.
### Compatibility
Fully backward compatible with the original single-file implementation, in
both directions:
- `reference` and `fast` produce payloads byte-for-byte identical to the
original algorithm; only the header is new.
- `turbo` and `raw` define their own formats, and their payloads differ by
`out` and `mid`. An artefact is read with the parameters it carries, and there
is no fallback between them.
- The key file format carries a header but is otherwise unchanged.
- Artefacts written before the header existed are readable only through the
compatibility API.
- key files produced by either implementation load in the other;
- ciphertext produced by either implementation decrypts in the other;
- the default key file name remains `encryption.key`, and the on-disk key
file format is unchanged.
---
## [2.1.6] — 2026-09-27
Packaging release. The algorithm is unchanged; this version restructures the
project as a distributable library.
### Added
- Bilingual interactive TUI, English and Chinese, switchable at runtime.
- Full command line interface: `keygen`, `encrypt`, `decrypt`, `file`, `info`,
`chpwd`, `tui`.
- Unified error hierarchy: sixteen named types rendering as
`ChaosCrypt_ERR_<Detail>_<Code>`.
- Binary and file support.
- Key management helpers: `change_key_password`, `export_key_text`,
`load_from_text`.
- Packaging: `pyproject.toml`, `setup.py` shim, `MANIFEST.in`, `py.typed`.
### Fixed
1. The `-p` and `-P` option groups both registered `--pwd`, so the CLI could
not start at all.
2. A wrong key file password was reported as file corruption (`E002`).
3. `export_key_text` produced key material without loading it into the instance.
4. **Non-hexadecimal input was accepted as ciphertext and silently decrypted
into garbage**, making corruption indistinguishable from success.
5. The TUI failed under piped input, because `getpass` requires a TTY.
6. After generating a key file, the TUI kept looking at the default file name.
---
@@ -103,13 +192,12 @@ The original single-file implementation of the ChaosCrypt cipher.
### Added
- Random system key generation: independent uppercase and lowercase
bijections, digit and padding substitutions, a 10-bit case-flip pattern, a
4096-nibble long key and a 512-nibble short key.
- Random system key generation.
- Password-protected key file wrapping.
- Layered encryption: Base64 encoding, substitution, conditional case
flipping, whole-message reversal, and two XOR stages.
- Layered encryption: Base64, substitution, conditional case flipping,
whole-message reversal, and two XOR stages.
- Interactive menu for encryption, decryption and key management.
[4.2.6]: https://git.dvscloud.net/dvs/ChaosCrypt/releases/tag/v4.2.6
[2.1.6]: https://git.dvscloud.net/dvs/ChaosCrypt/releases/tag/v2.1.6
[2.0.0]: https://git.dvscloud.net/dvs/ChaosCrypt/releases/tag/v2.0.0