v4.2.6: Self-describing format, byte pipeline, and a performance pass

Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
This commit is contained in:
dvs
2026-09-27 22:06:34 +08:00
parent 24a36d6fb4
commit dd49043171
25 changed files with 5348 additions and 852 deletions
+177 -282
View File
@@ -1,6 +1,7 @@
# ChaosCrypt
**A chaos-based obfuscation cipher with a bilingual TUI, a full CLI, and a clean Python API.**
**A chaos-based obfuscation cipher with a self-describing format, four
performance modes, a bilingual TUI, a full CLI, and a clean Python API.**
[![Python](https://img.shields.io/badge/python-3.8%2B-blue)](https://www.python.org/)
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
@@ -22,66 +23,119 @@ purposes:
**If you need standardised cryptography, use a standardised cipher.** AES-GCM
or ChaCha20-Poly1305 from a vetted library such as
[`cryptography`](https://cryptography.io) are the right tools for production
systems, regulated data, and long-lived secrets. ChaosCrypt is not a
replacement for them and does not claim to be.
systems, regulated data, and long-lived secrets.
See [SECURITY.md](SECURITY.md) for the complete design description, key-space
derivation, and attack analysis.
See [SECURITY.md](SECURITY.md) for the full design and threat analysis, and
[docs/FORMAT.md](docs/FORMAT.md) for the normative format specification.
---
## Why it is worth a look
## Every artefact describes itself
The absence of an audit says nothing about the strength of the construction.
ChaosCrypt is deliberately built to remove every kind of structure that
classical cryptanalysis relies on, and the result is a design with real
substance behind it:
**A key space of roughly 10^5621.** The system key is the product of two
independent 26-letter bijections, a 10-into-62 injection, an ordered selection
of 4 padding symbols, a 10-bit flip pattern, a 4096-nibble long key and a
512-nibble short key. For comparison: AES-256 offers about 10^77, and the
observable universe contains about 10^80 atoms.
**Five layers, each removing a different class of structure.**
Each key file and ciphertext begins with a header that states how it was made.
A reader never has to be told the mode, the encoding, or anything else — it
reads them from the artefact.
```
plaintext
| Base64 lift arbitrary bytes into printable ASCII
| substitution destroy frequency structure
| conditional case flip remove linearity
| whole-message reversal destroy locality
| XOR with long key erase residual substituted structure
| XOR with derived key sever any link to the key file alone
ciphertext
69CES
385aeff1f4179a744c8fb0c5ae4702d325f6230d059f4251135c2536409dee93
{"alg":"chaoscrypt","kind":"ciphertext","mode":"turbo","out":"hex","mid":"bt",...}
2025ea4e4b4b93803f2ccadeaf318d1957ce42ff5409410f660348be96a4c0f2
<payload>
```
**No algebraic structure to attack.** The substitution tables are arbitrary
permutations stored as data, so there is no polynomial representation.
Differential, linear and algebraic cryptanalysis all require a compact
algebraic description that simply does not exist here.
The consequence is deliberate: **input without a valid header is rejected**
(`E017`), rather than guessed at. A ciphertext decrypted under the wrong
parameters produces garbage that looks like a wrong password, and the header
removes that ambiguity entirely.
**No number-theoretic structure either.** There is no modular arithmetic, no
group operation and no lattice, so Shor's algorithm has nothing to solve.
Grover's algorithm reduces the search to about 10^2810 — still far beyond any
physically realisable computation.
```python
from chaoscrypt import ChaosCrypt
**The two XOR layers shield each other.** An attacker who recovers the outer
keystream still faces the 4096-nibble long key, and no keystream recovered
from one message is reusable against another, because the derivation is
parameterised by message length.
cc = ChaosCrypt(key_file="my.key", key_password="key-file-password",
auto_load=True)
**Lossless for arbitrary binary data.** Every stage is a bijection, so images,
archives and database dumps round-trip byte-for-byte. Verified in the test
suite from 0 to 100,000 bytes, including NUL bytes and invalid UTF-8.
# whatever produced this, it will be read correctly
plain = cc.decrypt(open("message.ces").read(), "encryption-password")
```
**Zero third-party runtime dependencies.** Pure standard library. Nothing to
audit in your dependency tree, nothing to break on upgrade, and `pip install`
is immediately usable.
---
**A real interface.** A bilingual interactive TUI for interactive use, a
complete argument-driven CLI for automation, and a typed Python API for
embedding — all three backed by the same sixteen-code error hierarchy.
## Performance modes
Four modes, selectable at construction, on the command line, and from the TUI.
| Mode | Algorithm | Ciphertext | Compatible with pre-4.2.6 material |
|---|---|---|---|
| `reference` | the original algorithm, character pipeline | 5.33x | payload identical |
| `fast` *(default)* | same algorithm, bulk operations | 5.33x | payload identical |
| `turbo` | SHAKE256 keystream, byte pipeline | 2.00x / 1.33x | no |
| `raw` | SHAKE256 plus the long-key layer | 2.00x / 1.33x | no |
`turbo` and `raw` take two extra settings:
| Setting | Values | Meaning |
|---|---|---|
| `encoding` | `hex` (default), `base64` | final output encoding: 2.00x or 1.33x expansion |
| `mid` | `bt` (default), `hex` | whether bytes or hexadecimal text are carried between the XOR layers |
```python
from chaoscrypt import ChaosCrypt
cc = ChaosCrypt(mode="raw", encoding="base64", mid="bt")
```
```bash
chaoscrypt encrypt "hello" -p pw -k my.key -P kp --mode raw --encoding base64
```
### Measured throughput
Hex output, `raw` mode, 1 MiB payloads and above:
| mid | Encrypt | Decrypt | Ciphertext |
|---|---:|---:|---:|
| `bt` (default) | **57–63 MB/s** | **61–73 MB/s** | 2.00x |
| `hex` | 18–21 MB/s | 18–23 MB/s | 8.00x |
Throughput holds steady from 64 KiB to 1 GiB, which is what a linear algorithm
looks like. Full figures, including the profiling behind them, are in
[docs/benchmark.md](docs/benchmark.md).
For scale: the original implementation managed **0.08 MB/s**, about twelve
seconds per megabyte.
---
## Reading artefacts from before 4.2.6
Artefacts written before the header existed have none. They are read through a
compatibility API, which synthesises a minimal header and then uses the ordinary
path — so the strict path stays strict, and there is no lenient branch inside
it.
```python
from chaoscrypt import ChaosCrypt
cc = ChaosCrypt(key_file="my.key", key_password="key-file-password",
auto_load=True)
# a ciphertext with no header
plain = cc.decrypt_legacy("72370655031311011c111f261365172716020b69",
"encryption-password")
# a key file with no header
cc.load_legacy_keys(open("old.key", encoding="utf-8").read(), "key-password")
# or modernise one, once
open("new.key", "w", encoding="utf-8").write(
cc.wrap_legacy(open("old.key", encoding="utf-8").read(), is_key=True)
)
```
`decrypt_legacy_auto` probes whether the artefact is a key file or a ciphertext,
so the caller does not have to know. See
[docs/FORMAT.md §12](docs/FORMAT.md) for the legacy standard.
---
@@ -91,14 +145,6 @@ embedding — all three backed by the same sixteen-code error hierarchy.
pip install chaoscrypt
```
From source:
```bash
git clone https://git.dvscloud.net/dvs/ChaosCrypt
cd ChaosCrypt
pip install .
```
**Requires** Python 3.8 or newer. No third-party runtime dependencies.
---
@@ -111,130 +157,69 @@ pip install .
from chaoscrypt import ChaosCrypt
cc = ChaosCrypt()
# Generate a key file once, then keep it somewhere safe.
cc.create_key("key-file-password", "my.key")
# Encrypt and decrypt.
token = cc.encrypt("hello world", "encryption-password")
assert cc.decrypt(token, "encryption-password") == "hello world"
```
One-shot helpers, when you do not want to manage an instance:
```python
import chaoscrypt
chaoscrypt.create_key("key-file-pwd", "my.key")
ct = chaoscrypt.encrypt("secret", "user-pwd", "my.key", "key-file-pwd")
pt = chaoscrypt.decrypt(ct, "user-pwd", "my.key", "key-file-pwd")
```
Binary data and files:
```python
from chaoscrypt import ChaosCrypt
cc = ChaosCrypt("my.key", "key-file-pwd", auto_load=True)
cc = ChaosCrypt("my.key", "key-file-password", auto_load=True)
raw = open("photo.png", "rb").read()
ct = cc.encrypt_bytes(raw, "encryption-password")
open("photo.cc", "w").write(ct)
open("photo.cc", "w").write(cc.encrypt_bytes(raw, "encryption-password"))
restored = cc.decrypt_bytes(ct, "encryption-password")
open("photo.restored.png", "wb").write(restored)
# Or use the file helpers directly.
cc.encrypt_file("photo.png", "encryption-password") # -> photo.png.cc
cc.decrypt_file("photo.png.cc", "encryption-password") # -> photo.png
cc.encrypt_file("photo.png", "encryption-password") # -> photo.png.cc
cc.decrypt_file("photo.png.cc", "encryption-password") # -> photo.png
```
**Error handling.** Every failure is a subclass of `ChaosCryptError` and
renders as `ChaosCrypt_ERR_<Detail>_<Code>`:
Error handling — every failure is a `ChaosCryptError` and renders as
`ChaosCrypt_ERR_<Detail>_<Code>`:
```python
from chaoscrypt import ChaosCrypt, ChaosCryptError
cc = ChaosCrypt()
try:
cc.load("my.key", "wrong-password")
ChaosCrypt().load("my.key", "wrong-password")
except ChaosCryptError as e:
print(e) # ChaosCrypt_ERR_KeyPasswordIncorrect_E003
print(e.code) # E003
print(e.detail) # KeyPasswordIncorrect
```
Catching `ChaosCryptError` handles everything the library can raise.
### 2. As a command line tool
**No arguments** launches the interactive TUI:
**No arguments** launches the interactive TUI. **With arguments** it is a
scriptable CLI:
```bash
chaoscrypt keygen -k my.key --key-pwd-env KEY_PWD
chaoscrypt encrypt "hello world" -p mypass -k my.key -P keypass
chaoscrypt decrypt "1a2b3c..." -p mypass -k my.key -P keypass
chaoscrypt file encrypt photo.png -p mypass -k my.key -P keypass
chaoscrypt info -k my.key -P keypass --json
chaoscrypt chpwd -k my.key -P oldpass --new-pwd newpass
```
`-p`/`--pwd` is the **encryption password**; `-P`/`--key-pwd` is the **key file
password**. Passwords may come inline, from an environment variable, or from a
file.
**Exit codes:** `0` success, `1` runtime error, `2` usage error. See
[docs/CLI.md](docs/CLI.md) for the full reference.
### 3. As an interactive TUI
```bash
chaoscrypt
```
**With arguments** it is a scriptable CLI:
```bash
# Generate a key file.
chaoscrypt keygen -k my.key --key-pwd-env KEY_PWD
# Encrypt and decrypt text.
chaoscrypt encrypt "hello world" -p mypass -k my.key -P keypass
chaoscrypt decrypt "1a2b3c..." -p mypass -k my.key -P keypass
# Files (binary-safe).
chaoscrypt file encrypt photo.png -p mypass -k my.key -P keypass
chaoscrypt file decrypt photo.png.cc -p mypass -k my.key -P keypass
# Key file metadata.
chaoscrypt info -k my.key -P keypass --json
# Rotate the key file password.
chaoscrypt chpwd -k my.key -P oldpass --new-pwd newpass
```
Working with pipes and files:
```bash
# JSON output for programmatic use.
chaoscrypt --json encrypt "data" -p pw -k my.key -P kp
# Read from stdin, write to stdout.
echo "hello" | chaoscrypt encrypt - -p pw -k my.key -P kp
chaoscrypt decrypt - -p pw -k my.key -P kp < secret.cc
# Read and write files directly.
chaoscrypt encrypt -i in.txt -o out.cc -p pw -k my.key -P kp
```
**Supplying passwords.** Three options, in ascending order of safety:
| Method | Example | Notes |
|---|---|---|
| Inline | `-p mypass` | Visible in shell history — not recommended |
| Environment variable | `--pwd-env MY_PWD` | Suitable for CI/CD |
| Password file | `--pwd-file ~/.cc.pwd` | Reads the last non-empty line — recommended |
Note the distinction: `-p`/`--pwd` is the **encryption password** that protects
your data, while `-P`/`--key-pwd` is the **key file password** that unlocks the
key file.
**Exit codes:** `0` success, `1` runtime error, `2` usage error.
### 3. As an interactive TUI
Run `chaoscrypt` with no arguments (or `python -m chaoscrypt`):
```
+================================================================+
| ChaosCrypt - Chaos Obfuscation Cipher v2.1.6 |
| ChaosCrypt - Chaos Obfuscation Cipher v4.2.6 |
+================================================================+
ChaosCrypt is an unaudited, non-standardised cipher. It is intended for
obfuscation, personal use and study -- not for production systems.
[ ] No key loaded yet. [encryption.key]
[*] Key file loaded my.key [fast]
------------------------------------------------------------------
Select an action
@@ -248,136 +233,57 @@ obfuscation, personal use and study -- not for production systems.
[7] Show key information
[8] Change key file password
[9] Switch language
[m] Performance mode
[e] Output encoding
[a] About / security notice
[q] Quit
------------------------------------------------------------------
```
- **Bilingual**: option `[9]` switches between English and Chinese at runtime,
and every label follows.
- **Passwords are not echoed**: input uses `getpass` on a terminal.
- **Helpful when a key is missing**: offers to create one on the spot.
- **Errors are self-explanatory**: shown as `ChaosCrypt_ERR_..._Exxx` with a
short hint about the likely cause.
- **Degrades cleanly**: redirected or piped output drops colour and switches
to plain input, so the UI remains scriptable.
Bilingual (option `[9]`), passwords are not echoed, and the interface degrades
cleanly when output is piped.
---
## The two-password model
## Error codes
ChaosCrypt separates the secret that protects the key file from the secret
used for each message. Keeping them distinct is what makes key rotation cheap:
| Code | Meaning |
|---|---|
| `E001` | Key file not found |
| `E002` | Key file corrupted |
| `E003` | Key file password incorrect |
| `E004` | No key loaded |
| `E005` | Key file already exists |
| `E006` | A required password was missing |
| `E007` | Password mismatch |
| `E008` | Empty plaintext |
| `E009` | Empty ciphertext |
| `E010` | Malformed payload (not valid hexadecimal, or bad length) |
| `E011` | Decryption failed (wrong password, wrong key, or tampered) |
| `E012` | Encoding failure |
| `E013` | Invalid argument |
| `E014` | File read failed |
| `E015` | File write failed |
| `E016` | Unsupported key file version |
| `E017` | **Missing magic header** — not a CES artefact |
| `E018` | **Malformed header** — marker missing, JSON unusable, or `kind` mismatch |
| Password | Purpose | Where it lives |
|---|---|---|
| **Key file password** | Wraps and unwraps `encryption.key` | Only in your head |
| **Encryption password** | Used per encryption/decryption operation | Only in your head |
The system key material — substitution tables, long key, short key, flip
pattern — is stored in the key file, protected by the key file password. The
encryption password is mixed with the short key at runtime to derive the
session keystream.
> **To decrypt your data you need all three:** the key file, the key file
> password, and the encryption password. Back them up separately. Losing any
> one makes recovery impossible — there is no backdoor.
Codes are stable and safe to branch on.
---
## How it works
## Documentation
```
plaintext P
|
+-- Base64 encode -----------------> printable ASCII
|
+-- Letter substitution -----------> two independent 26! bijections
|
+-- Conditional case flip ---------> 10-bit pattern, adds non-linearity
|
+-- Whole-message reversal --------> destroys local correlation
|
+-- XOR with long key -------------> 4096-nibble keystream
|
+-- XOR with derived keystream ----> 512-nibble short key + password
|
ciphertext C (hexadecimal)
```
Decryption applies the exact inverse of each step in reverse order.
The packaged library is **bit-for-bit equivalent** to the original
single-file implementation. Old key files, old ciphertexts, and the old
script remain mutually compatible in both directions — this is enforced by
the test suite, not merely asserted.
---
## FAQ
<details>
<summary><b>Why not just use AES?</b></summary>
If your goal is security, use AES-GCM or ChaCha20-Poly1305. They are audited,
standardised, and available in mature libraries. ChaosCrypt exists to explore
a different design premise — replacing algebraic structure with randomness —
and to serve obfuscation, personal use and teaching.
Judging a personal project against the standard applied to AES-256-GCM is a
category error, in the same way that judging a bicycle against Formula 1
regulations tells you nothing useful about the bicycle.
</details>
<details>
<summary><b>There is no IV. Is that a problem?</b></summary>
An IV creates output variation when the key is fixed. Here the system key is
freshly generated per key file and combined with your password, so variation
is supplied structurally rather than per message.
The real consequence is **deterministic output**: the same plaintext, key file
and encryption password produce the same ciphertext. Confidentiality is
unaffected. If you need two identical messages to be distinguishable — for
replay detection, say — use a fresh key file, or prepend your own nonce and
strip it after decryption. This is a documented design trade-off, not an
oversight. See [SECURITY.md §9](SECURITY.md).
</details>
<details>
<summary><b>What happens if I lose the key file or a password?</b></summary>
**Your data is unrecoverable.** This is by design: there is no backdoor, no
escrow, and no recovery path. Back up the key file, and keep both passwords in
a password manager.
</details>
<details>
<summary><b>Can I use this to protect cryptocurrency keys?</b></summary>
**No.** Use audited tooling designed for that purpose — a hardware wallet, or
`age` / `gpg` with a hardened configuration. ChaosCrypt is explicitly not
suitable for this.
</details>
<details>
<summary><b>Is it compatible with the original single-file script?</b></summary>
Yes, completely. The algorithm is implemented bit-for-bit identically. Old
key files load directly, old ciphertexts decrypt directly, and anything the
library produces is readable by the original script. The original
implementation is preserved under `chaoscrypt/legacy/` for exactly this
purpose, and interoperability is covered by dedicated tests.
</details>
<details>
<summary><b>How do I handle a password for a service that stores credentials?</b></summary>
Do not store user passwords with ChaosCrypt. Password storage has its own
requirements — a deliberately slow, memory-hard KDF with a per-user salt —
and the right choices are Argon2id, scrypt or bcrypt. ChaosCrypt is not a
password hashing function.
</details>
| Document | Contents |
|---|---|
| [SECURITY.md](SECURITY.md) | Design, key space, threat model, attack analysis |
| [docs/FORMAT.md](docs/FORMAT.md) | **Normative format specification**, with examples |
| [docs/benchmark.md](docs/benchmark.md) | Measured throughput and the profiling behind it |
| [docs/API.md](docs/API.md) | Complete Python API reference |
| [docs/CLI.md](docs/CLI.md) | Command line reference and troubleshooting |
| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Internal design and module map |
| [docs/BUILDING.md](docs/BUILDING.md) | Building, testing and releasing |
| [CHANGELOG.md](CHANGELOG.md) | Release history |
---
@@ -395,39 +301,28 @@ twine check dist/* # validate package metadata
```
ChaosCrypt/
├── README.md SECURITY.md LICENSE CHANGELOG.md
├── docs/ additional documentation
├── docs/
├── pyproject.toml
├── dist/ build artefacts
└── chaoscrypt/ source tree
├── dist/
└── chaoscrypt/
├── __init__.py public API and the ChaosCrypt facade
├── _core.py the algorithm
├── _core.py the algorithm and the pipelines
├── _header.py the self-describing header
├── cli.py argument-driven command line interface
├── tui.py interactive bilingual interface
├── errors.py the unified error hierarchy
├── i18n.py UI strings
├── tests/ test suite
├── tests/ test suite, including frozen vectors
├── examples/ runnable examples
└── legacy/ the original single-file implementation
└── legacy/ the original implementation, for reference
```
---
## Documentation
| Document | Contents |
|---|---|
| [SECURITY.md](SECURITY.md) | Design, key space, threat model, attack analysis |
| [CHANGELOG.md](CHANGELOG.md) | Release history |
| [docs/API.md](docs/API.md) | Complete API reference |
| [docs/CLI.md](docs/CLI.md) | Command line reference |
| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Internal design and module map |
---
## Licence
[MIT](LICENSE) — provided as-is, without warranty of any kind.
ChaosCrypt has not been independently audited. It is intended for
obfuscation, personal use, and research or teaching. Do not deploy it where
an audited, standardised cryptographic primitive is required.
ChaosCrypt has not been independently audited. It is intended for obfuscation,
personal use, and research or teaching. Do not deploy it where an audited,
standardised cryptographic primitive is required.