v4.2.6: Self-describing format, byte pipeline, and a performance pass
Every key file and ciphertext now carries a header that states how it was produced: magic, two fixed markers, and a complete JSON configuration record. A reader takes its parameters from the artefact rather than from the caller. This is an incompatible format change. Headerless input is rejected with E017; pre-4.2.6 material is read through a compatibility API that wraps it in a minimal header and then uses the ordinary path, so the strict path stays strict. Added - Self-describing header on all artefacts, with a normative specification in docs/FORMAT.md - E017 MissingMagicHeader and E018 MalformedHeader - Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising throughput about 4x. The old behaviour is available as mid="hex". - Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x) - Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto, load_legacy_keys - Frozen reference vectors covering every mode, encoding and intermediate - docs/benchmark.md with measured throughput and the profiling behind it Changed - Default mode is fast: byte-for-byte compatible with the original algorithm, roughly 30x quicker - Substitution, case flipping and the printable fold use bytes.translate over precomputed tables - XOR layers use one big-integer operation instead of a per-byte loop - Keystream derivation caches length-independent base material; expansion uses cached translation tables and list accumulation rather than str += - _get_dynamic_key repeats by multiplication. The loop version was quadratic: covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged. - _derive_final_key decodes only the prefix the output consumes - The decrypt path no longer copies or scans the whole artefact: a redundant strip(), a full-string int(payload, 16), and three whole-string slices in strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s. - Documentation rewritten for 4.2.6; all eleven markdown files are English Fixed 1. Key derivation is no longer quadratic (_get_dynamic_key and _expand_keystream_slow shared the str += pattern) 2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact written with mid="hex" could not be read back 3. change_key_password and export_key_text wrote artefacts without a header 4. The constructor accepted encoding="base64" while the header records out="base"; both spellings are now accepted 5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext) rather than E011 (DecryptionFailed) 6. The TUI encoding menu referenced a missing translation Security - Format break from pre-4.2.6 artefacts, documented rather than accidental - fast is not a weaker mode: the frozen vectors assert identical output - The header adds no secrecy and is not authenticated; an application that must detect header tampering should MAC the whole artefact Performance - raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s decryption, ciphertext 2.00x the payload - Throughput holds steady from 64 KiB to 1 GiB - The original implementation managed 0.08 MB/s
This commit is contained in:
+122
@@ -21,6 +21,12 @@ from chaoscrypt import ChaosCryptError # base error class
|
||||
| `decrypt` | function | One-shot decrypt using a key file |
|
||||
| `create_key` | function | One-shot key file generation |
|
||||
| `DEFAULT_KEY_FILE` | constant | `"encryption.key"` |
|
||||
| `MODES` | constant | Tuple of the four mode names |
|
||||
| `DEFAULT_MODE` | constant | `"fast"` |
|
||||
| `MODE_REFERENCE` | constant | `"reference"` |
|
||||
| `MODE_FAST` | constant | `"fast"` |
|
||||
| `MODE_TURBO` | constant | `"turbo"` |
|
||||
| `MODE_RAW` | constant | `"raw"` |
|
||||
| `KEYFILE_VERSION` | constant | On-disk key file format version (`1`) |
|
||||
| `__version__` | constant | Library version string |
|
||||
| `__license__` | constant | `"MIT"` |
|
||||
@@ -50,6 +56,9 @@ ChaosCrypt(
|
||||
| `key_password` | `str \| None` | `None` | Password protecting that key file |
|
||||
| `auto_load` | `bool` | `False` | Load the key file during construction |
|
||||
| `default_key_file` | `str` | `"encryption.key"` | File name used when `key_file` is omitted |
|
||||
| `mode` | `str` | `"fast"` | Performance mode: `reference`, `fast`, `turbo` or `raw` |
|
||||
| `encoding` | `str` | `"hex"` | Output encoding for `turbo`/`raw`: `hex` or `base64` (also accepted as `base`) |
|
||||
| `mid` | `str` | `"bt"` | Intermediate representation for `turbo`/`raw`: `bt` or `hex` (also accepted as `bytes`) |
|
||||
|
||||
When `auto_load=True` and the file cannot be loaded, the corresponding
|
||||
exception is raised from the constructor.
|
||||
@@ -292,6 +301,117 @@ compatibility guarantee.
|
||||
|
||||
---
|
||||
|
||||
## Performance modes
|
||||
|
||||
Four modes, plus two settings that apply to the two byte-pipeline modes.
|
||||
|
||||
| Constant | Value | Pipeline | Ciphertext |
|
||||
|---|---|---|---|
|
||||
| `MODE_REFERENCE` | `"reference"` | original algorithm, character pipeline | hex, 5.33x |
|
||||
| `MODE_FAST` | `"fast"` | same algorithm, bulk operations | hex, 5.33x |
|
||||
| `MODE_TURBO` | `"turbo"` | SHAKE256 keystream, byte pipeline | 2.00x / 1.33x |
|
||||
| `MODE_RAW` | `"raw"` | SHAKE256 plus the long-key layer | 2.00x / 1.33x |
|
||||
|
||||
```python
|
||||
from chaoscrypt import ChaosCrypt, MODES, DEFAULT_MODE
|
||||
|
||||
print(MODES) # ('reference', 'fast', 'turbo', 'raw')
|
||||
print(DEFAULT_MODE) # 'fast'
|
||||
```
|
||||
|
||||
### Settings for the byte-pipeline modes
|
||||
|
||||
| Constant | Value | Applies to | Meaning |
|
||||
|---|---|---|---|
|
||||
| `ENC_HEX` | `"hex"` | `turbo`, `raw` | Two characters per octet (2.00x) |
|
||||
| `ENC_BASE64` | `"base64"` | `turbo`, `raw` | Four characters per three octets (1.33x) |
|
||||
| `MID_BYTES` | `"bt"` | `turbo`, `raw` | XOR layers exchange raw bytes |
|
||||
| `MID_HEX` | `"hex"` | `turbo`, `raw` | XOR layers exchange hexadecimal text |
|
||||
|
||||
```python
|
||||
from chaoscrypt import ChaosCrypt
|
||||
|
||||
cc = ChaosCrypt(mode="raw", encoding="base64", mid="bt")
|
||||
cc.load("my.key", "key-password")
|
||||
|
||||
token = cc.encrypt("hello", "user-password")
|
||||
```
|
||||
|
||||
```python
|
||||
from chaoscrypt import ChaosCryptCore, MODE_TURBO, ENC_BASE64, MID_HEX
|
||||
|
||||
core = ChaosCryptCore(MODE_TURBO, "hash", ENC_BASE64, MID_HEX)
|
||||
core.generate_keymaterial()
|
||||
token = core.encrypt_text("hello", "password")
|
||||
assert core.decrypt_text(token, "password") == "hello"
|
||||
```
|
||||
|
||||
### Which mode to choose
|
||||
|
||||
| If you… | Use |
|
||||
|---|---|
|
||||
| Are not sure | `fast` — the default |
|
||||
| Need maximum speed from a standardised keystream primitive | `raw` with `mid="bt"` |
|
||||
| Need the smallest ciphertext | `turbo` or `raw` with `encoding="base64"` |
|
||||
| Need to interoperate with the character pipeline's shape | `mid="hex"` |
|
||||
| Need a payload byte-compatible with pre-4.2.6 material | `reference` or `fast` |
|
||||
|
||||
`fast` performs the same arithmetic as `reference` and produces the same bytes;
|
||||
only the implementation differs. `turbo` and `raw` change the algorithm and
|
||||
define their own formats.
|
||||
|
||||
> **Artefacts are not interchangeable between mode families**, and `turbo`/`raw`
|
||||
> artefacts are also specific to their `encoding` and `mid`. The header records
|
||||
> all three, so a reader configured differently still reads the artefact
|
||||
> correctly — the ordinary decrypt takes its parameters from the header.
|
||||
|
||||
An unrecognised `mode`, `encoding` or `mid` raises `ValueError` at construction.
|
||||
|
||||
## Reading pre-4.2.6 artefacts
|
||||
|
||||
Artefacts written before the header existed have none, and are rejected by the
|
||||
ordinary path with `E017`. They are read through the compatibility API, which
|
||||
synthesises a minimal header and then uses the ordinary path.
|
||||
|
||||
```python
|
||||
from chaoscrypt import ChaosCrypt
|
||||
|
||||
cc = ChaosCrypt(key_file="my.key", key_password="kf-pwd", auto_load=True)
|
||||
|
||||
# a ciphertext with no header
|
||||
plain = cc.decrypt_legacy(old_ciphertext, "user-password")
|
||||
|
||||
# if you know it is a legacy key file
|
||||
cc.load_legacy_keys(old_key_text, "key-password")
|
||||
|
||||
# let the library work out which it is
|
||||
plain = cc.decrypt_legacy_auto(old_ciphertext, "user-password")
|
||||
|
||||
# migrate once, then use the modern path
|
||||
open("new.key", "w", encoding="utf-8").write(
|
||||
cc.wrap_legacy(old_key_text, is_key=True)
|
||||
)
|
||||
```
|
||||
|
||||
| Method | Purpose |
|
||||
|---|---|
|
||||
| `wrap_legacy(text, is_key=False)` | Prefix a minimal header; the payload is untouched. Idempotent. |
|
||||
| `decrypt_legacy(text, password, is_key=False)` | Read an artefact that may predate the header |
|
||||
| `decrypt_legacy_auto(text, password)` | As above, probing whether the input is a key file or a ciphertext |
|
||||
| `load_legacy_keys(text, key_password)` | Load a pre-header key file |
|
||||
|
||||
`is_old` in the synthesised header means every other field is ignored; the
|
||||
legacy path tries `fast` and falls back to `reference`. Both implement the
|
||||
original algorithm.
|
||||
|
||||
> The compatibility API is a **wrapper** around the ordinary path, never a
|
||||
> lenient branch inside it. Passing headerless input to `decrypt` still raises
|
||||
> `E017`.
|
||||
|
||||
See [FORMAT.md §12](FORMAT.md) for the legacy standard.
|
||||
|
||||
---
|
||||
|
||||
## Error hierarchy
|
||||
|
||||
Every error is a subclass of `ChaosCryptError`, so a single `except` clause
|
||||
@@ -326,6 +446,8 @@ except ChaosCryptError as e:
|
||||
| `E014` | `ChaosCrypt_ERR_FileReadFailed_E014` | Reading a file failed |
|
||||
| `E015` | `ChaosCrypt_ERR_FileWriteFailed_E015` | Writing a file failed |
|
||||
| `E016` | `ChaosCrypt_ERR_UnsupportedKeyVersion_E016` | Key file version not supported |
|
||||
| `E017` | `ChaosCrypt_ERR_MissingMagicHeader_E017` | Input is not a CES artefact |
|
||||
| `E018` | `ChaosCrypt_ERR_MalformedHeader_E018` | Header unusable, or `kind` mismatch |
|
||||
|
||||
Codes are stable and safe to branch on. The human-readable portion of the
|
||||
message may be reworded between minor releases.
|
||||
|
||||
Reference in New Issue
Block a user