Files
dvs dd49043171 v4.2.6: Self-describing format, byte pipeline, and a performance pass
Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
2026-09-27 22:06:34 +08:00

586 lines
19 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# API Reference
Complete reference for the `chaoscrypt` Python API.
```python
from chaoscrypt import ChaosCrypt # high-level facade
from chaoscrypt import ChaosCryptCore # raw algorithm
from chaoscrypt import ChaosCryptError # base error class
```
---
## Package contents
| Name | Kind | Description |
|---|---|---|
| `ChaosCrypt` | class | Key file management plus encryption helpers |
| `ChaosCryptCore` | class | The algorithm, with no file handling |
| `ChaosCryptError` | exception | Base class for every error raised |
| `encrypt` | function | One-shot encrypt using a key file |
| `decrypt` | function | One-shot decrypt using a key file |
| `create_key` | function | One-shot key file generation |
| `DEFAULT_KEY_FILE` | constant | `"encryption.key"` |
| `MODES` | constant | Tuple of the four mode names |
| `DEFAULT_MODE` | constant | `"fast"` |
| `MODE_REFERENCE` | constant | `"reference"` |
| `MODE_FAST` | constant | `"fast"` |
| `MODE_TURBO` | constant | `"turbo"` |
| `MODE_RAW` | constant | `"raw"` |
| `KEYFILE_VERSION` | constant | On-disk key file format version (`1`) |
| `__version__` | constant | Library version string |
| `__license__` | constant | `"MIT"` |
---
## `class ChaosCrypt`
The facade most callers should use. Wraps key file handling around
`ChaosCryptCore`.
### Constructor
```python
ChaosCrypt(
key_file: str | None = None,
key_password: str | None = None,
*,
auto_load: bool = False,
default_key_file: str = "encryption.key",
)
```
| Parameter | Type | Default | Description |
|---|---|---|---|
| `key_file` | `str \| None` | `None` | Path to a key file |
| `key_password` | `str \| None` | `None` | Password protecting that key file |
| `auto_load` | `bool` | `False` | Load the key file during construction |
| `default_key_file` | `str` | `"encryption.key"` | File name used when `key_file` is omitted |
| `mode` | `str` | `"fast"` | Performance mode: `reference`, `fast`, `turbo` or `raw` |
| `encoding` | `str` | `"hex"` | Output encoding for `turbo`/`raw`: `hex` or `base64` (also accepted as `base`) |
| `mid` | `str` | `"bt"` | Intermediate representation for `turbo`/`raw`: `bt` or `hex` (also accepted as `bytes`) |
When `auto_load=True` and the file cannot be loaded, the corresponding
exception is raised from the constructor.
```python
cc = ChaosCrypt("my.key", "key-password", auto_load=True)
```
### Properties
#### `loaded -> bool`
Whether usable key material is currently held in memory.
```python
cc = ChaosCrypt()
assert cc.loaded is False
cc.create_key("pwd", "my.key")
assert cc.loaded is True
```
#### `is_loaded() -> bool`
Method form of the same check, for call sites that read better that way.
### Key management
#### `create_key(key_password, key_file=None, *, overwrite=False) -> str`
Generate a fresh key file.
| Parameter | Type | Default | Description |
|---|---|---|---|
| `key_password` | `str` | required | Password that will protect the file |
| `key_file` | `str \| None` | `None` | Destination; defaults to the configured name |
| `overwrite` | `bool` | `False` | Replace an existing file rather than refusing |
**Returns** the path written.
**Raises** `ChaosCrypt_ERR_EmptyPassword_E006` for an empty password, or
`ChaosCrypt_ERR_KeyFileExists_E005` if the file exists and `overwrite` is
false.
```python
cc = ChaosCrypt()
cc.create_key("strong-password", "my.key")
cc.create_key("strong-password", "my.key", overwrite=True) # replace
```
Aliases `generate_keys` and `create_key_file` are provided for readability.
#### `load(key_file=None, key_password=None) -> ChaosCrypt`
Load a key file from disk. Falls back to the instance attributes, then to the
default file name. Returns `self`, so calls can be chained.
**Raises** `ChaosCrypt_ERR_KeyFileNotFound_E001`,
`ChaosCrypt_ERR_EmptyPassword_E006`, `ChaosCrypt_ERR_KeyPasswordIncorrect_E003`,
or `ChaosCrypt_ERR_KeyFileCorrupted_E002`.
```python
cc = ChaosCrypt().load("my.key", "key-password")
```
#### `load_from_text(key_text, key_password) -> ChaosCrypt`
Load key material directly from a key file's contents, without touching disk.
Useful when the key is stored in a password manager or retrieved from a vault.
```python
text = vault.read("chaoscrypt-key")
cc = ChaosCrypt().load_from_text(text, "key-password")
```
#### `export_key_text(key_password) -> str`
Generate key material and return it as text, writing nothing to disk. The
instance becomes loaded and immediately usable.
```python
cc = ChaosCrypt()
text = cc.export_key_text("key-password")
vault.write("chaoscrypt-key", text)
token = cc.encrypt("data", "encryption-password") # usable right away
```
#### `change_key_password(old_password, new_password, key_file=None) -> str`
Re-wrap an existing key file under a new password. The key material is
unchanged, so previously encrypted data remains readable.
**Raises** `ChaosCrypt_ERR_EmptyPassword_E006` if the new password is empty,
or `ChaosCrypt_ERR_KeyFileNotFound_E001` / `ChaosCrypt_ERR_KeyPasswordIncorrect_E003`
if the old password does not unlock the file.
```python
cc.change_key_password("old-password", "new-password", "my.key")
```
#### `info() -> dict`
Return a summary of the loaded key material.
```python
{
"key_file": "my.key",
"version": 1,
"upper_mapping": "QWERTYUIOPASDFGHJKLZXCVBNM",
"lower_mapping": "qwertyuiopasdfghjklzxcvbnm",
"digit_mapping": {...},
"equal_mapping": {...},
"flip_pattern": [1, 0, 1, 0, 0, 1, 0, 0, 1, 0],
"long_key_length": 4096,
"short_key_length": 512,
"loaded": True,
}
```
The substitution tables describe the structure of your key file. Treat the
result as sensitive.
**Raises** `ChaosCrypt_ERR_KeyNotLoaded_E004` if nothing is loaded.
### Encryption
#### `encrypt(plaintext, password) -> str`
Encrypt a string, returning a hexadecimal ciphertext.
```python
token = cc.encrypt("hello world", "encryption-password")
```
**Raises** `ChaosCrypt_ERR_KeyNotLoaded_E004`,
`ChaosCrypt_ERR_EmptyPassword_E006`, or `ChaosCrypt_ERR_EmptyPlaintext_E008`.
#### `decrypt(ciphertext, password) -> str`
Decrypt a hexadecimal ciphertext back into a string.
```python
plain = cc.decrypt(token, "encryption-password")
```
**Raises** `ChaosCrypt_ERR_MalformedCiphertext_E010` if the input is not
valid hexadecimal, or `ChaosCrypt_ERR_DecryptionFailed_E011` if the password
is wrong, the key file does not match, or the ciphertext was altered.
#### `encrypt_bytes(data, password) -> str`
Encrypt arbitrary binary data. Returns a hexadecimal ciphertext.
```python
raw = open("archive.zip", "rb").read()
token = cc.encrypt_bytes(raw, "encryption-password")
```
#### `decrypt_bytes(ciphertext, password) -> bytes`
Decrypt back into the original binary data.
```python
restored = cc.decrypt_bytes(token, "encryption-password")
open("archive.restored.zip", "wb").write(restored)
```
### File helpers
#### `encrypt_file(src, password, dst=None) -> str`
Encrypt a file on disk, writing UTF-8 hexadecimal ciphertext. When `dst` is
omitted, the output path is `src + ".cc"`. Returns the output path.
**Raises** `ChaosCrypt_ERR_FileReadFailed_E014` or
`ChaosCrypt_ERR_FileWriteFailed_E015`.
#### `decrypt_file(src, password, dst=None) -> str`
Decrypt a `.cc` ciphertext file, restoring the original bytes. When `dst` is
omitted, a trailing `.cc` is stripped, or `.out` is appended. Returns the
output path.
```python
cc.encrypt_file("report.pdf", "encryption-password")
cc.decrypt_file("report.pdf.cc", "encryption-password")
```
---
## `class ChaosCryptCore`
The algorithm itself, with no file handling and no user interaction. Use it
when you manage key material yourself — for example by loading it from a
database or a secrets service.
```python
from chaoscrypt import ChaosCryptCore
core = ChaosCryptCore()
core.generate_keymaterial() # random key material
token = core.encrypt_text("hello", "password")
assert core.decrypt_text(token, "password") == "hello"
```
### Key material methods
| Method | Description |
|---|---|
| `generate_keymaterial() -> dict` | Generate a complete random key set; leaves the instance loaded |
| `serialize_keys(key_password, save_path) -> str` | Generate and write a key file |
| `load_keys(key_file, key_password) -> None` | Load a key file from disk |
| `load_keys_from_text(text, key_password) -> None` | Load from key file contents |
### Encryption methods
| Method | Description |
|---|---|
| `encrypt_text(plaintext, password) -> str` | Encrypt a string to hexadecimal |
| `decrypt_text(ciphertext, password) -> str` | Decrypt to a string |
| `encrypt_bytes(data, password) -> str` | Encrypt binary data |
| `decrypt_bytes(ciphertext, password) -> bytes` | Decrypt to binary data |
### Attributes
Set by `generate_keymaterial()` or one of the `load_keys*` methods:
| Attribute | Type | Description |
|---|---|---|
| `keys_loaded` | `bool` | Whether key material is present |
| `upper_mapping` | `str` | Bijection over `A–Z`, 26 characters |
| `lower_mapping` | `str` | Bijection over `a–z`, 26 characters |
| `digit_mapping` | `dict` | Digits `0–9` to distinct alphanumerics |
| `equal_mapping` | `dict` | Padding counts `0–3` to distinct characters |
| `flip_pattern` | `list[int]` | Ten bits controlling case flipping |
| `long_key` | `str` | 4096 hexadecimal nibbles |
| `short_key` | `str` | 512 hexadecimal nibbles |
Attributes prefixed with an underscore are internal and not covered by the
compatibility guarantee.
---
## Performance modes
Four modes, plus two settings that apply to the two byte-pipeline modes.
| Constant | Value | Pipeline | Ciphertext |
|---|---|---|---|
| `MODE_REFERENCE` | `"reference"` | original algorithm, character pipeline | hex, 5.33x |
| `MODE_FAST` | `"fast"` | same algorithm, bulk operations | hex, 5.33x |
| `MODE_TURBO` | `"turbo"` | SHAKE256 keystream, byte pipeline | 2.00x / 1.33x |
| `MODE_RAW` | `"raw"` | SHAKE256 plus the long-key layer | 2.00x / 1.33x |
```python
from chaoscrypt import ChaosCrypt, MODES, DEFAULT_MODE
print(MODES) # ('reference', 'fast', 'turbo', 'raw')
print(DEFAULT_MODE) # 'fast'
```
### Settings for the byte-pipeline modes
| Constant | Value | Applies to | Meaning |
|---|---|---|---|
| `ENC_HEX` | `"hex"` | `turbo`, `raw` | Two characters per octet (2.00x) |
| `ENC_BASE64` | `"base64"` | `turbo`, `raw` | Four characters per three octets (1.33x) |
| `MID_BYTES` | `"bt"` | `turbo`, `raw` | XOR layers exchange raw bytes |
| `MID_HEX` | `"hex"` | `turbo`, `raw` | XOR layers exchange hexadecimal text |
```python
from chaoscrypt import ChaosCrypt
cc = ChaosCrypt(mode="raw", encoding="base64", mid="bt")
cc.load("my.key", "key-password")
token = cc.encrypt("hello", "user-password")
```
```python
from chaoscrypt import ChaosCryptCore, MODE_TURBO, ENC_BASE64, MID_HEX
core = ChaosCryptCore(MODE_TURBO, "hash", ENC_BASE64, MID_HEX)
core.generate_keymaterial()
token = core.encrypt_text("hello", "password")
assert core.decrypt_text(token, "password") == "hello"
```
### Which mode to choose
| If you… | Use |
|---|---|
| Are not sure | `fast` — the default |
| Need maximum speed from a standardised keystream primitive | `raw` with `mid="bt"` |
| Need the smallest ciphertext | `turbo` or `raw` with `encoding="base64"` |
| Need to interoperate with the character pipeline's shape | `mid="hex"` |
| Need a payload byte-compatible with pre-4.2.6 material | `reference` or `fast` |
`fast` performs the same arithmetic as `reference` and produces the same bytes;
only the implementation differs. `turbo` and `raw` change the algorithm and
define their own formats.
> **Artefacts are not interchangeable between mode families**, and `turbo`/`raw`
> artefacts are also specific to their `encoding` and `mid`. The header records
> all three, so a reader configured differently still reads the artefact
> correctly — the ordinary decrypt takes its parameters from the header.
An unrecognised `mode`, `encoding` or `mid` raises `ValueError` at construction.
## Reading pre-4.2.6 artefacts
Artefacts written before the header existed have none, and are rejected by the
ordinary path with `E017`. They are read through the compatibility API, which
synthesises a minimal header and then uses the ordinary path.
```python
from chaoscrypt import ChaosCrypt
cc = ChaosCrypt(key_file="my.key", key_password="kf-pwd", auto_load=True)
# a ciphertext with no header
plain = cc.decrypt_legacy(old_ciphertext, "user-password")
# if you know it is a legacy key file
cc.load_legacy_keys(old_key_text, "key-password")
# let the library work out which it is
plain = cc.decrypt_legacy_auto(old_ciphertext, "user-password")
# migrate once, then use the modern path
open("new.key", "w", encoding="utf-8").write(
cc.wrap_legacy(old_key_text, is_key=True)
)
```
| Method | Purpose |
|---|---|
| `wrap_legacy(text, is_key=False)` | Prefix a minimal header; the payload is untouched. Idempotent. |
| `decrypt_legacy(text, password, is_key=False)` | Read an artefact that may predate the header |
| `decrypt_legacy_auto(text, password)` | As above, probing whether the input is a key file or a ciphertext |
| `load_legacy_keys(text, key_password)` | Load a pre-header key file |
`is_old` in the synthesised header means every other field is ignored; the
legacy path tries `fast` and falls back to `reference`. Both implement the
original algorithm.
> The compatibility API is a **wrapper** around the ordinary path, never a
> lenient branch inside it. Passing headerless input to `decrypt` still raises
> `E017`.
See [FORMAT.md §12](FORMAT.md) for the legacy standard.
---
## Error hierarchy
Every error is a subclass of `ChaosCryptError`, so a single `except` clause
catches everything:
```python
from chaoscrypt import ChaosCryptError
try:
cc.decrypt(token, "password")
except ChaosCryptError as e:
print(e.message) # ChaosCrypt_ERR_DecryptionFailed_E011
print(e.code) # E011
print(e.detail) # DecryptionFailed
```
| Code | Class | Raised when |
|---|---|---|
| `E001` | `ChaosCrypt_ERR_KeyFileNotFound_E001` | The key file does not exist |
| `E002` | `ChaosCrypt_ERR_KeyFileCorrupted_E002` | The key file cannot be parsed |
| `E003` | `ChaosCrypt_ERR_KeyPasswordIncorrect_E003` | Wrong key file password |
| `E004` | `ChaosCrypt_ERR_KeyNotLoaded_E004` | Operation needs a key, none loaded |
| `E005` | `ChaosCrypt_ERR_KeyFileExists_E005` | File exists and overwrite not permitted |
| `E006` | `ChaosCrypt_ERR_EmptyPassword_E006` | A required password was missing |
| `E007` | `ChaosCrypt_ERR_PasswordMismatch_E007` | Two password entries differed |
| `E008` | `ChaosCrypt_ERR_EmptyPlaintext_E008` | The plaintext was empty |
| `E009` | `ChaosCrypt_ERR_EmptyCiphertext_E009` | The ciphertext was empty |
| `E010` | `ChaosCrypt_ERR_MalformedCiphertext_E010` | Not valid hexadecimal, or odd length |
| `E011` | `ChaosCrypt_ERR_DecryptionFailed_E011` | Decryption produced invalid data |
| `E012` | `ChaosCrypt_ERR_EncodingFailed_E012` | Base64 or UTF-8 conversion failed |
| `E013` | `ChaosCrypt_ERR_InvalidArgument_E013` | An argument had an invalid value |
| `E014` | `ChaosCrypt_ERR_FileReadFailed_E014` | Reading a file failed |
| `E015` | `ChaosCrypt_ERR_FileWriteFailed_E015` | Writing a file failed |
| `E016` | `ChaosCrypt_ERR_UnsupportedKeyVersion_E016` | Key file version not supported |
| `E017` | `ChaosCrypt_ERR_MissingMagicHeader_E017` | Input is not a CES artefact |
| `E018` | `ChaosCrypt_ERR_MalformedHeader_E018` | Header unusable, or `kind` mismatch |
Codes are stable and safe to branch on. The human-readable portion of the
message may be reworded between minor releases.
### Custom details
Each error accepts an optional `detail` and `code`, which is useful for
adding context such as a file path:
```python
from chaoscrypt.errors import ChaosCrypt_ERR_KeyFileNotFound_E001
raise ChaosCrypt_ERR_KeyFileNotFound_E001(
detail="KeyFileNotFound:/etc/keys/prod.key", code="E001"
)
```
---
## Module-level functions
Convenience wrappers for scripts that perform a single operation.
### `encrypt(plaintext, password, key_file="encryption.key", key_password=None) -> str`
Load a key file and encrypt in one call. When `key_password` is omitted it
falls back to `password`, which suits setups where one secret guards both
layers.
```python
import chaoscrypt
token = chaoscrypt.encrypt("data", "user-pwd", "my.key", "key-pwd")
```
### `decrypt(ciphertext, password, key_file="encryption.key", key_password=None) -> str`
The inverse of the above.
```python
plain = chaoscrypt.decrypt(token, "user-pwd", "my.key", "key-pwd")
```
### `create_key(key_password, key_file="encryption.key", *, overwrite=False) -> str`
Generate a key file in one call. Returns the path written.
```python
chaoscrypt.create_key("key-pwd", "my.key")
```
---
## Complete example
```python
"""Encrypt a config file, then restore it."""
from chaoscrypt import ChaosCrypt, ChaosCryptError
KEY_FILE = "secrets.key"
KEY_PASSWORD = "key-file-password"
DATA_PASSWORD = "encryption-password"
def main() -> None:
cc = ChaosCrypt()
# Reuse an existing key file, or create one on first run.
try:
cc.load(KEY_FILE, KEY_PASSWORD)
except ChaosCryptError:
cc.create_key(KEY_PASSWORD, KEY_FILE)
# Text round trip.
token = cc.encrypt("API_KEY=abc123", DATA_PASSWORD)
print(f"Ciphertext: {token}")
print(f"Restored: {cc.decrypt(token, DATA_PASSWORD)}")
# File round trip.
cc.encrypt_file("config.json", DATA_PASSWORD)
cc.decrypt_file("config.json.cc", DATA_PASSWORD, "config.restored.json")
# A wrong password is reported clearly.
try:
cc.decrypt(token, "not-the-password")
except ChaosCryptError as e:
print(f"Rejected as expected: {e} (code {e.code})")
if __name__ == "__main__":
main()
```
---
## Thread safety
`ChaosCrypt` and `ChaosCryptCore` instances are **not** thread-safe. Key
material is mutable state on the instance, and loading a key while an
encryption is in progress is a data race.
Either use one instance per thread, or create the instance once and guard it
with a lock:
```python
import threading
from chaoscrypt import ChaosCrypt
cc = ChaosCrypt().load("my.key", "key-password")
lock = threading.Lock()
def encrypt_safely(text: str, password: str) -> str:
with lock:
return cc.encrypt(text, password)
```
The module-level functions create a fresh instance on each call, so they are
safe to use concurrently without a lock — at the cost of reloading the key
file each time.
---
## Compatibility guarantee
Within a major version:
- the public API described here will not change incompatibly;
- error codes will not be renumbered;
- the key file format will not change;
- ciphertexts produced by any earlier release will continue to decrypt.
The underscore-prefixed members of `ChaosCryptCore` are internal and may
change without notice.