Every key file and ciphertext now carries a header that states how it was produced: magic, two fixed markers, and a complete JSON configuration record. A reader takes its parameters from the artefact rather than from the caller. This is an incompatible format change. Headerless input is rejected with E017; pre-4.2.6 material is read through a compatibility API that wraps it in a minimal header and then uses the ordinary path, so the strict path stays strict. Added - Self-describing header on all artefacts, with a normative specification in docs/FORMAT.md - E017 MissingMagicHeader and E018 MalformedHeader - Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising throughput about 4x. The old behaviour is available as mid="hex". - Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x) - Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto, load_legacy_keys - Frozen reference vectors covering every mode, encoding and intermediate - docs/benchmark.md with measured throughput and the profiling behind it Changed - Default mode is fast: byte-for-byte compatible with the original algorithm, roughly 30x quicker - Substitution, case flipping and the printable fold use bytes.translate over precomputed tables - XOR layers use one big-integer operation instead of a per-byte loop - Keystream derivation caches length-independent base material; expansion uses cached translation tables and list accumulation rather than str += - _get_dynamic_key repeats by multiplication. The loop version was quadratic: covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged. - _derive_final_key decodes only the prefix the output consumes - The decrypt path no longer copies or scans the whole artefact: a redundant strip(), a full-string int(payload, 16), and three whole-string slices in strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s. - Documentation rewritten for 4.2.6; all eleven markdown files are English Fixed 1. Key derivation is no longer quadratic (_get_dynamic_key and _expand_keystream_slow shared the str += pattern) 2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact written with mid="hex" could not be read back 3. change_key_password and export_key_text wrote artefacts without a header 4. The constructor accepted encoding="base64" while the header records out="base"; both spellings are now accepted 5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext) rather than E011 (DecryptionFailed) 6. The TUI encoding menu referenced a missing translation Security - Format break from pre-4.2.6 artefacts, documented rather than accidental - fast is not a weaker mode: the frozen vectors assert identical output - The header adds no secrecy and is not authenticated; an application that must detect header tampering should MAC the whole artefact Performance - raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s decryption, ciphertext 2.00x the payload - Throughput holds steady from 64 KiB to 1 GiB - The original implementation managed 0.08 MB/s
8.7 KiB
Benchmark
Performance reference for chaoscrypt 4.2.6.
Read this first. These numbers are not a claim that ChaosCrypt competes with a C-backed cipher. It is pure Python. AES-GCM in
cryptographyruns at hundreds of megabytes per second; ChaosCrypt's fastest mode manages tens. What the tables below show is what the implementation costs, how the modes differ, and which parameters are realistic for the workloads this library is meant for.
1. Headline figures
raw mode, hex output, keystream cache warm, best of several runs.
| Payload | bt encrypt |
bt decrypt |
hex encrypt |
hex decrypt |
|---|---|---|---|---|
| 64 KiB | 26.9 | 60.6 | 20.7 | 20.2 |
| 128 KiB | 54.9 | 85.5 | 15.8 | 14.3 |
| 1 MiB | 57.3 | 61.0 | 19.3 | 21.8 |
| 16 MiB | 63.4 | 69.0 | 18.9 | 22.7 |
| 128 MiB | 59.4 | 64.5 | 18.2 | 18.0 |
| 1024 MiB | 47.9 | 31.7 | — | — |
MB/s. Ciphertext expansion: bt 2.00x, hex 8.00x.
The one-line summary: raw with the default bt intermediate runs at
roughly 60 MB/s encryption and 70 MB/s decryption, with ciphertext twice the
payload size. Throughput holds steady across four orders of magnitude, which is
what a linear algorithm looks like.
Against the original implementation
| Original | raw bt |
Improvement | |
|---|---|---|---|
| 1 MiB encrypt | 12.050 s (0.08 MB/s) | 0.017 s (57 MB/s) | ≈700x |
| Ciphertext | 5.33x | 2.00x | 62% smaller |
2. Test environment
| Item | Value |
|---|---|
| CPU | 16 logical cores |
| Python | CPython 3.13.4 |
| OS | Windows 10 (10.0.26200) |
| Library | chaoscrypt 4.2.6, pure Python, zero dependencies |
| Payload | ASCII, "A" * n |
| Measurement | Best of several runs; each size warmed once so the keystream cache is populated |
Best-of-N is used because the machine is not otherwise idle. That makes the figures optimistic; a loaded system will be slower, and the ordering between modes is unaffected.
1024 MiB with
hexneeds an 8 GB ciphertext as a single Python string, which is not addressable. That cell is left blank rather than measured in a way that would only report the allocator's behaviour.
3. Throughput across four orders of magnitude
raw, bt intermediate, encryption:
| Size | MB/s | Ratio to previous |
|---|---|---|
| 64 KiB | 26.9 | — |
| 128 KiB | 54.9 | 2.04x |
| 1 MiB | 57.3 | 1.04x |
| 16 MiB | 63.4 | 1.11x |
| 128 MiB | 59.4 | 0.94x |
| 1024 MiB | 47.9 | 0.81x |
From 128 KiB upward the figure is flat within ±15%, which is measurement noise plus cache effects rather than algorithmic behaviour. 64 KiB reads low because fixed setup costs are a larger share of a short operation, not because something is quadratic.
The 1024 MiB figure is lower because a 2 GB ciphertext string puts the process under memory pressure. It is an observation about the machine, not the algorithm.
The quadratic bug that used to be here
An earlier revision was genuinely super-linear, and it showed up exactly this
way: throughput fell as payload grew. The cause was _get_dynamic_key:
key = self.long_key
while len(key) < text_length:
key += self.long_key # reallocates the whole accumulator every round
Covering a 5.6 MB intermediate took 866 ms. Multiplication instead:
return (lk * (text_length // ll + 1))[:text_length]
The same work takes 3 ms, with byte-identical output. _expand_keystream_slow
shared the pattern and was fixed the same way.
4. Intermediate representation
The setting with the largest effect on both speed and size.
| Payload | bt enc |
bt dec |
hex enc |
hex dec |
Ciphertext bt / hex |
|---|---|---|---|---|---|
| 64 KiB | 26.9 | 60.6 | 20.7 | 20.2 | 0.1 / 0.5 MB |
| 1 MiB | 57.3 | 61.0 | 19.3 | 21.8 | 2 / 8 MB |
| 16 MiB | 63.4 | 69.0 | 18.9 | 22.7 | 32 / 128 MB |
| 128 MiB | 59.4 | 64.5 | 18.2 | 18.0 | 256 / 1024 MB |
hex is about 3x slower and produces 4x more ciphertext. It re-encodes
the payload between the XOR layers, which doubles it twice. It exists for
interoperability with the character pipeline's shape, not for performance.
5. Output encoding
| Encoding | Expansion | Notes |
|---|---|---|
hex |
2.00x | Two characters per octet |
base64 |
1.33x | Four characters per three octets |
Base64 produces a third less output at broadly the same speed. The choice affects only the final encode.
6. How the modes compare
reference |
fast |
turbo |
raw |
|
|---|---|---|---|---|
| 1 MiB encrypt | 12.05 s | 0.35 s | 0.05 s | 0.017 s |
| Ciphertext | 5.33x | 5.33x | 2.00x | 2.00x |
| Payload compatible with pre-4.2.6 | yes | yes | no | no |
| Keystream primitive | hand-rolled expansion | hand-rolled expansion | SHAKE256 | SHAKE256 |
fast is reference with the same arithmetic performed by C-level bulk
operations; its payload is byte-identical. The frozen vectors assert this, so
the equivalence cannot silently stop holding.
turbo and raw replace the keystream derivation with SHAKE256 and stop
re-encoding between layers. raw keeps the long-key XOR layer, turbo does not.
7. Where the time goes
Profiling raw with the bt intermediate, 10 MiB payload, encryption:
| Stage | Time | Share |
|---|---|---|
_bytes_stages (substitution, flip, reversal) |
29 ms | 24% |
| XOR with the long key | 38 ms | 32% |
| XOR with the SHAKE keystream | 38 ms | 32% |
| Keystream derivation (cached) | ~0 ms | 0% |
Output encode (.hex()) |
9 ms | 8% |
| Total | 119 ms |
Decryption is the mirror image, at 122 ms, and is if anything slightly quicker end to end because it does not encode its output.
The changes that mattered
1. Keeping the payload in bytes between XOR layers (4x). The original re-encoded to hexadecimal after each XOR, doubling the payload at every layer. At 1 MiB that meant a 5.6 MB intermediate. Bytes between layers means a 1.4 MB intermediate instead.
2. Per-character loops to bytes.translate (61x on substitution alone).
Substitution was a dictionary lookup per character; it is now one translation
over a 256-entry table. The case flip is periodic, so it became ten strided
slice translations instead of a per-character branch.
3. Per-byte XOR to one big-integer XOR (43x).
# before
for i, char in enumerate(text):
result.append(f"{ord(char) ^ ord(key[i % key_len]):02x}")
# after
xored = int.from_bytes(data, 'big') ^ int.from_bytes(key, 'big')
return xored.to_bytes(n, 'big').hex()
4. Not copying or scanning the whole artefact on decrypt (2x).
The decrypt path used to call strip(), then int(payload, 16), and
strip_header sliced the whole string three times. On a 20 MB artefact that was
about 65 ms of pure copying. Removing it took decryption at 512 KiB from
34 MB/s to 87 MB/s.
8. What did not work
An attempt to build all expansion chunks with numpy as a two-dimensional
array reached only 2.9x and added a dependency. Cached translation tables were
faster and dependency-free.
A later attempt to fold the hex round trip in _derive_final_key by processing
only the prefix the output consumes gained 1.6x — worth keeping, but far less
than the byte-pipeline change that made the whole stage cheaper.
9. Reproducing these numbers
python bench/benchmark.py
python bench/benchmark.py --sizes 64,1024,65536 --budget 0.05 --repeats 3
python bench/benchmark.py --groups Text,Binary
The harness dispatches every task to a process pool, one worker per core, and reports the best of N rounds per task.
A note on large payloads
Beyond roughly 128 MiB the ciphertext itself becomes the practical limit: at
1024 MiB inputs, bt needs a 2 GB string and hex needs 8 GB. For payloads
that large, encrypt in blocks. The per-block throughput is the same, as §3
shows.
10. Summary
| Dimension | Result |
|---|---|
| Fastest throughput | ~63 MB/s encryption, ~85 MB/s decryption (raw, bt, hex) |
| Ciphertext overhead | 2.00x (bt) or 1.33x (base64) |
| Speed with a payload compatible with pre-4.2.6 | fast, ~30x the original |
| Linear across | 64 KiB to 1 GiB |
| Dependencies | none, in every mode |
| Best suited for | Notes, credentials, documents, personal backups |
| Not suited for | Bulk media at gigabyte scale, high-throughput services, KDF stretching |
Bottom line. The original implementation's cost was almost entirely Python-level per-byte work, not the cipher's arithmetic. Keeping the payload in bytes between the XOR layers removed the largest remaining factor, and the result is a library that runs at tens of megabytes per second rather than fractions of one — with the same construction, and with the modes and their trade-offs documented in SECURITY.md.