Files
ChaosCrypt/docs/benchmark.md
T
dvs dd49043171 v4.2.6: Self-describing format, byte pipeline, and a performance pass
Every key file and ciphertext now carries a header that states how it was
produced: magic, two fixed markers, and a complete JSON configuration record.
A reader takes its parameters from the artefact rather than from the caller.

This is an incompatible format change. Headerless input is rejected with E017;
pre-4.2.6 material is read through a compatibility API that wraps it in a
minimal header and then uses the ordinary path, so the strict path stays strict.

Added
- Self-describing header on all artefacts, with a normative specification in
  docs/FORMAT.md
- E017 MissingMagicHeader and E018 MalformedHeader
- Byte pipeline for turbo and raw: the XOR layers exchange raw bytes rather
  than hexadecimal text, cutting expansion from 5.33x to 2.00x and raising
  throughput about 4x. The old behaviour is available as mid="hex".
- Selectable output encoding for turbo and raw: hex (2.00x) or base64 (1.33x)
- Legacy compatibility API: wrap_legacy, decrypt_legacy, decrypt_legacy_auto,
  load_legacy_keys
- Frozen reference vectors covering every mode, encoding and intermediate
- docs/benchmark.md with measured throughput and the profiling behind it

Changed
- Default mode is fast: byte-for-byte compatible with the original algorithm,
  roughly 30x quicker
- Substitution, case flipping and the printable fold use bytes.translate over
  precomputed tables
- XOR layers use one big-integer operation instead of a per-byte loop
- Keystream derivation caches length-independent base material; expansion uses
  cached translation tables and list accumulation rather than str +=
- _get_dynamic_key repeats by multiplication. The loop version was quadratic:
  covering a 5.6 MB intermediate took 866 ms, now 3 ms, output unchanged.
- _derive_final_key decodes only the prefix the output consumes
- The decrypt path no longer copies or scans the whole artefact: a redundant
  strip(), a full-string int(payload, 16), and three whole-string slices in
  strip_header were removed. Decryption at 512 KiB went from 34 to 87 MB/s.
- Documentation rewritten for 4.2.6; all eleven markdown files are English

Fixed
1. Key derivation is no longer quadratic (_get_dynamic_key and
   _expand_keystream_slow shared the str += pattern)
2. Two hex-intermediate pipelines were asymmetric, so a turbo or raw artefact
   written with mid="hex" could not be read back
3. change_key_password and export_key_text wrote artefacts without a header
4. The constructor accepted encoding="base64" while the header records
   out="base"; both spellings are now accepted
5. A payload that is not valid hexadecimal reports E010 (MalformedCiphertext)
   rather than E011 (DecryptionFailed)
6. The TUI encoding menu referenced a missing translation

Security
- Format break from pre-4.2.6 artefacts, documented rather than accidental
- fast is not a weaker mode: the frozen vectors assert identical output
- The header adds no secrecy and is not authenticated; an application that must
  detect header tampering should MAC the whole artefact

Performance
- raw with the default bt intermediate: ~60 MB/s encryption, ~70 MB/s
  decryption, ciphertext 2.00x the payload
- Throughput holds steady from 64 KiB to 1 GiB
- The original implementation managed 0.08 MB/s
2026-09-27 22:06:34 +08:00

8.7 KiB

Benchmark

Performance reference for chaoscrypt 4.2.6.

Read this first. These numbers are not a claim that ChaosCrypt competes with a C-backed cipher. It is pure Python. AES-GCM in cryptography runs at hundreds of megabytes per second; ChaosCrypt's fastest mode manages tens. What the tables below show is what the implementation costs, how the modes differ, and which parameters are realistic for the workloads this library is meant for.


1. Headline figures

raw mode, hex output, keystream cache warm, best of several runs.

Payload bt encrypt bt decrypt hex encrypt hex decrypt
64 KiB 26.9 60.6 20.7 20.2
128 KiB 54.9 85.5 15.8 14.3
1 MiB 57.3 61.0 19.3 21.8
16 MiB 63.4 69.0 18.9 22.7
128 MiB 59.4 64.5 18.2 18.0
1024 MiB 47.9 31.7 — —

MB/s. Ciphertext expansion: bt 2.00x, hex 8.00x.

The one-line summary: raw with the default bt intermediate runs at roughly 60 MB/s encryption and 70 MB/s decryption, with ciphertext twice the payload size. Throughput holds steady across four orders of magnitude, which is what a linear algorithm looks like.

Against the original implementation

Original raw bt Improvement
1 MiB encrypt 12.050 s (0.08 MB/s) 0.017 s (57 MB/s) ≈700x
Ciphertext 5.33x 2.00x 62% smaller

2. Test environment

Item Value
CPU 16 logical cores
Python CPython 3.13.4
OS Windows 10 (10.0.26200)
Library chaoscrypt 4.2.6, pure Python, zero dependencies
Payload ASCII, "A" * n
Measurement Best of several runs; each size warmed once so the keystream cache is populated

Best-of-N is used because the machine is not otherwise idle. That makes the figures optimistic; a loaded system will be slower, and the ordering between modes is unaffected.

1024 MiB with hex needs an 8 GB ciphertext as a single Python string, which is not addressable. That cell is left blank rather than measured in a way that would only report the allocator's behaviour.


3. Throughput across four orders of magnitude

raw, bt intermediate, encryption:

Size MB/s Ratio to previous
64 KiB 26.9 —
128 KiB 54.9 2.04x
1 MiB 57.3 1.04x
16 MiB 63.4 1.11x
128 MiB 59.4 0.94x
1024 MiB 47.9 0.81x

From 128 KiB upward the figure is flat within ±15%, which is measurement noise plus cache effects rather than algorithmic behaviour. 64 KiB reads low because fixed setup costs are a larger share of a short operation, not because something is quadratic.

The 1024 MiB figure is lower because a 2 GB ciphertext string puts the process under memory pressure. It is an observation about the machine, not the algorithm.

The quadratic bug that used to be here

An earlier revision was genuinely super-linear, and it showed up exactly this way: throughput fell as payload grew. The cause was _get_dynamic_key:

key = self.long_key
while len(key) < text_length:
    key += self.long_key        # reallocates the whole accumulator every round

Covering a 5.6 MB intermediate took 866 ms. Multiplication instead:

return (lk * (text_length // ll + 1))[:text_length]

The same work takes 3 ms, with byte-identical output. _expand_keystream_slow shared the pattern and was fixed the same way.


4. Intermediate representation

The setting with the largest effect on both speed and size.

Payload bt enc bt dec hex enc hex dec Ciphertext bt / hex
64 KiB 26.9 60.6 20.7 20.2 0.1 / 0.5 MB
1 MiB 57.3 61.0 19.3 21.8 2 / 8 MB
16 MiB 63.4 69.0 18.9 22.7 32 / 128 MB
128 MiB 59.4 64.5 18.2 18.0 256 / 1024 MB

hex is about 3x slower and produces 4x more ciphertext. It re-encodes the payload between the XOR layers, which doubles it twice. It exists for interoperability with the character pipeline's shape, not for performance.


5. Output encoding

Encoding Expansion Notes
hex 2.00x Two characters per octet
base64 1.33x Four characters per three octets

Base64 produces a third less output at broadly the same speed. The choice affects only the final encode.


6. How the modes compare

reference fast turbo raw
1 MiB encrypt 12.05 s 0.35 s 0.05 s 0.017 s
Ciphertext 5.33x 5.33x 2.00x 2.00x
Payload compatible with pre-4.2.6 yes yes no no
Keystream primitive hand-rolled expansion hand-rolled expansion SHAKE256 SHAKE256

fast is reference with the same arithmetic performed by C-level bulk operations; its payload is byte-identical. The frozen vectors assert this, so the equivalence cannot silently stop holding.

turbo and raw replace the keystream derivation with SHAKE256 and stop re-encoding between layers. raw keeps the long-key XOR layer, turbo does not.


7. Where the time goes

Profiling raw with the bt intermediate, 10 MiB payload, encryption:

Stage Time Share
_bytes_stages (substitution, flip, reversal) 29 ms 24%
XOR with the long key 38 ms 32%
XOR with the SHAKE keystream 38 ms 32%
Keystream derivation (cached) ~0 ms 0%
Output encode (.hex()) 9 ms 8%
Total 119 ms

Decryption is the mirror image, at 122 ms, and is if anything slightly quicker end to end because it does not encode its output.

The changes that mattered

1. Keeping the payload in bytes between XOR layers (4x). The original re-encoded to hexadecimal after each XOR, doubling the payload at every layer. At 1 MiB that meant a 5.6 MB intermediate. Bytes between layers means a 1.4 MB intermediate instead.

2. Per-character loops to bytes.translate (61x on substitution alone). Substitution was a dictionary lookup per character; it is now one translation over a 256-entry table. The case flip is periodic, so it became ten strided slice translations instead of a per-character branch.

3. Per-byte XOR to one big-integer XOR (43x).

# before
for i, char in enumerate(text):
    result.append(f"{ord(char) ^ ord(key[i % key_len]):02x}")

# after
xored = int.from_bytes(data, 'big') ^ int.from_bytes(key, 'big')
return xored.to_bytes(n, 'big').hex()

4. Not copying or scanning the whole artefact on decrypt (2x). The decrypt path used to call strip(), then int(payload, 16), and strip_header sliced the whole string three times. On a 20 MB artefact that was about 65 ms of pure copying. Removing it took decryption at 512 KiB from 34 MB/s to 87 MB/s.


8. What did not work

An attempt to build all expansion chunks with numpy as a two-dimensional array reached only 2.9x and added a dependency. Cached translation tables were faster and dependency-free.

A later attempt to fold the hex round trip in _derive_final_key by processing only the prefix the output consumes gained 1.6x — worth keeping, but far less than the byte-pipeline change that made the whole stage cheaper.


9. Reproducing these numbers

python bench/benchmark.py
python bench/benchmark.py --sizes 64,1024,65536 --budget 0.05 --repeats 3
python bench/benchmark.py --groups Text,Binary

The harness dispatches every task to a process pool, one worker per core, and reports the best of N rounds per task.

A note on large payloads

Beyond roughly 128 MiB the ciphertext itself becomes the practical limit: at 1024 MiB inputs, bt needs a 2 GB string and hex needs 8 GB. For payloads that large, encrypt in blocks. The per-block throughput is the same, as §3 shows.


10. Summary

Dimension Result
Fastest throughput ~63 MB/s encryption, ~85 MB/s decryption (raw, bt, hex)
Ciphertext overhead 2.00x (bt) or 1.33x (base64)
Speed with a payload compatible with pre-4.2.6 fast, ~30x the original
Linear across 64 KiB to 1 GiB
Dependencies none, in every mode
Best suited for Notes, credentials, documents, personal backups
Not suited for Bulk media at gigabyte scale, high-throughput services, KDF stretching

Bottom line. The original implementation's cost was almost entirely Python-level per-byte work, not the cipher's arithmetic. Keeping the payload in bytes between the XOR layers removed the largest remaining factor, and the result is a library that runs at tens of megabytes per second rather than fractions of one — with the same construction, and with the modes and their trade-offs documented in SECURITY.md.