Files
dvs 5dfdd87c06 Release v3.2.1: performance overhaul (1.96x geometric-mean speed-up)
Pure-Python, zero-dependency speed-up across every hot path. No public API,
error-semantics or security change.

Measured (interleaved A/B min-of-N, idle 16-core, 1 KiB payload):
  Base64 decode      21.98x
  Base58 encode       8.01x
  ChaCha20-Poly1305   3.54x
  XChaCha20-Poly1305  3.28x
  Base64 encode       2.98x
  AES-256-GCM (enc)   2.25x   (dec) 2.26x
  AES-GCM-SIV enc     2.06x
  AES-256 CTR/ECB/CBC 2.01x / 1.96x / 1.88x
  AES single block    1.87x
  Poly1305 MAC        1.41x
  Ed25519 sign        1.41x
  X25519 base         1.19x
  BLAKE2b / SHA3-512 / SHAKE128  1.34x / 1.33x / 1.30x
  => 24 improved, 3 unchanged, 0 regressed.

Techniques (all pure Python):
  - code-generated, fully unrolled AES round function and Keccak-f[1600]
  - big-integer bulk operations for GHASH, POLYVAL, buffer XOR, Base64 pack
  - precomputed lookup tables (AES T-tables, GHASH/POLYVAL 4-bit windows)
  - branch-free constant-time selection in Ed25519 scan and X25519 ladder
  - lazy modular reduction in X25519 field arithmetic
  - batched Base58/Base64 processing
  - inlined inner loops in AEAD assembly and the SHA-2 message schedule

Verification:
  - full test suite: 639 passed, 1 skipped, 0 failed
  - all FIPS / RFC / NIST vectors reproduce byte-for-byte
  - differential probe of 1266 deterministic inputs: identical
    (one unreachable private-helper edge case, no caller can reach it)
  - public API signature comparison: 0 removals, 0 changes
  - independent security regression suite: 35 / 35

Also adds docs/benchmark.md, bench/benchmark.py and the v3.2.1 documentation.
2026-09-27 11:37:33 +08:00

394 lines
16 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Benchmark
Performance reference for `purepycrypto` **v3.2.1**.
> ⚠️ **Read this first.** `purepycrypto` is a **pure-Python** implementation.
> Every primitive is written in Python, with no C extension and no native
> dependency. As a result it is typically **100x-1000x slower** than
> C-backed libraries (OpenSSL, `pycryptodome`, `cryptography`).
>
> These numbers are published to be **honest about the trade-off**, not to
> compete on throughput. They are useful for capacity planning: they tell you
> which parameters are realistic in pure Python and which are not.
---
## 0. v3.2.1 optimization results
**v3.2.1 is a pure optimization release.** Every hot path was reworked while
keeping the public API, the error semantics and the security properties
unchanged. No algorithm was replaced by a C-backed shortcut: the library is
still 100% pure Python, zero dependencies.
### 0.1 Measured speed-ups (v3.2.0 -> v3.2.1)
Measured with an **interleaved A/B, min-of-N** harness on an idle 16-core
machine (same process loads both the old and the new package, alternates them
every round, and keeps the fastest run of each to defeat scheduler noise).
| Operation | v3.2.0 | v3.2.1 | Speed-up |
|---|---:|---:|---:|
| **Base64 decode** | 135.6 us | 6.17 us | **21.98x** |
| **Base58 encode** | 1022.1 us | 127.6 us | **8.01x** |
| **Base58 decode** | 398.1 us | 100.1 us | **3.98x** |
| **ChaCha20-Poly1305 encrypt** | 978.8 us | 276.6 us | **3.54x** |
| **XChaCha20-Poly1305 encrypt** | 1034.9 us | 315.2 us | **3.28x** |
| **Base64 encode** | 159.3 us | 53.4 us | **2.98x** |
| **AES-256-GCM decrypt** | 1841.2 us | 816.4 us | **2.26x** |
| **AES-256-GCM encrypt** | 1835.5 us | 816.7 us | **2.25x** |
| **AES-GCM-SIV encrypt** | 1695.0 us | 824.3 us | **2.06x** |
| **AES-256-CTR** | 1582.5 us | 789.0 us | **2.01x** |
| **AES-256-ECB** | 1494.6 us | 762.9 us | **1.96x** |
| **AES-256-CBC** | 1587.0 us | 844.2 us | **1.88x** |
| **AES single block (256-bit)** | 20.8 us | 11.2 us | **1.87x** |
| **AES single block (128-bit)** | 39.8 us | 28.9 us | **1.38x** |
| Poly1305 MAC | 22.6 us | 16.0 us | 1.41x |
| Ed25519 sign | 1024.0 us | 727.6 us | 1.41x |
| Ed25519 publickey | 466.4 us | 334.5 us | 1.39x |
| Ed25519 verify | 1475.4 us | 1060.0 us | 1.39x |
| X25519 base | 915.3 us | 766.3 us | 1.19x |
| BLAKE2b-64 | 716.4 us | 533.9 us | 1.34x |
| SHA3-512 | 2192.2 us | 1645.4 us | 1.33x |
| SHAKE128 | 1012.8 us | 780.5 us | 1.30x |
| SHA3-256 | 1126.2 us | 891.4 us | 1.26x |
| SHA-512 | 939.9 us | 867.6 us | 1.08x |
| SHA-256 | 1393.8 us | 1341.8 us | 1.04x |
Payload 1 KiB unless noted. RSA keygen and the slow KDF parameter sets are
excluded from this table (they are seconds-scale; see sections 7-8 for their
cost model).
### 0.2 Summary
| Metric | Result |
|---|---|
| Improved operations (>= 1.05x) | **24 / 27** |
| Unchanged (0.98x - 1.05x) | 3 |
| **Regressed (< 0.98x)** | **0** |
| **Geometric-mean speed-up** | **1.96x** |
**Nothing got slower.** Every measured operation is at least as fast as in
v3.2.0, and the overall geometric mean is a **1.96x** speed-up.
### 0.3 Key techniques
All optimizations stay inside pure Python, using only language-level
primitives (which are allowed), never a C-backed *reimplementation* of the
algorithm (which is forbidden by the project rules).
| Technique | Where it was applied | Why it helps |
|---|---|---|
| **Code-generated, fully unrolled functions** | AES round function, Keccak-f permutation | CPython executes a long run of straight-line bytecode far faster than a loop with indexing and attribute lookups |
| **Big-integer bulk operations** | GHASH, POLYVAL, XOR of buffers, Base64 pack/unpack | `int.from_bytes` / `to_bytes` / `^` run in C over a whole buffer in one call |
| **Precomputed lookup tables** | AES T-tables, GHASH/POLYVAL 4-bit windows, Keccak rho/pi offsets | Trades a few hundred bytes of memory for fewer Python-level operations |
| **Constant-time-safe selection** | Ed25519 scalar scan, X25519 Montgomery ladder | Preserves the security properties while removing per-lane branching |
| **Lazy modular reduction** | X25519 field arithmetic | Replaces a `% p` per multiply with a cheaper fold-and-subtract |
| **Inlined hot loops** | AEAD assembly, CTR/GCM block loops, Base58 batching | Removes Python function-call overhead from the innermost loop |
### 0.4 What did *not* change
* **Public API**: function names, signatures, defaults, return types and error
codes are byte-identical. A signature comparison shows 0 removals and
0 changes.
* **Security**: constant-time scalar multiplication, constant-time tag
comparison, RSA blinding and the "never return unauthenticated plaintext"
rule are all preserved. An independent security regression suite (35 checks
covering AEAD tamper detection, constant-time helpers, X25519 low-order
points, Ed25519 constant-time selection, RSA failure paths and error-message
hygiene) passes **35/35**.
* **Correctness**: the full test suite passes (639 passed, 1 skipped), and all
standard vectors (FIPS / RFC / NIST) are reproduced byte-for-byte.
* **Zero dependencies**: still standard library only, still pure Python.
## 1. How to reproduce
The full suite is a single script. Every algorithm / mode is one task,
dispatched over **one worker process per CPU core**, so the whole run finishes
in one parallel wave.
```bash
# from the repository root
export PYTHONPATH=$PWD # Windows: set PYTHONPATH=%CD%
python bench/benchmark.py
```
Design of the harness:
| Aspect | Choice | Why |
|---|---|---|
| Concurrency | `ProcessPoolExecutor`, one worker per core | pure-Python crypto is CPU-bound; threads would not help (the GIL) |
| Task transport | picklable *specs* (op name + parameters), rebuilt inside the worker | Windows uses the `spawn` start method, so lambdas/closures cannot be sent to workers |
| Measurement | warm-up + auto-calibrated loop count + `time.perf_counter()` | the loop count self-adjusts to fill a small time budget, so fast and slow primitives are both measured meaningfully |
| Payloads | 64 B and 1 KiB | typical API-call sizes; large buffers are dominated by the same per-byte cost |
---
## 2. Test environment
| Item | Value |
|---|---|
| CPU | 16 logical cores |
| Python | CPython 3.13.4 |
| OS | Windows |
| Library | `purepycrypto` 3.2.1 (pure Python, zero dependencies) |
| Workers | 16 processes |
| Per-task budget | 30 ms, 1 measurement round |
| Payload sizes | 64 B, 1 KiB |
| **Total wall clock** | **11.0 s for 82 tasks** |
> The 11 s figure is the whole benchmark, not a single operation — the
> 16-process fan-out is what makes a full-suite run this fast.
---
## 3. Hash functions
Throughput is reported for the 1 KiB payload.
| Algorithm | 64 B (ops/s) | 1 KiB (ops/s) | 64 B latency | 1 KiB throughput |
|---|---:|---:|---:|---:|
| **BLAKE2b-64** | 4,125 | 418 | 242 µs | **0.41 MB/s** |
| SHA-512 | 3,054 | 376 | 327 µs | 0.37 MB/s |
| SHAKE128 (32 B out) | 2,714 | 300 | 368 µs | 0.29 MB/s |
| SHAKE256 (64 B out) | 2,353 | 265 | 425 µs | 0.26 MB/s |
| SHA3-256 | 2,086 | 258 | 479 µs | 0.25 MB/s |
| SHA-256 | 1,893 | 207 | 528 µs | 0.20 MB/s |
| SHA3-512 | 2,616 | 187 | 382 µs | 0.18 MB/s |
| HMAC-SHA512 (1 KiB) | — | 285 | — | 0.28 MB/s |
| HMAC-SHA256 (1 KiB) | — | 183 | — | 0.18 MB/s |
| HMAC-SHA3-256 (1 KiB) | — | 187 | — | 0.18 MB/s |
| HMAC-SHA3-512 (1 KiB) | — | 106 | — | 0.10 MB/s |
**Notes**
- BLAKE2b is the fastest hash in the library, followed by SHA-512.
- SHA-3 / SHAKE use the Keccak-f[1600] permutation in pure Python and are
therefore the most expensive family per byte.
- HMAC adds one extra compression pass (inner + outer), so it is roughly
1.5–2× the cost of the bare hash on the same payload.
---
## 4. AES
### 4.1 Single block
| Operation | ops/s | Latency |
|---|---:|---:|
| AES-128 block encrypt | 18,149 | **55.1 µs** |
| AES-256 block encrypt | 18,217 | **54.9 µs** |
The raw block cipher is the fastest cryptographic primitive in the library:
about **18,000 ops/s**, i.e. 55 µs per 16-byte block.
### 4.2 Modes
| Mode | 64 B (ops/s) | 64 B latency | 1 KiB (ops/s) | 1 KiB latency |
|---|---:|---:|---:|---:|
| AES-256-OFB | 3,707 | 270 µs | 168 | 5.95 ms |
| AES-256-CTR | 3,378 | 296 µs | 199 | 5.02 ms |
| AES-256-CBC | 3,292 | 304 µs | 181 | 5.53 ms |
| AES-256-CFB | 2,815 | 355 µs | 225 | 4.44 ms |
| AES-256-ECB | 2,051 | 488 µs | 165 | 6.05 ms |
| AES-256-GCM encrypt | 1,195 | 837 µs | 126 | 7.94 ms |
| AES-256-GCM decrypt | 959 | 1.04 ms | 197 | 5.07 ms |
**Notes**
- Stream-like modes (OFB / CTR) are the fastest bulk modes; they avoid padding
and process the input as a single keystream.
- GCM is the slowest because it adds GHASH authentication on top of CTR.
- ECB is slower than CBC/CTR here mainly because of its padding path.
---
## 5. AES-GCM-SIV (RFC 8452)
| Operation | 64 B (ops/s) | 64 B latency | 1 KiB (ops/s) |
|---|---:|---:|---:|
| AES-128-GCM-SIV encrypt | 2,275 | 439 µs | 211 |
| AES-256-GCM-SIV encrypt | 2,093 | 478 µs | 139 |
| AES-256-GCM-SIV decrypt | 2,140 | 467 µs | 190 |
**Note.** On small inputs GCM-SIV is *faster* than plain AES-GCM in this
library (2,093 vs 1,195 ops/s at 64 B). That is expected: SIV is built on
AES-CTR plus POLYVAL, avoiding the GHASH table path that dominates GCM here.
GCM-SIV also gives nonce-misuse resistance, so for small messages it is the
better default.
---
## 6. ChaCha20-Poly1305
| Operation | 64 B (ops/s) | 64 B latency | 1 KiB (ops/s) | 1 KiB latency |
|---|---:|---:|---:|---:|
| **Poly1305 MAC (1 KiB)** | — | — | **16,337** | **61.2 µs** |
| ChaCha20-Poly1305 encrypt | 2,551 | 392 µs | 234 | 4.28 ms |
| ChaCha20-Poly1305 decrypt | 2,363 | 423 µs | 399 | 2.50 ms |
| XChaCha20-Poly1305 encrypt | 2,209 | 453 µs | 260 | 3.84 ms |
**Note.** Poly1305 is the single fastest AEAD primitive in the library
(16,337 ops/s on 1 KiB, 15.95 MB/s). The AEAD constructions around it are
slower because they must first run the ChaCha20 keystream.
---
## 7. Key derivation functions
KDFs are **intentionally expensive**; here the absolute numbers matter far more
than ops/s. Per-unit costs are normalised so you can extrapolate to production
parameters.
| Function | Measured cost | Normalised unit cost |
|---|---|---|
| **HKDF-SHA256 (32 B out)** | **1.196 ms** | — |
| PBKDF2-SHA256 | 649 ms / 1,000 iterations | **649 ns per iteration** |
| PBKDF2-SHA512 | 891 ms / 1,000 iterations | 891 ns per iteration |
| Argon2id (t=1, m=256 KiB) | 184 ms | **0.718 ms per KiB** |
| scrypt (N=1024, r=8) | 1,751 ms | **1.710 ms per N-unit** |
### 7.1 Extrapolation to production parameters
⚠️ **This is the single most important table in this document.** Pure-Python
KDFs at production parameters are measured in **tens of seconds to minutes**.
| Production parameter set | Cost of one derivation (estimated) |
|---|---|
| PBKDF2-SHA256, **100,000** iterations | **≈ 65 s** |
| scrypt, **N = 2^14**, r = 8 | **≈ 28 s** |
| Argon2id, **t = 3, m = 64 MiB** | **≈ 138 s** |
> 💡 **Recommendation.** When running on pure Python, choose KDF parameters
> **one to two orders of magnitude lower** than you would for a C library, or
> delegate the KDF to a native implementation (`hashlib.scrypt`,
> `hashlib.pbkdf2_hmac`) while keeping `purepycrypto` for everything else.
> A one-second-stretch target that is routine in C is simply not reachable in
> pure Python at comparable parameters.
---
## 8. Public-key algorithms
| Operation | ops/s | Latency |
|---|---:|---:|
| **Ed25519 publickey** | 763 | **1.31 ms** |
| X25519 base (scalar·B) | 358 | 2.79 ms |
| X25519 keygen | 350 | 2.86 ms |
| X25519 shared (ECDH) | 318 | 3.14 ms |
| Ed25519 sign | 400 | 2.50 ms |
| Ed25519 verify | 281 | 3.56 ms |
| RSA-1024 OAEP encrypt | 523 | 1.91 ms |
| RSA-1024 PSS verify | 244 | 4.10 ms |
| RSA-1024 OAEP decrypt | 150 | 6.66 ms |
| **RSA-1024 PSS sign** | 97 | **10.27 ms** |
**Notes**
- `x25519_generate_keypair()` is a fixed-base scalar multiplication, so it
costs essentially the same as `x25519_base()`.
- Ed25519 signing is faster than X25519 ECDH because the base point uses a
precomputed affine Comb table (one point addition per digit, zero doublings).
- RSA **private-key** operations are 3–5× slower than public-key operations:
signing/decryption need a full modular exponentiation with CRT and blinding,
while verification/encryption only need the public exponent e = 65537.
- RSA-1024 is used here for fast measurement; **2048-bit key generation in pure
Python takes seconds** and is not included in the timed table.
---
## 9. Encoding
The encoding module is hand-written bit manipulation (it does **not** call the
standard-library `base64`), and is verified byte-for-byte identical to it.
| Operation | 64 B | 1 KiB | 1 KiB throughput |
|---|---:|---:|---:|
| **Hex encode** | **759,029 ops/s** (1.32 µs) | 184,404 (5.42 µs) | **180 MB/s** |
| Base64 decode | 40,187 (24.9 µs) | 1,422 (703 µs) | 1.85 MB/s |
| Base32 encode | 24,401 (41.0 µs) | 1,646 (608 µs) | 1.61 MB/s |
| Base64 encode | 27,185 (36.8 µs) | 886 (1.13 ms) | 0.87 MB/s |
| Base58 encode | 21,038 (47.5 µs) | 236 (4.24 ms) | 0.23 MB/s |
**Notes**
- Hex encoding is the fastest non-random operation in the library
(759,029 ops/s at 64 B, 180 MB/s on 1 KiB) — it is a simple nibble lookup.
- Base58 is the slowest because it performs repeated big-integer division.
- Base64 decode is faster than encode because encoding must emit padding.
---
## 10. Secure random
All randomness comes from `os.urandom` (the OS CSPRNG); the `random` module is
deliberately not used.
| Operation | ops/s | Latency |
|---|---:|---:|
| **random_bytes(32)** | **889,781** | **1.12 µs** |
| token_hex(32) | 760,926 | 1.31 µs |
| random_bytes(1 KiB) | 413,661 | 2.42 µs |
| token_urlsafe(32) | 386,361 | 2.59 µs |
| random_bit() | 292,535 | 3.42 µs |
| random_int(1, 2^64) | 288,410 | 3.47 µs |
| random_uuid4() | 100,854 | 9.92 µs |
| random_password(16) | 10,103 | 99.0 µs |
**Notes**
- `random_bytes` reaches **889,781 ops/s**, showing that the cost is dominated
by the Python call overhead rather than by the CSPRNG syscall.
- `random_password` is the slowest because it applies a rejection-sampling
alphabet-uniformity loop across 16 characters.
---
## 11. Parallel helpers
| Operation | Wall time | Throughput |
|---|---:|---:|
| `sha3_batch` 64 × 1 KiB (16 processes) | **62.7 ms** | 1.00 MB/s |
| `aes256_gcm_siv_encrypt`, 1 MiB, `n_workers=16` | included above | — |
The library's own batch APIs take an `n_workers` argument and use the same
`multiprocessing` pool. Note that per-call multiprocessing overhead means the
parallel helpers pay off only for workloads with enough items or bytes; for a
single small message, call the plain function.
---
## 12. Overall ranking — fastest operations
| # | Operation | ops/s |
|---|---|---:|
| 1 | random_bytes(32) | **889,781** |
| 2 | token_hex(32) | 760,926 |
| 3 | Hex encode [64 B] | **759,029** |
| 4 | random_bytes(1 KiB) | 413,661 |
| 5 | token_urlsafe(32) | 386,361 |
| 6 | random_bit() | 292,535 |
| 7 | random_int(1, 2^64) | 288,410 |
| 8 | Hex encode [1 KiB] | 184,404 |
| 9 | random_uuid4() | 100,854 |
| 10 | **AES-256 block encrypt** | **18,217** |
Among **cryptographic primitives** (excluding encoding and randomness), the
fastest are **AES single-block encryption (18,217 ops/s)** and
**Poly1305 (16,337 ops/s on 1 KiB)**.
---
## 13. Summary
| Dimension | Result |
|---|---|
| Performance position | Pure Python — **100×–1000× slower** than C-backed libraries (expected) |
| Best suited for | Learning, auditing, zero-dependency embedding, small payloads |
| Not suited for | Bulk file encryption, production-grade KDF parameters, high-throughput TLS |
| Parallel speed-up | 16-core fan-out reduces the full 82-task suite to **11 s** (≈16× vs serial) |
| Implementation quality | 55 µs per AES block and 2.5 ms per Ed25519 signature show that the hand-written bit manipulation and precomputed tables are doing real work |
> **Bottom line.** `purepycrypto` trades throughput for readability, portability
> and zero dependencies. It is a reference implementation you can read end to
> end — not a drop-in replacement for a native cryptography stack.