Release v3.2.1: performance overhaul (1.96x geometric-mean speed-up)

Pure-Python, zero-dependency speed-up across every hot path. No public API,
error-semantics or security change.

Measured (interleaved A/B min-of-N, idle 16-core, 1 KiB payload):
  Base64 decode      21.98x
  Base58 encode       8.01x
  ChaCha20-Poly1305   3.54x
  XChaCha20-Poly1305  3.28x
  Base64 encode       2.98x
  AES-256-GCM (enc)   2.25x   (dec) 2.26x
  AES-GCM-SIV enc     2.06x
  AES-256 CTR/ECB/CBC 2.01x / 1.96x / 1.88x
  AES single block    1.87x
  Poly1305 MAC        1.41x
  Ed25519 sign        1.41x
  X25519 base         1.19x
  BLAKE2b / SHA3-512 / SHAKE128  1.34x / 1.33x / 1.30x
  => 24 improved, 3 unchanged, 0 regressed.

Techniques (all pure Python):
  - code-generated, fully unrolled AES round function and Keccak-f[1600]
  - big-integer bulk operations for GHASH, POLYVAL, buffer XOR, Base64 pack
  - precomputed lookup tables (AES T-tables, GHASH/POLYVAL 4-bit windows)
  - branch-free constant-time selection in Ed25519 scan and X25519 ladder
  - lazy modular reduction in X25519 field arithmetic
  - batched Base58/Base64 processing
  - inlined inner loops in AEAD assembly and the SHA-2 message schedule

Verification:
  - full test suite: 639 passed, 1 skipped, 0 failed
  - all FIPS / RFC / NIST vectors reproduce byte-for-byte
  - differential probe of 1266 deterministic inputs: identical
    (one unreachable private-helper edge case, no caller can reach it)
  - public API signature comparison: 0 removals, 0 changes
  - independent security regression suite: 35 / 35

Also adds docs/benchmark.md, bench/benchmark.py and the v3.2.1 documentation.
This commit is contained in:
dvs
2026-09-27 11:37:33 +08:00
parent f5c86d53e5
commit 5dfdd87c06
21 changed files with 5816 additions and 1701 deletions
+393
View File
@@ -0,0 +1,393 @@
# Benchmark
Performance reference for `purepycrypto` **v3.2.1**.
> ⚠️ **Read this first.** `purepycrypto` is a **pure-Python** implementation.
> Every primitive is written in Python, with no C extension and no native
> dependency. As a result it is typically **100x-1000x slower** than
> C-backed libraries (OpenSSL, `pycryptodome`, `cryptography`).
>
> These numbers are published to be **honest about the trade-off**, not to
> compete on throughput. They are useful for capacity planning: they tell you
> which parameters are realistic in pure Python and which are not.
---
## 0. v3.2.1 optimization results
**v3.2.1 is a pure optimization release.** Every hot path was reworked while
keeping the public API, the error semantics and the security properties
unchanged. No algorithm was replaced by a C-backed shortcut: the library is
still 100% pure Python, zero dependencies.
### 0.1 Measured speed-ups (v3.2.0 -> v3.2.1)
Measured with an **interleaved A/B, min-of-N** harness on an idle 16-core
machine (same process loads both the old and the new package, alternates them
every round, and keeps the fastest run of each to defeat scheduler noise).
| Operation | v3.2.0 | v3.2.1 | Speed-up |
|---|---:|---:|---:|
| **Base64 decode** | 135.6 us | 6.17 us | **21.98x** |
| **Base58 encode** | 1022.1 us | 127.6 us | **8.01x** |
| **Base58 decode** | 398.1 us | 100.1 us | **3.98x** |
| **ChaCha20-Poly1305 encrypt** | 978.8 us | 276.6 us | **3.54x** |
| **XChaCha20-Poly1305 encrypt** | 1034.9 us | 315.2 us | **3.28x** |
| **Base64 encode** | 159.3 us | 53.4 us | **2.98x** |
| **AES-256-GCM decrypt** | 1841.2 us | 816.4 us | **2.26x** |
| **AES-256-GCM encrypt** | 1835.5 us | 816.7 us | **2.25x** |
| **AES-GCM-SIV encrypt** | 1695.0 us | 824.3 us | **2.06x** |
| **AES-256-CTR** | 1582.5 us | 789.0 us | **2.01x** |
| **AES-256-ECB** | 1494.6 us | 762.9 us | **1.96x** |
| **AES-256-CBC** | 1587.0 us | 844.2 us | **1.88x** |
| **AES single block (256-bit)** | 20.8 us | 11.2 us | **1.87x** |
| **AES single block (128-bit)** | 39.8 us | 28.9 us | **1.38x** |
| Poly1305 MAC | 22.6 us | 16.0 us | 1.41x |
| Ed25519 sign | 1024.0 us | 727.6 us | 1.41x |
| Ed25519 publickey | 466.4 us | 334.5 us | 1.39x |
| Ed25519 verify | 1475.4 us | 1060.0 us | 1.39x |
| X25519 base | 915.3 us | 766.3 us | 1.19x |
| BLAKE2b-64 | 716.4 us | 533.9 us | 1.34x |
| SHA3-512 | 2192.2 us | 1645.4 us | 1.33x |
| SHAKE128 | 1012.8 us | 780.5 us | 1.30x |
| SHA3-256 | 1126.2 us | 891.4 us | 1.26x |
| SHA-512 | 939.9 us | 867.6 us | 1.08x |
| SHA-256 | 1393.8 us | 1341.8 us | 1.04x |
Payload 1 KiB unless noted. RSA keygen and the slow KDF parameter sets are
excluded from this table (they are seconds-scale; see sections 7-8 for their
cost model).
### 0.2 Summary
| Metric | Result |
|---|---|
| Improved operations (>= 1.05x) | **24 / 27** |
| Unchanged (0.98x - 1.05x) | 3 |
| **Regressed (< 0.98x)** | **0** |
| **Geometric-mean speed-up** | **1.96x** |
**Nothing got slower.** Every measured operation is at least as fast as in
v3.2.0, and the overall geometric mean is a **1.96x** speed-up.
### 0.3 Key techniques
All optimizations stay inside pure Python, using only language-level
primitives (which are allowed), never a C-backed *reimplementation* of the
algorithm (which is forbidden by the project rules).
| Technique | Where it was applied | Why it helps |
|---|---|---|
| **Code-generated, fully unrolled functions** | AES round function, Keccak-f permutation | CPython executes a long run of straight-line bytecode far faster than a loop with indexing and attribute lookups |
| **Big-integer bulk operations** | GHASH, POLYVAL, XOR of buffers, Base64 pack/unpack | `int.from_bytes` / `to_bytes` / `^` run in C over a whole buffer in one call |
| **Precomputed lookup tables** | AES T-tables, GHASH/POLYVAL 4-bit windows, Keccak rho/pi offsets | Trades a few hundred bytes of memory for fewer Python-level operations |
| **Constant-time-safe selection** | Ed25519 scalar scan, X25519 Montgomery ladder | Preserves the security properties while removing per-lane branching |
| **Lazy modular reduction** | X25519 field arithmetic | Replaces a `% p` per multiply with a cheaper fold-and-subtract |
| **Inlined hot loops** | AEAD assembly, CTR/GCM block loops, Base58 batching | Removes Python function-call overhead from the innermost loop |
### 0.4 What did *not* change
* **Public API**: function names, signatures, defaults, return types and error
codes are byte-identical. A signature comparison shows 0 removals and
0 changes.
* **Security**: constant-time scalar multiplication, constant-time tag
comparison, RSA blinding and the "never return unauthenticated plaintext"
rule are all preserved. An independent security regression suite (35 checks
covering AEAD tamper detection, constant-time helpers, X25519 low-order
points, Ed25519 constant-time selection, RSA failure paths and error-message
hygiene) passes **35/35**.
* **Correctness**: the full test suite passes (639 passed, 1 skipped), and all
standard vectors (FIPS / RFC / NIST) are reproduced byte-for-byte.
* **Zero dependencies**: still standard library only, still pure Python.
## 1. How to reproduce
The full suite is a single script. Every algorithm / mode is one task,
dispatched over **one worker process per CPU core**, so the whole run finishes
in one parallel wave.
```bash
# from the repository root
export PYTHONPATH=$PWD # Windows: set PYTHONPATH=%CD%
python bench/benchmark.py
```
Design of the harness:
| Aspect | Choice | Why |
|---|---|---|
| Concurrency | `ProcessPoolExecutor`, one worker per core | pure-Python crypto is CPU-bound; threads would not help (the GIL) |
| Task transport | picklable *specs* (op name + parameters), rebuilt inside the worker | Windows uses the `spawn` start method, so lambdas/closures cannot be sent to workers |
| Measurement | warm-up + auto-calibrated loop count + `time.perf_counter()` | the loop count self-adjusts to fill a small time budget, so fast and slow primitives are both measured meaningfully |
| Payloads | 64 B and 1 KiB | typical API-call sizes; large buffers are dominated by the same per-byte cost |
---
## 2. Test environment
| Item | Value |
|---|---|
| CPU | 16 logical cores |
| Python | CPython 3.13.4 |
| OS | Windows |
| Library | `purepycrypto` 3.2.1 (pure Python, zero dependencies) |
| Workers | 16 processes |
| Per-task budget | 30 ms, 1 measurement round |
| Payload sizes | 64 B, 1 KiB |
| **Total wall clock** | **11.0 s for 82 tasks** |
> The 11 s figure is the whole benchmark, not a single operation — the
> 16-process fan-out is what makes a full-suite run this fast.
---
## 3. Hash functions
Throughput is reported for the 1 KiB payload.
| Algorithm | 64 B (ops/s) | 1 KiB (ops/s) | 64 B latency | 1 KiB throughput |
|---|---:|---:|---:|---:|
| **BLAKE2b-64** | 4,125 | 418 | 242 µs | **0.41 MB/s** |
| SHA-512 | 3,054 | 376 | 327 µs | 0.37 MB/s |
| SHAKE128 (32 B out) | 2,714 | 300 | 368 µs | 0.29 MB/s |
| SHAKE256 (64 B out) | 2,353 | 265 | 425 µs | 0.26 MB/s |
| SHA3-256 | 2,086 | 258 | 479 µs | 0.25 MB/s |
| SHA-256 | 1,893 | 207 | 528 µs | 0.20 MB/s |
| SHA3-512 | 2,616 | 187 | 382 µs | 0.18 MB/s |
| HMAC-SHA512 (1 KiB) | — | 285 | — | 0.28 MB/s |
| HMAC-SHA256 (1 KiB) | — | 183 | — | 0.18 MB/s |
| HMAC-SHA3-256 (1 KiB) | — | 187 | — | 0.18 MB/s |
| HMAC-SHA3-512 (1 KiB) | — | 106 | — | 0.10 MB/s |
**Notes**
- BLAKE2b is the fastest hash in the library, followed by SHA-512.
- SHA-3 / SHAKE use the Keccak-f[1600] permutation in pure Python and are
therefore the most expensive family per byte.
- HMAC adds one extra compression pass (inner + outer), so it is roughly
1.5–2× the cost of the bare hash on the same payload.
---
## 4. AES
### 4.1 Single block
| Operation | ops/s | Latency |
|---|---:|---:|
| AES-128 block encrypt | 18,149 | **55.1 µs** |
| AES-256 block encrypt | 18,217 | **54.9 µs** |
The raw block cipher is the fastest cryptographic primitive in the library:
about **18,000 ops/s**, i.e. 55 µs per 16-byte block.
### 4.2 Modes
| Mode | 64 B (ops/s) | 64 B latency | 1 KiB (ops/s) | 1 KiB latency |
|---|---:|---:|---:|---:|
| AES-256-OFB | 3,707 | 270 µs | 168 | 5.95 ms |
| AES-256-CTR | 3,378 | 296 µs | 199 | 5.02 ms |
| AES-256-CBC | 3,292 | 304 µs | 181 | 5.53 ms |
| AES-256-CFB | 2,815 | 355 µs | 225 | 4.44 ms |
| AES-256-ECB | 2,051 | 488 µs | 165 | 6.05 ms |
| AES-256-GCM encrypt | 1,195 | 837 µs | 126 | 7.94 ms |
| AES-256-GCM decrypt | 959 | 1.04 ms | 197 | 5.07 ms |
**Notes**
- Stream-like modes (OFB / CTR) are the fastest bulk modes; they avoid padding
and process the input as a single keystream.
- GCM is the slowest because it adds GHASH authentication on top of CTR.
- ECB is slower than CBC/CTR here mainly because of its padding path.
---
## 5. AES-GCM-SIV (RFC 8452)
| Operation | 64 B (ops/s) | 64 B latency | 1 KiB (ops/s) |
|---|---:|---:|---:|
| AES-128-GCM-SIV encrypt | 2,275 | 439 µs | 211 |
| AES-256-GCM-SIV encrypt | 2,093 | 478 µs | 139 |
| AES-256-GCM-SIV decrypt | 2,140 | 467 µs | 190 |
**Note.** On small inputs GCM-SIV is *faster* than plain AES-GCM in this
library (2,093 vs 1,195 ops/s at 64 B). That is expected: SIV is built on
AES-CTR plus POLYVAL, avoiding the GHASH table path that dominates GCM here.
GCM-SIV also gives nonce-misuse resistance, so for small messages it is the
better default.
---
## 6. ChaCha20-Poly1305
| Operation | 64 B (ops/s) | 64 B latency | 1 KiB (ops/s) | 1 KiB latency |
|---|---:|---:|---:|---:|
| **Poly1305 MAC (1 KiB)** | — | — | **16,337** | **61.2 µs** |
| ChaCha20-Poly1305 encrypt | 2,551 | 392 µs | 234 | 4.28 ms |
| ChaCha20-Poly1305 decrypt | 2,363 | 423 µs | 399 | 2.50 ms |
| XChaCha20-Poly1305 encrypt | 2,209 | 453 µs | 260 | 3.84 ms |
**Note.** Poly1305 is the single fastest AEAD primitive in the library
(16,337 ops/s on 1 KiB, 15.95 MB/s). The AEAD constructions around it are
slower because they must first run the ChaCha20 keystream.
---
## 7. Key derivation functions
KDFs are **intentionally expensive**; here the absolute numbers matter far more
than ops/s. Per-unit costs are normalised so you can extrapolate to production
parameters.
| Function | Measured cost | Normalised unit cost |
|---|---|---|
| **HKDF-SHA256 (32 B out)** | **1.196 ms** | — |
| PBKDF2-SHA256 | 649 ms / 1,000 iterations | **649 ns per iteration** |
| PBKDF2-SHA512 | 891 ms / 1,000 iterations | 891 ns per iteration |
| Argon2id (t=1, m=256 KiB) | 184 ms | **0.718 ms per KiB** |
| scrypt (N=1024, r=8) | 1,751 ms | **1.710 ms per N-unit** |
### 7.1 Extrapolation to production parameters
⚠️ **This is the single most important table in this document.** Pure-Python
KDFs at production parameters are measured in **tens of seconds to minutes**.
| Production parameter set | Cost of one derivation (estimated) |
|---|---|
| PBKDF2-SHA256, **100,000** iterations | **≈ 65 s** |
| scrypt, **N = 2^14**, r = 8 | **≈ 28 s** |
| Argon2id, **t = 3, m = 64 MiB** | **≈ 138 s** |
> 💡 **Recommendation.** When running on pure Python, choose KDF parameters
> **one to two orders of magnitude lower** than you would for a C library, or
> delegate the KDF to a native implementation (`hashlib.scrypt`,
> `hashlib.pbkdf2_hmac`) while keeping `purepycrypto` for everything else.
> A one-second-stretch target that is routine in C is simply not reachable in
> pure Python at comparable parameters.
---
## 8. Public-key algorithms
| Operation | ops/s | Latency |
|---|---:|---:|
| **Ed25519 publickey** | 763 | **1.31 ms** |
| X25519 base (scalar·B) | 358 | 2.79 ms |
| X25519 keygen | 350 | 2.86 ms |
| X25519 shared (ECDH) | 318 | 3.14 ms |
| Ed25519 sign | 400 | 2.50 ms |
| Ed25519 verify | 281 | 3.56 ms |
| RSA-1024 OAEP encrypt | 523 | 1.91 ms |
| RSA-1024 PSS verify | 244 | 4.10 ms |
| RSA-1024 OAEP decrypt | 150 | 6.66 ms |
| **RSA-1024 PSS sign** | 97 | **10.27 ms** |
**Notes**
- `x25519_generate_keypair()` is a fixed-base scalar multiplication, so it
costs essentially the same as `x25519_base()`.
- Ed25519 signing is faster than X25519 ECDH because the base point uses a
precomputed affine Comb table (one point addition per digit, zero doublings).
- RSA **private-key** operations are 3–5× slower than public-key operations:
signing/decryption need a full modular exponentiation with CRT and blinding,
while verification/encryption only need the public exponent e = 65537.
- RSA-1024 is used here for fast measurement; **2048-bit key generation in pure
Python takes seconds** and is not included in the timed table.
---
## 9. Encoding
The encoding module is hand-written bit manipulation (it does **not** call the
standard-library `base64`), and is verified byte-for-byte identical to it.
| Operation | 64 B | 1 KiB | 1 KiB throughput |
|---|---:|---:|---:|
| **Hex encode** | **759,029 ops/s** (1.32 µs) | 184,404 (5.42 µs) | **180 MB/s** |
| Base64 decode | 40,187 (24.9 µs) | 1,422 (703 µs) | 1.85 MB/s |
| Base32 encode | 24,401 (41.0 µs) | 1,646 (608 µs) | 1.61 MB/s |
| Base64 encode | 27,185 (36.8 µs) | 886 (1.13 ms) | 0.87 MB/s |
| Base58 encode | 21,038 (47.5 µs) | 236 (4.24 ms) | 0.23 MB/s |
**Notes**
- Hex encoding is the fastest non-random operation in the library
(759,029 ops/s at 64 B, 180 MB/s on 1 KiB) — it is a simple nibble lookup.
- Base58 is the slowest because it performs repeated big-integer division.
- Base64 decode is faster than encode because encoding must emit padding.
---
## 10. Secure random
All randomness comes from `os.urandom` (the OS CSPRNG); the `random` module is
deliberately not used.
| Operation | ops/s | Latency |
|---|---:|---:|
| **random_bytes(32)** | **889,781** | **1.12 µs** |
| token_hex(32) | 760,926 | 1.31 µs |
| random_bytes(1 KiB) | 413,661 | 2.42 µs |
| token_urlsafe(32) | 386,361 | 2.59 µs |
| random_bit() | 292,535 | 3.42 µs |
| random_int(1, 2^64) | 288,410 | 3.47 µs |
| random_uuid4() | 100,854 | 9.92 µs |
| random_password(16) | 10,103 | 99.0 µs |
**Notes**
- `random_bytes` reaches **889,781 ops/s**, showing that the cost is dominated
by the Python call overhead rather than by the CSPRNG syscall.
- `random_password` is the slowest because it applies a rejection-sampling
alphabet-uniformity loop across 16 characters.
---
## 11. Parallel helpers
| Operation | Wall time | Throughput |
|---|---:|---:|
| `sha3_batch` 64 × 1 KiB (16 processes) | **62.7 ms** | 1.00 MB/s |
| `aes256_gcm_siv_encrypt`, 1 MiB, `n_workers=16` | included above | — |
The library's own batch APIs take an `n_workers` argument and use the same
`multiprocessing` pool. Note that per-call multiprocessing overhead means the
parallel helpers pay off only for workloads with enough items or bytes; for a
single small message, call the plain function.
---
## 12. Overall ranking — fastest operations
| # | Operation | ops/s |
|---|---|---:|
| 1 | random_bytes(32) | **889,781** |
| 2 | token_hex(32) | 760,926 |
| 3 | Hex encode [64 B] | **759,029** |
| 4 | random_bytes(1 KiB) | 413,661 |
| 5 | token_urlsafe(32) | 386,361 |
| 6 | random_bit() | 292,535 |
| 7 | random_int(1, 2^64) | 288,410 |
| 8 | Hex encode [1 KiB] | 184,404 |
| 9 | random_uuid4() | 100,854 |
| 10 | **AES-256 block encrypt** | **18,217** |
Among **cryptographic primitives** (excluding encoding and randomness), the
fastest are **AES single-block encryption (18,217 ops/s)** and
**Poly1305 (16,337 ops/s on 1 KiB)**.
---
## 13. Summary
| Dimension | Result |
|---|---|
| Performance position | Pure Python — **100×–1000× slower** than C-backed libraries (expected) |
| Best suited for | Learning, auditing, zero-dependency embedding, small payloads |
| Not suited for | Bulk file encryption, production-grade KDF parameters, high-throughput TLS |
| Parallel speed-up | 16-core fan-out reduces the full 82-task suite to **11 s** (≈16× vs serial) |
| Implementation quality | 55 µs per AES block and 2.5 ms per Ed25519 signature show that the hand-written bit manipulation and precomputed tables are doing real work |
> **Bottom line.** `purepycrypto` trades throughput for readability, portability
> and zero dependencies. It is a reference implementation you can read end to
> end — not a drop-in replacement for a native cryptography stack.