solokey_tests.md · 21 KiB · Markdown Raw

# SoloKey 2 (stock firmware from 2022)

Basically cannot handle more than 1-2 workers at the same time.

```bash
$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw8
CTAP versions: [U2F_V2 FIDO_2_0 FIDO_2_1_PRE], AAGUID: 2369d4d0-13ce-48cb-9f26-f7ed8c9a6068, maxMsgSize: 3072
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=2s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=3s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=4s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=5s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=6s attempts=4 ok=0 failed=4 rate=0.7/s
elapsed=7s attempts=4 ok=0 failed=4 rate=0.6/s
elapsed=8s attempts=4 ok=0 failed=4 rate=0.5/s
elapsed=9s attempts=4 ok=0 failed=4 rate=0.4/s
elapsed=10s attempts=4 ok=0 failed=4 rate=0.4/s
elapsed=11s attempts=8 ok=0 failed=8 rate=0.7/s
elapsed=12s attempts=8 ok=0 failed=8 rate=0.7/s
elapsed=13s attempts=8 ok=0 failed=8 rate=0.6/s
elapsed=14s attempts=8 ok=0 failed=8 rate=0.6/s
elapsed=15s attempts=8 ok=0 failed=8 rate=0.5/s
elapsed=16s attempts=12 ok=0 failed=12 rate=0.7/s
elapsed=17s attempts=12 ok=0 failed=12 rate=0.7/s
elapsed=18s attempts=12 ok=0 failed=12 rate=0.7/s
elapsed=19s attempts=12 ok=0 failed=12 rate=0.6/s
elapsed=20s attempts=12 ok=0 failed=12 rate=0.6/s
elapsed=21s attempts=16 ok=0 failed=16 rate=0.8/s
elapsed=22s attempts=16 ok=0 failed=16 rate=0.7/s
elapsed=23s attempts=16 ok=0 failed=16 rate=0.7/s
elapsed=24s attempts=16 ok=0 failed=16 rate=0.7/s
elapsed=25s attempts=16 ok=0 failed=16 rate=0.6/s
elapsed=26s attempts=20 ok=0 failed=20 rate=0.8/s
elapsed=27s attempts=20 ok=0 failed=20 rate=0.7/s
elapsed=28s attempts=20 ok=0 failed=20 rate=0.7/s
elapsed=29s attempts=20 ok=0 failed=20 rate=0.7/s
elapsed=30s attempts=20 ok=0 failed=20 rate=0.7/s

Result: elapsed=30.003s attempts=20 ok=0 failed=20 rate=0.7/s opens=20 open_failures=20 close_failures=0
OPERATION  ATTEMPTS  OK  FAILED  AVG        P50<=  P95<=  P99<=  MAX        PING BYTES
ping       20        0   20      5.001254s  10s    10s    10s    5.003794s  0
Failure kinds:
  open/timeout: 20
Failure samples:
  2026-07-21T19:43:53.363721036Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:53.363722396Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:53.363727746Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:53.364329749Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465444739Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465457369Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465460949Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465503238Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566434825Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566443045Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566447745Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566456735Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667418842Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667419862Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667439831Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667444001Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.76831307Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.768322999Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.768333469Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.768338869Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
exit status 1
```

# SoloKey 2 (2.2363.0)

Classic test with random ping sizes but with the limit of max message to 1200 (YubiKey's default).
Those Error(0) were actually supposed to be CHANNEL_BUSY; stress util would retry them in this case.

```bash
$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw11
CTAP versions: [FIDO_2_0 FIDO_2_1 FIDO_2_3], AAGUID: 41414755-4944-3031-3233-343536373839, maxMsgSize: 7609
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=28 ok=25 failed=3 rate=28.0/s
elapsed=2s attempts=54 ok=51 failed=3 rate=27.0/s
elapsed=3s attempts=74 ok=71 failed=3 rate=24.7/s
elapsed=4s attempts=100 ok=97 failed=3 rate=25.0/s
elapsed=5s attempts=127 ok=124 failed=3 rate=25.4/s
elapsed=6s attempts=149 ok=143 failed=6 rate=24.8/s
elapsed=7s attempts=176 ok=170 failed=6 rate=25.1/s
elapsed=8s attempts=202 ok=196 failed=6 rate=25.2/s
elapsed=9s attempts=221 ok=215 failed=6 rate=24.6/s
elapsed=10s attempts=248 ok=242 failed=6 rate=24.8/s
elapsed=11s attempts=274 ok=265 failed=9 rate=24.9/s
elapsed=12s attempts=299 ok=290 failed=9 rate=24.9/s
elapsed=13s attempts=320 ok=311 failed=9 rate=24.6/s
elapsed=14s attempts=346 ok=337 failed=9 rate=24.7/s
elapsed=15s attempts=372 ok=363 failed=9 rate=24.8/s
elapsed=16s attempts=392 ok=383 failed=9 rate=24.5/s
elapsed=17s attempts=418 ok=409 failed=9 rate=24.6/s
elapsed=18s attempts=445 ok=436 failed=9 rate=24.7/s
elapsed=19s attempts=464 ok=455 failed=9 rate=24.4/s
elapsed=20s attempts=491 ok=482 failed=9 rate=24.5/s
elapsed=21s attempts=518 ok=506 failed=12 rate=24.7/s
elapsed=22s attempts=545 ok=533 failed=12 rate=24.8/s
elapsed=23s attempts=564 ok=552 failed=12 rate=24.5/s
elapsed=24s attempts=591 ok=579 failed=12 rate=24.6/s
elapsed=25s attempts=617 ok=605 failed=12 rate=24.7/s
elapsed=26s attempts=637 ok=624 failed=13 rate=24.5/s
elapsed=27s attempts=664 ok=651 failed=13 rate=24.6/s
elapsed=28s attempts=690 ok=677 failed=13 rate=24.6/s
elapsed=29s attempts=713 ok=700 failed=13 rate=24.6/s
elapsed=30s attempts=736 ok=723 failed=13 rate=24.5/s

Result: elapsed=30.003s attempts=736 ok=723 failed=13 rate=24.5/s opens=16 open_failures=10 close_failures=0
OPERATION  ATTEMPTS  OK   FAILED  AVG        P50<=  P95<=  P99<=  MAX         PING BYTES
ping       736       723  13      128.362ms  25ms   250ms  10s    10.000984s  210173
Failure kinds:
  open/ctaphid-Error(0): 3
  open/timeout: 7
  timeout: 3
Failure samples:
  2026-07-21T19:54:27.098325955Z worker=1 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:54:27.098956297Z worker=2 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:54:27.218331751Z worker=2 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:54:32.086286732Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:32.202273757Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:32.319981161Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:37.18694175Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:37.310221726Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:37.422253429Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:47.259846541Z worker=4 op=ping bytes=1024 kind=timeout: context deadline exceeded
  2026-07-21T19:54:47.355898648Z worker=3 op=ping bytes=1200 kind=timeout: context deadline exceeded
  2026-07-21T19:54:47.523835704Z worker=1 op=ping bytes=116 kind=timeout: context deadline exceeded
  2026-07-21T19:54:52.362819581Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
exit status 1
```

# SoloKey 2 (2.2362.0)

Test with a bigger message; this stresses SoloKey even more. It ended up completely freezing,
and I had to remove the key to print the report.

```
$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw11
CTAP versions: [FIDO_2_0 FIDO_2_1 FIDO_2_3], AAGUID: 41414755-4944-3031-3233-343536373839, maxMsgSize: 7609
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=11 ok=8 failed=3 rate=11.0/s
elapsed=2s attempts=20 ok=17 failed=3 rate=10.0/s
elapsed=3s attempts=29 ok=26 failed=3 rate=9.7/s
elapsed=4s attempts=38 ok=35 failed=3 rate=9.5/s
elapsed=5s attempts=38 ok=35 failed=3 rate=7.6/s
elapsed=6s attempts=57 ok=47 failed=10 rate=9.5/s
elapsed=7s attempts=66 ok=53 failed=13 rate=9.4/s
elapsed=8s attempts=75 ok=62 failed=13 rate=9.4/s
elapsed=9s attempts=83 ok=70 failed=13 rate=9.2/s
elapsed=10s attempts=84 ok=71 failed=13 rate=8.4/s
elapsed=11s attempts=93 ok=80 failed=13 rate=8.5/s
elapsed=12s attempts=108 ok=92 failed=16 rate=9.0/s
elapsed=13s attempts=111 ok=95 failed=16 rate=8.5/s
elapsed=14s attempts=120 ok=104 failed=16 rate=8.6/s
elapsed=15s attempts=129 ok=113 failed=16 rate=8.6/s
elapsed=16s attempts=138 ok=121 failed=17 rate=8.6/s
elapsed=17s attempts=139 ok=122 failed=17 rate=8.2/s
elapsed=18s attempts=148 ok=131 failed=17 rate=8.2/s
elapsed=19s attempts=157 ok=140 failed=17 rate=8.3/s
elapsed=20s attempts=166 ok=149 failed=17 rate=8.3/s
elapsed=21s attempts=175 ok=157 failed=18 rate=8.3/s
elapsed=22s attempts=177 ok=158 failed=19 rate=8.0/s
elapsed=23s attempts=186 ok=167 failed=19 rate=8.1/s
elapsed=24s attempts=195 ok=176 failed=19 rate=8.1/s
elapsed=25s attempts=204 ok=185 failed=19 rate=8.2/s
elapsed=26s attempts=214 ok=194 failed=20 rate=8.2/s
elapsed=27s attempts=220 ok=199 failed=21 rate=8.1/s
elapsed=28s attempts=224 ok=203 failed=21 rate=8.0/s
elapsed=29s attempts=233 ok=212 failed=21 rate=8.0/s
^C
Result: elapsed=2m10.567s attempts=242 ok=221 failed=21 rate=1.9/s opens=22 open_failures=9 close_failures=0
OPERATION  ATTEMPTS  OK   FAILED  AVG        P50<=  P95<=  P99<=  MAX        PING BYTES
ping       242       221  21      325.937ms  25ms   2.5s   10s    10.00101s  227587
Failure kinds:
  ctaphid-ERR_INVALID_SEQ: 10
  open/ctaphid-Error(0): 3
  open/timeout: 6
  timeout: 2
Failure samples:
  2026-07-21T19:55:10.121967309Z worker=4 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:55:10.122592982Z worker=3 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:55:10.241970217Z worker=3 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:55:15.2259163Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:15.349930039Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:15.441907386Z worker=4 op=ping bytes=58 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.537902704Z worker=3 op=ping bytes=64 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.633904972Z worker=4 op=ping bytes=116 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.737900434Z worker=3 op=ping bytes=117 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.985916892Z worker=4 op=ping bytes=1024 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:16.209902722Z worker=4 op=ping bytes=58 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:16.401902978Z worker=4 op=ping bytes=64 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:16.573892607Z worker=3 op=ping bytes=7609 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:21.67452534Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:21.889847024Z worker=3 op=ping bytes=58 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:22.085854812Z worker=3 op=ping bytes=64 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:25.319056801Z worker=1 op=ping bytes=7609 kind=timeout: context deadline exceeded
  2026-07-21T19:55:30.43044571Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:31.691526024Z worker=4 op=ping bytes=7609 kind=timeout: context deadline exceeded
  2026-07-21T19:55:35.531714626Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
exit status 1
```

# SoloKey 2 (2.2362.0 with my patches applied)

```bash
$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw11
CTAP versions: [FIDO_2_0 FIDO_2_1 FIDO_2_3], AAGUID: 41414755-4944-3031-3233-343536373839, maxMsgSize: 7609
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=87 ok=87 failed=0 rate=87.0/s
elapsed=2s attempts=158 ok=158 failed=0 rate=79.0/s
elapsed=3s attempts=229 ok=229 failed=0 rate=76.3/s
elapsed=4s attempts=302 ok=302 failed=0 rate=75.5/s
elapsed=5s attempts=383 ok=383 failed=0 rate=76.6/s
elapsed=6s attempts=471 ok=471 failed=0 rate=78.5/s
elapsed=7s attempts=546 ok=546 failed=0 rate=78.0/s
elapsed=8s attempts=625 ok=625 failed=0 rate=78.1/s
elapsed=9s attempts=703 ok=703 failed=0 rate=78.1/s
elapsed=10s attempts=785 ok=785 failed=0 rate=78.5/s
elapsed=11s attempts=869 ok=869 failed=0 rate=79.0/s
elapsed=12s attempts=949 ok=949 failed=0 rate=79.1/s
elapsed=13s attempts=1031 ok=1031 failed=0 rate=79.3/s
elapsed=14s attempts=1106 ok=1106 failed=0 rate=79.0/s
elapsed=15s attempts=1193 ok=1193 failed=0 rate=79.5/s
elapsed=16s attempts=1274 ok=1274 failed=0 rate=79.6/s
elapsed=17s attempts=1356 ok=1356 failed=0 rate=79.8/s
elapsed=18s attempts=1444 ok=1444 failed=0 rate=80.2/s
elapsed=19s attempts=1531 ok=1531 failed=0 rate=80.6/s
elapsed=20s attempts=1616 ok=1616 failed=0 rate=80.8/s
elapsed=21s attempts=1703 ok=1703 failed=0 rate=81.1/s
elapsed=22s attempts=1787 ok=1787 failed=0 rate=81.2/s
elapsed=23s attempts=1876 ok=1876 failed=0 rate=81.6/s
elapsed=24s attempts=1958 ok=1958 failed=0 rate=81.6/s
elapsed=25s attempts=2039 ok=2039 failed=0 rate=81.6/s
elapsed=26s attempts=2120 ok=2120 failed=0 rate=81.5/s
elapsed=27s attempts=2201 ok=2201 failed=0 rate=81.5/s
elapsed=28s attempts=2288 ok=2288 failed=0 rate=81.7/s
elapsed=29s attempts=2372 ok=2372 failed=0 rate=81.8/s
elapsed=30s attempts=2451 ok=2451 failed=0 rate=81.7/s

Result: elapsed=30.002s attempts=2451 ok=2451 failed=0 rate=81.7/s opens=4 open_failures=0 close_failures=0
OPERATION  ATTEMPTS  OK    FAILED  AVG       P50<=  P95<=  P99<=  MAX       PING BYTES
ping       2451      2451  0       46.591ms  5ms    250ms  2.5s   4.74196s  715096
```

# YubiKey 5C (F/W 5.7.4)

```bash
$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: Yubico YubiKey FIDO+CCID (1050:0406), path=/dev/hidraw8
CTAP versions: [U2F_V2 FIDO_2_0 FIDO_2_1_PRE FIDO_2_1], AAGUID: d7781e5d-e353-46aa-afe2-3ca49f13332a, maxMsgSize: 1536
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=41 ok=41 failed=0 rate=41.0/s
elapsed=2s attempts=85 ok=85 failed=0 rate=42.5/s
elapsed=3s attempts=111 ok=111 failed=0 rate=37.0/s
elapsed=4s attempts=148 ok=148 failed=0 rate=37.0/s
elapsed=5s attempts=178 ok=178 failed=0 rate=35.6/s
elapsed=6s attempts=220 ok=220 failed=0 rate=36.7/s
elapsed=7s attempts=255 ok=255 failed=0 rate=36.4/s
elapsed=8s attempts=292 ok=292 failed=0 rate=36.5/s
elapsed=9s attempts=329 ok=329 failed=0 rate=36.6/s
elapsed=10s attempts=365 ok=365 failed=0 rate=36.5/s
elapsed=11s attempts=401 ok=401 failed=0 rate=36.5/s
elapsed=12s attempts=436 ok=436 failed=0 rate=36.3/s
elapsed=13s attempts=482 ok=482 failed=0 rate=37.1/s
elapsed=14s attempts=518 ok=518 failed=0 rate=37.0/s
elapsed=15s attempts=561 ok=561 failed=0 rate=37.4/s
elapsed=16s attempts=600 ok=600 failed=0 rate=37.5/s
elapsed=17s attempts=634 ok=634 failed=0 rate=37.3/s
elapsed=18s attempts=671 ok=671 failed=0 rate=37.3/s
elapsed=19s attempts=706 ok=706 failed=0 rate=37.2/s
elapsed=20s attempts=742 ok=742 failed=0 rate=37.1/s
elapsed=21s attempts=757 ok=757 failed=0 rate=36.0/s
elapsed=22s attempts=757 ok=757 failed=0 rate=34.4/s
elapsed=23s attempts=757 ok=757 failed=0 rate=32.9/s
elapsed=24s attempts=757 ok=757 failed=0 rate=31.5/s
elapsed=25s attempts=758 ok=758 failed=0 rate=30.3/s
elapsed=26s attempts=788 ok=788 failed=0 rate=30.3/s
elapsed=27s attempts=825 ok=825 failed=0 rate=30.6/s
elapsed=28s attempts=868 ok=868 failed=0 rate=31.0/s
elapsed=29s attempts=895 ok=895 failed=0 rate=30.9/s

Result: elapsed=30.001s attempts=926 ok=926 failed=0 rate=30.9/s opens=4 open_failures=0 close_failures=0
OPERATION  ATTEMPTS  OK   FAILED  AVG        P50<=  P95<=  P99<=  MAX        PING BYTES
ping       926       926  0       116.966ms  10ms   1s     2.5s   6.345952s  269330
```

# Token2 PIN+ Bio3 (R3.2)

```bash
$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: TOKEN2 FIDO2 Security Key(0206) (349e:0206), path=/dev/hidraw11
CTAP versions: [U2F_V2 FIDO_2_0 FIDO_2_1 FIDO_2_1_PRE], AAGUID: eabb46cc-e241-80bf-ae9e-96fa6d2975cf, maxMsgSize: 2048
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=43 ok=43 failed=0 rate=43.0/s
elapsed=2s attempts=88 ok=88 failed=0 rate=44.0/s
elapsed=3s attempts=132 ok=132 failed=0 rate=44.0/s
elapsed=4s attempts=177 ok=177 failed=0 rate=44.2/s
elapsed=5s attempts=222 ok=222 failed=0 rate=44.4/s
elapsed=6s attempts=268 ok=266 failed=2 rate=44.7/s
elapsed=7s attempts=313 ok=311 failed=2 rate=44.7/s
elapsed=8s attempts=350 ok=348 failed=2 rate=43.7/s
elapsed=9s attempts=399 ok=397 failed=2 rate=44.3/s
elapsed=10s attempts=447 ok=445 failed=2 rate=44.7/s
elapsed=11s attempts=494 ok=491 failed=3 rate=44.9/s
elapsed=12s attempts=530 ok=527 failed=3 rate=44.2/s
elapsed=13s attempts=573 ok=569 failed=4 rate=44.1/s
elapsed=14s attempts=619 ok=615 failed=4 rate=44.2/s
elapsed=15s attempts=664 ok=660 failed=4 rate=44.3/s
elapsed=16s attempts=709 ok=705 failed=4 rate=44.3/s
elapsed=17s attempts=754 ok=750 failed=4 rate=44.4/s
elapsed=18s attempts=795 ok=790 failed=5 rate=44.2/s
elapsed=19s attempts=843 ok=838 failed=5 rate=44.4/s
elapsed=20s attempts=889 ok=884 failed=5 rate=44.4/s
elapsed=21s attempts=934 ok=929 failed=5 rate=44.5/s
elapsed=22s attempts=974 ok=968 failed=6 rate=44.3/s
elapsed=23s attempts=1017 ok=1010 failed=7 rate=44.2/s
elapsed=24s attempts=1060 ok=1053 failed=7 rate=44.2/s
elapsed=25s attempts=1106 ok=1099 failed=7 rate=44.2/s
elapsed=26s attempts=1151 ok=1144 failed=7 rate=44.3/s
elapsed=27s attempts=1196 ok=1189 failed=7 rate=44.3/s
elapsed=28s attempts=1240 ok=1232 failed=8 rate=44.3/s
elapsed=29s attempts=1285 ok=1277 failed=8 rate=44.3/s
elapsed=30s attempts=1330 ok=1322 failed=8 rate=44.3/s

Result: elapsed=30.001s attempts=1330 ok=1322 failed=8 rate=44.3/s opens=12 open_failures=3 close_failures=0
OPERATION  ATTEMPTS  OK    FAILED  AVG       P50<=  P95<=  P99<=  MAX         PING BYTES
ping       1330      1322  8       74.398ms  10ms   100ms  250ms  10.001031s  388049
Failure kinds:
  open/timeout: 3
  timeout: 5
Failure samples:
  2026-07-21T19:42:29.526163481Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:42:29.526175491Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:42:34.628393762Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:42:37.186044931Z worker=3 op=ping bytes=64 kind=timeout: context deadline exceeded
  2026-07-21T19:42:41.862042747Z worker=4 op=ping bytes=64 kind=timeout: context deadline exceeded
  2026-07-21T19:42:46.304021887Z worker=2 op=ping bytes=0 kind=timeout: context deadline exceeded
  2026-07-21T19:42:47.406024904Z worker=1 op=ping bytes=116 kind=timeout: context deadline exceeded
  2026-07-21T19:42:52.010979951Z worker=3 op=ping bytes=1 kind=timeout: context deadline exceeded
exit status 1
```

Those timeouts are kind of normal, and both YubiKey and Token2 have them from time to time,
but it's fine unless those are CTAPHID errors.CTAPHID errors.

SoloKey 2 (stock firmware from 2022)

Basically cannot handle more than 1-2 workers at the same time.

$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw8
CTAP versions: [U2F_V2 FIDO_2_0 FIDO_2_1_PRE], AAGUID: 2369d4d0-13ce-48cb-9f26-f7ed8c9a6068, maxMsgSize: 3072
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=2s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=3s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=4s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=5s attempts=0 ok=0 failed=0 rate=0.0/s
elapsed=6s attempts=4 ok=0 failed=4 rate=0.7/s
elapsed=7s attempts=4 ok=0 failed=4 rate=0.6/s
elapsed=8s attempts=4 ok=0 failed=4 rate=0.5/s
elapsed=9s attempts=4 ok=0 failed=4 rate=0.4/s
elapsed=10s attempts=4 ok=0 failed=4 rate=0.4/s
elapsed=11s attempts=8 ok=0 failed=8 rate=0.7/s
elapsed=12s attempts=8 ok=0 failed=8 rate=0.7/s
elapsed=13s attempts=8 ok=0 failed=8 rate=0.6/s
elapsed=14s attempts=8 ok=0 failed=8 rate=0.6/s
elapsed=15s attempts=8 ok=0 failed=8 rate=0.5/s
elapsed=16s attempts=12 ok=0 failed=12 rate=0.7/s
elapsed=17s attempts=12 ok=0 failed=12 rate=0.7/s
elapsed=18s attempts=12 ok=0 failed=12 rate=0.7/s
elapsed=19s attempts=12 ok=0 failed=12 rate=0.6/s
elapsed=20s attempts=12 ok=0 failed=12 rate=0.6/s
elapsed=21s attempts=16 ok=0 failed=16 rate=0.8/s
elapsed=22s attempts=16 ok=0 failed=16 rate=0.7/s
elapsed=23s attempts=16 ok=0 failed=16 rate=0.7/s
elapsed=24s attempts=16 ok=0 failed=16 rate=0.7/s
elapsed=25s attempts=16 ok=0 failed=16 rate=0.6/s
elapsed=26s attempts=20 ok=0 failed=20 rate=0.8/s
elapsed=27s attempts=20 ok=0 failed=20 rate=0.7/s
elapsed=28s attempts=20 ok=0 failed=20 rate=0.7/s
elapsed=29s attempts=20 ok=0 failed=20 rate=0.7/s
elapsed=30s attempts=20 ok=0 failed=20 rate=0.7/s

Result: elapsed=30.003s attempts=20 ok=0 failed=20 rate=0.7/s opens=20 open_failures=20 close_failures=0
OPERATION  ATTEMPTS  OK  FAILED  AVG        P50<=  P95<=  P99<=  MAX        PING BYTES
ping       20        0   20      5.001254s  10s    10s    10s    5.003794s  0
Failure kinds:
  open/timeout: 20
Failure samples:
  2026-07-21T19:43:53.363721036Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:53.363722396Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:53.363727746Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:53.364329749Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465444739Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465457369Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465460949Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:43:58.465503238Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566434825Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566443045Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566447745Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:03.566456735Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667418842Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667419862Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667439831Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:08.667444001Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.76831307Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.768322999Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.768333469Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:44:13.768338869Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
exit status 1

SoloKey 2 (2.2363.0)

Classic test with random ping sizes but with the limit of max message to 1200 (YubiKey's default). Those Error(0) were actually supposed to be CHANNEL_BUSY; stress util would retry them in this case.

$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw11
CTAP versions: [FIDO_2_0 FIDO_2_1 FIDO_2_3], AAGUID: 41414755-4944-3031-3233-343536373839, maxMsgSize: 7609
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=28 ok=25 failed=3 rate=28.0/s
elapsed=2s attempts=54 ok=51 failed=3 rate=27.0/s
elapsed=3s attempts=74 ok=71 failed=3 rate=24.7/s
elapsed=4s attempts=100 ok=97 failed=3 rate=25.0/s
elapsed=5s attempts=127 ok=124 failed=3 rate=25.4/s
elapsed=6s attempts=149 ok=143 failed=6 rate=24.8/s
elapsed=7s attempts=176 ok=170 failed=6 rate=25.1/s
elapsed=8s attempts=202 ok=196 failed=6 rate=25.2/s
elapsed=9s attempts=221 ok=215 failed=6 rate=24.6/s
elapsed=10s attempts=248 ok=242 failed=6 rate=24.8/s
elapsed=11s attempts=274 ok=265 failed=9 rate=24.9/s
elapsed=12s attempts=299 ok=290 failed=9 rate=24.9/s
elapsed=13s attempts=320 ok=311 failed=9 rate=24.6/s
elapsed=14s attempts=346 ok=337 failed=9 rate=24.7/s
elapsed=15s attempts=372 ok=363 failed=9 rate=24.8/s
elapsed=16s attempts=392 ok=383 failed=9 rate=24.5/s
elapsed=17s attempts=418 ok=409 failed=9 rate=24.6/s
elapsed=18s attempts=445 ok=436 failed=9 rate=24.7/s
elapsed=19s attempts=464 ok=455 failed=9 rate=24.4/s
elapsed=20s attempts=491 ok=482 failed=9 rate=24.5/s
elapsed=21s attempts=518 ok=506 failed=12 rate=24.7/s
elapsed=22s attempts=545 ok=533 failed=12 rate=24.8/s
elapsed=23s attempts=564 ok=552 failed=12 rate=24.5/s
elapsed=24s attempts=591 ok=579 failed=12 rate=24.6/s
elapsed=25s attempts=617 ok=605 failed=12 rate=24.7/s
elapsed=26s attempts=637 ok=624 failed=13 rate=24.5/s
elapsed=27s attempts=664 ok=651 failed=13 rate=24.6/s
elapsed=28s attempts=690 ok=677 failed=13 rate=24.6/s
elapsed=29s attempts=713 ok=700 failed=13 rate=24.6/s
elapsed=30s attempts=736 ok=723 failed=13 rate=24.5/s

Result: elapsed=30.003s attempts=736 ok=723 failed=13 rate=24.5/s opens=16 open_failures=10 close_failures=0
OPERATION  ATTEMPTS  OK   FAILED  AVG        P50<=  P95<=  P99<=  MAX         PING BYTES
ping       736       723  13      128.362ms  25ms   250ms  10s    10.000984s  210173
Failure kinds:
  open/ctaphid-Error(0): 3
  open/timeout: 7
  timeout: 3
Failure samples:
  2026-07-21T19:54:27.098325955Z worker=1 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:54:27.098956297Z worker=2 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:54:27.218331751Z worker=2 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:54:32.086286732Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:32.202273757Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:32.319981161Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:37.18694175Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:37.310221726Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:37.422253429Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:54:47.259846541Z worker=4 op=ping bytes=1024 kind=timeout: context deadline exceeded
  2026-07-21T19:54:47.355898648Z worker=3 op=ping bytes=1200 kind=timeout: context deadline exceeded
  2026-07-21T19:54:47.523835704Z worker=1 op=ping bytes=116 kind=timeout: context deadline exceeded
  2026-07-21T19:54:52.362819581Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
exit status 1

SoloKey 2 (2.2362.0)

Test with a bigger message; this stresses SoloKey even more. It ended up completely freezing, and I had to remove the key to print the report.

$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw11
CTAP versions: [FIDO_2_0 FIDO_2_1 FIDO_2_3], AAGUID: 41414755-4944-3031-3233-343536373839, maxMsgSize: 7609
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=11 ok=8 failed=3 rate=11.0/s
elapsed=2s attempts=20 ok=17 failed=3 rate=10.0/s
elapsed=3s attempts=29 ok=26 failed=3 rate=9.7/s
elapsed=4s attempts=38 ok=35 failed=3 rate=9.5/s
elapsed=5s attempts=38 ok=35 failed=3 rate=7.6/s
elapsed=6s attempts=57 ok=47 failed=10 rate=9.5/s
elapsed=7s attempts=66 ok=53 failed=13 rate=9.4/s
elapsed=8s attempts=75 ok=62 failed=13 rate=9.4/s
elapsed=9s attempts=83 ok=70 failed=13 rate=9.2/s
elapsed=10s attempts=84 ok=71 failed=13 rate=8.4/s
elapsed=11s attempts=93 ok=80 failed=13 rate=8.5/s
elapsed=12s attempts=108 ok=92 failed=16 rate=9.0/s
elapsed=13s attempts=111 ok=95 failed=16 rate=8.5/s
elapsed=14s attempts=120 ok=104 failed=16 rate=8.6/s
elapsed=15s attempts=129 ok=113 failed=16 rate=8.6/s
elapsed=16s attempts=138 ok=121 failed=17 rate=8.6/s
elapsed=17s attempts=139 ok=122 failed=17 rate=8.2/s
elapsed=18s attempts=148 ok=131 failed=17 rate=8.2/s
elapsed=19s attempts=157 ok=140 failed=17 rate=8.3/s
elapsed=20s attempts=166 ok=149 failed=17 rate=8.3/s
elapsed=21s attempts=175 ok=157 failed=18 rate=8.3/s
elapsed=22s attempts=177 ok=158 failed=19 rate=8.0/s
elapsed=23s attempts=186 ok=167 failed=19 rate=8.1/s
elapsed=24s attempts=195 ok=176 failed=19 rate=8.1/s
elapsed=25s attempts=204 ok=185 failed=19 rate=8.2/s
elapsed=26s attempts=214 ok=194 failed=20 rate=8.2/s
elapsed=27s attempts=220 ok=199 failed=21 rate=8.1/s
elapsed=28s attempts=224 ok=203 failed=21 rate=8.0/s
elapsed=29s attempts=233 ok=212 failed=21 rate=8.0/s
^C
Result: elapsed=2m10.567s attempts=242 ok=221 failed=21 rate=1.9/s opens=22 open_failures=9 close_failures=0
OPERATION  ATTEMPTS  OK   FAILED  AVG        P50<=  P95<=  P99<=  MAX        PING BYTES
ping       242       221  21      325.937ms  25ms   2.5s   10s    10.00101s  227587
Failure kinds:
  ctaphid-ERR_INVALID_SEQ: 10
  open/ctaphid-Error(0): 3
  open/timeout: 6
  timeout: 2
Failure samples:
  2026-07-21T19:55:10.121967309Z worker=4 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:55:10.122592982Z worker=3 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:55:10.241970217Z worker=3 op=ping kind=open/ctaphid-Error(0): open device: Error(0)
  2026-07-21T19:55:15.2259163Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:15.349930039Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:15.441907386Z worker=4 op=ping bytes=58 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.537902704Z worker=3 op=ping bytes=64 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.633904972Z worker=4 op=ping bytes=116 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.737900434Z worker=3 op=ping bytes=117 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:15.985916892Z worker=4 op=ping bytes=1024 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:16.209902722Z worker=4 op=ping bytes=58 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:16.401902978Z worker=4 op=ping bytes=64 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:16.573892607Z worker=3 op=ping bytes=7609 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:21.67452534Z worker=3 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:21.889847024Z worker=3 op=ping bytes=58 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:22.085854812Z worker=3 op=ping bytes=64 kind=ctaphid-ERR_INVALID_SEQ: ERR_INVALID_SEQ
  2026-07-21T19:55:25.319056801Z worker=1 op=ping bytes=7609 kind=timeout: context deadline exceeded
  2026-07-21T19:55:30.43044571Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:55:31.691526024Z worker=4 op=ping bytes=7609 kind=timeout: context deadline exceeded
  2026-07-21T19:55:35.531714626Z worker=1 op=ping kind=open/timeout: open device: context deadline exceeded
exit status 1

SoloKey 2 (2.2362.0 with my patches applied)

$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: SoloKeys Solo 2 Security Key (1209:beee), path=/dev/hidraw11
CTAP versions: [FIDO_2_0 FIDO_2_1 FIDO_2_3], AAGUID: 41414755-4944-3031-3233-343536373839, maxMsgSize: 7609
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=87 ok=87 failed=0 rate=87.0/s
elapsed=2s attempts=158 ok=158 failed=0 rate=79.0/s
elapsed=3s attempts=229 ok=229 failed=0 rate=76.3/s
elapsed=4s attempts=302 ok=302 failed=0 rate=75.5/s
elapsed=5s attempts=383 ok=383 failed=0 rate=76.6/s
elapsed=6s attempts=471 ok=471 failed=0 rate=78.5/s
elapsed=7s attempts=546 ok=546 failed=0 rate=78.0/s
elapsed=8s attempts=625 ok=625 failed=0 rate=78.1/s
elapsed=9s attempts=703 ok=703 failed=0 rate=78.1/s
elapsed=10s attempts=785 ok=785 failed=0 rate=78.5/s
elapsed=11s attempts=869 ok=869 failed=0 rate=79.0/s
elapsed=12s attempts=949 ok=949 failed=0 rate=79.1/s
elapsed=13s attempts=1031 ok=1031 failed=0 rate=79.3/s
elapsed=14s attempts=1106 ok=1106 failed=0 rate=79.0/s
elapsed=15s attempts=1193 ok=1193 failed=0 rate=79.5/s
elapsed=16s attempts=1274 ok=1274 failed=0 rate=79.6/s
elapsed=17s attempts=1356 ok=1356 failed=0 rate=79.8/s
elapsed=18s attempts=1444 ok=1444 failed=0 rate=80.2/s
elapsed=19s attempts=1531 ok=1531 failed=0 rate=80.6/s
elapsed=20s attempts=1616 ok=1616 failed=0 rate=80.8/s
elapsed=21s attempts=1703 ok=1703 failed=0 rate=81.1/s
elapsed=22s attempts=1787 ok=1787 failed=0 rate=81.2/s
elapsed=23s attempts=1876 ok=1876 failed=0 rate=81.6/s
elapsed=24s attempts=1958 ok=1958 failed=0 rate=81.6/s
elapsed=25s attempts=2039 ok=2039 failed=0 rate=81.6/s
elapsed=26s attempts=2120 ok=2120 failed=0 rate=81.5/s
elapsed=27s attempts=2201 ok=2201 failed=0 rate=81.5/s
elapsed=28s attempts=2288 ok=2288 failed=0 rate=81.7/s
elapsed=29s attempts=2372 ok=2372 failed=0 rate=81.8/s
elapsed=30s attempts=2451 ok=2451 failed=0 rate=81.7/s

Result: elapsed=30.002s attempts=2451 ok=2451 failed=0 rate=81.7/s opens=4 open_failures=0 close_failures=0
OPERATION  ATTEMPTS  OK    FAILED  AVG       P50<=  P95<=  P99<=  MAX       PING BYTES
ping       2451      2451  0       46.591ms  5ms    250ms  2.5s   4.74196s  715096

YubiKey 5C (F/W 5.7.4)

$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: Yubico YubiKey FIDO+CCID (1050:0406), path=/dev/hidraw8
CTAP versions: [U2F_V2 FIDO_2_0 FIDO_2_1_PRE FIDO_2_1], AAGUID: d7781e5d-e353-46aa-afe2-3ca49f13332a, maxMsgSize: 1536
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=41 ok=41 failed=0 rate=41.0/s
elapsed=2s attempts=85 ok=85 failed=0 rate=42.5/s
elapsed=3s attempts=111 ok=111 failed=0 rate=37.0/s
elapsed=4s attempts=148 ok=148 failed=0 rate=37.0/s
elapsed=5s attempts=178 ok=178 failed=0 rate=35.6/s
elapsed=6s attempts=220 ok=220 failed=0 rate=36.7/s
elapsed=7s attempts=255 ok=255 failed=0 rate=36.4/s
elapsed=8s attempts=292 ok=292 failed=0 rate=36.5/s
elapsed=9s attempts=329 ok=329 failed=0 rate=36.6/s
elapsed=10s attempts=365 ok=365 failed=0 rate=36.5/s
elapsed=11s attempts=401 ok=401 failed=0 rate=36.5/s
elapsed=12s attempts=436 ok=436 failed=0 rate=36.3/s
elapsed=13s attempts=482 ok=482 failed=0 rate=37.1/s
elapsed=14s attempts=518 ok=518 failed=0 rate=37.0/s
elapsed=15s attempts=561 ok=561 failed=0 rate=37.4/s
elapsed=16s attempts=600 ok=600 failed=0 rate=37.5/s
elapsed=17s attempts=634 ok=634 failed=0 rate=37.3/s
elapsed=18s attempts=671 ok=671 failed=0 rate=37.3/s
elapsed=19s attempts=706 ok=706 failed=0 rate=37.2/s
elapsed=20s attempts=742 ok=742 failed=0 rate=37.1/s
elapsed=21s attempts=757 ok=757 failed=0 rate=36.0/s
elapsed=22s attempts=757 ok=757 failed=0 rate=34.4/s
elapsed=23s attempts=757 ok=757 failed=0 rate=32.9/s
elapsed=24s attempts=757 ok=757 failed=0 rate=31.5/s
elapsed=25s attempts=758 ok=758 failed=0 rate=30.3/s
elapsed=26s attempts=788 ok=788 failed=0 rate=30.3/s
elapsed=27s attempts=825 ok=825 failed=0 rate=30.6/s
elapsed=28s attempts=868 ok=868 failed=0 rate=31.0/s
elapsed=29s attempts=895 ok=895 failed=0 rate=30.9/s

Result: elapsed=30.001s attempts=926 ok=926 failed=0 rate=30.9/s opens=4 open_failures=0 close_failures=0
OPERATION  ATTEMPTS  OK   FAILED  AVG        P50<=  P95<=  P99<=  MAX        PING BYTES
ping       926       926  0       116.966ms  10ms   1s     2.5s   6.345952s  269330

Token2 PIN+ Bio3 (R3.2)

$ go run -buildvcs=false ./cmd/ctap-stress \
      -device 1 -workers 4 -duration 30s -ops ping \
      -ping-sizes 0,1,57,58,64,116,117,1024,1200 \
      -timeout 10s
device: TOKEN2 FIDO2 Security Key(0206) (349e:0206), path=/dev/hidraw11
CTAP versions: [U2F_V2 FIDO_2_0 FIDO_2_1 FIDO_2_1_PRE], AAGUID: eabb46cc-e241-80bf-ae9e-96fa6d2975cf, maxMsgSize: 2048
load: workers=4 ops=ping timeout=10s duration=30s requests=0
suite is read-only; Ctrl-C stops it and still prints the collected report
elapsed=1s attempts=43 ok=43 failed=0 rate=43.0/s
elapsed=2s attempts=88 ok=88 failed=0 rate=44.0/s
elapsed=3s attempts=132 ok=132 failed=0 rate=44.0/s
elapsed=4s attempts=177 ok=177 failed=0 rate=44.2/s
elapsed=5s attempts=222 ok=222 failed=0 rate=44.4/s
elapsed=6s attempts=268 ok=266 failed=2 rate=44.7/s
elapsed=7s attempts=313 ok=311 failed=2 rate=44.7/s
elapsed=8s attempts=350 ok=348 failed=2 rate=43.7/s
elapsed=9s attempts=399 ok=397 failed=2 rate=44.3/s
elapsed=10s attempts=447 ok=445 failed=2 rate=44.7/s
elapsed=11s attempts=494 ok=491 failed=3 rate=44.9/s
elapsed=12s attempts=530 ok=527 failed=3 rate=44.2/s
elapsed=13s attempts=573 ok=569 failed=4 rate=44.1/s
elapsed=14s attempts=619 ok=615 failed=4 rate=44.2/s
elapsed=15s attempts=664 ok=660 failed=4 rate=44.3/s
elapsed=16s attempts=709 ok=705 failed=4 rate=44.3/s
elapsed=17s attempts=754 ok=750 failed=4 rate=44.4/s
elapsed=18s attempts=795 ok=790 failed=5 rate=44.2/s
elapsed=19s attempts=843 ok=838 failed=5 rate=44.4/s
elapsed=20s attempts=889 ok=884 failed=5 rate=44.4/s
elapsed=21s attempts=934 ok=929 failed=5 rate=44.5/s
elapsed=22s attempts=974 ok=968 failed=6 rate=44.3/s
elapsed=23s attempts=1017 ok=1010 failed=7 rate=44.2/s
elapsed=24s attempts=1060 ok=1053 failed=7 rate=44.2/s
elapsed=25s attempts=1106 ok=1099 failed=7 rate=44.2/s
elapsed=26s attempts=1151 ok=1144 failed=7 rate=44.3/s
elapsed=27s attempts=1196 ok=1189 failed=7 rate=44.3/s
elapsed=28s attempts=1240 ok=1232 failed=8 rate=44.3/s
elapsed=29s attempts=1285 ok=1277 failed=8 rate=44.3/s
elapsed=30s attempts=1330 ok=1322 failed=8 rate=44.3/s

Result: elapsed=30.001s attempts=1330 ok=1322 failed=8 rate=44.3/s opens=12 open_failures=3 close_failures=0
OPERATION  ATTEMPTS  OK    FAILED  AVG       P50<=  P95<=  P99<=  MAX         PING BYTES
ping       1330      1322  8       74.398ms  10ms   100ms  250ms  10.001031s  388049
Failure kinds:
  open/timeout: 3
  timeout: 5
Failure samples:
  2026-07-21T19:42:29.526163481Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:42:29.526175491Z worker=4 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:42:34.628393762Z worker=2 op=ping kind=open/timeout: open device: context deadline exceeded
  2026-07-21T19:42:37.186044931Z worker=3 op=ping bytes=64 kind=timeout: context deadline exceeded
  2026-07-21T19:42:41.862042747Z worker=4 op=ping bytes=64 kind=timeout: context deadline exceeded
  2026-07-21T19:42:46.304021887Z worker=2 op=ping bytes=0 kind=timeout: context deadline exceeded
  2026-07-21T19:42:47.406024904Z worker=1 op=ping bytes=116 kind=timeout: context deadline exceeded
  2026-07-21T19:42:52.010979951Z worker=3 op=ping bytes=1 kind=timeout: context deadline exceeded
exit status 1

Those timeouts are kind of normal, and both YubiKey and Token2 have them from time to time, but it's fine unless those are CTAPHID errors.CTAPHID errors.