Torch Hammer benchmark report

Sirius — Sirius 4-GPU all-GPUs fleet (A100 40GB, stress) — fleet_sirius_allgpus
source: fleet_sirius_allgpus GPU: NVIDIA A100-SXM4-40GB nodes: 1 GPUs: 4 benchmarks: 4
nodes
1
GPUs
4
benchmarks
4
outliers
0

(a) Batched GEMM (float32) [TFLOP/s]

fleet mean: 18.9 TFLOP/s | CV: 0.2% | mean power: 313W | max temp: 48 C
Fleet distribution of GPU mean performance (TFLOP/s)
0 0 1 1 2 GPU count 18.9–18.9 TFLOP/s: 2 GPUs 18.9–18.9 TFLOP/s: 2 GPUs 18.9 18.9 18.9 18.9 TFLOP/s mean
Fleet distribution of GPU mean power (W)
0 1 2 GPU count 305–309 W: 1 GPUs 309–314 W: 2 GPUs 318–323 W: 1 GPUs 307 312 316 321 W mean
Fleet distribution of GPU max temperature (°C)
0 1 2 GPU count 43.0–44.2 °C: 2 GPUs 46.8–48.0 °C: 2 GPUs 43.6 44.9 46.1 47.4 °C mean
Fleet distribution of GPU mean SM utilization (%)
0 2 4 GPU count 100–100 %: 4 GPUs 100 100 101 101 % mean
Fleet distribution of GPU mean memory BW utilization (%)
0 1 2 GPU count 17.0–18.5 %: 1 GPUs 20.0–21.5 %: 1 GPUs 21.5–23.0 %: 2 GPUs 17.8 19.2 20.8 22.2 % mean
Fleet distribution of GPU mean GPU clock (MHz)
0 2 4 GPU count 1,410–1,410 MHz: 4 GPUs 1,410 1,410 1,411 1,411 MHz mean
node GPU 0GPU 1GPU 2GPU 3 node mean vs fleet thrt
x3200c0s13b1n01919191919+0.0%

(b) Convolution (float32) [img/s]

fleet mean: 9,392,486.6 img/s | CV: 0.2% | mean power: 386W | max temp: 52 C
Fleet distribution of GPU mean performance (img/s)
0 0 1 1 2 GPU count 9.4M–9.4M img/s: 1 GPUs 9.4M–9.4M img/s: 1 GPUs 9.4M–9.4M img/s: 2 GPUs 9.4M 9.4M 9.4M 9.4M img/s mean
Fleet distribution of GPU mean power (W)
0 1 2 GPU count 384–385 W: 1 GPUs 385–386 W: 1 GPUs 387–388 W: 2 GPUs 385 386 386 387 W mean
Fleet distribution of GPU max temperature (°C)
0 1 2 GPU count 47.0–48.2 °C: 2 GPUs 50.8–52.0 °C: 2 GPUs 47.6 48.9 50.1 51.4 °C mean
Fleet distribution of GPU mean SM utilization (%)
0 1 2 3 GPU count 97.0–97.5 %: 1 GPUs 98.5–99.0 %: 3 GPUs 97.2 97.8 98.2 98.8 % mean
Fleet distribution of GPU mean memory BW utilization (%)
0 1 2 3 GPU count 70.0–70.5 %: 1 GPUs 71.5–72.0 %: 3 GPUs 70.2 70.8 71.2 71.8 % mean
Fleet distribution of GPU mean GPU clock (MHz)
0 1 2 GPU count 1,395–1,399 MHz: 2 GPUs 1,402–1,406 MHz: 1 GPUs 1,406–1,410 MHz: 1 GPUs 1,397 1,401 1,404 1,408 MHz mean
node GPU 0GPU 1GPU 2GPU 3 node mean vs fleet thrt
x3200c0s13b1n09,366,2909,407,2579,396,3839,400,0169,392,487+0.0%

(c) Einsum Attention (float32) [TFLOP/s]

fleet mean: 17.1 TFLOP/s | CV: 0.2% | mean power: 339W | max temp: 53 C
Fleet distribution of GPU mean performance (TFLOP/s)
0 0 1 1 2 GPU count 17.1–17.1 TFLOP/s: 2 GPUs 17.1–17.1 TFLOP/s: 1 GPUs 17.1–17.1 TFLOP/s: 1 GPUs 17.1 17.1 17.1 17.1 TFLOP/s mean
Fleet distribution of GPU mean power (W)
0 1 2 3 GPU count 326–331 W: 1 GPUs 342–347 W: 3 GPUs 329 334 339 344 W mean
Fleet distribution of GPU max temperature (°C)
0 1 2 GPU count 47.0–48.5 °C: 1 GPUs 48.5–50.0 °C: 1 GPUs 51.5–53.0 °C: 2 GPUs 47.8 49.2 50.8 52.2 °C mean
Fleet distribution of GPU mean SM utilization (%)
0 2 4 GPU count 100–100 %: 4 GPUs 100 100 101 101 % mean
Fleet distribution of GPU mean memory BW utilization (%)
0 1 2 GPU count 26.0–26.2 %: 2 GPUs 26.8–27.0 %: 2 GPUs 26.1 26.4 26.6 26.9 % mean
Fleet distribution of GPU mean GPU clock (MHz)
0 2 4 GPU count 1,410–1,410 MHz: 4 GPUs 1,410 1,410 1,411 1,411 MHz mean
node GPU 0GPU 1GPU 2GPU 3 node mean vs fleet thrt
x3200c0s13b1n01717171717+0.0%

(d) Memory Traffic (float32) [GB/s]

fleet mean: 29.6 GB/s | CV: 0.0% | mean power: 191W | max temp: 52 C
Fleet distribution of GPU mean performance (GB/s)
0 0 1 1 2 GPU count 29.6–29.6 GB/s: 1 GPUs 29.6–29.6 GB/s: 2 GPUs 29.6–29.6 GB/s: 1 GPUs 29.6 29.6 29.6 29.6 GB/s mean
Fleet distribution of GPU mean power (W)
0 1 2 GPU count 184–187 W: 1 GPUs 190–192 W: 1 GPUs 192–195 W: 2 GPUs 186 188 191 193 W mean
Fleet distribution of GPU max temperature (°C)
0 1 2 GPU count 44.0–46.0 °C: 1 GPUs 48.0–50.0 °C: 1 GPUs 50.0–52.0 °C: 2 GPUs 45.0 47.0 49.0 51.0 °C mean
Fleet distribution of GPU mean SM utilization (%)
0 2 4 GPU count 100–100 %: 4 GPUs 100 100 101 101 % mean
Fleet distribution of GPU mean memory BW utilization (%)
0 1 2 GPU count 51.0–51.2 %: 2 GPUs 51.8–52.0 %: 2 GPUs 51.1 51.4 51.6 51.9 % mean
Fleet distribution of GPU mean GPU clock (MHz)
0 2 4 GPU count 1,410–1,410 MHz: 4 GPUs 1,410 1,410 1,411 1,411 MHz mean
node GPU 0GPU 1GPU 2GPU 3 node mean vs fleet thrt
x3200c0s13b1n03030303030+0.0%

Outliers

None detected