What the app does, and what it never does

• It downloads llama.cpp from its official GitHub releases page (github.com/ggml-org/llama.cpp), unmodified, and models from Hugging Face. The addresses are shown in the app while downloading.
• It runs llama-bench.exe, llama.cpp's benchmarking tool, on the chosen model. That is what you see in Task Manager during a run, and what uses your graphics card, for two to three minutes per measurement.
• On an NVIDIA card, it queries nvidia-smi (the driver's tool) to record temperature, power draw and memory during the run.
• It sends the result to tokenbench.fr: card name, model, measured figures, llama.cpp and OS versions. Every upload is listed in “My account”.
• At startup it checks tokenbench.fr for a newer version, and only installs it if you accept.

• It does not run when its window is closed: no service, no scheduled task, no autostart with Windows.
• It does not ask for administrator rights. The portable version does not even install: a single file, delete it to remove everything.
• It only talks to tokenbench.fr, github.com and huggingface.co. No other server, no ad network, no tracker.
• It does not read your files, your browser or your clipboard; it sends neither your Windows user name, nor your IP address, nor your installed programs.
• It does not mine anything. A miner wants to run permanently and unseen; TokenBench uses the card for the duration of one measurement, in plain sight, then stops.
• It does not keep your password: signing in exchanges it once for a key that you can revoke from the site at any time.

None of this requires trusting us:

• Task Manager (Ctrl+Shift+Esc), Performance → GPU: load appears when a measurement starts and drops when it ends. Close the app: nothing left.
• Task Manager's Startup tab and Task Scheduler: TokenBench is not there.
• Resource Monitor (Network tab) or Windows Firewall: TokenBench's only connections go to tokenbench.fr, github.com and huggingface.co.
• Compare the SHA-256 of the downloaded file with the one published below, then scan it on VirusTotal: the link looks up exactly that file across some sixty antivirus engines.

Computed on the server from the files actually served. A file altered in transit would have a different fingerprint.

• TokenBench-Setup-0.6.3.exe v0.6.3 · 93.7 Mo

  SHA-256 2377835db64bb84ca9a5513e474f0abf5ee247e0b2796afcd66d8f31f925368c

  Scan on VirusTotal ↗

• TokenBench-0.6.3-portable.exe v0.6.3 · 84.4 Mo

  SHA-256 8b66e0d81be13ae0bbeb7b285374ca48cc9ecdb636d82cfef7284083f3b117a5

  Scan on VirusTotal ↗

To compute the fingerprint of the downloaded file, in PowerShell:

Get-FileHash .\TokenBench-Setup.exe -Algorithm SHA256

The same measurement can be made with the CLI, a readable Node.js package where every line is open source code (tokenbench-0.5.4.tgz)

See the command →

TokenBench is a personal project, with no company and no ads, published in France. Contributors are individuals measuring their own card. Questions: contact@tokenbench.fr. See also the privacy page.