Benchmark

yof against apps made to be attacked

We run yof against two open-source practice apps full of known problems, OWASP Juice Shop and OWASP NodeGoat, and score it against each app's own official list: what yof found, what it missed and what it flagged wrongly. The misses are part of the result.

This page is in English for now. A translation is coming.

Results

Run of 2026-11-08, yof 0.2.0 (commit dbb3967), on a GitHub-hosted ubuntu-latest machine.

OWASP Juice Shop (v20.3.0, commit f189f52)

Result	Items on the official list (116)
Found, same weakness	1
Found, same root cause	4
Missed	7
Not tested by yof	81
Not available in this run	23
False positives	2

Found

Admin Section (same root cause): The administration page is hidden only in the browser. The user list it shows comes from /api/Users, which any signed-in member can read.
Forged Signed JWT (same root cause): The forged token works because the pinned jsonwebtoken 0.4.0 accepts a token signed with the public key as an HMAC secret, the advisory published for versions before 4.2.2.
Password Strength (same root cause): Nothing slows down password guessing at sign-in, which is how the admin's weak password can be found by trying.
Unsigned JWT (same root cause): The shop accepts a token signed with the 'none' algorithm because it pins jsonwebtoken 0.4.0 and express-jwt 0.1.3, which have published advisories for that token check.
Vulnerable Library (same weakness): The challenge's accepted answers are express-jwt 0.1.3 and sanitize-html 1.4.2, both pinned in package.json. yof's package check looks up exact pinned versions in the OSV advisory database.

Missed

Forged Review: Reviews are changed with the review id in the request body. yof's tenant test needs the id in the address, so it cannot be configured for reviews.
Frontend Typosquatting: yof flags a lookalike only when the name is one or two letters from a package on its bundled popular list. ngx-cookie, the package it imitates, is not on that list.
Legacy Typosquatting: The package is named only in ftp/package.json.bak, a backup file. yof reads package.json, requirements files and pyproject.toml, not backups.
Manipulate Basket: The same limit as View Basket: yof's tenant test needs a record with a text field it can mark, and basket items have none.
Supply Chain Attack: eslint-scope is not pinned in a manifest yof reads; the challenge points at the forgotten backup files under ftp/.
View Basket: yof's tenant test creates a record holding a marker text and reads it back as account B. A basket holds products, not text yof can mark, so the test cannot be set up for baskets.
Allowlist Bypass: The shop allows a redirect when an allowed address appears anywhere in the link. yof's probe sends a plain outside address, which the shop refuses.

Findings that are not on the official list

yof result	Where	Review
`attack.auth_bypass`	`GET /api/Feedbacks`	False positive: Public by design: the About Us page lists customer feedback to anyone. server.ts at the pinned commit guards only /api/Feedbacks/:id, not the list.
`attack.auth_bypass`	`GET /rest/user/whoami`	False positive: Public by design: whoami answers every visitor and returns an empty user when no one is signed in (server.ts line 628 has no sign-in check).
`attack.missing_headers`	`GET /`	Real, not on the official list: server.ts sets only nosniff and frameguard; there is no Content-Security-Policy, and the official list has no challenge for missing headers.
`secret.private_key`	`infrastructure/terraform/networking.tf:171`	Real, not on the official list: The same inline TLS private key in the second copy of the Terraform file.
`secret.private_key`	`lib/insecurity.ts:21`	Real, not on the official list: The RSA private key that signs the shop's sign-in tokens is written in the source. The list's JWT challenges use other weaknesses (the 'none' algorithm and key confusion).
`secret.private_key`	`terraform/networking.tf:171`	Real, not on the official list: A TLS private key written inline in the Terraform file. 'Misplaced IaC Files' is about reaching these files over the web, not about the key inside them.

OWASP NodeGoat (commit c5cb68a)

Result	Items on the official list (12)
Found, same weakness	3
Found, same root cause	0
Missed	3
Not tested by yof	6
Not available in this run	0
False positives	0

Found

A5 Misconfig (same weakness): The fix the app documents for this topic (server.js, "Fix for A5") adds the security headers that yof's surface group checks for.
A7 Access Controls (same weakness): A normal member opening the admin-only benefits page.
A9 Insecure Components (same weakness): Dependencies with published advisories, which yof's package check looks up for exact pinned versions.

Missed

A2 Broken Auth: yof's sign-in checks need a sign-in route that answers 2xx with a token or a cookie. NodeGoat answers its sign-in with a redirect, so the benchmark signs in itself and those two checks cannot run.
A4 Insecure DOR: yof's tenant test creates its own record through a JSON API and reads it back as account B. NodeGoat's pages are server-rendered forms with no such API, so the test cannot be set up.
A10 Redirects: The redirect sits behind sign-in, and yof's redirect probe runs signed out.

Findings that are not on the official list

yof result	Where	Review
`secret.private_key`	`artifacts/cert/server.key:1`	Real, not on the official list: The HTTPS server's private key is committed to the repository. NodeGoat's tutorial topics do not name it.

Method

Both apps run on a fresh GitHub-hosted machine, on localhost only, and are thrown away with it. Nothing is attacked on the internet.
The run creates two test accounts in each app through the app's own sign-up, with fresh passwords that exist only for that run.
yof check reads each app's source at the pinned commit. yof attack runs against the app with its normal rules: written consent naming localhost, the default budget of 300 requests and 110 seconds, and clean-up of the records it creates.
Each item of the app's official list, read from the same commit, is one of: found (a yof result named for it fired), missed (yof tests this kind of problem and did not find it here), not tested (yof does not test this kind of problem) or not available (it cannot happen in this run, for example Juice Shop turns some challenges off in Docker).
Which yof result counts for which item, and why we expect each miss, was written down before the first run. "Same root cause" means yof found what makes the item possible (for example the pinned library version with the advisory), not the exploit itself.
Every yof result that matches no item is checked by hand and marked as a false positive or as real but not on the list.

Pinned versions

OWASP Juice Shop v20.3.0, commit f189f52: the official Docker image of the same release, pinned by digest (bkimminich/juice-shop:v20.3.0@sha256:cea5a4af4aacee498c491491af0408934ec81d004035c0a02ef3e54395e3bf27). Official list: the challenge list in data/static/challenges.yml at the same commit.
OWASP NodeGoat, commit c5cb68a: built from its own Dockerfile at that commit, with MongoDB 4.4.29 (mongo:4.4.29@sha256:52c42cbab240b3c5b1748582cc13ef46d521ddacae002bbbda645cebed270ec0). Official list: the tutorial topics (OWASP Top 10 2013, ReDoS, SSRF) in app/views/tutorial/layout.html at the same commit.

How to reproduce

The scoring scripts are open source in the yof repository under benchmark/, with the mapping from yof results to each list. Run them only on a machine you can throw away, never on a computer you use: these apps are vulnerable on purpose.

# on a throwaway Linux machine with Docker and Node 22
git clone https://github.com/yof-dev/yof && cd yof && npm install && npm run build
git clone https://github.com/juice-shop/juice-shop ../juice-shop
git -C ../juice-shop checkout f189f521266f42c6bd05884468292000cee852ed
docker run -d -p 127.0.0.1:3000:3000 bkimminich/juice-shop:v20.3.0
node benchmark/cli.ts setup juice-shop --work /tmp/js --env-file /tmp/js.env
set -a && . /tmp/js.env && set +a
(cd ../juice-shop && node "$OLDPWD/dist/bin.js" check . --json) > /tmp/js-check.json
(cd /tmp/js && node "$OLDPWD/dist/bin.js" attack http://localhost:3000 --json) > /tmp/js-attack.json
node benchmark/cli.ts score juice-shop --list ../juice-shop/data/static/challenges.yml \
  --check /tmp/js-check.json --attack /tmp/js-attack.json --out /tmp/js-score.json

The full run, NodeGoat included, is one GitHub Actions job of about ten minutes. Its steps are listed in benchmark/cli.ts.

What yof does not test

yof looks for leaked keys and risky packages in source code, and attacks a running app in five groups: signing in, one user reaching another user's records, members reaching admin routes, AI assistant probes, and redirects, headers and sign-in limits. It does not test:

injection of any kind (SQL, NoSQL, commands, templates, XML), cross-site scripting or request forgery;
file uploads, deserialization, server-side request forgery or denial of service;
weak cryptography, business logic (prices, coupons, quantities) or anything a person finds by reading the app;
pages its config does not list: it does not crawl the app looking for hidden routes;
Dockerfiles, container images or infrastructure files, and passwords written in code (they have no shape it can tell apart from ordinary text).

A clean yof run shows what it tried. It does not show that nothing can go wrong. More on what yof does today: yof docs.