Language
Measuring

Source in, structured token stream out. No HTML, no rendering — just the parse.

Our corpus, three sizes

Every language at roughly 1KB, 10KB and 100KB. The small tier is the docs-snippet case, where fixed per-call cost still dominates; the large tier is a big vendored file. These are inputs we assembled, so read them alongside the set below.

small

799B · 28 lines
  • Prism
    6.2k ops/s
    fastest
  • sugar-high
    6.1k ops/s
    1.0× slower
  • twinkleplop
    5.3k ops/s
    1.2× slower
  • Shiki (wasm)
    359.7 ops/s
    17× slower
  • Shiki (JS engine)
    279.9 ops/s
    22× slower

tokens emitted: Prism 256 · sugar-high 325 · twinkleplop 609 · Shiki (wasm) 216 · Shiki (JS engine) 216

medium

8KB · 394 lines
  • twinkleplop
    895.1 ops/s
    fastest
  • sugar-high
    692.1 ops/s
    1.3× slower
  • Prism
    605.5 ops/s
    1.5× slower
  • Shiki (wasm)
    65.6 ops/s
    14× slower
  • Shiki (JS engine)
    46.2 ops/s
    19× slower

tokens emitted: twinkleplop 5325 · sugar-high 3269 · Prism 2542 · Shiki (wasm) 2176 · Shiki (JS engine) 2176

large

122KB · 3322 lines
  • twinkleplop
    112.2 ops/s
    fastest
  • sugar-high
    74.9 ops/s
    1.5× slower
  • Prism
    36.0 ops/s
    3.1× slower
  • Shiki (wasm)
    7.5 ops/s
    15× slower
  • Shiki (JS engine)
    0.9 ops/s
    131× slower

tokens emitted: twinkleplop 41688 · sugar-high 25162 · Prism 19298 · Shiki (wasm) 15424 · Shiki (JS engine) 15424

Shiki's own benchmark inputs

The sample files from shikijs/textmate-grammars-themes, vendored and pinned. Shiki's own engine benchmark runs on these. We did not choose them, which is the point: a highlighter benchmark published by the people who wrote the highlighter is worth exactly as much as its inputs. shikijs/textmate-grammars-themes at 45e292ef.

typescript

2KB · 78 lines
  • twinkleplop
    5.0k ops/s
    fastest
  • sugar-high
    4.0k ops/s
    1.3× slower
  • Prism
    2.2k ops/s
    2.3× slower
  • Shiki (wasm)
    263.7 ops/s
    19× slower
  • Shiki (JS engine)
    173.0 ops/s
    29× slower

tokens emitted: twinkleplop 957 · sugar-high 580 · Prism 438 · Shiki (wasm) 387 · Shiki (JS engine) 387

How to read this

Token counts are the caveat on every bar. A library that emits half as many tokens for the same file is doing less work per byte, not the same work faster. The counts are printed under each chart so you can see which is which rather than taking the bar on trust.

The HTML numbers are not measuring identical output. twinkleplop and Prism emit classes and leave colour to a stylesheet; Shiki resolves a theme and writes inline styles. That is strictly more string work, and it is a real difference in what you get, not a handicap we imposed.

These numbers do not travel. They describe one machine on one day. Comparing a bar here against a number from somewhere else — another run, another runner, another node version — is not a comparison. Within a single chart, the interleaving makes them fair.

⚠️ The machine drifted 6.8% during this run. The ratios still hold; the absolute ops/s figures should be treated as approximate.

  • twinkleplop 0.1.0 full pipeline: grammar plus the language's reclassifier stack, as shipped.
  • Shiki (wasm) 3.23.0 Oniguruma wasm engine, shiki's default. HTML output carries inline styles rather than classes, which is strictly more string work than a class-based emitter does.
  • Shiki (JS engine) 3.23.0 same grammars, RegExp engine instead of wasm. Shiki's own benchmark treats these as separate arms; the wasm payload is a real deployment cost the JS engine avoids.
  • Prism 1.30.0 class-based HTML output, same as twinkleplop's.
  • sugar-high 1.1.0 JavaScript-shaped tokenizer with no grammar registry: it has no language argument, so it appears only in the JS-family charts. Coarser output than the others.