Shiki code highlighting

For some reason, I always limited myself to using either Prism.js or highlight.js for code highlighting, and recently I've learned of Shiki, a better (but much heavier) highlighter, and decided to try it out. It uses the same engine and grammars to parse code as VS Code, so it'd go really well with the VS Code theme I've been using!

In this post I'll compare TypeScript support specifically of the three mentioned highlighters and highlight some issues in them. I've also implemented switching between highlighters in my code blocks, so you'll be able to see how each one performs right here in this post!

Preparations

Specifying the highlighter

To properly compare all three highlighters, I decided to make my code blocks more configurable and added a highlighter prop, so that I could easily go back to Prism, or even switch to another highlighter more easily. All three highlighters are lazy-loaded as needed.

```tsx @prism
const thisWillBe: HighlightedWith = "Prism.js";
```
```tsx @hljs
const thisWillBe: HighlightedWith = "Highlight.js";
```
```tsx
const thisWillBe: HighlightedWith = "Shiki" as ByDefault;
```

It took me a while to figure out a nice-looking API for this. In the end, I decided to put all of the code highlighting logic in a <CodeRenderer> component. <CodeBlock> and <InlineCode> components simply wrap its output in either <pre><code> or <code>, like this:

/components/Common/Code/index.tsx

return withHighlighter(highlighter, props.lang, (highlighter, lang) => {
  return (
    <CodeBlockContainer {...props}>
      <code className={className}>
        <CodeRenderer code={code} highlighter={highlighter} lang={lang} />
      </code>
    </CodeBlockContainer>
  );
});

You can find the full code in the website's repository.

Here withHighlighter is a special function, that behaves differently depending on where it's imported from (see "RSC and Hybrid Shared Components"). If it's from a server component, it imports the required highlighter and grammars, and then renders the provided function. If it's from a client component, it uses a hook to lazily import the highlighter and grammars.

Highlighting inline code

I also added support for highlighting inline code in markdown:

const name: null — const name: null​//​lang=tsx@prism
const name: null — const name: null​//​lang=tsx@hljs
const name: null — const name: null​//​lang=tsx

It makes sense to use comments for this kind of stuff, since no one is going to actually use comments in inline code. If you need to comment on the code, you can do it on a paragraph, outside of the inline code block. But, just to be safe, and to make the directives easier to understand, I added lang= as part of the pattern.

Sometimes I'd also need to provide some context to the highlighter, such as when rendering undefined as a type and undefined as a value, so I implemented another comment-directive:

undefined — undefined//​lang=tsx
undefined — /*​pre=let x:*/undefined//​lang=tsx
undefined — /*​pre=function*/undefined//​lang=tsx

This directive prepends a new line, let x:, to the code block's "source". The highlighter then highlights two lines: let x: and undefined, inferring that the latter is a type. And since an inline code block can only output one line, only the last line is actually rendered.

With the preparations complete, let's finally compare the highlighters!

Prism.js

I've seen Prism.js being used on quite a few websites, and it does its job as a syntax highlighter pretty well. Prism uses CSS classes, so it's easy to write your own themes, and integrating it with React's SSR/RSC could not be simpler with the help of refractor (a wrapper over Prism).

But the tokenizer isn't perfect. It's not really understanding the languages, after all. And there are also a few things in some languages, that Prism's grammars do not account for. I'll focus specifically on TypeScript, since it's the language I'm gonna use the most here.

Issues with highlighting

1. Interfaces in TypeScript can extend multiple other types, but Prism detects and properly highlights only the first base type. (see issue, 2023)

interface Bird extends Animal, Flying {}
//                             ^^^^^^

2. The nested types aren't highlighted correctly, — only the parent/namespace part is.

interface Props extends React.HTMLAttributes<any> {}
//                            ^^^^^^^^^^^^^^
function Component(): JSX.Element {}
//                        ^^^^^^^

3. And then, the type parameters of generic types and functions would often get highlighted as "constants", since they often have short uppercase-only names.

interface Interface<T> extends Base1<T>, Base2<T> {}
//                 ^^^              ^^^       ^^^
function sort<T>(list: T[]): T[] {}
//           ^^^      ^^^   ^^^

4. Prism doesn't differentiate null/undefined values from null/undefined types.

const a = or("prism", null, undefined);

const b: string | null | undefined;

1. And finally, Prism doesn't highlight typed variables and parameters. Yeah... Prism's TypeScript support is surprisingly lacking. (see issue, 2018)

type Value<T> = string | Date | T;
//                       ^^^^
const value: Value<T> | null = null;
//           ^^^^^^^^

function Component({ value }: { value: Value<T> }): JSX.Element {
  //                                   ^^^^^^^^     ^^^^^^^^^^^
  const asString = value as Value<T>;
  //                        ^^^^^^^^
}

Prism's highlighting sample

Prism.js/refractor highlighting

const num = 5 * 1.2e4, regex = /^([a-z]+)|--(\d*)$/gi;
console.log(num); // log the result

type Value<T> = T | { value: T };
interface Props extends React.BaseProps<T>, Base { value: Value<T>; }

function Component({ value, className }: Props): JSX.Element {
  const [state, setState] = useState<number | null>(null);
  return (
    <>
      <label>{`Current state: ${state}`}</label>
      <StateInput data={"data-" + state} action={setState} />
    </>
  );
}

Highlight.js

Highlight.js is also quite a popular highlighter. Like Prism, it uses CSS classes, only shallowly parses the code, and also has an AST wrapper lowlight. Highlight.js handles TypeScript's types a bit better, but it has a few of its own issues.

Issues with highlighting

1. Generic functions' names aren't highlighted correctly, when their type parameters are explicitly specified right after the name.

const sorted1 = sort([3, 1, 2]);
const sorted2 = sort<number>([3, 1, 2]);
//              ^^^^

function sort1(list: any[]): any[] {}
function sort2<T>(list: T[]): T[] {}
//       ^^^^^

1. If a generic function's name starts with an uppercase letter, it's highlighted as a type.

function Component(props: Props<any>): JSX.Element {}

function Component<T>(props: Props<T>): JSX.Element {}
//       ^^^^^^^^^

3. Highlight.js doesn't differentiate between HTML tags and React Components in JSX.

const label = <label>Name:</label>;

const input = <ComponentInput />;

4. JSX attributes aren't interpreted correctly at all. Apparently, when reading JSX, Highlight.js uses XML's grammar, which obviously isn't enough.

return (
  <div role="tabpanel" className={clsx("div", flag && "flag")}>
    {children ? children : "(none)"}
  </div>
);

In some themes that have very little colors and barely any contrast, you probably wouldn't notice this. But with VS Code's theme here, where strings, punctuation and property names are of different enough colors, it's really noticeable.

1. Some tokens aren't highlighted at all. And, even though highlight.js's theme guide contains .punctuation and .operator classes, they're not used in the highlighting (I've seen them on highlight.js' docs website though, so maybe I'm missing something in my setup). And most identifiers (parameters, untyped variables) don't have any classes.

const value1: number, value2 = 5;

HLJS's highlighting sample

Highlight.js/lowlight highlighting

const num = 5 * 1.2e4, regex = /^([a-z]+)|--(\d*)$/gi;
console.log(num); // log the result

type Value<T> = T | { value: T };
interface Props extends React.BaseProps<T>, Base { value: Value<T>; }

function Component({ value, className }: Props): JSX.Element {
  const [state, setState] = useState<number | null>(null);
  return (
    <>
      <label>{`Current state: ${state}`}</label>
      <StateInput data={"data-" + state} action={setState} />
    </>
  );
}

Shiki

Shiki is rather heavy, but it uses the same engine and grammars as VS Code, so if you've used VS Code, then you've already seen them in action!

It uses TextMate grammars, and understands all the constructs and syntaxes without fail. And the theming/styling system is incredibly detailed, allowing you to style pretty much every little thing in the language, from specific types of keywords to opening/closing parentheses/brackets of class/method/property parameters/bodies.

Shiki also has a "debug" mode (includeExplanation: true option), which outputs all the scopes that every single token has, and all theme colors matches. For example, here's an explanation for the comma (,) between state and setState in the below sample:

({
  content: ",",
  scopes: [
    { scopeName: "source.tsx", themeMatches: [] },
    { scopeName: "meta.function.tsx", themeMatches: [] },
    { scopeName: "meta.block.tsx", themeMatches: [] },
    { scopeName: "meta.var.expr.tsx", themeMatches: [] },
    { scopeName: "meta.array-binding-pattern-variable.tsx", themeMatches: [] },
    { scopeName: "punctuation.separator.comma.tsx", themeMatches: [{…}] }
  ]
})

Shiki's highlighting sample

Shiki highlighting

const num = 5 * 1.2e4, regex = /^([a-z]+)|--(\d*)$/gi;
console.log(num); // log the result

type Value<T> = T | { value: T };
interface Props extends React.BaseProps<T>, Base { value: Value<T>; }

function Component({ value, className }: Props): JSX.Element {
  const [state, setState] = useState<number | null>(null);
  return (
    <>
      <label>{`Current state: ${state}`}</label>
      <StateInput data={"data-" + state} action={setState} />
    </>
  );
}

Compare with Prism.js

Prism.js/refractor highlighting

const num = 5 * 1.2e4, regex = /^([a-z]+)|--(\d*)$/gi;
console.log(num); // log the result

type Value<T> = T | { value: T };
interface Props extends React.BaseProps<T>, Base { value: Value<T>; }

function Component({ value, className }: Props): JSX.Element {
  const [state, setState] = useState<number | null>(null);
  return (
    <>
      <label>{`Current state: ${state}`}</label>
      <StateInput data={"data-" + state} action={setState} />
    </>
  );
}

Compare with Highlight.js

Highlight.js/lowlight highlighting

const num = 5 * 1.2e4, regex = /^([a-z]+)|--(\d*)$/gi;
console.log(num); // log the result

type Value<T> = T | { value: T };
interface Props extends React.BaseProps<T>, Base { value: Value<T>; }

function Component({ value, className }: Props): JSX.Element {
  const [state, setState] = useState<number | null>(null);
  return (
    <>
      <label>{`Current state: ${state}`}</label>
      <StateInput data={"data-" + state} action={setState} />
    </>
  );
}

Size comparison

Now let's compare the costs that these highlighters come with:

Both Prism.js and Highlight.js are pretty light, so they start up quickly and are ready to highlight even before the page loads. Shiki, on the other hand, weighs in at a quarter of a megabyte, and also includes a WASM dependency, making it not ideal for client-side rendering.

Compared to average page sizes nowadays, 280 KiB isn't that much. For comparison, a Next.js + React bundle weighs about 100 KiB (compressed size), a Supabase db/auth client — 40 KiB, Inter font — 320 KiB (or 50 KiB, latin subset). If you import it lazily only when rendering client-input code, it should be fine. It wouldn't highlight everything right on page load though.

Speed comparison

I've used the highlighting sample used previously as a benchmark, and got this:

Prism.js is the fastest, Highlight.js is half as fast, and Shiki is 7x slower than Prism.

Conclusion

In terms of quality, Shiki is definitely superior, but it's also rather heavy, and probably shouldn't be used on the client side too often. But, if a big share of your users are programmers, then you should definitely consider using it. A nice code highlighter makes a huge difference! It's also worth it, if you're rendering the code on the server, — great highlighting at the cost of a few extra milliseconds. And it's even better, if you're caching the render results!

Both Prism.js and Highlight.js are light and fast, but their parsers/grammars are pretty different. It's hard to say which one is better or worse, so I'd recommend trying both and seeing which one works best for your set of languages.

Or... you can just implement all three, like I did on this website! Even though I'll probably only use Shiki from now on, now I have the option to switch any time I want!

(this post will probably remain the only place where I use this feature)