JXL Art

7 min read Original article ↗

JXL Art is the practice of using JPEG XL’s prediction tree to generate art. If you have questions, you can join the #jxl-art channel on the JPEG XL Discord.

The oversimplified summary is that JPEG XL has a modular mode that divides the image it encodes into squares called groups, up to 1024x1024 each. JPEG XL uses a prediction tree to make a prediction what the value of each pixel is in such a square, based on neighboring pixels and their gradients. As a result only the difference (or error) between the actual image and the prediction needs to be encoded. The better the predictions, the smaller the error, the more compressible the data, the smaller the file size. Profit.

In the context of JXL art, however, the error is always assumed to be zero, which makes the image consist of only the prediction tree. That means the predictions effectively generate the image. The process of creating JXL art is writing that prediction tree. A prediction tree is a tree of if-else statements, branching on different properties and selecting a predictor for each leaf of the tree. The flexibility of these prediction trees is intentionally limited because they need to be small and execution needs to be fast, as it is part of the image decoding process. The prediction tree is run for every channel (red, green and blue) and for every pixel. Some predictors incorporate the values of the current pixel’s neighbors, specifically to the left, top-left, top and top-right, which dictates in which order the pixels have to be predicted: Row-by-row, top-to-bottom, left-to-right — just like Westerners read text.

A program starts with an optional header that specifies image properties and transformations to apply. The default header looks like this (everything is optional):

It is then followed by a tree description, which starts with a decision node — an if-else-like statement. Technically you can also just give a single predictor, but that’s rarely interesting. A decision node looks like this: 

if [property] > [value:int]
   (THEN BRANCH)
   (ELSE BRANCH)

Both the THEN branch and the ELSE branch can either be another decision node or a leaf node.

The following properties can be used in a decision node:

Leaf nodes are of the following form:

  - [predictor] +-[offset:int]

The following predictors are supported in leaf nodes:

Edge cases: