Testing Modern HTTP Compressors

Text compression has long been a standard practice in web applications. Shrinking the amount of data cuts transfer time and lowers the load on the network channel. Compression setup, however, often comes down to dynamic gzip and default settings. This article takes a deeper look at compression. First, a quick review of the main compression technologies available on the web.

Video version

For convenience, a video version of this article is available on YouTube.

Compression options for the web

Several formats solve the problem of compressing text resources, and they all have reasonably wide support in browsers and other clients:

• gzip, deflate — the traditional compression format with the broadest support;
• brotli (br) — a newer format optimized for high compression ratios;
• Zstandard (zstd) — a newer format optimized for high compression and decompression speed.

In the context of a web server, Angie ships with a built-in module for gzip and third-party modules for brotli and zstd. Alongside the compression algorithms there are two modes: dynamic and static. Dynamic compression means compressing the response in real time, while the request is being processed. In this mode CPU resources are spent on every request. Static compression means creating compressed copies of resources on disk; serving a response only requires reading that copy from disk and sending it over the network. CPU resources are then used once (when the compressed copy is created) — not during request processing, but when the resource is published (or the static build runs).

Choosing settings for static compression is straightforward: use the maximum compression settings (CPU cost does not matter) and the format with the best compression ratio. Skipping ahead, that will be brotli at compression level 11 (the maximum). Static compression should of course be used everywhere possible. Also remember file minification where it applies: minification improves file size even when compression is used.

With dynamic compression, the choice of algorithm and compression settings is less obvious. A comparative test will help make that choice.

Testing the compressors

The topic of web compressors was covered earlier, in an article on text compression. The focus of that article was on applying compression modules in Angie. Now it is time to return to the question of choosing the optimal compression level and compressor, accounting for the costs of transferring, compressing, and decompressing the text resource. To make an informed choice, different compressors need to be compared under identical conditions, on a dataset close to real-world data.

As a benchmark, the open-source project lzbench will be used. It makes it easy to compare the speed of all the options of interest (brotli, zstd, gzip) and presents the results in a convenient form. As test data, large files with HTML, CSS and JS code are used (1 MB and up). The HTML file comes from the Silesia corpus (file webster, 40.5 MB). The CSS and JS files were taken from a real site and are 1.0 MB and 1.8 MB respectively. The lzbench benchmark is highly repeatable and uses current library versions. Its results agree well with testing the compression modules in the Angie server. The benchmark always uses a single CPU core, which suits the purpose. The CPU also has a high clock frequency, so expect lower absolute numbers on server-class systems.

Test conditions:

• current lzbench (built from source);
• zlib 1.3.1;
• brotli 1.2.0;
• zstd 1.5.7;
• CPU: AMD Ryzen 5800x;
• OS: Ubuntu 24.04.
• Virtualization: KVM.

The tests were run with the following command (only the file was changed):

./lzbench -o4 -ezlib/brotli/zstd 1.css

The parameters request a test of the three libraries of interest and enable CSV output format for easy import into a spreadsheet, where the analysis takes place. The output is three datasets (one per file) that need to be analyzed. The source files and results are in the repository.

Test results

The first test runs on the HTML file, webster (40.5 MB). The first column lists the compressor, its version, and the compression level. The second column shows the compression speed in MB/s, the third the decompression speed. The fourth column is the resulting compression ratio as a percentage of the original file size. Let's look at the table in detail.

Table 1. Test results for HTML.

The absolute leader in compression speed is zstd -1, followed by brotli -0. At the same time, zstd at low compression settings reaches a good ratio. Overall, zlib (gzip) shows a complete failure: both in speed and in compression ratio. High compression settings for brotli (10–11) and zstd (14–22) are clearly unsuitable for dynamic compression because of their low speed. As for decompression, it barely varies within a single library and stays consistently high; the absolute leader in this category is zstd. Note that decompression of the resource will most likely be done by the client, and the library it uses may differ.

For static compression, the results hold a surprise: the winner is zstd -22. brotli usually wins this category, but for this large file (over 40 MB) zstd turned out best.

Let's see whether the picture changes with CSS compression.

Table 2. Test results for CSS.

Here maximum brotli compression takes its place as the absolute leader in compression ratio.

In dynamic mode (low levels) everything stays as in the previous test: zstd leads in speed and compression quality, with brotli following at a clear distance behind in speed.

Finally, let's complete the picture with the JS file results.

Table 3. Test results for JS.

The JS file test results confirm all the conclusions from the CSS test; the ranking of compression options does not change.

The data above supports the conclusions needed for using each compression option in static mode, where CPU resources are not constrained. Also remember older clients that only support gzip/deflate. The highest compression ratio for gzip can be obtained with zopfli or the zstd ‑format=gzip utility. These questions were covered in more detail in the earlier article on text compression.

But how do you choose the optimal compression level for real use in a web server? Several additional factors need to be taken into account.

Choosing the optimal dynamic compression level

The test results point to a clear leader in dynamic compression — zstd. But which level is reasonable to use for this codec?

From the standpoint of minimizing CPU resource usage, the minimum settings for each compressor should be used. This gives a decent compression ratio (about 3x) and maximum compression speed.

The other approach is a compromise between CPU load and the size of the transferred resource. There is no simple answer here; calculations are required. The target parameter to optimize can be the total request processing time, including compression, decompression, and network transfer.

Let's model the resource transfer process with dynamic compression as the following components, expressed in time:

• request send (RTT);
• resource compression (C);
• compressed response send (T);
• response decompression (D).

To get the resulting time, several variables are needed: file size (S), network latency (RTT), and channel bandwidth (BW). The test results provide the compression speed (Cs), decompression speed (Ds), and compression ratio (R). From these, a formula for the full request processing time can be assembled:

This formula accounts for the costs of compressing and decompressing the resource and of transferring it over the network, given the file and channel parameters.

To apply this model to this test, the following values are used:

• RTT 10 ms;
• BW: 1 MB/s;
• S: 1 MB.

In the results file, the parameters are set on the "Settings" sheet. These parameters are equivalent to transferring an average bundle of JS code for a site over a fairly fast mobile connection (about 8 Mbit/s or 10 ms latency). The calculated time for each compression option is added to the results tables (the "Total time" column).

If options with compression speed below 50 MB/s are discarded (as too slow), the following distribution results (the CSS test):

The same in table form:

Table 4. Total time for CSS.

In terms of total processing time, the mid-range levels brotli (5-7) or zstd (7-9) are optimal, with zstd slightly ahead of brotli. Under these conditions the transfer component makes the main contribution to the total time, which is why the ranking looks like this. Let's look at the same calculations for the JS file:

And in table form:

Table 5. Total time for JS.

The overall distribution is similar to the CSS case, but brotli 5–6 breaks into the lead thanks to the small file size. zstd 7–8 follows just behind. Of course, the testing was done on a CPU with high single-threaded performance.

When the parameters change (for example, channel bandwidth increases), the ranking will shift, because the transfer time component gets smaller and compression/decompression time carries more weight. The parameters can be changed in the spreadsheet from the repository.

As an example, consider a case with low CPU speed (roughly 2x lower — an Intel Core i5-8300H at about 2.0 Ghz) and a fast channel (12 MB/s, about 100 Mbit/s), keeping the resource size at 1 MB. The results are shown below.

For the CSS file, the optimal settings by total transfer time are zstd 3–4; for brotli the best option is 0–1. Further increases in channel speed or decreases in CPU speed will shift the settings toward lower compression levels even more.

Summing up

The test results show a crushing defeat for gzip compression (zlib in the test). The only advantage of this compression format is compatibility: on both the client and web server side. Otherwise, brotli is in close competition with zstd. Depending on the dataset, one or the other comes out ahead. For static compression, brotli 11 will be the better choice in most cases.

Choosing dynamic compression settings is harder: the optimal choice depends on channel bandwidth and resource sizes. The most compatible recommendation: use mid-to-low compression settings with brotli and/or zstd (3–5). If the server has spare CPU resources, the levels can be raised by a few notches.