ZFS Compression
The preferred way to compress QuestDB data is to convert partitions to
Parquet. QuestDB compresses Parquet
partitions natively and keeps them fully queryable, with no file system
requirements. In QuestDB Enterprise, a
storage policy converts partitions
automatically as they age, and can also tier them to object storage. In QuestDB
open source, convert partitions manually with
ALTER TABLE CONVERT PARTITION.
ZFS compression works at the file system level, outside the database. This page remains for existing deployments that already run on ZFS.
QuestDB can use ZFS for system-level compression, reducing disk usage without application changes.
The following example shows how to set up ZFS on Ubuntu:
sudo apt update
sudo apt install zfsutils-linux
To enable compression, create a ZPool with compression enabled:
zpool create -m legacy -o feature@lz4_compress=enabled -o autoexpand=on -O compression=lz4 -t volume1 questdb-pool sdf
The exact commands depend on which version of ZFS you are running. Use the ZFS docs to customize your ZFS to meet your requirements.
Once created, ZFS provides system-level compression.
Compression choices, LZ4 and zstd
ZFS offers a number of compression choices when constructing the volume.
LZ4 offers a good balance of compression ratio versus increased CPU usage, and slowed performance. For general usage, we recommend using LZ4.
zstd is another strong option. This will provide higher compression ratios, but take longer to decompress. We recommend this when storage size is an absolute priority, or for embedded-style deployments (i.e. Raspberry Pi, home IoT setups).
As always, it is best to benchmark your choice to ensure that the performance matches your use case.
We regularly test ZFS with LZ4 compression. If you encounter issues with other compression algorithms, please let us know.