Add compression benchmark to ZFS

[PrivatePuffin]

• Add compression benchmark script to ZFS
• Create a location to but benchmarks
• Document it

Co-authored-by: Daniel Andersen dan-and@users.noreply.github.com
Signed-off-by: Daniel Andersen dan-and@users.noreply.github.com
Signed-off-by: Kjeld Schouten-Lebbing kjeld@schouten-lebbing.nl

Motivation and Context

During the work on ZSTD we came to realise there is no easy-reachable standardised way to make a benchmark of a compression algorithm in such a way that at least the compression results (ratio) is repeatable accoss runs and platforms.

Description

Research showed a good way to get benchmark results are the enwik suit of testfiles, which are basically a highly compressable plain-text export of wikipedia. This script uses a 1GB file (enwik9) to lower the margin of error.

To test if said results are in fact correct, I got the suggestion to use checksums which is included. @dan-and also added uncompressable testing and some code cleaning to ensure proper functioning.

Why include this with zfs?

• It's good to have a standardised test if people have issues about compression performance. As long as said test has ben thoroughly tested. Due to our many test and benchmark runs for ZSTD we can be very sure this test functions very well. As it even brought interesting behavorial differences between zstd and other algorithms to light (Introduce ZSTD compression to ZFS #9735 (comment))

How Has This Been Tested?

Manually, running it and crossreferencing values.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the ZFS on Linux code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• All commit messages are properly formatted and contain Signed-off-by.

[behlendorf]

Definitely useful for performance testing. What I'd like to suggest is that we move these benchmarks under the existing tests/zfs-tests/tests/perf/ ZTS framework. This is where we've added a similar set of fio based performance tests. This could be placed under tests/zfs-tests/tests/perf/compression/ and updated accordingly to be portable. Additionally this would all ow it to be easily run from installed ZFS packages or in-tree like the other tests.

[PrivatePuffin]

@behlendorf I decided against doing so.
Because the file downloads exceed the execution-time limits of the zfstestsuite.
(unless said limits don't apply to perf tests)

But i'll close it then, It would take me significant more time investment to figure out how that system works and integrating it.

[behlendorf]

unless said limits don't apply to perf tests

They're significantly longer since the perf tests are used for long running performance benchmarks, we also don't run the performance tests for each PR.

[PrivatePuffin]

@behlendorf Thanks for the headsup, I might or might not look into it.
Is there ANY documentation at all about the perf test system?

Additionally this would all ow it to be easily run from installed ZFS packages or in-tree like the other tests.

In essence i'm all for it, but as long as barely anything about it has clear and accessable user-side documentation, barely anyone is going to use it anyway (just as it is now, basically).