rze_stage.h File Reference

Recursive Zero-byte Elimination stage — lossless byte-stream compressor. More...

#include "stage/stage.h"
#include "fzm_format.h"
#include <cuda_runtime.h>
#include <cstdint>
#include <cstring>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <vector>

Go to the source code of this file.

Classes
class	fz::RZEStage

Namespaces
namespace	fz

Detailed Description

Recursive Zero-byte Elimination stage — lossless byte-stream compressor.

Operates on raw byte streams (e.g. BitshuffleStage output). Each chunk is processed with up to 4 recursive levels:

• Level 1 (ZE): compact non-zero bytes; emit N/8-byte bitmap.
• Levels 2–4 (RE): compact non-repeated bytes of the previous bitmap.

Output stream layout:

[uint32_t: original byte count]
[uint32_t: num_chunks]
[uint32_t × n_chunks: per-chunk compressed sizes (high bit → stored raw)]
[compressed chunk data...]

Serialized header (9 bytes): [0..3] chunk_size (uint32_t LE), [4] levels (uint8_t), [5..8] cached_orig_bytes (uint32_t LE).