rare_stage.h File Reference

Repetition-Adaptive Reduction Encoding stage — lossless byte-stream compressor. More...

#include "stage/stage.h"
#include "fzm_format.h"
#include "backend/types.h"
#include <cstdint>
#include <cstring>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <vector>

Go to the source code of this file.

Classes
class	fz::RAREStage

Namespaces
namespace	fz

Detailed Description

Repetition-Adaptive Reduction Encoding stage — lossless byte-stream compressor.

Standalone port of the LC framework RARE component. RARE is the auto-k generalization of RRE (see RREStage): rather than a binary "word repeats its predecessor in full, or is dropped entirely" test, it histograms how many top bits of word ^ predecessor are zero across the whole chunk, picks one global cut keep (0 <= keep < word_size*8) that maximizes total bit savings, then bit-packs the bottom keep bits of every word whose top bits match its predecessor (words that don't match are stored in full, same as RRE). The 4-level recursive bitmap compression is identical to RRE.

Output stream layout and serialized header are identical to RREStage — the per-chunk keep value lives inside the chunk's own compressed bytes (accounted for in its csize), not in the container/host format, so the two stages share the same host-side orchestration byte-for-byte.

[uint32_t: original byte count]
[uint32_t: num_chunks]
[uint32_t × n_chunks: per-chunk compressed sizes (high bit → stored raw)]
[compressed chunk data...]

Serialized header (9 bytes): [0..3] chunk_size (uint32_t LE), [4] word_size (uint8_t), [5..8] cached_orig_bytes (uint32_t LE).