bitpack_stage.h File Reference

Bit-packing stage: packs N-bit integers into a dense byte stream. More...

#include "stage/stage.h"
#include "fzm_format.h"
#include "backend/types.h"
#include <cstdint>
#include <cstring>
#include <stdexcept>
#include <string>
#include <type_traits>
#include <unordered_map>
#include <vector>

Go to the source code of this file.

Classes
class	fz::BitpackStage< T >

Namespaces
namespace	fz

Detailed Description

Bit-packing stage: packs N-bit integers into a dense byte stream.

Supported input types: uint8_t, uint16_t, uint32_t. Output is always uint8_t[] (byte-transparent to downstream stages).

nbits must be a power of two and satisfy 1 <= nbits <= 8*sizeof(T). Allowed values per type: uint8_t : 1, 2, 4, 8 uint16_t : 1, 2, 4, 8, 16 uint32_t : 1, 2, 4, 8, 16, 32

Shift (frame-of-reference base + low-bit right shift)

Each element is transformed before packing and inverted on unpack:

forward:  packed = (v - base) >> shift        (low `nbits` bits kept)
inverse:  v      = (packed << shift) + base

The two knobs attack opposite ends of the word and compose:

• base removes a constant offset, i.e. dead high bits (values clustered far from zero — a "frame of reference" transform). Always lossless.
• shift removes dead low bits. Lossless only when every value has shift trailing zeros after the base subtraction; otherwise it is a lossy truncation (v is restored to a multiple of 1 << shift plus base).

Both default to 0, which is the previous behaviour exactly.

Serialized header layout (15 bytes): [0] DataType of T (1 byte) [1] nbits (1 byte) [2..9] num_elements (uint64_t, little-endian) [10] shift (1 byte) [11..14] base (uint32_t, little-endian; zero-extended T)

num_elements is written during forward compression and used by the inverse to know how many elements to unpack (byte count alone is ambiguous). Headers shorter than 15 bytes (pre-shift archives) decode with shift = base = 0.