fz Namespace Reference

Classes
struct	AllocationInfo
class	BitshuffleStage
struct	BufferInfo
class	CompressionDAG
struct	DAGNode
class	DifferenceStage
struct	FZMBufferEntry
	Per-buffer metadata record written into the FZM header (256 bytes). More...
struct	FZMHeaderCore
	Fixed-size FZM file header core (80 bytes). More...
struct	FZMStageInfo
	Per-stage metadata record written into the FZM header (256 bytes). More...
struct	LevelTimingResult
struct	LorenzoConfig
class	LorenzoStage
class	MemoryPool
struct	MemoryPoolConfig
class	NegabinaryStage
class	Pipeline
struct	PipelinePerfResult
struct	QuantizerConfig
class	QuantizerStage
struct	ReconstructionStats
class	RLEStage
class	RZEStage
class	Stage
struct	StageTimingResult
class	ZigzagStage

Enumerations
enum class	StageType : uint16_t { }
	Stage type identifiers written into the FZM header. More...
enum class	DataType : uint8_t { }
	Element data type identifiers used in buffer and stage descriptors. More...
enum class	LogLevel : int {   TRACE = 0 , DEBUG = 1 , INFO = 2 , WARN = 3 ,   SILENT = 255 }
enum class	MemoryStrategy { MINIMAL , PREALLOCATE }
enum class	ErrorBoundMode : uint8_t { ABS = 0 , REL = 1 , NOA = 2 }

Functions
constexpr uint8_t	fzmVersionMajor (uint16_t v)
constexpr uint8_t	fzmVersionMinor (uint16_t v)
size_t	getDataTypeSize (DataType type)
std::string	dataTypeToString (DataType type)
std::string	stageTypeToString (StageType type)
template<typename TInput = float, typename TCode = uint16_t>
LorenzoStage< TInput, TCode > *	addLorenzo (Pipeline &pipeline, float error_bound, int quant_radius=32768, float outlier_capacity=0.2f)
template<typename T >
ReconstructionStats	calculateStatistics (const T *d_original, const T *d_decompressed, size_t n)
Stage *	createStage (StageType type, const uint8_t *config, size_t config_size)
template<typename TInput , typename TCode >
void	launchLorenzoKernel2D (const TInput *d_input, size_t nx, size_t ny, TInput ebx2_r, TCode quant_radius, TCode *d_codes, TInput *d_outlier_errors, uint32_t *d_outlier_indices, uint32_t *d_outlier_count, size_t max_outliers, bool zigzag_codes, cudaStream_t stream)
	2-D forward Lorenzo kernel launcher. nx is the fast (x) dimension.
template<typename TInput , typename TCode >
void	launchLorenzoInverseKernel2D (const TCode *d_codes, const TInput *d_outlier_errors, const uint32_t *d_outlier_indices, const uint32_t *d_outlier_count, size_t nx, size_t ny, size_t max_outliers, TInput ebx2, TCode quant_radius, TInput *d_output, bool zigzag_codes, cudaStream_t stream, MemoryPool *pool)
	2-D inverse Lorenzo kernel launcher.
template<typename TInput , typename TCode >
void	launchLorenzoKernel3D (const TInput *d_input, size_t nx, size_t ny, size_t nz, TInput ebx2_r, TCode quant_radius, TCode *d_codes, TInput *d_outlier_errors, uint32_t *d_outlier_indices, uint32_t *d_outlier_count, size_t max_outliers, bool zigzag_codes, cudaStream_t stream)
	3-D forward Lorenzo kernel launcher.
template<typename TInput , typename TCode >
void	launchLorenzoInverseKernel3D (const TCode *d_codes, const TInput *d_outlier_errors, const uint32_t *d_outlier_indices, const uint32_t *d_outlier_count, size_t nx, size_t ny, size_t nz, size_t max_outliers, TInput ebx2, TCode quant_radius, TInput *d_output, bool zigzag_codes, cudaStream_t stream, MemoryPool *pool)
	3-D inverse Lorenzo kernel launcher.

Variables
constexpr uint32_t	FZM_MAGIC = 0x464D5A32
constexpr uint8_t	FZM_VERSION_MAJOR = 3
constexpr size_t	FZM_LEGACY_HEADER_CORE_SIZE = 72
constexpr uint16_t	FZM_FLAG_HAS_DATA_CHECKSUM = 0x0001u
	data_checksum field is valid
constexpr uint16_t	FZM_FLAG_HAS_HEADER_CHECKSUM = 0x0002u
	header_checksum field is valid
constexpr size_t	FZM_MAX_BUFFERS = 32
	Maximum pipeline output buffers per file.
constexpr size_t	FZM_MAX_NAME_LEN = 64
	Maximum output port name length (bytes, null-terminated)
constexpr size_t	FZM_STAGE_CONFIG_SIZE = 128
	Per-stage serialized config slot (bytes)
constexpr size_t	FZM_MAX_SOURCES = 4
	Maximum source stages per pipeline.

Detailed Description

include/log.h — FZGPUModules logging infrastructure

Design goals:

1. Zero overhead when disabled — FZ_LOG calls below the compile-time threshold expand to ((void)0); arguments are never evaluated.
2. Zero overhead in release builds by default — FZ_LOG_MIN_LEVEL defaults to INFO (2), so TRACE and DEBUG calls compile out completely.
3. Flexible output — a single callback receives all log messages; the caller decides where they go (stderr, file, test buffer, etc.).
4. Explicit diagnostic helpers (printDAG, printBufferLifetimes) always produce output via FZ_PRINT, which uses the callback if set or stdout.

Compile-time control (set via CMake -DFZ_LOG_MIN_LEVEL=N or \#define before including this header):

FZ_LOG_MIN_LEVEL=0 (TRACE) — all log calls compiled in FZ_LOG_MIN_LEVEL=1 (DEBUG) — TRACE compiled out FZ_LOG_MIN_LEVEL=2 (INFO) — TRACE+DEBUG compiled out ← default FZ_LOG_MIN_LEVEL=3 (WARN) — only WARN compiled in FZ_LOG_MIN_LEVEL=255 (SILENT) — all log calls compiled out

Runtime control: fz::Logger::setCallback(cb) — set output sink (nullptr = silent) fz::Logger::setMinLevel(level) — runtime filter (≥ compile-time floor) fz::Logger::enableStderr(level) — convenience: log ≥ level to stderr

Usage: FZ_LOG(INFO, "finalize: %zu stages", n); FZ_LOG(DEBUG, "buffer %s allocated %.1f KB", tag, kb); FZ_LOG(TRACE, "execute: input=%zu output=%zu", in, out); FZ_LOG(WARN, "outlier overflow: %u > capacity", count);

FZ_PRINT(" node [%d] %s", id, name); // always outputs (diagnostic)

Enumeration Type Documentation

StageType

enum class fz::StageType : uint16_t

strong

Stage type identifiers written into the FZM header.

Each concrete Stage subclass reports one of these values via getStageTypeId(). Used by StageFactory::createStage() to reconstruct the pipeline during decompression.

Enumerator
LORENZO_1D	LorenzoStage — 1-D predictor.
DIFFERENCE	DifferenceStage — first-order differencing.
SCALE	ScaleStage (test utility)
PASSTHROUGH	PassThroughStage (test utility)
RLE	RLEStage — run-length encoding.
HUFFMAN	Reserved (not yet implemented)
BITPACK	Reserved (not yet implemented)
SPLIT	SplitStage (test utility)
MERGE	MergeStage (test utility)
LORENZO_2D	LorenzoStage — 2-D predictor.
LORENZO_3D	LorenzoStage — 3-D predictor.
QUANTIZER	QuantizerStage — direct-value quantization.
ZIGZAG	ZigzagStage — zigzag encode/decode.
NEGABINARY	NegabinaryStage — negabinary encode/decode.
BITSHUFFLE	BitshuffleStage — GPU bit-matrix transpose.
RZE	RZEStage — recursive zero-byte elimination.

DataType

enum class fz::DataType : uint8_t

strong

Element data type identifiers used in buffer and stage descriptors.

Returned by Stage::getOutputDataType() and Stage::getInputDataType(). UNKNOWN is returned by byte-transparent stages (Bitshuffle, RZE) to opt out of Pipeline::finalize() type-compatibility checking.

Enumerator
UNKNOWN	Byte-transparent stages: skip type checking at finalize()

LogLevel

enum class fz::LogLevel : int

strong

Enumerator
TRACE	Per-stage execute(), per-chunk details — very verbose.
DEBUG	Pipeline construction, buffer allocation, data stats.
INFO	High-level milestones: finalize, compress, decompress.
WARN	Unexpected but recoverable: outlier overflow, fallbacks.
SILENT	Compile-time sentinel — do not pass to log()

MemoryStrategy

enum class fz::MemoryStrategy

strong

Memory allocation strategy for pipeline execution.

Enumerator
MINIMAL	Allocate on-demand, free at last consumer. Lowest peak memory.
PREALLOCATE	Allocate everything upfront at finalize(). Required for graph mode.

ErrorBoundMode

enum class fz::ErrorBoundMode : uint8_t

strong

Interpretation of the user-specified error bound.

• ABS — |x_orig - x_recon| <= eb (default).
• REL — point-wise relative (PFPL): |error| / |x_orig| <= eb. For Lorenzo this is a global approximation: abs_eb = eb × max(|data|). Values much smaller than max(|data|) may exceed the per-element ratio. For an exact per-element REL bound use QuantizerStage with REL mode.
• NOA — norm-of-absolute / value-range relative (PFPL): abs_eb = eb × (max(data) - min(data)). Equivalent to what most other compressors call "relative".

Enumerator
ABS	Absolute error bound.
REL	Global-approximate point-wise relative bound.
NOA	Value-range relative bound (norm-of-absolute).

Function Documentation

fzmVersionMajor()

constexpr uint8_t fz::fzmVersionMajor ( uint16_t  v )

constexpr

Extract major version from a raw on-disk version field. Pre-split files stored small integers (e.g. 3); values ≤ 0xFF are treated as (major=value, minor=0).

fzmVersionMinor()

constexpr uint8_t fz::fzmVersionMinor ( uint16_t  v )

constexpr

Extract minor version from a raw on-disk version field (see fzmVersionMajor).

getDataTypeSize()

size_t fz::getDataTypeSize ( DataType  type )

inline

Returns the size in bytes of the given DataType. Throws for DataType::UNKNOWN.

dataTypeToString()

std::string fz::dataTypeToString ( DataType  type )

inline

Returns a human-readable string for the given DataType (e.g. "float32").

stageTypeToString()

std::string fz::stageTypeToString ( StageType  type )

inline

Returns a human-readable string for the given StageType (e.g. "Lorenzo1D").

addLorenzo()

template<typename TInput = float, typename TCode = uint16_t>

LorenzoStage< TInput, TCode > * fz::addLorenzo ( Pipeline &  pipeline, float  error_bound, int  quant_radius = 32768, float  outlier_capacity = 0.2f  )

inline

Add a Lorenzo predictor stage with automatic dimensionality selection.

Chooses the correct 1-D / 2-D / 3-D variant based on the pipeline's current dims (set via Pipeline::setDims()). Defaults to 1-D if setDims() was never called.

Parameters

pipeline	The pipeline to add the stage to.
error_bound	Absolute pointwise error tolerance.
quant_radius	Quantization radius (default 32768 for uint16_t).
outlier_capacity	Fraction of data to reserve for outliers (default 0.2).

Returns

Pointer to the created stage (owned by the pipeline).

calculateStatistics()

template<typename T >

ReconstructionStats fz::calculateStatistics	(	const T *	d_original,
		const T *	d_decompressed,
		size_t	n
	)

Compute reconstruction statistics between two device arrays.

Parameters

d_original	Device pointer to original data.
d_decompressed	Device pointer to reconstructed data.
n	Number of elements.

createStage()

Stage * fz::createStage ( StageType  type, const uint8_t *  config, size_t  config_size  )

inline

Reconstruct a Stage from a serialized FZM header. Used by the decompressor to rebuild the inverse pipeline from the file.

Parameters

type	Stage type read from FZMStageInfo.
config	Serialized config bytes.
config_size	Number of valid bytes in config.

Returns

Heap-allocated Stage; caller takes ownership.

Variable Documentation

FZM_MAGIC

constexpr uint32_t fz::FZM_MAGIC = 0x464D5A32

constexpr

FZM magic number ("FZM2" in little-endian).

FZM_VERSION_MAJOR

constexpr uint8_t fz::FZM_VERSION_MAJOR = 3

constexpr

Version encoding: high byte = major, low byte = minor.

Major mismatch → throw. Minor mismatch → warn and continue. Pre-split files stored a bare integer (e.g. 3); those are treated as major = value, minor = 0, so FZM_VERSION = 0x0300 is backward-compatible.

v3.0 → v3.1: FZMHeaderCore grew from 72 to 80 bytes; added flags, data_checksum, and header_checksum fields.

FZM_LEGACY_HEADER_CORE_SIZE

constexpr size_t fz::FZM_LEGACY_HEADER_CORE_SIZE = 72

constexpr

FZMHeaderCore size for v3.0 files (before checksums). Used by readHeader() to avoid overrunning the stage array.