fz Namespace Reference

Classes
struct	AllocationInfo
class	BitpackStage
class	BitshuffleStage
struct	BufferInfo
class	CompressionDAG
struct	DAGNode
class	DifferenceStage
struct	FZMBufferEntry
	Per-buffer metadata record written into the FZM header (256 bytes). More...
struct	FZMHeaderCore
	Fixed-size FZM file header core (80 bytes). More...
struct	FZMStageInfo
	Per-stage metadata record written into the FZM header (256 bytes). More...
class	HuffmanStage
struct	LevelTimingResult
class	Logger
struct	LorenzoConfig
struct	LorenzoQuantConfig
class	LorenzoQuantStage
class	LorenzoStage
class	MemoryPool
struct	MemoryPoolConfig
struct	Negabinary
class	NegabinaryStage
struct	PersistentAllocInfo
class	Pipeline
struct	PipelinePerfResult
struct	QuantizerConfig
class	QuantizerStage
struct	ReconstructionStats
class	RLEStage
class	RZEStage
class	Stage
struct	StageTimingResult
struct	Zigzag
class	ZigzagStage

Enumerations
enum class	StageType : uint16_t { }
	Stage type identifiers written into the FZM header. More...
enum class	DataType : uint8_t { }
	Element data type identifiers used in buffer and stage descriptors. More...
enum class	LogLevel : int {   TRACE = 0 , DEBUG = 1 , INFO = 2 , WARN = 3 ,   SILENT = 255 }
enum class	MemoryStrategy { MINIMAL , PREALLOCATE }
enum class	HuffmanEncodeMode { Coarse , Fine }
enum class	ErrorBoundMode : uint8_t { ABS = 0 , REL = 1 , NOA = 2 }
enum class	ADMDtype : uint8_t

Functions
constexpr uint8_t	fzmVersionMajor (uint16_t v)
constexpr uint8_t	fzmVersionMinor (uint16_t v)
size_t	getDataTypeSize (DataType type)
std::string	dataTypeToString (DataType type)
std::string	stageTypeToString (StageType type)
template<typename T >
ReconstructionStats	calculateStatistics (const T *d_original, const T *d_decompressed, size_t n)
Stage *	createStage (StageType type, const uint8_t *config, size_t config_size)
template<typename TInput , typename TCode >
void	launchLorenzoKernel2D (const TInput *d_input, size_t nx, size_t ny, TInput ebx2_r, TCode quant_radius, TCode *d_codes, TInput *d_outlier_errors, uint32_t *d_outlier_indices, uint32_t *d_outlier_count, size_t max_outliers, bool zigzag_codes, cudaStream_t stream)
	2-D forward Lorenzo kernel launcher. nx is the fast (x) dimension.
template<typename TInput , typename TCode >
void	launchLorenzoInverseKernel2D (const TCode *d_codes, const TInput *d_outlier_errors, const uint32_t *d_outlier_indices, const uint32_t *d_outlier_count, size_t nx, size_t ny, size_t max_outliers, TInput ebx2, TCode quant_radius, TInput *d_output, bool zigzag_codes, cudaStream_t stream, MemoryPool *pool)
	2-D inverse Lorenzo kernel launcher.
template<typename TInput , typename TCode >
void	launchLorenzoKernel3D (const TInput *d_input, size_t nx, size_t ny, size_t nz, TInput ebx2_r, TCode quant_radius, TCode *d_codes, TInput *d_outlier_errors, uint32_t *d_outlier_indices, uint32_t *d_outlier_count, size_t max_outliers, bool zigzag_codes, cudaStream_t stream)
	3-D forward Lorenzo kernel launcher.
template<typename TInput , typename TCode >
void	launchLorenzoInverseKernel3D (const TCode *d_codes, const TInput *d_outlier_errors, const uint32_t *d_outlier_indices, const uint32_t *d_outlier_count, size_t nx, size_t ny, size_t nz, size_t max_outliers, TInput ebx2, TCode quant_radius, TInput *d_output, bool zigzag_codes, cudaStream_t stream, MemoryPool *pool)
	3-D inverse Lorenzo kernel launcher.

Variables
constexpr uint32_t	FZM_MAGIC = 0x464D5A32
constexpr uint8_t	FZM_VERSION_MAJOR = 3
constexpr size_t	FZM_LEGACY_HEADER_CORE_SIZE = 72
constexpr uint16_t	FZM_FLAG_HAS_DATA_CHECKSUM = 0x0001u
	data_checksum field is valid
constexpr uint16_t	FZM_FLAG_HAS_HEADER_CHECKSUM = 0x0002u
	header_checksum field is valid
constexpr size_t	FZM_MAX_BUFFERS = 32
	Maximum pipeline output buffers per file.
constexpr size_t	FZM_MAX_NAME_LEN = 64
	Maximum output port name length (bytes, null-terminated)
constexpr size_t	FZM_STAGE_CONFIG_SIZE = 128
	Per-stage serialized config slot (bytes)
constexpr size_t	FZM_MAX_SOURCES = 4
	Maximum source stages per pipeline.

Detailed Description

Copyright (c) Meta Platforms, Inc. and affiliates.

This source code is licensed under the MIT license found in the LICENSE file in the root directory of this source tree.

Adapted for FZGPUModules: namespace renamed from multibyte_ans to fz::ans.

Copyright (c) Meta Platforms, Inc. and affiliates.

This source code is licensed under the MIT license found in the LICENSE file in the root directory of this source tree.

Adapted for FZGPUModules: namespace renamed from multibyte_ans to fz::ans; stripped ansDecode() host function (called directly from ans_stage.cu instead).

Copyright (c) Meta Platforms, Inc. and affiliates.

This source code is licensed under the MIT license found in the LICENSE file in the root directory of this source tree.

Adapted for FZGPUModules: namespace renamed from multibyte_ans to fz::ans; stripped ansEncode() host function (called directly from ans_stage.cu instead).

Copyright (c) Meta Platforms, Inc. and affiliates.

This source code is licensed under the MIT license found in the LICENSE file in the root directory of this source tree.

Adapted for FZGPUModules: namespace renamed from multibyte_ans to fz::ans; stripped histogramSingle, histogramBatch, ansHistogramBatch (replaced by fz::module::GPU_histogram_generic in ans_stage.cu).

Enumeration Type Documentation

StageType

enum class fz::StageType : uint16_t

strong

Stage type identifiers written into the FZM header.

Each concrete Stage subclass reports one of these values via getStageTypeId(). Used by StageFactory::createStage() to reconstruct the pipeline during decompression.

Enumerator
ANS	rANS entropy coder (GPU, via dietGPU)
ADM	Adaptive Data Mapping transform (MANS)

DataType

enum class fz::DataType : uint8_t

strong

Element data type identifiers used in buffer and stage descriptors.

Returned by Stage::getOutputDataType() and Stage::getInputDataType(). UNKNOWN is returned by byte-transparent stages (Bitshuffle, RZE) to opt out of Pipeline::finalize() type-compatibility checking.

Enumerator
UNKNOWN	Byte-transparent stages: skip type checking at finalize()

LogLevel

enum class fz::LogLevel : int

strong

Enumerator
TRACE	Per-stage execute(), per-chunk details — very verbose.
DEBUG	Pipeline construction, buffer allocation, data stats.
INFO	High-level milestones: finalize, compress, decompress.
WARN	Unexpected but recoverable: outlier overflow, fallbacks.
SILENT	Compile-time sentinel — do not pass to log()

MemoryStrategy

enum class fz::MemoryStrategy

strong

Memory allocation strategy for pipeline execution.

Enumerator
MINIMAL	Allocate on-demand, free at last consumer. Lowest peak memory.
PREALLOCATE	Allocate everything upfront at finalize(). Required for graph mode.

HuffmanEncodeMode

enum class fz::HuffmanEncodeMode

strong

Selects the PHF encode algorithm used by HuffmanStage on the forward path.

Enumerator
Coarse	Multi-kernel coarse path; CPU prefix-sum sync in phase 3 (default).
Fine	ReVISIT-lite single kernel; fully GPU-async phase 3, no mid-encode CPU sync.

ErrorBoundMode

enum class fz::ErrorBoundMode : uint8_t

strong

Interpretation of the user-specified error bound.

• ABS — |x_orig - x_recon| <= eb (default).
• REL — point-wise relative (PFPL): |error| / |x_orig| <= eb. For Lorenzo this is a global approximation: abs_eb = eb × max(|data|). Values much smaller than max(|data|) may exceed the per-element ratio. For an exact per-element REL bound use QuantizerStage with REL mode.
• NOA — norm-of-absolute / value-range relative (PFPL): abs_eb = eb × (max(data) - min(data)). Equivalent to what most other compressors call "relative".

Enumerator
ABS	Absolute error bound.
REL	Global-approximate point-wise relative bound.
NOA	Value-range relative bound (norm-of-absolute).

ADMDtype

enum class fz::ADMDtype : uint8_t

strong

Input element type for ADMStage.

Function Documentation

fzmVersionMajor()

constexpr uint8_t fz::fzmVersionMajor ( uint16_t  v )

constexpr

Extract major version from a raw on-disk version field. Pre-split files stored small integers (e.g. 3); values ≤ 0xFF are treated as (major=value, minor=0).

fzmVersionMinor()

constexpr uint8_t fz::fzmVersionMinor ( uint16_t  v )

constexpr

Extract minor version from a raw on-disk version field (see fzmVersionMajor).

getDataTypeSize()

size_t fz::getDataTypeSize ( DataType  type )

inline

Returns the size in bytes of the given DataType. Throws for DataType::UNKNOWN.

dataTypeToString()

std::string fz::dataTypeToString ( DataType  type )

inline

Returns a human-readable string for the given DataType (e.g. "float32").

stageTypeToString()

std::string fz::stageTypeToString ( StageType  type )

inline

Returns a human-readable string for the given StageType (e.g. "LorenzoQuant").

calculateStatistics()

template<typename T >

ReconstructionStats fz::calculateStatistics	(	const T *	d_original,
		const T *	d_decompressed,
		size_t	n
	)

Compute reconstruction statistics between two device arrays.

Parameters

d_original	Device pointer to original data.
d_decompressed	Device pointer to reconstructed data.
n	Number of elements.

createStage()

Stage * fz::createStage ( StageType  type, const uint8_t *  config, size_t  config_size  )

inline

Reconstruct a Stage from a serialized FZM header. Used by the decompressor to rebuild the inverse pipeline from the file.

Parameters

type	Stage type read from FZMStageInfo.
config	Serialized config bytes.
config_size	Number of valid bytes in config.

Returns

Heap-allocated Stage; caller takes ownership.

Variable Documentation

FZM_MAGIC

constexpr uint32_t fz::FZM_MAGIC = 0x464D5A32

constexpr

FZM magic number ("FZM2" in little-endian).

FZM_VERSION_MAJOR

constexpr uint8_t fz::FZM_VERSION_MAJOR = 3

constexpr

Version encoding: high byte = major, low byte = minor.

Major mismatch → throw. Minor mismatch → warn and continue. Pre-split files stored a bare integer (e.g. 3); those are treated as major = value, minor = 0, so FZM_VERSION = 0x0300 is backward-compatible.

v3.0 → v3.1: FZMHeaderCore grew from 72 to 80 bytes; added flags, data_checksum, and header_checksum fields.

FZM_LEGACY_HEADER_CORE_SIZE

constexpr size_t fz::FZM_LEGACY_HEADER_CORE_SIZE = 72

constexpr

FZMHeaderCore size for v3.0 files (before checksums). Used by readHeader() to avoid overrunning the stage array.