Huggingface.js documentation
Interface: MlxQuantizationConfig
Interface: MlxQuantizationConfig
Hierarchy
-
↳
MlxQuantizationConfig
Indexable
▪ [key: string]: unknown
Properties
bits
• Optional bits: number
Inherited from
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:930
config _ groups
• Optional config_groups: Record<string, { format?: string ; targets?: string[] ; weights?: { num_bits?: number } }>
Inherited from
QuantizationConfig.config_groups
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:935
expert _ dtype
• Optional expert_dtype: string
Routed expert precision when it differs from the main quantizer (e.g. FP4 experts with FP8 attention).
Inherited from
QuantizationConfig.expert_dtype
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:920
format
• Optional format: string
Inherited from
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:934
group _ size
• Optional group_size: number
Inherited from
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:928
ignore
• Optional ignore: string[]
compressed-tensors names its exclusion list ignore rather than modules_to_not_convert,
using the same re:-prefixed target syntax as config_groups[].targets.
Inherited from
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:940
load _ in _ 4bit
• Optional load_in_4bit: boolean
Inherited from
QuantizationConfig.load_in_4bit
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:931
load _ in _ 8bit
• Optional load_in_8bit: boolean
Inherited from
QuantizationConfig.load_in_8bit
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:932
mode
• Optional mode: string
MLX quantization mode (e.g. affine); MLX configs do not declare quant_method.
Inherited from
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:927
modules _ to _ not _ convert
• Optional modules_to_not_convert: string[]
Inherited from
QuantizationConfig.modules_to_not_convert
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:929
quant _ algo
• Optional quant_algo: string
NVIDIA ModelOpt (quant_method: "modelopt") algorithm, e.g. "NVFP4", "W4A16_NVFP4", "FP8",
or "MIXED_PRECISION" — in which case the per-module algorithm is in quantized_layers.
Inherited from
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:945
quant _ method
• Optional quant_method: string
Inherited from
QuantizationConfig.quant_method
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:918
quantized _ layers
• Optional quantized_layers: Record<string, { quant_algo?: string }>
ModelOpt mixed-precision: module name (or a parent module, e.g. …mlp.experts) -> its algorithm.
Inherited from
QuantizationConfig.quantized_layers
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:947
store _ dtype
• Optional store_dtype: string
Same role as expert_dtype under another name: MiMo-V2.6 is quant_method: "fp8" for its
dense layers but stores the routed experts as store_dtype: "mxfp4", packed two per U8.
Inherited from
QuantizationConfig.store_dtype
Defined in
packages/hub/src/lib/parse-safetensors-metadata.ts:925
Update on GitHub