Back to AIBriefs
AnalysisAI Models

NVIDIA introduces 4-bit pretraining methodology NVFP4

NVIDIA introduces 4-bit pretraining methodology NVFP4MarkTechPost

NVFP4, a 4-bit floating-point format, enables pretraining a 12B hybrid Mamba-Transformer model up to 10 trillion tokens. The method aims to reduce memory and compute while maintaining model quality at long token horizons.

·
29 days ago
NVIDIA introduces 4-bit pretraining methodology NVFP4 — AIBriefs