Comments on: Finding NeMo Features for Fresh LLM Building Boost https://www.nextplatform.com/2023/12/05/finding-nemo-features-for-fresh-llm-building-boost/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Thu, 07 Dec 2023 03:31:41 +0000 hourly 1 https://wordpress.org/?v=6.7.1 By: Slim Albert https://www.nextplatform.com/2023/12/05/finding-nemo-features-for-fresh-llm-building-boost/#comment-217264 Thu, 07 Dec 2023 03:31:41 +0000 https://www.nextplatform.com/?p=143351#comment-217264 In reply to Hubert.

Hmmm, your hf16-hf32-FP64 mixed-precision proposition is quite intriguing. In the same vein, for computational problems that don’t need the gigantic dynamic range afforded by 11-bit exponents (eg. 1e-308 to 1e308), but do benefit from a more precise (longer) mantissa to tackle roundoff error, I think that a system of bf16-FP32-bf64 could be a mixed-precision winner, with bf64 having 1 sign bit, 8 exponent bits, and 55 mantissa bits (a precision-only extension of FP32).

]]> By: Hubert https://www.nextplatform.com/2023/12/05/finding-nemo-features-for-fresh-llm-building-boost/#comment-217229 Wed, 06 Dec 2023 13:47:11 +0000 https://www.nextplatform.com/?p=143351#comment-217229 Interesting presentation! I’m glad they are incorporating mixed-precision in NeMo, it made me think about bf16 (bfloat16, brain floating point) and a trip to WikiPedia that promptly sent me back to TNP ( https://www.nextplatform.com/2018/05/10/tearing-apart-googles-tpu-3-0-ai-coprocessor/ )! bf16 now makes sense to me, for FP32-oriented mixed-precision, as it has the same sign and exponent bits as FP32, but a shorter mantissa (7 bits vs 23 bits). Might some hf16 and hf32 encodings, with 1 sign bit and 11 exponent bits, be similarly useful for FP64-oriented mixed-precision (eg. in HPC, with 4, 20, and 52 mantissa bits) … “Inquisition Minds” … ?

And BioNeMo! That sure sounds interesting in view of yesterday’s TNP “Test of Time” article on the Battle Royale between MD and genAI in the ring of protein folding, and ironing … (eh-eh!)

]]>