https://www.nextplatform.com/2024/07/25/meta-lets-its-largest-llama-ai-model-loose-into-the-open-field/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Wed, 31 Jul 2024 18:46:49 +0000 hourly 1 https://wordpress.org/?v=6.7.1 https://www.nextplatform.com/2024/07/25/meta-lets-its-largest-llama-ai-model-loose-into-the-open-field/#comment-229549 Fri, 26 Jul 2024 09:12:45 +0000 https://www.nextplatform.com/?p=144452#comment-229549 Great to see Groq’s dataflow LPU Groqchip applied to inferencing of these Llama 3.1 models (especially 405B). Their 10x power efficiency advantage, and 10x speed boost should be quite valuable here ( https://www.nextplatform.com/2023/11/27/groq-says-it-can-deploy-1-million-ai-inference-chips-in-two-years/ ). The flexibility of GPUs might be needed for training (and culling, quantizing, etc …), but for inference, the more efficient dataflow arch seems to win out nicely (also Cerebras, SambaNova, …).

]]>