NVIDIA has become the benchmark in the GPU market for training large-scale language models (LLMs). A.I And economic figures released by the company a few weeks ago showed that this market is much more profitable for the company than its domestic market. Intel and AMD still have a long way to go, but we’re doing our best to follow in their footsteps.
Intel’s latest offerings for this market include: Gaudi 3 The latest generation of accelerators, just presented at the Vision 2024 event, has significantly higher specifications than previous generations. Gaudi 2Focus on higher performance with superior energy efficiency.
Plus, according to Intel, this product is significantly cheaper than its predecessor. H100 GPU This can mean significant cost savings as large enterprises do not purchase individual units of this type of GPU to train language models as long as they are equivalent in terms of performance.
Intel Guadi3 Wants NVIDIA GPU Market
Intel is confident enough in Gaudi3 that during its presentation it compared its performance with the H100 and H200, which are based on NVIDIA’s Hopper architecture, but without accelerators. AMD MI300A/X Announced by the company in late 2023, it was more powerful than NVIDIA’s H100 GPU, according to the manufacturer.
The Gaudi3 accelerator has two 5nm matrices. SST Uses 64 5th generation Tensor cores. This is a significant increase compared to the previous generation. tensor core 40 and used a 7nm manufacturing process.
As for memory, the Gaudi3 has 128GB HBM2w providing 3.7TB/s of bandwidth compared to the 2.45TB/s provided by Gaudi2 with 96GB of HBM2e memory. These new Intel accelerators are available in form factors: PCIe The HL-388 model, which is compatible with PCIe 5.0, has a TDP of 450~600W, which is very high for this format. OAM models are also available in HL-328, HL-325, and HL-335 versions with TDPs ranging from 450 to 900 W, compatible with air and liquid cooling.
Performance
As seen in the comparison Intel showed during its presentation, Gaudi3 is up to 1.7 times faster (depending on the language model used) for the following tasks: Language model training Compared to NVIDIA’s H100 GPU.
Inference speed can vary depending on the language and can be up to 70% higher. reference energy efficiencyGaudi3, a very important part of data centers and something that the head of NVIDIA did not consider according to his latest statement, is 1.2 to 2.3 times more efficient.
Intel’s problem with Gaudi3 is that the giant B200 GPU based on blackwell architectureThis model is equipped with about three times more transistors than the H100 model, and is scheduled to be released on the market in the second quarter of 2024, when Gaudi 3 is also scheduled to be released.
