Yeah, I just looked at my CDW list and it's coming in at $33,000 CAD, also as back ordered and discontinued.https://www.tomshardware.com/news/nvidia-to-sell-550000-h100-compute-gpus-in-2023-report
"While we don't know the precise mix of GPUs sold, each Nvidia H100 80GB HBM2E compute GPU add-in-card (14,592 CUDA cores, 26 FP64 TFLOPS, 1,513 FP16 TFLOPS) retails for around $30,000 in the U.S. However, this is not the company's highest-performing Hopper architecture-based part. In fact, this is the cheapest one, at least for now. Meanwhile in China, one such card can cost as much as $70,000."
CDW was reportedly selling them for $30,000 (discontinued as of Nov 2023)
Microsoft is getting a discount so there cost is $10,000 for each MI300X.
We'll see if the part gets reinstated at their new facilities in Mexico once they have the H200 production up and running but I doubt they will.
Microsoft has a lot of leverage on AMD for pricing, Microsoft recently spun up their own Maia AI accelerators in their data centers, and while they are slower than the MI300, they are cheap and are still fast enough for many of the canned workloads the existing customer base is already doing so the MI300 is competing against them in that respect.
Maia is built on TSMC 5nm, and has strong TOPS and FLOPS, but was designed before the LLM explosion (it takes ~3 years to develop, fab, and test an ASIC). It is massive, with 105 B transistors (vs. 80B in H100). It cranks out 1600 TFLOPS of MXInt8 and 3200 TFLOPS of MXFP4. Its most significant deficit is that it only has 64 GB of HBM but a ton of SRAM. Ie. It looks like it is designed for older AI models like CNNs. Microsoft went with only four stacks of HBM instead of 6 like Nvidia and 8 like AMD. The second generation Memory bandwidth is 1.6 TB/s, which beats out AWS Trainium/Inferentia at 820 GB/s and is well under NVIDIA, which has 2x3.9 TB/s.
But the Maia accelerators are functioning in GPT 3.5, CoPilot, and Bing.