Skip Navigation
LocalLLaMA @sh.itjust.works noneabove1182 @sh.itjust.works

TensorRT-LLM evaluation of the new H200 GPU achieves 11,819 tokens/s on Llama2-13B

github.com /NVIDIA/TensorRT-LLM/blob/release/0.5.0/docs/source/blogs/H200launch.md

H200 is up to 1.9x faster than H100. This performance is enabled by H200's larger, faster HBM3e memory.

https://nvidianews.nvidia.com/news/nvidia-supercharges-hopper-the-worlds-leading-ai-computing-platform

0
0 comments