GPU Performance Analysis and Optimization - GPU Technology ...

Summary: Shared Memory• Shared memory is a tremendous resource– Very high bandwidth (terabytes per second)– 20-30x lower latency than accessing GMEM– Data is programmer-managed, no evictions by hardware• Performance issues to look out for:– Bank conflicts add latency and reduce throughput– Many-way bank conflicts can be very expensive• Replay latency adds up• However, few code patterns have high conflicts, padding is a very simple and effective solution• Kepler has 2x SMEM throughput compared to Fermi:– SMEM throughput is doubled by increasing bank width to 8 bytes– Kernels with 8-byte words will benefit without changing kernel code• Put GPU into 8-byte bank mode with cudaSetSharedMemConfig() call– Kernels with smaller words will benefit if words are grouped into 8-byte structures© 2012, NVIDIA96