Tutorial CUDA

Expose Parallelism:
CPU/GPU Parallelism
Take advantage of asynchronous kernel launches by
overlapping CPU computations with kernel execution
Take advantage of asynchronous CPU ↔ GPU memory
transfers (cudaMemcpyAsync()) that overlap with kernel
execution (only available for G84 and up)
© NVIDIA Corporation 2008
Overlap implemented by using a CUDA stream
CUDA Stream = Sequence of CUDA operations that execute in order
Example:
cudaStreamCreate(&stream1);
cudaStreamCreate(&stream2);
cudaMemcpyAsync(dst, src, size, stream1);
overlapped
kernel(…);
cudaMemcpyAsync(dst2, src2, size, stream2);
cudaStreamQuery(stream2);