IBM Israel Research Seminars

Originally suited only for visualization workloads, GPUs evolved into a powerful general purpose data-parallel platform, hypothetically allowing for orders-of-magnitude speedups over ordinary CPUs. Still, efficient utilization of GPU's highly parallel hardware is challenging in particular for the memory-bound applications with high data reuse, due to the complex GPU memory hierarchy design.

In this talk we show that efficient use of the GPU memory hierarchy by implementing user-managed cache, and structuring the computations to take advantage of this cache plays crucial role in achieving high performance. We demonstrate our user-managed cache approach by implementing GPU version of a solver for sum-product, or "marginalize a product of functions (MPF)" problem, which can be viewed as a generalization of a chain matrix multiplication problem for multi-dimensional matrices. Efficient implementation of MPF is complicated by the complex memory access pattern and data-dependent memory reuse.

We show both analytically and experimentally that poor use of memory hierarchy yields no performance gains for GPU-based version of the solver. User-managed cache implementation achieves up to 50-fold speedup over an optimized CPU version (single core). We also show promising results when applying GPU-based sum product solver to the inference in Bayesian networks (up to 700-fold speedup in log-scale computations). Finally, we suggest a few hardware improvements to the GPU memory hierarhcy which appear to be applicable not only to GPUs, but to the future generations of manycore systems in general.

About the speaker
Mark Silberstein is a PhD student at the CS department in the Technion under the joint supervision of Prof. Assaf Schuster and Dan Geiger. His main research focus has been efficient serial and parallel algorithms for inference in Bayesian networks (in the context of genetic linkage analysis), and their execution in large-scale opportunistic computing environments, aka Grids. This is a joint research with Prof. John Owens. It was conducted when Mark was a visiting researcher in the University of California in Davis.