SCPY403/571 Parallel Programming

Instructor

Chaiwoot Boonyasiriwat (chaiwoot@gmail.com)

Class Hours

Friday (9:00AM - 12:00PM)

Grading

Homework 50, Project 50 (marking scheme)

Slides

Resources

AMD OpenCL SDK (AMD APP SDK)

Intel OpenCL SDK

Old video clips

Course Schedule

Date	Topics	Videos	Assignments
Jan 13, 2023	Parallel computer architectures, levels of parallelism, performance measurement (execution time, throughput, Amdahl's law, Gustafson's law, speedup, efficiency, scalability), C++ programming	View
Jan 20, 2023	C++ programming, computational complexity, big O notation, timing your codes	View
Jan 27, 2023	C++ programming, writing/reading binary files, writing/reading image files, preprocessor directives Codes: stb_image_write.h, mandelbrot.cpp, hello_openmp.cpp,	View
Feb 3, 2023	OpenMP: parallel regions, parallel loop and loop scheduling, data race and synchronization, performance consideration	View	Submit HW1
Feb 10, 2023	C++: compiling multiple source files, using third-party libraries OpenCL: platform model	View
Feb 17, 2023	OpenCL: platform and device queries, execution model, kernel programming model, memory model, vector addition Codes: vecadd.cpp, vecadd.cl, stb_image.h, building.png	View
Feb 24, 2023	OpenCL: matrix addition and 2D index space, timing kernel executions, checking for OpenCL errors, Codes: cl_util.h, query3.cpp	View
Mar 3, 2023	OpenCL: vector addition, matrix addition, image histogram Codes: stair-gray.jpg, histogram4.cpp, histogram.cl	View
Mar 17, 2023	OpenCL: histogram equalization, image rotation, image filtering, python programming, PyOpenCL Codes: histogram_equalization.cpp, pyOpenCL.ipynb	View
Mar 24, 2023	CUDA: programming model and interface, CPU and GPU timing, vector and matrix addition, performance guidelines, PyCUDA, CuPy, Numba, pyOpenCL Codes: vecadd.cu, matadd.cu, a1d.m, pyCUDA.ipynb, CuPy.ipynb, numba.ipynb, pyOpenCL2.ipynb (matrix multiplication in pyOpenCL)	View
Mar 31, 2023	MPI: point-to-point and collective communications, timing, synchronization Codes: mpi_hello.cpp, mpi_timing.cpp, send_recv.cpp, nonblocking.cpp, mpi4py_send_recv.py,	View
Apr 7, 2023	Parallel programming in MATLAB Parallel algorithms: wave propagation, Julia set fractal, cellular automata	View
Apr 21, 2023	MPI, Slurm Codes: integrate_serial.cpp, integrate_mpi.cpp, job.txt,	View
Apr 28, 2023	Direct and inverse scattering, finite-difference time-domain (FDTD) method, n-body problem and particle simulations, ocean simulation	View1, View2
TBA	Project presentation

References

Rajaraman, V., and Murthy, C.S.R. (2016) Parallel Computers: Architecture and Programming. PHI Learning Private Ltd.
Trobec, R., Slivnik, B., Bulic, P., and Robic, B. (2018) Introduction to Parallel Computing: From Algorithms to Programming on State-of-the-Art Platforms, Springer.
Gropp, W., Lusk, E., and Skjellum, A. (2014) Using MPI: Portable Parallel Programming with the Message-Passing Interface, Third Edition, MIT Press.
NVIDIA (2021) CUDA C++ Programming Guide, https://docs.nvidia.com/cuda/pdf/CUDA_C_Programming_Guide.pdf
Han, J., and Sharma, B. (2019) Learn CUDA Programming: A Beginner’s Guide to GPU Programming and Parallel Computing with CUDA 10.x and C/C++, Packt Publishing.
Kaeli, D., Mistry, P., Schha, D., and Zhang, D.P. (2015) Heterogeneous Computing with OpenCL 2.0, Morgan Kaufmann.
Nielsen, F. (2016) Introduction to HPC with MPI for Data Science, Springer.
Van Der Pas, R., Stotzer, E., and Terboven, C. (2017) Using OpenMP – The Next Step: Affinity, Accelerators, Tasking, and SIMD, MIT Press.