A while ago, after having taught MPI & OpenMP umpteen times, I finally realized that students often have an incorrect mental model of parallelism. Especially with MPI this is quite easy. So I started wondering how we can teach so as to prevent this from happening.
With MPI I find that students have a hard time getting away from the notion that there is a central process, or a central data store. So I redid my teaching materials to emphasize the intrinsic symmetry of an MPI run.
I wrote a technical report about this, which it seems I can not attach to this forum, so I’ll just give the link to my parallelism-book-in-progress. You can download the article there, as well as my revamped MPI course slides.
Comments very much appreciated.