The Book On Parallel Programming
By rchrd on Nov 07, 2007
Wilkinson and Allen's PARALLEL PROGRAMMING was published in 1999 (Prentice Hall). A revised edition came out in 2005. And I finally got my hands on it.
It IS expensive. Maybe you can now find used copies (be sure it's the 2nd edition!)
Used as a computer science text, it is a complete course on parallel programming techniques and algorithms, and covers MPI, Pthreads, and MPI. The new material covers distributed shared memory systems (DSM).
And, as a course, it is rich with examples and end-of-chapter homework problems. Some of them are really creative, like: "Write a multithreaded program to simulate two automatic teller machines being accessed by different persons on a single shared account."
Some of the typical parallel applications explored are image processing, numerical algorithms, and sorting and searching techniques.
I went thru the first edition (minus the problems) when it first came out. Now this expanded revised edition has me totally entranced, again. This is a great book, and I highly recommend it to anyone thinking seriously about parallel programming, especially for technical computing applications.