Parallel processing is the future. It is how we will increase our processing capability without having to double the clock speed of processors every other year.
But parallel processing requires generating parallel processing programs. And writing parallel processing programs is difficult.
These are the two common beliefs in our industry now and they’ve been used as an excuse.
Last week we went to the famous and beautiful Paramount Theater in Oakland California. If you live within 50 miles of it you should go (I can also recommend a great nearby restaurant.)
At the theater we heard Brahms Requiem. A masterful collection of work performed by the Oakland East Bay Symphony and led by the talented and always entertaining Michael Morgan.
Morgan had 157 to 161performers on stage, 95 of them in a choir.
There were 41 strings, and 21 horns and flutes and the rest were percussion.
They played beautifully for almost two hours.
All those musicians and singers, reading the score, were intent on what they doing; everyone doing something different. Watching them, I had an epiphany – they were all processing units, executing code (the sheet music). They would at just the right time start their process, usually very complicated , and end their process in perfect synchronization with the others. A glorious example of parallel processing – think of the musicians as shaders, their music the code.
Now think of Brahms as the programmer. He wrote the shader code for each and every instrument, each singer, applied clock synch flags, and made it all come out as desired on time.
Brahms’ music is rooted in the structures and compositional techniques of the Baroque and Classical masters. He liked counterpoint, and complex and highly disciplined composition, the same kind of structure Bach used and is famous for. And he probably did his finest work in the 1870s-80s – 140 years ago, 70 years after Jacuard’s loom, another parallel processing program machine.
Look at the code in the accompanying example. Now imagine multiplying it 80x to match the program that ran at the Paramount – that’s a lot of simultaneously running code.
The composer of such music is the compiler, taking his vision and converting it into machine language the processors can read. The conductor is an interpreter, and the system clock.
At the symphony, it all fell in place for me. I used to worry about the industry’s slow progress in programming parallel processes. Yet, parallel programming is absolutely required to realize the acceleration of existing code and to get us to the next level of complexity. We can’t get to the singularity without parallel processing.
And if people could program looms and orchestras 140 to 210 years ago, why can’t we do it today?
I’ve often heard that programmers are usually good musicians too. That the structure of music naturally appeals to them.
Maybe that’s the hiring criteria for a parallel processing program programmer we should use. Is the candidate a musician? (Okay, likes music?) And if he or she is one who likes the classics such as Brahms, then all the better – he or she will understand something about massive parallel processing.
In the old days when computers had vacuum tubes you could hear the computer sing as the elements in the tubes vibrated and modulated each other’s sounds.
Imagine what a complex parallel processing program might sound like if all of the threads were down-sampled to audio frequencies? I wonder if discordant sounds would indicate bad or sloppy code. Maybe there’s another debugger opportunity here.
And think about the project manager commenting on the final bug fix with the strings of code humming and resonating. “Ah,” she says, “that’s music to my ears.”
Now for something completely different.
What’d you say?
I noticed how we have a new word in our vocabulary – “App.” It is an abbreviation of application of course and the world has come to understand it. Car of course means automobile. Cop mean police, carb is short for carbohydrates, yet cal is not popular for calories. We also don’t seem to say prog for program, or wag for wagon.
Us – not “us” but plural U – as in youse. But not you, just U, which is short for unit. We have a lot of units – CPU, GPU, MPU, HPU, APU, I’ll bet if U tried U could come up with even more Us. Send your Us to me and I’ll compile a list and post it.