Taking advantage of multiple cores
By Malcolm Kavalsky on Dec 14, 2009
I just got a new computer with an i7 processor ( 4 cores x 2 threads per core). I am now far more aware of the dismal state of user software that is not multi-threaded. Many tasks that I expected to run much faster, are just as slow since they are not able to take advantage of the extra cores.
One of the more time consuming tasks that I execute, is video encoding. After taking movies of my kids, I like to convert them, and save on my hard-drive in a more friendly format than that in my camera. Also, if I want to edit them with a non-linear editor, this would require me to convert them usually to mpeg2 format.
I use Mencoder for converting these files, and noticed that it is single-threaded. So, in order to take advantage of the multiple cores, I split the mpeg file into multiple pieces, and run multiple encoding streams in parallel.
Here is an example of running this on a short movie:
First without parallel encoding streams:
malcolm@Blue:~/Demo$ time mencoder -o dvd2.avi -ovc lavc -oac copy dvd.avi >& /dev/null
Now, running four mencoders in parallel:
malcolm@Blue:~/Demo$ time ./go
Avisplit: Mon Dec 14 12:27:12 IST 2009
Mencoder: Mon Dec 14 12:27:31 IST 2009
Avimerge: Mon Dec 14 12:29:26 IST 2009
Complete: Mon Dec 14 12:29:44 IST 2009
So, we have speeded up in total around 3x. This includes the extra overhead of splitting and recombining the files once encoding has completed.
This is a good example of how you can still improve application performance even without access to the source code itself.
Just in case you are wondering (or skeptical), here is my 'go' script:
echo -n "Avisplit: "
avisplit -i dvd.avi -s 400 >& /dev/null
echo -n "Mencoder: "
mencoder -o dvd-0000.avi -ovc lavc -oac copy dvd.avi-0000 >& /dev/null &
mencoder -o dvd-0001.avi -ovc lavc -oac copy dvd.avi-0001 >& /dev/null &
mencoder -o dvd-0002.avi -ovc lavc -oac copy dvd.avi-0002 >& /dev/null &
mencoder -o dvd-0003.avi -ovc lavc -oac copy dvd.avi-0003 >& /dev/null &
echo -n "Avimerge: "
avimerge -o dvd2.avi -i dvd-0000.avi dvd-0001.avi dvd-0002.avi dvd-0003.avi >& /dev/null
echo -n "Complete: "