It has been more than a year since I came to the graduate school. It’s hard to say how’s the lift in u.s. but one thing I can confirm is money is everything. Everybody could devote themselves, their dream for funding. It’s really a tragedy. Hope I can get UMICH’s PhD offer even if it seems hopeless now ToT.
This post is mainly about how to configure matlab for pbs or torque configured cluster. I struggled for a long time and finally find the solution to run Matlab parallely on cluster with multinodes and multicores. (B.T.W : MATLAB sucks! )
The main trick about Matlab is that it only transfers one argument matlabpool ^N^ to the parpool setting. The number ^N^ determines how many processes it will open. Most of the cluster configuration file sets the submission command as “qsub -l nodes=^N^:ppn=1”, which makes the multi-thread feature unavailable. The worse thing is that if you only has an access to a cluster which runs naccesspolicy as single mode, i.e. once you get access to one node, no matter how many ppns do you request, your usage of cores is increased by the cores on one node. This feature is really sucks.
Let’s make a summary of the above words. If you set “-l nodes=^N^:ppn=1″ for cluster configuration file submitting command and run matlabpool 5. You will request 5 nodes, each node uses one core. Suppose each node has 16 cores. The qlist will decrease 5*16 = 80 cores! but not 5*1=5 cores.
Thus our aim is to use all the cores on each node.
In this article, the author provides a smart way to do that http://hpclab.blogspot.com/2010/11/matlab-dcs-torque-whats-missing.html. But I don’t like to modify original code of Matlab.
The hard core method I used is as the following:
Suppose we want to use 2 nodes with all 16 cores on each node. Then modify the configuration file’s submission command to ” -l nodes=2:ppn=16″. Then get parpool by matlabpool open your_cluster_configuration_file_name 32.
Hope it helps!
B.T.W. NYU’s website of high performance computing is the best one I’ve ever seen. It has a great introduction of how to write pbs file.