ATLAS@Home: Tips on how to gain better performance on the ATLAS_MCORE app

According to our test, the CPU performance (We measure the CPU performance by seconds of CPU per event. For example, the current ATLAS job running on the ATlAS_MCORE app processes 100 events, if the overall CPU time for this job is 30000 seconds, then the CPU performance is 300 seconds/event) varies to the number of cores of the vm.

In the test, we also compared the CPU performance on different Virtualbox versions.

The test result can be seen here:

http://gilda117.ihep.ac.cn/wuimages/vbox3.jpg

The above test is done on 2 hosts, HOST1 has HT(HypherThreading) enabled, while HOST2 has HT disabled. The test result is consistent despite of whether disabling/enabling HT of the CPU.

Also, we conclude a result from the ATLAS job statistics based on the jobs from over 1 month period. The following result shows the average CPU performance on different number of cores. (ATLAS_MCORE supports up to 12 cores for now).

http://gilda117.ihep.ac.cn/wuimages/vbox2.jpg

The benefit of using more cores in one vm is to save memory usage, but using big number of cores can also significantly reduce the CPU performance. Our test result also concludes this is a case in all cloud computing platforms, not just on ATLAS@home.

In order to have a good tradeoff between memory usage and CPU performance, we advise that you configure the cores to each ATLAS_MCORE job(VM) according to the overall cores and memory allocated to BOINC.For example,if your host allocates 12 cores to BOINC, by default, ATLAS_MCORE creates a vm with 12 cores with 12.1GB memory, but if the host has enough memory, you can customize the usage with the app_config.xml file, i.e. each vm uses 6 cores with 7.3GB memory, so that your host runs 2 vms, and the overall memory usage is 14.6GB.

You can limit the MultiCore-App by using the app_config.xml (This file needs to be put in your project/atlasathome.cern.ch/ directory).

Below is an example to limit each ATLAS_MCORE job to use only 6 Cores:

ATLAS_MCORE 6.000000
vbox_64_mt_mcore --memory_size_mb 7300


You should change these two lines to your needs:

4.000000--memory_size_mb 7300


Memory usage calculated by the ATLAS_MCORE app is by this formula:

memory = 2500 + (800* NumerOfCores)

so it is 7300MB for 6 cores.

We will also make some changes on the server side very soon:
1. Require a minimum version(5.0.0) of Virtualbox for the ATLAS_MCORE app.
2. Limit the ATLAS_MCORE app to use at most 8 cores.

More...