Project accounting¶
Every project proposal is granted a number of compute and storage resources. Once these resources have been fully consumed, no further computations are possible, so it is important to keep an eye on your project's resource consumption.
Compute resources¶
The steering committe will grant your project a number of CPU hours and GPU hours when the proposal is accepted.
The granted and used amount of CPU/GPU hours can be queried using the following command:
$ kit_project_usage
The default output has the following format:
Project Resource Granted Used
hkt-project-test cpu 1000000 20000
hkt-project-test gres/gpu 5000 1000
If the amount of resources for a project is unlimited, the Granted column will show inf. If a project has been restricted (e.g. it was not accepted by the steering committee while in the preliminary phase), this column will show 0 instead. If the calling account is a member of multiple projects, the output will containe two lines (CPU/GPU resources) for every project.
When the sum of the used CPU or GPU hours of a project is higher or equal than the granted number CPU or GPU hours, new batch jobs will no longer start. Instead they will be held in the PENDING state and the error message AssocGrpTRESLimit will be shown. Please see the matching FAQ entry for what to do in this case
Accounting of resources
The cpuonly partition is running in exclusive mode, which means that Slurm always allocates and accounts whole nodes even if the number of requested cores is lower than the ones available. All project proposals should specify their resource consumption with the assumption that a node has 76 CPU cores, since every node hour on HoreKa will account as 76 CPU hours.
Storage resources¶
Please refer to the File systems chapter to get more information about storage quotas.