Skip to content

CESGA#

Overview#

CESGA (Centro de Supercomputación de Galicia) is a public supercomputing centre. It gives your solvers access to two machines, each wired to the platform as its own provider:

Provider Machine What it is
CESGA FinisTerrae III ft3.cesga.es General-purpose supercomputer: x86 CPU nodes, NVIDIA A100 and T4 GPU nodes, and ARM (A64FX) nodes
CESGA QMIO qmio.cesga.es CESGA’s quantum computer. Restricted access

Both are managed with Slurm, so your solver does not compute anything itself: it submits a batch job to the cluster, waits for it, and returns the output. That also means both are driven with exactly the same code — switching machine is a matter of which provider your solver uses, not of rewriting anything.

Note

This is the provider-specific page. The client API, the job options and the result format are the same for every HPC provider and are documented in How to create an HPC solver — read that page first, and come back here for what is specific to CESGA.

How it works#

CESGA’s login nodes are only reachable from authorised networks — those of member research centres and universities. From anywhere else the connection simply times out, so the platform reaches them through CESGA’s VPN.

You do not have to deal with any of that: when your solver uses a CESGA provider, the platform, before your run() is called:

  1. Brings up the VPN to CESGA.
  2. Logs in to the cluster and prepares Slurm access.
  3. Injects the cluster credentials as environment variables, which the HPC client reads for you.

Your run() → submit a script → HPC client → Slurm over the VPN → cluster runs it on a compute node → the job’s output comes back to your run().

Runs on the platform only

Solvers using these providers cannot be run locally with python app.py. The VPN, the cluster credentials and the HPC client only exist inside the QCentroid runtime. Develop the job logic locally if you want, but test the full solver on the platform.

Create a solver for CESGA#

1. Enable the provider#

Go to the Providers section, find CESGA FinisTerrae III (or CESGA QMIO) and click Enable.

  • When requested, add your CESGA username and password.

One account for everything at CESGA

The same credentials open the VPN and the cluster — CESGA issues a single account. They are not your QCentroid ones: you get them from CESGA when your access is granted. Enter the username on its own (myuser), never the full email address. You enter them once, here; from then on the platform injects them into your solvers and you never write them in your code.

QMIO access is granted separately

Being able to reach CESGA is not enough for QMIO: your account also has to be entitled to that machine. If it is not, the connection reaches the login node and is then refused. Ask CESGA to enable it before creating a QMIO solver.

2. Register your solver#

Go to My solvers → Add new solver, and then:

  1. Select one of the existing Use Cases.
  2. Select the CESGA provider you enabled as the hardware provider for this solver.
  3. Choose the HPC language (python_hpc) in the programming language dropdown. This is what tells the platform to build your solver with the HPC client and the VPN.
  4. Complete the rest of the information and click Add new solver.

3. Implement your solver#

Write your qcentroid.py using the HPC client (see the example below), and push it to a Git repository.

4. Connect your repository#

Go to the Repositories section, click Connect a new repository, select the solver you just created, fill in the rest of the details and click Connect repository.

Your solver is now registered, wired to the cluster and ready to run. To launch it, see How to run a job.

Working with these clusters#

Always state memory and time#

This is the rule that catches everyone first: a job that does not state both its memory and its time limit is rejected at submit time.

job_id = client.run_job(
    BATCH_SCRIPT,
    name="my-job",
    time_limit=10,           # minutes  — mandatory
    memory_per_node=4096,    # MB       — mandatory
    environment="login",     # so `module load` works
)

The provider configures a default memory, so a solver that passes nothing still submits a valid job. Set both explicitly anyway: they are the two values only you know.

Do not name a partition or a QOS#

FinisTerrae III has partitions (short, medium, long, ondemand, …) but you are not meant to pick one: Slurm routes the job automatically from the resources it asks for, and CESGA’s own guide advises against setting the QOS by hand. Naming either is a common cause of rejected submits, so the provider deliberately leaves both unset.

The exception is node type, which is selected with a Slurm constraint rather than a partition:

job_overrides={"constraints": "clk"}   # Intel Cascade Lake; "epyc" for AMD EPYC

Asking for a GPU#

FinisTerrae III’s GPU nodes are requested through gres, naming the GPU model:

gres="gpu:a100:1",     # one A100
cpus_per_task=32,      # cores scale with GPUs: 32 per GPU on the standard nodes

Cluster modules#

Software is loaded with module (Lmod). One thing is specific to CESGA: only the Core section is visible until you load a base module first, so module load <something> fails on its own until you do:

module load cesga/2022     # or cesga/2020, or cesga/system
module load <the software you need>

Two more things for module to work inside your job:

  • Start the script with #!/bin/bash -l — the -l makes it a login shell.
  • Pass environment="login" to run_job(), so the cluster’s login environment is propagated to the job.

Use module spider <name> on the cluster to find what is available: the catalogue changes over time.

Example solver#

A complete qcentroid.py that runs a job on FinisTerrae III:

qcentroid.py
import logging
from qcentroid_solver_runner.hpc import QCentroidHPCClient

logger = logging.getLogger("qcentroid-user-log")   # so progress shows in the dashboard

# "-l" runs a login shell, which is what makes `module` available. cesga/2022 is the base
# module that makes the rest of the software stack visible to Lmod.
BATCH_SCRIPT = """#!/bin/bash -l
module load cesga/2022

echo "Running on $(hostname)"
python3 -c 'print("Hello world from CESGA")'
"""


def run(input_data: dict, solver_params: dict, extra_arguments: dict) -> dict:
    logger.info("Submitting the job to CESGA...")

    with QCentroidHPCClient(logger=logger) as client:
        client.login()

        # Useful on a first run: this logs the partitions the cluster really has and what
        # your account may submit under.
        logger.info("Partitions: %s", client.list_partitions())

        job_id = client.run_job(
            BATCH_SCRIPT,
            name="my-hpc-job",
            time_limit=10,           # minutes — mandatory here
            memory_per_node=4096,    # MB      — mandatory here
            environment="login",     # so `module load` works
            # No partition and no QOS: Slurm picks both from the resources requested.
        )
        logger.info("Submitted job %s", job_id)

        # Bound the wait: the cluster is shared, so queues can be long.
        client.wait_for_completion(job_id, poll_interval=5, queue_timeout=3600, run_timeout=600)

        result = client.get_job_results(job_id)

    logger.info("Job %s finished: %s (exit code %s)", result.job_id, result.state, result.exit_code)

    return {
        "job_id": result.job_id,
        "state": result.state,
        "succeeded": result.succeeded,
        "stdout": result.stdout,
        "stderr": result.stderr,
    }

The same solver runs on QMIO by pointing it at the QMIO provider — only the batch script changes, since the quantum software stack has its own modules. The full client reference — every run_job() option, the JobResult fields and how to publish assets — is in How to create an HPC solver.

Good to know#

  • Queue times can be long. These are shared national facilities, so a job may sit queued for a while before it runs. wait_for_completion() logs the queue reason on each poll; bound the wait with queue_timeout / run_timeout if your solver should not wait indefinitely.
  • A failed cluster job is not automatically a failed solver. Your run() still returns normally, reporting the state. Raise an exception yourself if you want the whole execution marked as failed.
  • Keep the job’s output. Drop the raw stdout, reports or plots into an additional_output folder to publish them as downloadable assets.
  • CESGA documents its own machines. Node types, software catalogue, storage and quotas are covered by the FinisTerrae III user guide; this page only covers reaching the cluster from a solver.

What’s next#

How to create an HPC solver

Run your solver

Edit or update this solver