Compute worker reference data folder

compute_worker/compute_worker.py

@@ -501,6 +501,7 @@ def __init__(self, run_args):
         self.input_dir = os.path.join(self.root_dir, "input")
         self.output_dir = os.path.join(self.root_dir, "output")
         self.data_dir = os.path.join(Settings.HOST_DIRECTORY, "data")  # absolute path to data in the host
+        self.reference_dir = os.path.join(Settings.HOST_DIRECTORY, "reference")
         self.logs = {}
 
         # Details for submission
@@ -1199,6 +1200,10 @@ async def _run_program_directory(self, kind, program_dir):
             volumes_host.extend([self._get_host_path(self.input_dir)])
             volumes_config.update({volumes_host[-1]: {"bind": "/app/input"}})
 
+            # Reference data, only for the scoring phase.
+            volumes_host.extend([self._get_host_path(self.reference_dir)])
+            volumes_config.update({volumes_host[-1]: {"bind": "/app/reference", "mode": "ro"}})
+
         # During Ingestion: Add `/app/input_data` to volumes_host and update volumes_config
         if kind == ProgramKind.INGESTION_PROGRAM:
             # NOTE: self.input_data is valid when running an ingestion program and competition task has input data

documentation/docs/Organizers/Running_a_benchmark/Compute-Worker-Management---Setup.md

@@ -102,8 +102,6 @@ services:
 !!! note 
     `hostname: ${HOSTNAME}` allows you to set the hostname of the compute worker container, which will then be shown in the [server status](Server-status-page.md) page on Codabench. This can be set to anything you want, by setting the `HOSTNAME` environment variable on the machine hosting the Compute Worker, then uncommenting the line the `docker-compose.yml` before launching the compute worker.
 
-!!! note
-    Starting from `codalab/competitions-v2-compute-worker:v1.22` the images are now unifed for Podman and Docker CPU/GPU
 
 You can then launch the worker by running this command in the terminal where the `docker-compose.yml` file is located:
 ```bash
@@ -142,15 +140,12 @@ services:
 !!! note 
     `hostname: ${HOSTNAME}` allows you to set the hostname of the compute worker container, which will then be shown in the [server status](Server-status-page.md) page on Codabench. This can be set to anything you want, by setting the `HOSTNAME` environment variable on the machine hosting the Compute Worker, then uncommenting the line the `docker-compose.yml` before launching the compute worker.
 
-!!! note
-    Starting from `codalab/competitions-v2-compute-worker:v1.22` the images are now unifed for Podman and Docker CPU/GPU
-
 You can then launch the worker by running this command in the terminal where the `docker-compose.yml` file is located:
 ```bash
 docker compose up -d
 ```
-
-Note that a competition docker image including CUDA and other GPU libraries, such as `codalab/codalab-legacy:gpu`, is then required.
+!!! tip
+    Make sure to add the necessary packages inside the competition image to allow the use of the connected GPU
 
 ## Check logs
 
@@ -190,19 +185,22 @@ For each submission made to your queue, you can know what worker computed the in
 
 ---
 
-## Optional: put data directly inside the compute worker
-
-The folder `$HOST_DIRECTORY/data`, usually `/codabench/data`, is shared between the host (the compute worker) and the container running the submission (a new container is created for each submission). It is mounted inside the container as `/app/data`. This means that you can put data in your worker, in `$HOST_DIRECTORY/data`, so it can be read-only accessed during the job's process. You'll need to modify the scoring and/or ingestion programs accordingly, to points to `/app/data`. This is especially useful if you work with confidential data, or with a heavy dataset.
+## Optional: Put data directly inside the compute worker
+### Datasets
+The folder `$HOST_DIRECTORY/data`, usually `/codabench/data`, is shared between the host (the compute worker) and the container running the submission (a new container is created for each submission). It is mounted inside the container as `/app/data`. This means that you can put data in your worker, in `$HOST_DIRECTORY/data`, so it can be read-only accessed during the job's process. You'll need to modify the scoring and/or ingestion programs accordingly, to point to `/app/data`. This is especially useful if you work with confidential data, or with a heavy dataset.
 
 !!! note "If you have several workers in your queue, remember to have the data accessible for each one."
 
 ![](_attachments/4259c2e5-d119-4ca2-8fc8-b69196f1528c_17534367097493236.jpg)
 
+### Reference Data
+You can also add your reference data, used during the scoring phase, inside a folder on the compute worker. This folder, `$HOST_DIRECTORY/reference`, will be mounted during the scoring phase as `/app/reference` inside the container. Contrary to `$HOST_DIRECTORY/data`, this folder is only mounted during the scoring phase. 
+
 
 !!! warning 
     Make sure to make the owner of the folder(s) and file(s) the same as the one launching the compute worker.  
        - `root` for Docker rootfull  
-       - `codalab` for Podman and Docker rootless if you created a user name codalab to launch podman and docker rootless from
+       - `codabench` for Podman and Docker rootless if you created a user name `codabench` to launch podman and docker rootless from
 
 
 !!! tip "If you simply wish to set up some compute workers to increase the computing power of your benchmark, you don't need to scroll this page any further."
@@ -216,13 +214,13 @@ This is helpful only if you want to build the compute worker image. It is not ne
 To build the normal image:
 
 ```bash
-docker build -t codalab/codabench-compute-worker:latest -f packaging/container/Containerfile.compute_worker .
+docker build -t *account_name*/codabench-compute-worker:latest -f packaging/container/Containerfile.compute_worker .
 ```
 
-To update the image (add tag `:latest`, `:gpu` or else if needed)
+To update the image
 
 ```bash
-docker push codalab/codabench-compute-worker
+docker push *account_name*/codabench-compute-worker:latest
 ```
 
 !!! note "If you have running compute workers, you'll need to pull again the image and to restart the workers to take into account the changes."

documentation/docs/Organizers/Running_a_benchmark/Compute-worker-installation-with-Podman.md

@@ -82,10 +82,6 @@ USE_GPU=True
 
 
 ## Compute worker installation 
-!!! note
-    Starting from `codalab/competitions-v2-compute-worker:v1.22` the images are now unifed for Podman and Docker CPU/GPU and has been renamed to `codalab/codabench-compute-worker:latest`
-
-
 Create a `run_podman.sh` file with the following content:
 
 ```bash title="run_podman.sh"