Skip to content

DAL Persistence Configuration🔗

This section describes the configuration options for enabling persistence of DAL (Data Access Layer) intermediate results in the Apheris Compute Gateway.

This option is disabled by default.

Datasets saved this way will have the same level of security/access control as the original datasets. These Derived Datasets are local to the gateway in which they were created.

Management of the volume is the responsibility of the gateway owner.

Enable Persistence🔗

To enable persistence for the DAL intermediate results, set the following value:

dal:
  persistence:
    enabled: true

Persistence Configuration🔗

dal:
  persistence:
    name: "dal-persistence"
    dataPath: /mnt/storage
    existingClaim: "example-pv"

Existing Claim🔗

Specify an existing volume claim.

In this scenario other options need not be specified.

This is the canonical approach, other specific options are provided for certain environments/troubleshooting.

HostPath🔗

If using a HostPath volume, folder ownership must match the fsGroup in PodSecurityContext (default 1000). This can be set with chown -R 1000:1000 /path/to/local/storage.

Volume Name🔗

Specify the name of the DAL persistence volume. This is typically not something that should be changed.

Data Path🔗

Set the filesystem path where the DAL volume will be mounted. This is typically not something that should be changed.

Local Storage Configuration🔗

For local storage persistence:

dal:
  persistence:
    local:
      hostPath: "/example/path"
      capacity: 1Gi
  • hostPath: Specify the hostPath for the DAL persistent volume. This requires the local-path provisioner.
  • capacity: Set the storage capacity limit for the DAL persistent volume.

The section on HostPath applies here.

S3 Storage Configuration🔗

For S3 storage persistence:

dal:
  persistence:
    s3:
      bucket: "s3://example-bucket"
      accessMode: ReadWriteMany
  • bucket: Specify the name of the S3 bucket used by the DAL persistent volume. This requires the S3 CSI driver to be installed.
  • accessMode: Set the access mode for the S3 PV and PVC. Supported options are ReadWriteMany and ReadOnlyMany.

Please ensure that you have the necessary provisioners or CSI drivers installed when using S3 storage options.