Skip to content

Authentication with DiracX

This document explains how authentication is handled within DiracX, including the interactions with the legacy DIRAC server.

At a high level, DiracX acts as an IdP for all DIRAC/DiracX-specific activity. Users are authorised using an external IdP, typically Indigo IAM for real VOs. Other OIDC-compliant IdPs can be used for the admin pesudo-VO.

Overview

Let's start with an overview how authentication works in the legacy DIRAC software. We'll only cover what DIRAC handles directly, for communication with third party resources see the dedicated page.

What authentication methods do we have in DIRAC?

In the legacy DIRAC software there are 5 modes of authentication:

  • User certificates: X.509 certificates issued by a trusted Certificate Authority (CA) that identify individual users.
  • Proxies: Proxies are additional X.509 certificates derived from user certificates. This is used to allow users delegate their credentials to services or other processes without sharing their complete private key. In DIRAC proxies are further restricted by the addition of the dirac group extension.
  • Proxies with VOMS extensions: Proxies that include Virtual Organization Membership Service (VOMS) extensions, providing additional authorization information such as VO membership and role.
  • Host certificates: X.509 certificates that identify and authenticate hosts rather than individual users. These are used for service-to-service and agent-to-service authentication.
  • User impersonation via host certificates: A mechanism where services with host certificates can impersonate users, typically used for administrative operations or when services need to act on behalf of users. This mechanism is used by the legacy DIRAC webapp to "forward" requests to other DIRAC services. This is only available if the host has the TRUSTED_HOST property.

In DIRAC there are a series of properties which map to functionality which the current X509 credentials are permitted to do (e.g. JOB_ADMINISTRATOR, NORMAL_USER, ...) Properties are not added directly in the X509 certificates used by DIRAC and are instead mapped from the central configuration service (CS) by:

  • If the identity is a host properties are directly mapped to the host certificate's DN.
  • If the identity is a user without a group assigned (e.g. a direct user certificate) a default group is picked from the CS.
  • If the identity is a user (or a TRUSTED_HOST host imersonating a user) the associated group is mapped to a set of properties.

Additionally, some parts of DIRAC apply custom authourisation logic (e.g. the global job monitoring option in the CS).

DIRAC/DiracX equivalents

Purpose DIRAC DiracX
User authentication X509 certificates from CAs JWTs obtained via DIRAC OAuth2 flows
Service-to-service X509 host certificates Not applicable
Agent-to-service X509 host certificates or "shifter" proxies Not applicable - Uses direct DB access
First pilot → DIRAC interaction Proxy included in pilot payload Unique secret included in pilot payload
Pilot → communication (e.g. job matching) Proxy included in pilot payload Secret obtained from first pilot → DIRAC interaction
Obtaining job specific credentials Proxy included in pilot payload Job specific credentials included in matching request response
Job payload → DIRAC (e.g. status updates) Proxy included in pilot payload and job specific proxy Job specific credentials

User authentication

There are two ways in which users can authenticate with DiracX, inside their web browser or their terminal. In both cases standard OAuth2 authorization flows are used to obtain an access and a refresh token.

Properties

In DiracX access is handled with access polices which look at the content of an access token and make an authorisation decision. The primary content inside an access token are managed by OAuth2 scopes which mirror the existing DIRAC properties. This will likely evolve over time as more DiracX-only functionality appears. When authenticating with DiracX there are two ways by which these scopes are decided:

  • No property scopes requested: The default properties are taken to match the corresponding DIRAC group.
  • Some property scopes requested: Only the specific properties are included in the token. This allows users to create tokens with reduced permissions (e.g. can read data but can't submit jobs).

DiracX Web

When authenticating against DiracX web we effectively do two OAuth Authorization code flows, one between the web client and DiracX and a second between DiracX and the VO's IdP. Both authorizations code flows use PKCE.

What is "PKCE" and why is it used?

PKCE (Proof Key for Code Exchange) is an extension to the OAuth 2.0 Authorization Code flow designed to enhance security, especially for public clients such as single-page applications and mobile apps that cannot securely store client secrets.

PKCE works by having the client generate a random value called a "code verifier" and then deriving a "code challenge" from it (usually by hashing). The code challenge is sent in the initial authorization request. When exchanging the authorization code for tokens, the client must present the original code verifier. The server checks that the code verifier matches the code challenge sent earlier.

This mechanism prevents authorization code interception attacks, where a malicious actor could otherwise steal the authorization code and exchange it for tokens. By requiring the code verifier, only the legitimate client that initiated the flow can complete the process.

In DiracX, PKCE is used to ensure that authentication flows—especially those initiated from browsers or other public clients—are protected against such attacks, providing an additional layer of security on top of standard OAuth 2.0.

sequenceDiagram autonumber participant W as DiracX web box Server side participant DX as DiracX participant IdP as Identity Provider end W->>DX: GET /authorize DX->>IdP: Redirect to VO's IdP Note right of IdP: User authenticates with IdP<br />Might redirect to a second IdP<br />(e.g. CERN SSO) IdP-->>W: Authorization grant W-->>DX: GET /authorize/complete DX->>IdP: Request ID token IdP->>DX: ID token DX-->>W: Authorization grant W-->>DX: POST /token DX->>W: Access+Refresh token
  1. The flow is initiated by the client making a request to the DIRAC auth service with PKCE.
  2. The DIRAC auth service stores the request details and redirects the user to the IAM's authorization flow by performing an authorization request on behalf of the user.
  3. Once done, the IAM redirects the user back to the DIRAC auth service.
  4. The user's browser follows the redirect back to the DIRAC auth service.
  5. The DIRAC auth service exchanges the code for an ID token by making a token request to the IAM.
  6. The IAM responds with an access, a refresh and an ID tokens.
  7. The DIRAC auth service only stores the ID token and redirects the user to the client's redirect URI.
  8. The client requests an access token from the DIRAC auth service by making a request to the token endpoint (POST /token).
  9. The DIRAC auth service responds with a DIRAC access/refresh token, based on the IdP's ID token.

The client can:

CLI clients

When authenticating CLI clients there are two authorization flows involved:

  1. Device Authorization Grant flow between the CLI client and DiracX - this is the OAuth 2.0 Device Authorization Grant flow, designed for devices with limited input capabilities.
  2. Authorization Code flow between DiracX and the VO's Identity Provider - this is the standard OAuth Authorization Code flow that DiracX uses to authenticate with the external IdP.

The first flow is done between a user's terminal and DiracX. The second flow is done entirely inside the user's web browser as is done for DiracX web.

sequenceDiagram autonumber participant CD as Client Device participant U as User participant B as Browser box Server side participant DX as DiracX participant IdP as Identity Provider end CD->>DX: POST /device DX-->>CD: User Code, Device Code Note right of CD: Display user code to user U->>B: Navigate to verification URI B->>DX: GET /device (with user code) DX->>IdP: Authorization Request Note left of IdP: User authenticates with IdP IdP-->>DX: Authorization Grant DX->>IdP: ID Token Request IdP->>DX: ID Token DX-->>B: Device Grant Complete CD->>DX: POST /token (with device code) DX-->>CD: Access+Refresh token
  1. The device flow is initiated by the client, which is typically a device with limited input capabilities (POST /device).
  2. The DIRAC auth service responds with a user code and a device code.
  3. The client then displays the user code to the user and instructs the user to go to a verification URI on another device (e.g. a smartphone or computer) and enter the user code.
  4. The user then authenticates on the other device. In the meantime, the client polls the DIRAC authorization server to check if the user has authorized the client.
  5. The DIRAC auth service stores the request details and redirects the user to the IAM's authorization flow by performing an authorization request on behalf of the user.
  6. Once done, the IAM redirects the user back to the DIRAC auth service.
  7. The DIRAC auth service exchanges the code for an ID token by making a token request to the IAM.
  8. The IAM responds with an access, a refresh and an ID tokens.
  9. The DIRAC auth service stores the ID token and redirects the user to the redirect URI.
  10. The client requests an access token from the DIRAC auth service by making a request to the token endpoint.
  11. The DIRAC auth service responds with a DIRAC access token, based on the ID token.

The client can then:

Using a token

In OAuth2 there are three kinds of token defined, see below for their use within DiracX.

Access tokens

Access tokens are short-lived credentials that allow clients to authenticate with DiracX services. They contain information about the user's identity and permissions, and are used to authorize API requests.

Key characteristics of access tokens in DiracX:

  • Short-lived: Typically valid for a limited time (e.g., 1 hour) to minimize security risks
  • Bearer tokens: Included in the Authorization header as Bearer <token>
  • JWT format: JSON Web Tokens that can be decoded to inspect claims
  • User identification: Contains the user's VO, username, and other identity information
  • Permission scopes: Include information about what the user is authorized to do

When an access token expires, clients must use a refresh token to obtain a new access token.

Refresh tokens

Refresh tokens are long-lived credentials used to obtain new access tokens without requiring the user to re-authenticate. They provide a seamless user experience by automatically renewing access when needed.

Key characteristics of refresh tokens in DiracX:

  • Long-lived: Valid for extended periods (e.g., days) to minimize user re-authentication
  • Rotation: New refresh tokens are often issued when refreshing access tokens
  • Secure storage: Should be stored securely on the client side
  • Revocation: Can be revoked if compromised or when the user logs out
sequenceDiagram autonumber participant C as Client participant DX as DiracX Note over C,DX: Access token has expired C->>DX: POST /token<br/>(grant_type=refresh_token,<br/>refresh_token=...) DX-->>C: New access token + refresh token (optional) Note over C: Client can now make<br/>authenticated requests C->>DX: API request with new access token DX-->>C: API response
  1. The client detects that its access token has expired.
  2. The client sends a POST request to the /token endpoint with the refresh token.
  3. DiracX validates the refresh token and generates a new access token. If token rotation is enabled, DiracX also generates a new refresh token.
  4. The client can now make authenticated API requests using the new access token. If a new refresh token was returned it MUST be used for future refreshes as the previous token will have been revoked.

ID tokens

Currently there is no use of ID tokens within DiracX. The primary use for ID tokens within DiracX would be to use DiracX as an IdP for an independent service. However, this independent service could likely be registered as a separate client within the VO's IdP (e.g. IAM). If you have a distinct use case for this, please open an issue.

Job authentication

Differs from reality

This section represents the expected final workflow for pilots. Currently pilots follow the standard historical DIRAC workflow with the only addition being the inclusion of the DiracX token inside the proxy's PEM file. See here for details.

This section only explains after pilots start!

To learn how a pilot job is submitted see the external authentication docs.

Only run jobs on trusted resources

While every effort is made to minimise the trust places in remote compute resources, DiracX cannot provide complete security against malicious sites. Only submit jobs to compute resources which are being securely managed by admins you trust.

Job authentication in DiracX uses a multi-stage credential system to secure the execution of payloads on external computing resources. The sequence diagram is as follows:

sequenceDiagram autonumber participant DX as DiracX participant P as Pilot Agent participant AC as Apptainer container rect DiracX Pilot Startup Phase Note over P: Job started by<br/>Spontaneous or CE P->>DX: POST /pilot/start<br/>(with unique secret) DX->>P: Pilot-matching credentials end loop rect Job Matching Phase Note over P, DX: Payload matching P->>DX: Request job matching<br/>(with pilot credentials) DX->>P: Job details + limited user credentials<br/>(minimal capabilities, job lifetime) end rect Job Execution Phase Note over DX, AC: Job execution P->>AC: Limited user credentials + JobID loop AC->>AC: Execute job payload AC->>DX: Send status updates end loop P->>DX: Send heartbeats/logs/status updates DX->>P: Job commands/updates end end rect Job Completion Phase Note over P, DX: Job finalisation P->>DX: Job completed/failed DX->>DX: Invalidate job credentials<br/>(payload JWT revoked) end end
  1. The pilot agent is started by spontaneously or the Computing Element and uses a unique secret to authenticate with DiracX. This secret can typically be used only once, though this is configurable for Spontaneous-style situations.
  2. DiracX responds with a JWT which is used for all future operations, such as job matching or sending pilot logs.
  3. When ready, the pilot uses it's JWT to request jobs from the DiracX matching service.
  4. If a suitable job is available, DiracX responds with the job to be executed and a JWT representing the owner of the job. This "payload JWT" is limited to only the properties it needs for running the job.
  5. The pilot starts a container to isolate the job payload from the main pilot process.
  6. Inside the container, only the "payload JWT" is available.
  7. Application status updates are sent by the payload using the "payload JWT".
  8. Heartbeats, logs and status updates are send by the pilot.
  9. DiracX responds to heartbeats with job commands if needed (e.g. send a kill signals to the payload).
  10. When the job finishes the final status update is sent to DiracX.
  11. When receiving the final status update, DiracX revokes the refresh token of the "payload JWT".

DIRAC → DiracX communication

For the migration from DIRAC to DiracX there are several scenarios in which DIRAC needs to communicate with DiracX:

  • To use the legacy adaptor mechanism
  • To proxy sandbox upload/downloads from legacy DIRAC clients.

Legacy exchange

The underpinning of DIRAC → DiracX communication is the legacy exchange mechanism. A secret string is added to the local dirac.cfg file for DIRAC services (/DiracX/LegacyExchangeApiKey). This secret is included as a bearer token in calls to /api/auth/legacy-exchange to be able to request a valid DiracX token for a specific user and set of scopes.

Implementation detail

The /api/auth/legacy-exchange is ONLY for internal use within DIRAC. Implementation details may change at any time and the route is not intended to be secure enough for other purposes.

DiracX tokens in proxies

When functionality has been migrated from DIRAC to DiracX the legacy adaptor mechanism is used to redirect traffic to DiracX. To authenticate this requests a DiracX refresh token is stored inside the X509 proxy file in a section with the header -----BEGIN DIRACX-----. The contents of this section is obtained using the exchangeProxyForToken RPC call in the ProxyManager. When used, the DiracX token is extracted and placed into a separate file in the current temporary directory with the prefix dx_.

SanboxStoreHandler

The DIRAC SanboxStoreHandler has a special case for the migration where data is transparently proxied to the DiracX sandbox store. To authenticate the legacy exchange mechanism is used to obtain a DiracX token which is then used to impersonate users.

DiracX → DIRAC communication

There is no direct communication from DiracX to legacy DIRAC services. Any information in DIRAC which is needed by DiracX is passed indirectly, for example, via the databases and CS synchronisation from DIRAC.