Skip to content

[Feature]: HTCondorCEBackend — grid CE over the shared _htcondor core #23

Description

@aldbr

User Story

As the WMS consumer,
I want an HTCondorCEBackend submitting to an HTCondor-CE with IDTOKENS and spooled file
transfer,
So that the second major grid-CE path exists on the new contract, including the destructive-fetch
semantics that motivated IC-ADR-002.

Feature Description

  • HTCondorCEBackend: JobBackend + OutputRetriever + Cancellable.
    Deliberately no LoadReporter (DIRAC's getCEStatus is an explicit "not supported") and no
    live diagnostics — capability absence is the point, and the integration suite must show the
    narrowing works.
  • Reuses the _htcondor core (submit description, ClassAd parsing, state
    map); differs in transport/auth: remote schedd (-pool/-remote) or CE submission, grid
    universe routing, -spool.
  • Token + proxy together per IC-ADR-003 (its Motivation 7 / all-of kinds): SCITOKENS/IDTOKENS
    channel auth (token file + _CONDOR_* env injection, audience <ce>:9619) plus the X.509
    proxy materialised into the submit description (use_x509userproxy = true) — consumed
    site-side (VOMS attributes → APEL accounting), including the
    _CONDOR_DELEGATE_JOB_GSI_CREDENTIALS=false workaround (HTCONDOR-2904). The pure-IDTOKENS
    basic stack config exercises the token path; a proxy-bearing config is the ADR-002 auth axis.
  • Destructive retrieval: condor_transfer_data then declared destructive=True
    (HTCondor ≥ 25.8: completed spooled job leaves the queue); evaluate leave_in_queue as the
    re-fetchability override and record the decision. Coarse bounded materialisation (submit-time
    transfer_output_files allow-list, transfer timeout, post-hoc size/count check).
  • Phase 2 wiring: contract suite against the htcondor stack's CE path (markers
    remote and htcondor); the fetch-twice destructive_fetch test is the flagship regression
    (versions axis [lts, latest] makes 25.8-class changes red on the Renovate PR).

Definition of Done

  • Contract suite green against the htcondor stack CE path with IDTOKENS
  • Fetch-twice test asserts the declared one-shot semantics; leave_in_queue decision recorded
  • isinstance(be, LoadReporter) is False and the suite's capability-skip path exercises it
  • Kill path green; per-id outcomes on partial batches

Alternatives Considered

  • Expressing it as BatchBackend(HTCondorCETransport, HTCondorScheduler) — open ADR question;
    blocked on a transport-level "destructive get" notion. Start monolithic, revisit (share only
    _htcondor functions, never a subtype).
  • GSI/VOMS auth — out of scope; IDTOKENS-first per ADR-002 §4.

Related Issues

Blocked by: # (_htcondor), #. Validated by: #5 / htcondor stack.

Additional Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions