Skip to content

Latest commit

 

History

History
90 lines (49 loc) · 4.35 KB

TODO.md

File metadata and controls

90 lines (49 loc) · 4.35 KB
Raw

WfExS-backend official roadmap and informal TODO

  • Step 0: Materialize Nextflow and CWL workflows from WorkflowHub. Downloading an RO-Crate using the TRS API of WorkflowHub to materialize workflows.

  • Step 1: Materialize remote repositories that contains the materialized workflows.

  • Step 2: Materialize inputs to launch the workflows. Using a file that contains the ids or data references to the inputs, that we are going to use to instantiate the workflows. For example: (wetlab2variations_execution_cwl.yaml)

  • Step 3: Setup Nextflow and CWL engines.

  • Step 4: Validate workflows and materialize their containers.

  • Step 5: Launch the workflows in an execution environment.

    • Step 5.a: Launch workflows in a FUSE encrypted filesystem.

    • Step 5.b: Launch CWL workflows in an execution environment.

    • Step 5.c: Launch Nextflow workflows in an execution environment.

    • Step 5.d: Generalize the number of workflows which can share the same working directory, so outputs of one can feed inputs of others.

  • Step 6: Integrate or delegate the use of Crypt4GH into the process when requested, so outputs are encrypted for the researcher, allowing moving them with no data disclose.

    • Step 6.a: Use Crypt4GH to encrypt the passphrase of FUSE encrypted execution working directories.

    • Step 6.b: Create a micro CWL workflow which uses Crypt4GH to decrypt one or more files (depends on 5.d).

    • Step 6.c: Create a micro CWL workflow which uses Crypt4GH to encrypt one or more files (depends on 5.d).

  • Step 7: Add upload capabilities of results, which generate permanent identifiers.

    • Step 7.a: Upload to cache system.

    • Step 7.b: Upload to WebDAV server.

    • Step 7.c: Upload to FTP server.

    • Step 7.d: Upload to SFTP server.

    • Step 7.e: Upload to Nextcloud and generate share link.

    • Step 7.f: Upload workflow to WorkflowHub.

    • Step 7.g: Upload to Zenodo.

    • Step 7.h: Upload to B2SHARE.

    • Step 7.i: Upload to osf.io .

    • Setp 7.j: Explore datacite doi service.

    • Step 7.k: Upload to Dataverse.

  • Step 8: (partially implemented) Create execution provenance, which includes uploading URLs of results and / or DOIs / URIs.

  • Step 9: Generate RO-Crate from execution provenance and exported results.

    • Step 9.a: Generated RO-Crate should be consumable by WorkflowHub.

    • Step 9.c: Generated RO-Crate should be consumable by WfExS-backend.

    • Step 9.d: Add full circle capabilities. Re-execute workflow with the very same parameters from previously generated RO-Crate (only metadata).

    • Step 9.e: Add full circle capabilities. Re-execute workflow with the very same parameters from previously generated RO-Crate (reusing payloads).

Other features