Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wdl 1.1 sep() vs command block ~{sep} return different values #435

Open
Ben-Habermeyer opened this issue Apr 10, 2023 · 2 comments
Open
Labels
tracked_internally Issue is tracked internally

Comments

@Ben-Habermeyer
Copy link

Hi DXCompiler team, I recently came across a behavior when trying to convert an array of File into a CLI arg

I have an input Array[File] snp_vcfs

Originally in a command block I had given this to a CLI tool like the following
--sample-snp-vcfs ~{sep=' ' snp_vcfs}

Which resulted in the following resolution when run on DNAnexus --sample-snp-vcfs /home/dnanexus/inputs/input4994899743764944893/s_1_AACATACTGAGTGATCCGGA.final.vcf /home/dnanexus/inputs/input4994899743764944893/s_1_AATCACGGTTCGGATCGGTT.final.vcf

But then I needed to make this optional, so I put the following in my task block for the input Array[File] snp_vcfs = []

And added this arg
String snp_vcfs_args = if length(snp_vcfs) > 0 then "--sample-snp-vcfs " + sep(" ", snp_vcfs) else ""

Which I reference in command block ~{snp_vcfs_args} but this looks resolves as the following
--sample-snp-vcfs dx://file-GQZPx0Q0xKpKqJp3kxZ4XBP8::/s_1_AACATACTGAGTGATCCGGA.final.vcf dx://file-GQZKf9Q0GxY4K65ZPqGYkp4Y::/s_1_AATCACGGTTCGGATCGGTT.final.vcf

Is it possible for the 1.1 builtin sep() to resolve File objects to their path (like the ~{sep} does) i.e. /home/dnanexus/inputs/... rather than just converting them to strings (and rendering these paths potentially unusable)?

1.1 says ~{sep} is deprecated but I feel like they should have the same behavior unless there is something I'm missing? Thanks

@Gvaihir
Copy link
Contributor

Gvaihir commented Apr 10, 2023

This might be the implementation thing in dxC. I'll track internally, thanks for reporting

@Gvaihir Gvaihir added the tracked_internally Issue is tracked internally label Apr 10, 2023
@Gvaihir
Copy link
Contributor

Gvaihir commented Apr 20, 2023

@Ben-Habermeyer - I started working on this issue. Could you clarify:

  1. But then I needed to make this optional, so I put the following in my task block for the input Array[File] snp_vcfs = [] <- This does not look like an optional declaration. It's a defined array which is supposedly evaluated to an empty string in your conditional, if empty. Please clarify - is this what you mean? This is essential for the next point
  2. The difference between the Placeholder Options ~{sep=' ' snp_vcfs} (let's call it "case A") and sep() ("case B") - is that case A localized the files to the workers, but case B did not. Going back to point 1 - did you declare snp_vcfs as optional or not? I mean Array[File]? or Array[File?]. If they were optionals for Case B and not for case A and you chose to stream files - they may not be localized in the correct time or manner.
    Anyway - please provide a minimal wdl source and a compile flags you used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tracked_internally Issue is tracked internally
Projects
None yet
Development

No branches or pull requests

2 participants