Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error command harvest failed #89

Open
valeriaR89 opened this issue Dec 23, 2020 · 7 comments
Open

Error command harvest failed #89

valeriaR89 opened this issue Dec 23, 2020 · 7 comments

Comments

@valeriaR89
Copy link

Dear developers, I'm trying to use your tools for core SNP analysis. I use a native Ubuntu 20.04, the tutorial works fine but when I run my samples I have this error message. Before the error message seems that everything works (the reference and the genomes were recognised).

command line:
parsnp -g /storage/riferimento/reference.gbk -d /storage/AssemblyPlino/ -v

|--Parsnp v1.2--|
For detailed documentation please see --> http://harvest.readthedocs.org/en/latest

SETTINGS:
|-refgenome: /storage/riferimento/reference.gbk.fna
|-aligner: libMUSCLE
|-seqdir: /storage/AssemblyPlino/
|-outdir: /home/user/P_2020_12_23_110653922704
|-OS: Linux
|-threads: 32

<>

-->Reading Genome (asm, fasta) files from /storage/listeria_latina/caso_goretti/prova_snp/AssemblyPlino/..
|->[OK]
-->Reading Genbank file(s) for reference (.gbk) /storage/listeria_latina/caso_goretti/prova_snp/riferimento/reference.gbk..
|->[OK]
-->Calculating MUMi..

[...]

ParSNP: Preparing to construct global multiple alignment framework

Preparing to verify and process input sequences...
Searching for initial MUM anchors...

    Constructing compressed suffix graph...
    Performing initial search for exact matches in the sequences...

Performing recursive MUM search between MUM anchors...
Filtering spurious matches...
Creating and verifying final LCBs...
Writing output files & aligning LCBs...
Parsnp: Finished core genome alignment
|->[OK]
-->Running PhiPack on LCBs to detect recombination..
|->[SKIP]
ERROR
The following command failed:

/tmp/_MEIHlF43E/harvest -q -o /home/P_2020_12_23_110653922704/parsnp.ggr -x /home/P_2020_12_23_110653922704/parsnp.xmfa -g /storage/riferimento/reference.gbk
Please veryify input data and restart Parsnp. If the problem persists please contact the Parsnp development team.
ERROR

If I run the message I have this result:

-bash: /tmp/_MEIHlF43E/harvest: No such file or directory

if I check the folder:

ls -l /tmp/_MEIHlF43E/
total 0
lrwxrwxrwx 1 user user 33 Dec 23 11:06 harvest -> /tmp/_MEIHlF43E/bin/harvest_linux
lrwxrwxrwx 1 user user 29 Dec 23 11:06 nucmer -> /tmp/_MEIHlF43E/MUMmer/nucmer
lrwxrwxrwx 1 user user 26 Dec 23 11:06 parsnp -> /tmp/_MEIHlF43E/bin/parsnp
lrwxrwxrwx 1 user user 33 Dec 23 11:06 phiprofile -> /tmp/_MEIHlF43E/bin/Profile_linux
lrwxrwxrwx 1 user user 34 Dec 23 11:06 show-coords -> /tmp/_MEIHlF43E/MUMmer/show-coords

but the folder /tmp/_MEIHlF43E/bin/ does not exist.

How can I solve the problem?
Thank you for your help!

Valeria
@valeriaR89 valeriaR89 changed the title Error Error command harvest failed Dec 23, 2020
@bkille
Copy link
Contributor

bkille commented Dec 23, 2020 •
edited
Loading

Hi @valeriaR89, thanks for using Parsnp and reporting this issue! Unfortunately Parsnp v1.2 is no longer supported, so I can't help much with the issue you are currently facing. However, I have seen similar errors that have been fixed by updating to the most recent version of Parsnp.

Parsnp v1.5.4 is available via conda (recommended) as well as through GitHub releases. If you have conda installed on your system, you can install Parsnp by first adding the bioconda channel and then running conda install -c bioconda parsnp

@valeriaR89
Copy link
Author

Thank you for your answer, and happy new year!
I tried to install parsnp via conda following your advice, but in my personal laptop (WSL2) I had this error message "failed with repodata from current_repodata.json, will retry with next repodata source." and it still ask me to install the version 1.2.

In another machine (native linux) I manage to install via conda the version 1.5.3. (using the same way) but now I had the same problem of the issues #88, because the generated parsnp.snps.mblocks is empty and the run stucks. The other file seems to be ok.

the error message:

CRITICAL - The following command failed:
>>$ raxmlHPC-PTHREADS -m GTRCAT -p 12345 -T 8 -s /storage/output/parsnp.snps.mblocks -w /tmp/tmpa4avt5x5 -n OUTPUT
Please veryify input data and restart Parsnp.
If the problem persists please contact the Parsnp development team.

  STDOUT:
  Warning, you specified a working directory via "-w"

Keep in mind that RAxML only accepts absolute path names, not relative ones!

RAxML can't, parse the alignment file as phylip file
it will now try to parse it as FASTA file

TOO FEW SPECIES

The output file list:

ls output/

-rw-rw-r-- 1 users 147 Jan 12 09:36 all.mumi
-rw-rw-r-- 1 users 1.9K Jan 12 09:36 all_mumi.ini
drwxrwsr-x 693 users 16K Jan 12 09:37 blocks/
-rw-rw-r-- 1 users 1.9K Jan 12 09:36 parsnpAligner.ini
-rw-rw-r-- 1 users 3.8K Jan 12 09:38 parsnpAligner.log
-rw-r--r-- 1 users 83 Jan 12 09:38 parsnp.ggr
-rw-rw-r-- 1 users 2.2K Jan 12 09:38 parsnp.rec
-rw-rw-r-- 1 users 0 Jan 12 09:38 parsnp.snps.mblocks
-rw-rw-r-- 1 users 35M Jan 12 09:38 parsnp.xmfa
-rw-rw-r-- 1 users 1.9K Jan 12 09:36 psnn.ini
drwxrwsr-x 2 users 4.0K Jan 12 09:36 tmp/

Thank you

Valeria

@bkille
Copy link
Contributor

bkille commented Jan 13, 2021

@valeriaR89

I tried to install parsnp via conda following your advice, but in my personal laptop (WSL2) I had this error message "failed with repodata from current_repodata.json, will retry with next repodata source." and it still ask me to install the version 1.2.

This most likely means you are missing the bioconda channel setup for conda. Running the commands below should allow you to be able to install the most recent Parsnp version via conda:

conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge

With respect to error mentioned. Could you rerun the same command but with the --verbose flag and attach the output here?

@valeriaR89
Copy link
Author

This is the output:

parsnp -g /storage/snp/riferimento/reference.gbk -d /storage/snp/Assembly/ -o /storage/snp/output -q /storage/snp/Assembly/seq1.fna --verbose -p 8

|--Parsnp 1.5.3--|
For detailed documentation please see --> http://harvest.readthedocs.org/en/latest
10:16:58 - INFO -

SETTINGS:
|-refgenome: /storage/snp/riferimento/reference.gbk
|-genomes:
/storage/snp/Assembly/seq1.fna
/storage/snp/Assembly/seq2.fna
...15 more file(s)...
/storage/snp/Assembly/seq18.fna
/storage/snp/Assembly/seq19.fna
|-aligner: muscle
|-outdir: /storage/snp/output
|-OS: Linux
|-threads: 8

10:16:58 - INFO - <>
10:16:58 - DEBUG - Writing .ini file
10:16:58 - INFO - Recruiting genomes...
10:16:58 - DEBUG - /home/miniconda3/bin/bin/parsnp_core /storage/snp/output/all_mumi.ini
10:17:26 - DEBUG - 0
reference.gbk.fna,Len:2944528,GC:37.981
seq1.fna,Len:2975839,GC:37.9406
seq2.fna,Len:2985834,GC:37.9547
seq3.fna,Len:2990161,GC:38.0781
seq4.fna,Len:2919659,GC:37.9151
seq5.fasta,Len:2847393,GC:37.954
seq6.fasta,Len:2874584,GC:37.9796
seq7.fasta,Len:2871705,GC:38.0356
seq8.fna,Len:2922961,GC:38.0189
seq9.fna,Len:2952686,GC:37.948
seq10.fasta,Len:2889168,GC:38.0001
seq11.fna,Len:3131413,GC:38.0227
seq12.fna,Len:2982833,GC:38.0007
seq13.fna,Len:2974866,GC:37.8808
seq14.fasta,Len:2881583,GC:37.9931
seq15.fna,Len:2961987,GC:37.9747
seq16.fna,Len:2920369,GC:37.9095
seq17.fasta,Len:2859587,GC:37.9685
seq18.fna,Len:2931744,GC:37.9568
seq19.fna,Len:2967347,GC:38.0059
Finished processing input sequences, elapsed time: 2 seconds

             Compressed suffix graph construction elapsed time: 0 seconds

/storage/snp/output
/storage/snp/output/all.mumi
MUMi pairwise distance calculation finished: 22 seconds

10:17:26 - DEBUG -

parsnpAligner:: rapid whole genome SNP typing

ParSNP: Preparing to construct global multiple alignment framework

Preparing to verify and process input sequences...
Calculating mumi distances..

    Constructing compressed suffix graph...
    Calculting pairwise MUMi distances...

10:17:26 - INFO - Running Parsnp multi-MUM search and libMUSCLE aligner...
10:17:26 - DEBUG - /home/miniconda3/bin/bin/parsnp_core /storage/snp/output/parsnpAligner.ini
10:19:15 - DEBUG - 0
reference.gbk.fna,Len:2944528,GC:37.981
seq1.fna,Len:2975839,GC:37.9406
seq2.fna,Len:2985834,GC:37.9547
seq3.fna,Len:2990161,GC:38.0781
seq4.fna,Len:2919659,GC:37.9151
seq5.fasta,Len:2847393,GC:37.954
seq6.fasta,Len:2874584,GC:37.9796
seq7.fasta,Len:2871705,GC:38.0356
seq8.fna,Len:2922961,GC:38.0189
seq9.fna,Len:2952686,GC:37.948
seq10.fasta,Len:2889168,GC:38.0001
seq11.fna,Len:3131413,GC:38.0227
seq12.fna,Len:2982833,GC:38.0007
seq13.fna,Len:2974866,GC:37.8808
seq14.fasta,Len:2881583,GC:37.9931
seq15.fna,Len:2961987,GC:37.9747
seq16.fna,Len:2920369,GC:37.9095
seq17.fasta,Len:2859587,GC:37.9685
seq18.fna,Len:2931744,GC:37.9568
seq19.fna,Len:2967347,GC:38.0059
Finished processing input sequences, elapsed time: 2 seconds

             compressed suffix graph construction elapsed time: 4 seconds

             MUM anchor search elapsed time: 53 seconds

    Finished recursive MUM search, elapsed time: 15 seconds

    Finished filtering spurious matches, elapsed time: 0 seconds

    LCBs created, elapsed time: 20 seconds

    Output files updated, elapsed time: 12 seconds

    See log file for futher details. Total processing time: 109 seconds

10:19:15 - DEBUG -

parsnpAligner:: rapid whole genome SNP typing

ParSNP: Preparing to construct global multiple alignment framework

Preparing to verify and process input sequences...
Searching for initial MUM anchors...

    Constructing compressed suffix graph...
    Performing initial search for exact matches in the sequences...

Performing recursive MUM search between MUM anchors...
Filtering spurious matches...
Creating and verifying final LCBs...
Writing output files & aligning LCBs...
Parsnp: Finished core genome alignment

10:19:15 - DEBUG - harvesttools -q -o /storage/snp/output/parsnp.ggr -x /storage/snp/output/parsnp.xmfa -g /storage/snp/riferimento/reference.gbk
10:19:15 - DEBUG -
10:19:15 - DEBUG -
10:19:15 - DEBUG - harvesttools -q -i /storage/snp/output/parsnp.ggr -S /storage/snp/output/parsnp.snps.mblocks
10:19:15 - DEBUG -
10:19:15 - DEBUG -
10:19:15 - INFO - Reconstructing core genome phylogeny...
10:19:15 - DEBUG - raxmlHPC-PTHREADS -m GTRCAT -p 12345 -T 8 -s /storage/snp/output/parsnp.snps.mblocks -w /tmp/tmplpw5ktct -n OUTPUT
10:19:15 - CRITICAL - The following command failed:
>>$ raxmlHPC-PTHREADS -m GTRCAT -p 12345 -T 8 -s /storage/snp/output/parsnp.snps.mblocks -w /tmp/tmplpw5ktct -n OUTPUT
Please veryify input data and restart Parsnp.
If the problem persists please contact the Parsnp development team.

  STDOUT:
  Warning, you specified a working directory via "-w"

Keep in mind that RAxML only accepts absolute path names, not relative ones!

RAxML can't, parse the alignment file as phylip file
it will now try to parse it as FASTA file

TOO FEW SPECIES

  STDERR:

I notice that in the latest run also the folder "blocks" was empty

@bkille
Copy link
Contributor

bkille commented Jan 18, 2021

@valeriaR89 Thanks for running that. Looking at the output its not immediately clear what's causing the issue. Would you mind running ls -lh /storage/snp/output ? I'd like to see which files are being created/populated.

@valeriaR89
Copy link
Author

ls -lh /storage/snp/output

-rw-rw-r--	1	users	219	Jan	19	08:48	all.mumi
-rw-rw-r--	1	users	2.5K	Jan	19	08:48	all_mumi.ini
drwxrwsr-x	2	users	4.0K	Jan	19	08:48	blocks
-rw-rw-r--	1	users	2.5K	Jan	19	08:48	parsnpAligner.ini
-rw-rw-r--	1	users	5.0K	Jan	19	08:50	parsnpAligner.log
-rw-rw-r--	1	users	0	Jan	19	08:50	parsnp.snps.mblocks
-rw-rw-r--	1	users	50M	Jan	19	08:50	parsnp.xmfa
-rw-rw-r--	1	users	2.5K	Jan	19	08:48	psnn.ini
drwxrwsr-x	2	users	4.0K	Jan	19	08:48	tmp

the folder "tmp" contain the reference sequence reference.gbk.fna, while the folder "blocks" is empty

@bkille
Copy link
Contributor

bkille commented Jan 21, 2021

Hi,

It looks like the parsnp.ggr file is missing, which is causing the .mblocks file to be empty resulting in a failed RAxML run. If your intermediate files are still there, can you try running this command harvesttools -q -o /storage/snp/output/parsnp.ggr -x /storage/snp/output/parsnp.xmfa -g /storage/snp/riferimento/reference.gbk and see if the .ggr file appears (also please attach any output from harvesttools)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bkille @valeriaR89