Dowload data from EMBL-EBI via ftp/sftp or IBM-Aspera

IFB Core Cluster Support IFB Core Cluster
1

I need to download the following folder to the cluster for one of my projects:
Index of /biostudies/fire/E-MTAB-/529/E-MTAB-12529/Files (ebi.ac.uk)

There is this github repo suggesting how to download ,but proposing to download IBM's tool Aspera :

But I cannot download it because of following error:

ERROR: cannot verify download.asperasoft.com's certificate, issued by ‘/C=US/O=Let's Encrypt/CN=R3’:
  Issued certificate has expired.

Is this error depending on the server of Aspera, or is there sth I could do to skip it ?

Otherwise, is it possible to download securely via s-ftp , with a linux-equivalent of a script like this:

Run this file in cmd shell with this command:  ftp -i -s:./<filename>
open ftp.ebi.ac.uk
anonymous
youremail@domain.com
cd biostudies/fire/E-MTAB-/529/E-MTAB-12529/Files
binary
mget "E-MTAB-12529.idf.txt"
mget "S1_droplets_features.tsv"
mget "S1_droplets_barcodes.tsv"
mget "Gene_Annotations.txt"
#...
disconnect
quit

?
Thanks for any suggestion..

2

Hello,

Aspera is already available ont he cluster (no need to download it). Simply:

module load aspera-cli/3.9.6

So, I would try with aspera commmand line (faster) and otherwise with ftp like suggested.
Data are not so big, so wget or curl should also work.

1 « J'aime »
3

Hello @dbenaben ,
Thanks for the answer. I need one more information. The script suggested by EBI for downloading with aspera ->

Download_EBI-EMBL_data_with_Aspera
Download_EBI-EMBL_data_with_Aspera1562×442 49 KB

calls the main script :

asperaweb_id_dsa.openssh

but I cannot find where this is located in the cluster, since the aspera is created via singularity image.

Could you indicate me where is the source path of this script please ?

4

No need to specify the file (just remove -i option).
The file is located in: /usr/local/aspera/cli/etc/asperaweb_id_dsa.openssh

But in my test, I was not successful to download some data with ascp (Failed to open TCP connection for SSH).
Data is quiet small, so if it not works, I recommend to download the data as a zip file in your computer and then copy to the cluster. Or maybe just use FTP (you can use ftp or lftp commands).

5

It worked with Aspera finally !

module load aspera-cli/3.9.6

ascp -P33001 -i "/usr/local/aspera/cli/etc/asperaweb_id_dsa.openssh" --host=fasp.ebi.ac.uk --user=bsaspera --mode=recv  "fire/E-MTAB-/529/E-MTAB-12529/Files/S1_droplets_features.tsv" ./EMTAB_12529/
ascp -P33001 -i "/usr/local/aspera/cli/etc/asperaweb_id_dsa.openssh" --host=fasp.ebi.ac.uk --user=bsaspera --mode=recv  "fire/E-MTAB-/529/E-MTAB-12529/Files/S1_droplets_barcodes.tsv" ./EMTAB_12529/

and with LFTP:

cd /shared/projects/my_single_cell_project/raw_data/

lftp https://ftp.ebi.ac.uk/biostudies/fire/E-MTAB-/529/E-MTAB-12529/Files/
mget "S1_droplets_features.tsv" ;
mget "S1_droplets_barcodes.tsv"; 
#downloading....
bye #for exiting
6

Great. Thank you for your feedback.