Problème lancement MultiQC

IFB Core Cluster Tool or data bank request
,
1

Hello la team,

MultiQC ne s'exécute pas bien sur le cluster, que ce soit en module load ou via conda, pour les deux versions mises à disposition (1.6 et 1.7). J'ai déjà eu un pb analogue en local, à priori ça vient de la version de python installée avec par défaut via conda, fastqc en a besoin d'une plus ancienne (si mes souvenirs sont bons la 2.4 tournait bien, peut-être la 2.7 également). Actuellement sur le cluster c'est la 3.6 qui se lance.
Merci d'avance !

2

Pouvez-vous nous donner plus de précision ?
Comme un exemple de ligne de commande ?

FastQC a été utilisé une bonne 100 aine de fois sans problème cette semaine, nous sommes en pleine école EBAII.

3

FastQC ? Qui a parlé de FastQC ? Il s'agit de MultiQC voyons, Gildas, fais attention ...!
:stuck_out_tongue_winking_eye:

(désolé pour l'erreur)

4

Bonjour Bastien,

Les warning renvoyés par MultiQC sont sans danger. C'est pas très agréable mais pas impactant sur le fonctionnement.

$ multiqc --version
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:45: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  configs = yaml.load(f)
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:51: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  sp = yaml.load(f)
multiqc, version 1.7

Voir l'issue:

On va donc attendre la v1.8 si ça vous va :slight_smile:

Bonne journée

5

Pour moi, c'est juste des warnings.
C'est moche mais il semble tourner.

$ module load multiqc/1.7
$ multiqc .
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:45: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  configs = yaml.load(f)
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:51: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  sp = yaml.load(f)
[INFO   ]         multiqc : This is MultiQC v1.7
[INFO   ]         multiqc : Template    : default
[INFO   ]         multiqc : Searching '.'
[INFO   ]          fastqc : Found 2 reports
[INFO   ]         multiqc : Compressing plot data
[INFO   ]         multiqc : Report      : multiqc_report.html
[INFO   ]         multiqc : Data        : multiqc_data
[INFO   ]         multiqc : MultiQC complete
6

Je viens de retenter sur une session fraîche, j'ai toujours le même problème (mais cette fois-ci je vous mets le log) :

(base) [bjob@cpu-node-9 tgondii_phylo]$ conda deactivate
[bjob@cpu-node-9 tgondii_phylo]$ module purge
[bjob@cpu-node-9 tgondii_phylo]$ module load multiqc/1.7
[bjob@cpu-node-9 tgondii_phylo]$ multiqc -h
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:45: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
configs = yaml.load(f)
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:51: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
sp = yaml.load(f)
Traceback (most recent call last):
File "/shared/software/miniconda//envs/multiqc-1.7/bin/multiqc", line 766, in
multiqc()
File "/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/click/core.py", line 696, in main
_verify_python3_env()
File "/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/click/_unicodefun.py", line 124, in _verify_python3_env
' mitigation steps.' + extra
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult Python 3 Support — Click Documentation (7.x) for mitigation steps.

This system lists a couple of UTF-8 supporting locales that
you can pick from. The following suitable locales were
discovered: aa_DJ.utf8, aa_ER.utf8, aa_ET.utf8, af_ZA.utf8, am_ET.utf8, an_ES.utf8, ar_AE.utf8, ar_BH.utf8, ar_DZ.utf8, ar_EG.utf8, ar_IN.utf8, ar_IQ.utf8, ar_JO.utf8, ar_KW.utf8, ar_LB.utf8, ar_LY.utf8, ar_MA.utf8, ar_OM.utf8, ar_QA.utf8, ar_SA.utf8, ar_SD.utf8, ar_SY.utf8, ar_TN.utf8, ar_YE.utf8, as_IN.utf8, ast_ES.utf8, ayc_PE.utf8, az_AZ.utf8, be_BY.utf8, bem_ZM.utf8, ber_DZ.utf8, ber_MA.utf8, bg_BG.utf8, bho_IN.utf8, bn_BD.utf8, bn_IN.utf8, bo_CN.utf8, bo_IN.utf8, br_FR.utf8, brx_IN.utf8, bs_BA.utf8, byn_ER.utf8, ca_AD.utf8, ca_ES.utf8, ca_FR.utf8, ca_IT.utf8, crh_UA.utf8, cs_CZ.utf8, csb_PL.utf8, cv_RU.utf8, cy_GB.utf8, da_DK.utf8, de_AT.utf8, de_BE.utf8, de_CH.utf8, de_DE.utf8, de_LU.utf8, doi_IN.utf8, dv_MV.utf8, dz_BT.utf8, el_CY.utf8, el_GR.utf8, en_AG.utf8, en_AU.utf8, en_BW.utf8, en_CA.utf8, en_DK.utf8, en_GB.utf8, en_HK.utf8, en_IE.utf8, en_IN.utf8, en_NG.utf8, en_NZ.utf8, en_PH.utf8, en_SG.utf8, en_US.utf8, en_ZA.utf8, en_ZM.utf8, en_ZW.utf8, es_AR.utf8, es_BO.utf8, es_CL.utf8, es_CO.utf8, es_CR.utf8, es_CU.utf8, es_DO.utf8, es_EC.utf8, es_ES.utf8, es_GT.utf8, es_HN.utf8, es_MX.utf8, es_NI.utf8, es_PA.utf8, es_PE.utf8, es_PR.utf8, es_PY.utf8, es_SV.utf8, es_US.utf8, es_UY.utf8, es_VE.utf8, et_EE.utf8, eu_ES.utf8, fa_IR.utf8, ff_SN.utf8, fi_FI.utf8, fil_PH.utf8, fo_FO.utf8, fr_BE.utf8, fr_CA.utf8, fr_CH.utf8, fr_FR.utf8, fr_LU.utf8, fur_IT.utf8, fy_DE.utf8, fy_NL.utf8, ga_IE.utf8, gd_GB.utf8, gez_ER.utf8, gez_ET.utf8, gl_ES.utf8, gu_IN.utf8, gv_GB.utf8, ha_NG.utf8, he_IL.utf8, hi_IN.utf8, hne_IN.utf8, hr_HR.utf8, hsb_DE.utf8, ht_HT.utf8, hu_HU.utf8, hy_AM.utf8, ia_FR.utf8, id_ID.utf8, ig_NG.utf8, ik_CA.utf8, is_IS.utf8, it_CH.utf8, it_IT.utf8, iu_CA.utf8, iw_IL.utf8, ja_JP.utf8, ka_GE.utf8, kk_KZ.utf8, kl_GL.utf8, km_KH.utf8, kn_IN.utf8, ko_KR.utf8, kok_IN.utf8, ks_IN.utf8, ku_TR.utf8, kw_GB.utf8, ky_KG.utf8, lb_LU.utf8, lg_UG.utf8, li_BE.utf8, li_NL.utf8, lij_IT.utf8, lo_LA.utf8, lt_LT.utf8, lv_LV.utf8, mag_IN.utf8, mai_IN.utf8, mg_MG.utf8, mhr_RU.utf8, mi_NZ.utf8, mk_MK.utf8, ml_IN.utf8, mn_MN.utf8, mni_IN.utf8, mr_IN.utf8, ms_MY.utf8, mt_MT.utf8, my_MM.utf8, nb_NO.utf8, nds_DE.utf8, nds_NL.utf8, ne_NP.utf8, nhn_MX.utf8, niu_NU.utf8, niu_NZ.utf8, nl_AW.utf8, nl_BE.utf8, nl_NL.utf8, nn_NO.utf8, nr_ZA.utf8, nso_ZA.utf8, oc_FR.utf8, om_ET.utf8, om_KE.utf8, or_IN.utf8, os_RU.utf8, pa_IN.utf8, pa_PK.utf8, pap_AN.utf8, pl_PL.utf8, ps_AF.utf8, pt_BR.utf8, pt_PT.utf8, ro_RO.utf8, ru_RU.utf8, ru_UA.utf8, rw_RW.utf8, sa_IN.utf8, sat_IN.utf8, sc_IT.utf8, sd_IN.utf8, se_NO.utf8, shs_CA.utf8, si_LK.utf8, sid_ET.utf8, sk_SK.utf8, sl_SI.utf8, so_DJ.utf8, so_ET.utf8, so_KE.utf8, so_SO.utf8, sq_AL.utf8, sq_MK.utf8, sr_ME.utf8, sr_RS.utf8, ss_ZA.utf8, st_ZA.utf8, sv_FI.utf8, sv_SE.utf8, sw_KE.utf8, sw_TZ.utf8, szl_PL.utf8, ta_IN.utf8, ta_LK.utf8, te_IN.utf8, tg_TJ.utf8, th_TH.utf8, ti_ER.utf8, ti_ET.utf8, tig_ER.utf8, tk_TM.utf8, tl_PH.utf8, tn_ZA.utf8, tr_CY.utf8, tr_TR.utf8, ts_ZA.utf8, tt_RU.utf8, ug_CN.utf8, uk_UA.utf8, unm_US.utf8, ur_IN.utf8, ur_PK.utf8, ve_ZA.utf8, vi_VN.utf8, wa_BE.utf8, wae_CH.utf8, wal_ET.utf8, wo_SN.utf8, xh_ZA.utf8, yi_US.utf8, yo_NG.utf8, yue_HK.utf8, zh_CN.utf8, zh_HK.utf8, zh_SG.utf8, zh_TW.utf8, zu_ZA.utf8

Click discovered that you exported a UTF-8 locale
but the locale system could not pick up from it because
it does not exist. The exported locale is "C.UTF-8" but it
is not supported

7

Salut Bastien,

Tu dois avoir un bashrc ou quelque chose dans ton PATH de bizarre car je viens de tester et j'ai le même rendu que donné par Gildas ci dessous (un warning mais une aide qui s'affiche très bien, et je l'ai déjà lancé sur des données).

Elodie

8

Je ne reproduis pas l'erreur:

dbenaben@cpu-node-9:~$ module load multiqc/1.7
dbenaben@cpu-node-9:~$ multiqc -h
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:45: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  configs = yaml.load(f)
/shared/mfs/data/software/miniconda/envs/multiqc-1.7/lib/python3.6/site-packages/multiqc/utils/config.py:51: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  sp = yaml.load(f)
Usage: multiqc [OPTIONS] <analysis directory>

  MultiQC aggregates results from bioinformatics analyses across many
  samples into a single report.

  It searches a given directory for analysis logs and compiles a HTML
  report. It's a general use tool, perfect for summarising the output from
  numerous bioinformatics tools.

  To run, supply with one or more directory to scan for analysis results. To
  run here, use 'multiqc .'

  See http://multiqc.info for more details.

  Author: Phil Ewels (http://phil.ewels.co.uk)

Options:
  -f, --force                     Overwrite any existing reports
  -d, --dirs                      Prepend directory to sample names
  [...]
  --version                       Show the version and exit.
  -h, --help                      Show this message and exit.

Pouvez-vous nous renvoyer la sortie de env et locale (en message privé si vous prèfèrez) ?
Idem, je soupçonne un pb avec l'environnement (.bashrc ou autre).

9

Re,

Merci pour vos suggestions, j'ai checké mon .bashrc, la seule chose que j'avais éditée c'est l'ajout d'un TMOUT=0 (dans l'espoir vain de désactiver les timeouts fréquents)

Voici mon env :

(base) [bjob@clust-slurm-client ~]$ env
XDG_SESSION_ID=14795
HOSTNAME=clust-slurm-client.ifb.local
_LMFILES__modshare=/shared/software/modulefiles/multiqc/1.7:1
MODULEPATH_modshare=/shared/software/modulefiles:1
MODULES_CMD=/shared/software/modules/4.1.4/libexec/modulecmd.tcl
TERM=xterm-256color
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=193.52.39.1 61526 22
CONDA_SHLVL=1
CONDA_PROMPT_MODIFIER=(base)
SSH_TTY=/dev/pts/190
USER=bjob
LD_LIBRARY_PATH=/shared/software/miniconda//envs/multiqc-1.7/lib
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:.tar=38;5;9:.tgz=38;5;9:.arc=38;5;9:.arj=38;5;9:.taz=38;5;9:.lha=38;5;9:.lz4=38;5;9:.lzh=38;5;9:.lzma=38;5;9:.tlz=38;5;9:.txz=38;5;9:.tzo=38;5;9:.t7z=38;5;9:.zip=38;5;9:.z=38;5;9:.Z=38;5;9:.dz=38;5;9:.gz=38;5;9:.lrz=38;5;9:.lz=38;5;9:.lzo=38;5;9:.xz=38;5;9:.bz2=38;5;9:.bz=38;5;9:.tbz=38;5;9:.tbz2=38;5;9:.tz=38;5;9:.deb=38;5;9:.rpm=38;5;9:.jar=38;5;9:.war=38;5;9:.ear=38;5;9:.sar=38;5;9:.rar=38;5;9:.alz=38;5;9:.ace=38;5;9:.zoo=38;5;9:.cpio=38;5;9:.7z=38;5;9:.rz=38;5;9:.cab=38;5;9:.jpg=38;5;13:.jpeg=38;5;13:.gif=38;5;13:.bmp=38;5;13:.pbm=38;5;13:.pgm=38;5;13:.ppm=38;5;13:.tga=38;5;13:.xbm=38;5;13:.xpm=38;5;13:.tif=38;5;13:.tiff=38;5;13:.png=38;5;13:.svg=38;5;13:.svgz=38;5;13:.mng=38;5;13:.pcx=38;5;13:.mov=38;5;13:.mpg=38;5;13:.mpeg=38;5;13:.m2v=38;5;13:.mkv=38;5;13:.webm=38;5;13:.ogm=38;5;13:.mp4=38;5;13:.m4v=38;5;13:.mp4v=38;5;13:.vob=38;5;13:.qt=38;5;13:.nuv=38;5;13:.wmv=38;5;13:.asf=38;5;13:.rm=38;5;13:.rmvb=38;5;13:.flc=38;5;13:.avi=38;5;13:.fli=38;5;13:.flv=38;5;13:.gl=38;5;13:.dl=38;5;13:.xcf=38;5;13:.xwd=38;5;13:.yuv=38;5;13:.cgm=38;5;13:.emf=38;5;13:.axv=38;5;13:.anx=38;5;13:.ogv=38;5;13:.ogx=38;5;13:.aac=38;5;45:.au=38;5;45:.flac=38;5;45:.mid=38;5;45:.midi=38;5;45:.mka=38;5;45:.mp3=38;5;45:.mpc=38;5;45:.ogg=38;5;45:.ra=38;5;45:.wav=38;5;45:.axa=38;5;45:.oga=38;5;45:.spx=38;5;45:.xspf=38;5;45:
CONDA_EXE=/shared/mfs/data/software/miniconda/bin/conda
ENV=/shared/software/modules/4.1.4/init/profile.sh
CPATH=/shared/software/miniconda//envs/multiqc-1.7/include
_CE_CONDA=
MAIL=/var/spool/mail/bjob
PATH=/shared/software/miniconda//envs/multiqc-1.7/bin:/shared/mfs/data/software/miniconda/bin:/shared/mfs/data/software/miniconda/condabin:/shared/software/sinteractive:/shared/software/modules/4.1.4/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/go/1.11.5/bin:/opt/go/packages/bin:/shared/home/bjob/.local/bin:/shared/home/bjob/bin
CONDA_PREFIX=/shared/mfs/data/software/miniconda
PWD=/shared/home/bjob
LMFILES=/shared/software/modulefiles/multiqc/1.7
LANG=C.UTF-8
MODULEPATH=/shared/software/modulefiles
LOADEDMODULES=multiqc/1.7
CPATH_modshare=/shared/software/miniconda//envs/multiqc-1.7/include:1
_CE_M=
HISTCONTROL=ignoredups
PATH_modshare=/shared/software/sinteractive:1:/usr/bin:1:/shared/home/bjob/.local/bin:1:/usr/local/bin:1:/shared/mfs/data/software/miniconda/condabin:1:/shared/software/modules/4.1.4/bin:1:/opt/go/1.11.5/bin:1:/shared/home/bjob/bin:1:/shared/software/miniconda//envs/multiqc-1.7/bin:1:/usr/sbin:1:/opt/go/packages/bin:1:/usr/local/sbin:1:/shared/mfs/data/software/miniconda/bin:1
SHLVL=1
HOME=/shared/home/bjob
GOROOT=/opt/go/1.11.5
BASH_ENV=/shared/software/modules/4.1.4/init/bash
CONDA_PYTHON_EXE=/shared/mfs/data/software/miniconda/bin/python
LOGNAME=bjob
LD_LIBRARY_PATH_modshare=/shared/software/miniconda//envs/multiqc-1.7/lib:1
SSH_CONNECTION=193.52.39.1 61526 192.54.201.181 22
MODULESHOME=/shared/software/modules/4.1.4
CONDA_DEFAULT_ENV=base
LESSOPEN=||/usr/bin/lesspipe.sh %s
GOPATH=/opt/go/packages
LOADEDMODULES_modshare=multiqc/1.7:1
XDG_RUNTIME_DIR=/run/user/100071
BASH_FUNC_module()=() { _moduleraw "$@" 2>&1
}
BASH_FUNC_switchml()=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /shared/software/modules/4.1.4/libexec/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /shared/software/modules/4.1.4/libexec/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /shared/software/modules/4.1.4/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}
BASH_FUNC__moduleraw()=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
vx)
set +vx;
_mlshdbg='vx'
;;
v)
set +v;
_mlshdbg='v'
;;
x)
set +x;
mlshdbg='x'
;;
)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##[!A-Za-z0-9]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "eval 'echo ${'$_mlv'+x}'" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='eval 'echo ${'$_mlv'}'' ";
fi;
mlrv="MODULES_RUNENV${_mlv}";
_mlre="${_mlre:-}${_mlv}='eval 'echo ${'$_mlrv':-}'' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
eval eval ${_mlre}/usr/bin/tclsh /shared/software/modules/4.1.4/libexec/modulecmd.tcl bash '"$@"';
else
eval /usr/bin/tclsh /shared/software/modules/4.1.4/libexec/modulecmd.tcl bash "$@";
fi;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}
_=/usr/bin/env

et voici mon locale

(base) [bjob@clust-slurm-client ~]$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=C.UTF-8
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=
(base) [bjob@clust-slurm-client ~]$

10

C'est le paramètre de locale utilisé qui pose problème à python/multiqc.

Il faut donc définir la locale (variable LANG) comme suit:

export LANG=en_US.UTF-8

Soit pour le traitement avec multiqc, soit pour chaque session en le spécifiant/modifiant dans ton .bashrc.

11

Merci ça marche !
Une idée de ce qui a modifié cette variable d'environnement ?

12

C'est la valeur par défaut. Je ne connais pas de cas où la variable est modifié.
Si jamais vous trouvez, tenez-nous au courant !

Bonne après-midi

13

Pouquoi es-tu connecté sur un node ?