Problème aléatoire chargement module python/3.9

Bonjour,

En lançant un job array, j'ai noté depuis hier soir que certains jobs ne parviennent pas, apparemment, à charger le module python/3.9 (module load python/3.9), amenant ces jobs à planter.
Le phénomène a l'air aléatoire.

Voir par exemple le fichier :

/shared/ifbstor1/home/dfilloux/work/F146/Trimming.sh.45485475
############################################################################
Error processing line 1 of /shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/site-packages/google_auth-1.35.0-py3.9-nspkg.pth:

Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/site.py", line 169, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/importlib/util.py", line 2, in <module>
    from . import abc
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/importlib/abc.py", line 17, in <module>
    from typing import Protocol, runtime_checkable
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/typing.py", line 21, in <module>
    import collections
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/collections/__init__.py", line 36, in <module>
    from keyword import iskeyword as _iskeyword
ImportError: cannot import name 'iskeyword' from 'keyword' (/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/keyword.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/site.py", line 589, in <module>
    main()
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/site.py", line 576, in main
    known_paths = addsitepackages(known_paths)
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/site.py", line 359, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/site.py", line 208, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/site.py", line 179, in addpackage
    import traceback
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/traceback.py", line 3, in <module>
    import collections
  File "/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/collections/__init__.py", line 36, in <module>
    from keyword import iskeyword as _iskeyword
ImportError: cannot import name 'iskeyword' from 'keyword' (/shared/ifbstor1/software/miniconda/envs/python-pytorch-tensorflow-3.9-1.11.0-2.6.2/lib/python3.9/keyword.py)
mv: cannot stat 'F146_Trimming/F146_Trimmed-b1.fastq.fa.4_K-mer_36.tab': No such file or directory
cut: F146_Trimming/F146_Trimmed-b1.fastq_Hashed.tab.4: No such file or directory
awk: fatal: cannot open file `F146_Trimming/F146_Trimmed-b1.fastq_Hashed.tab.4' for reading (No such file or directory)
rm: cannot remove 'F146_Trimming/F146_Trimmed-b1.fastq_Hashed.tab.4': No such file or directory

Bonjour,

Le problème ne viendrait pas plutôt d'un fichier absent ?

F146_Trimming/F146_Trimmed-b1.fastq.fa.4_K-mer_36.tab': No such file or directory

Non, je ne crois pas.
Ce fichier n'est pas trouvé car il n'a pas pu être créé par suite à l'échec de chargement du module python.
Voir lignes 61 à 90 du fichier /shared/ifbstor1/home/dfilloux/work/F146/Trimming.sh

J'ai relancé le script pour voir si cela se reproduit

Bonjour,

Cette fois-ci, aucun problème constaté sur les quelques milliers de jobs lancés.
Le problème a dû être ponctuel.

Je pense qu'on peut classer l'affaire.... :grin:

Ca marche. Merci pour l'info et le retour.