Problem installing a git tool : libhts, libbzip2-dev missing

Hello, I need to test this tool : https://majiq.biociphers.org/
It asks to register by e-mail in order to see the installation page.
Basically the instructions say :

Before installation:

$ export HTSLIB_LIBRARY_DIR=/path/to/htslib/lib
$ export HTSLIB_INCLUDE_DIR=/path/to/htslib/include

#! Version of htslib > 1.10 required !

Installation:

$ python3 -m venv env
$ source env/bin/activate
$ pip install git+https://bitbucket.org/biociphers/majiq_academic.git

So I need to install a newer version of htslib :

curl -OL https://github.com/samtools/htslib/releases/download/1.13/htslib-1.13.tar.bz2
tar -xf htslib-1.13.tar.bz2
cd htslib-1.13
./configure --prefix=$HOME/Tools_Packages/htslib-1.13

...but ....

configure: error: ***libbzip2*** development files not found

The CRAM format may use bzip2 compression, which is implemented in HTSlib 
by using compression routines from libbzip2 <http://www.bzip.org/>.

Building HTSlib requires libbzip2 development files to be installed on the
build machine; you may need to ensure a package such as libbz2-dev (on Debian
or Ubuntu Linux) or bzip2-devel (on RPM-based Linux distributions or Cygwin)
is installed.

But ,when I try :
wget http://vault.centos.org/7.9.2009/os/Source/SPackages/bzip2-1.0.6-13.el7.src.rpm
or :
yum install bzip2

I get following message:

There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   /shared/ifbstor1/software/miniconda/envs/python-3.9/lib/liblzma.so.5: version `XZ_5.1.2alpha' not found (required by /lib64/librpmio.so.3)

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
2.7.5 (default, Nov 16 2020, 22:23:17)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

A bit tired of all these dependencies and different versions requirements ! :pensive: Don't know what to do anymore ! :tired_face:
Could you verify if newer version of htslib & bzip can be installed Globally..
Or, if I should do it locally, how ? I am not familiar with "yum" installations.

Je ne l'utilise jamais donc je ne sais pas. Vous pouvez essayer avec load module pour trouver la nouvelle version.

module avail

Voici la sortie.

-------------------------------------- /shared/software/modulefiles ---------------------------------------
abyss/2.2.1                 h5py/2.10.0                 pindel/0.2.5b9                ucsc-fatotwobit/377  
admixture/1.3.0             hicstuff/2.3.0              pipmir/1.1                    umap-learn/0.4.6     
adxv/1.9.14                 hifiasm/0.15.1              plink/1.90b4                  umi_tools/1.0.1      
albula/3.2.0                hisat2/2.1.0                plink/1.90b6.18               unicycler/0.4.8      
alientrimmer/0.4.1          hisat2/2.2.1                plink2/2.00a2.3               use.own              
alientrimmer/0.4.2          hmmer/3.2.1                 porechop/0.2.3_seqan2.1.1     valet/1.0            
alphafold/2.0.1             hmmlearn/0.2.4              ppanggolin/1.0.1              vamb/3.0.2           
alphafold/2.1.1             homer/4.9.1                 primer3/2.5.0                 varscan/2.4.3        
angsd/0.935                 homer/4.10                  prodigal/2.6.3                varscan/2.4.4        
anvio/6.1                   homer/4.11                  prokka/1.14.6                 vcftools/0.1.16      
anvio/6.2                   htseq/0.9.1                 psipred/4.01                  velocyto.py/0.17.17  
arcs/1.1.0                  htseq/0.11.2                pyani/0.2.10                  velvet/1.2.10        
aria2/1.36.0                htseq/0.11.3                pycrac/1.4.6                  vep/101.0            
ascatngs/2.1.0              htseq/0.12.4                pyega3/3.0.39                 virsorter/1.0.6      
aspera-cli/3.9.6            htseq/0.13.5                pygtftk/1.2.2                 virsorter/2.0        
assemblytics/1.2.1          htslib/1.7                  pygtftk/1.2.7                 virsorter/2.0.beta   
atlas/0.9.9                 htslib/1.9                  pypy/2.7-5.10.0               virsorter/2.1        
augustus/3.3.3              humann/3.0.0.alpha.3        pypy/3.5-7.0.0                virsorter/2.2        
bam2fastx/1.3.0             idba/1.1.3                  pyranges/0.0.98               vsearch/2.14.1       
barrnap/0.9                 idr/2.0.4.2                 pysam/0.16.0                  vsearch/2.18.0       
bc/1.07.1                   igraph/0.9.4                pyscenic/0.9.19               wolfpsort/0.2        
bcftools/1.9                igv/2.8.11                  pyscenic/0.10.0               wtdbg/2.5            
bcftools/1.10.2             igvtools/2.5.3              pyslim/0.501                  xgboost/0.82         
bcl2fastq/2.20.0            ima3/1.12                   python/2.7                    zenodo_get/1.3.2     
beagle-lib/3.1.2            imagemagick/7.1.0_5         python/3.7                    
beast/1.10.4                imod/4.11.2                 python/3.9                    
beast2/2.6.3                imoka/1.0                   pytorch/1.7.1                 
bedtools/2.26.0             instagraal/0.1.2            pytorch/1.10.0                
bedtools/2.27.1             interproscan/5.51-85.0      qdd/3.1.2                     
bedtools/2.29.2             interproscan/5.52-86.0      qiime2/2019.1.0               
berokka/0.2.3               iqtree/2.0.3                qiime2/2021.2.0               
bio-vcf/0.9.2               irods/4.2.10                qiime2/2021.8.0               
bioblend/0.16.0             isoseq3/3.3.0               qualimap/2.2.2b               
bioconductor-dada2/1.14.0   java-jdk/8.0.112            quast/5.0.2                   
biopet-seqstat/1.0.1        jcvi/1.1.17                 r-eacon/0.3.4_1               
bismark/0.22.2              jellyfish/0.8.2             r-eacon/0.3.5                 
blast-legacy/2.2.26         jpredapi/1.5.6              r-sartools/1.7.4              
blast/2.2.31                jq/1.6                      r/3.5.1                       
blast/2.7.1                 julia/1.5.3                 r/3.6.3                       
blast/2.9.0                 julia/1.6.2                 r/4.0.0                       
blobtools/1.1.1             kaiju/1.7.3                 r/4.0.2                       
bmge/1.12                   kallisto/0.44.0             r/4.0.3                       
bokeh/1.4.0                 kallisto/0.46.2             r/4.1.0                       
bowtie/1.2.2                kat/2.4.2                   racon/1.4.20                  
bowtie/1.2.3                keras/2.2.4                 ragout/2.3                    
bowtie/1.3.1                king/2.2.4                  ragtag/1.0.2                  
bowtie2/2.3.4.3             kmer-jellyfish/2.3.0        raven-assembler/1.4.0         
bowtie2/2.3.5               kodoja/0.0.10               raxml-ng/0.9.0                
bowtie2/2.4.1               kofamscan/1.3.0             raxml/8.2.12                  
:...skipping...

Essayez à nouveau d'installer with module load

J' ai déjà précisé que la version qui est installé ne suffit pas ! je connais la commande "module av" , et ça montre que la dernière version de htslib est la 1.9...S' il vous plaît de vérifier que vous avez compris la question avant de répondre.
J ai essayé d installer sur mon espace local/utilisateur, mais il me manque bzip2, que je sais pas non plus comment installer.

Désolé, je n'aurai pas le temps de regarder ça maintenant. Mais je fais
d'accord qu'il semble qu'il se passe plus de choses que je ne le pensais à l'origine.

wget http://vault.centos.org/7.9.2009/os/Source/SPackages/bzip2-1.0.6-13.el7.src.rpm

C'est pour installer les sources du paquet bzip2, afin de reconstruire les rpms, ce n'est pas ce que vous voulez faire...

yum install bzip2

cela demande des droits root et le paquet serait bzip2-devel pas bzip2. Ce serait aux admins du cluster de l'installer.

There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   /shared/ifbstor1/software/miniconda/envs/python-3.9/lib/liblzma.so.5: version `XZ_5.1.2alpha' not found (required by /lib64/librpmio.so.3)

yum est un programme en python (/usr/bin/python de CentOS: python2.7) mais votre environnment est visiblement un python 3.9 qui n'est pas compatible... Et c'est pas pas utilisable par un utilisateur normal.

1 « J'aime »

Hello Maria,

Sorry, but we will not modify system for any tools. Because, all tools have different need, and so we can't determine which version of which library is the most important.

But the good news, is that we have conda and singularity installed, and both can be used to customize your environement as you need.

For example with conda:

  • create a file env.yml with the following content
name: htslib
channels:
  - conda-forge
  - bioconda
  - defaults

dependencies:
  - bioconda::htslib=1.13

  • Then run conda to create the env:
conda env create -v -f ./env.yml
  • It will show an activate command you will be able to use after the env creation to load your tool:
conda activate htslib
  • And you can verify it work by running:
$ htsfile --version
htsfile (htslib) 1.13
Copyright (C) 2021 Genome Research Ltd.


  • ( if conda does not work on first launch you may need to run once:
/shared/software/miniconda/bin/conda init

then logout of the cluster (close cnx) and login again)

  • After some time using/learning conda, maybe you will want to try something like this for the env.yml file :
name: majiq
channels:
  - conda-forge
  - bioconda
  - defaults

dependencies:
  - bioconda::htslib=1.13
  - python=3.8
  - compilers # for gcc
  - setuptools
  - pip
  - pip:
      - git+https://bitbucket.org/biociphers/majiq_academic.git
conda env create -v -f ./env.yml
conda activate majiq
$ voila --help
usage: voila [-h] [-v] {tsv,view,modulize,filter,split,recombine} ...

VOILA is a visualization package for Alternative Local Splicing Events.

positional arguments:
  {tsv,view,modulize,filter,split,recombine}
    tsv                 Generate tsv output for the supplied files.
    view                Start service to view the visualization for the supplied files.
    modulize            Modulize splicing events and generate a breakdown of the modulization in multiple TSV files.
    filter              Make truncated versions of the input files of a more manageable file size for easier
                        collaboration.
    split               Split classifier input dataset (splicegraph and voila files) into <N> equal parts, for the
                        purpose of running on a compute cluster
    recombine           Used to combine output from a `voila split` run, after all initial runs are complete

optional arguments:
  -h, --help            show this help message and exit
  -v                    show program's version number and exit

:wink:

3 « J'aime »

And also, fyi, last htslib are being installed as module on the cluster :

2 « J'aime »

Merci ! I have it ! You are amazing !

Merci beaucoup pour le détail de l'installation dans un environnement conda, cela fonctionne bien pour majiq