Build Failure - Dependency Paths Not Recognized For Zlib-ng And Htslib

by ADMIN 71 views

Issue Summary

The build process fails to recognize properly installed zlib-ng and htslib libraries located in /opt directories. The Makefile hardcodes relative paths (zlib-ng/zlib.h) instead of respecting absolute paths or proper library installations. This makes the software incompatible with standard Linux package management practices and containerized workflows where dependencies are installed in non-relative paths.

Reproduction Steps

To reproduce this issue, follow these steps:

1. Installed dependencies to standard locations:

# zlib-ng installation
/opt/zlib-ng/v2.2.4
|-- include
|   |-- zconf-ng.h
|   |-- zlib-ng.h
|   `-- zlib_name_mangling-ng.h
|-- lib
|   |-- libz-ng.a
|   |-- libz-ng.so -> libz-ng.so.2.2.4
|   |-- libz-ng.so.2 -> libz-ng.so.2.2.4
|   |-- libz-ng.so.2.2.4
|   `-- pkgconfig
|       `-- zlib-ng.pc
`-- share
    `-- man
        `-- man3

# htslib installation
/opt/htslib/v1.21/
|-- bin
|   |-- annot-tsv
|   |-- bgzip
|   |-- htsfile
|   `-- tabix
|-- include
|   `-- htslib
|       |-- bgzf.h
|       |-- cram.h
|       |-- faidx.h
|       |-- hfile.h
|       |-- hts.h
|       |-- hts_defs.h
|       |-- hts_endian.h
|       |-- hts_expr.h
|       |-- hts_log.h
|       |-- hts_os.h
|       |-- kbitset.h
|       |-- kfunc.h
|       |-- khash.h
|       |-- khash_str2int.h
|       |-- klist.h
|       |-- knetfile.h
|       |-- kroundup.h
|       |-- kseq.h
|       |-- ksort.h
|       |-- kstring.h
|       |-- regidx.h
|       |-- sam.h
|       |-- synced_bcf_reader.h
|       |-- tbx.h
|       |-- thread_pool.h
|       |-- vcf.h
|       |-- vcf_sweep.h
|       `-- vcfutils.h
|-- lib
|   |-- libhts.a
|   |-- libhts.so -> libhts.so.1.21
|   |-- libhts.so.1.21
|   |-- libhts.so.3 -> libhts.so.1.21
|   `-- pkgconfig
|       `-- htslib.pc
`-- share
    `-- man
        |-- man1
        |   |-- annot-tsv.1
        |   |-- bgzip.1
        |   |-- htsfile.1
        |   `-- tabix.1
        |-- man5
        |   |-- faidx.5
        |   |-- sam.5
        |   `-- vcf.5
        `-- man7
            `-- htslib-s3-plugin.7

2. Attempted build with clean Makefile:

RUN apt-get install -y autoconf automake libtool
COPY fastcat-0.22.0 /tmp/fastcat
WORKDIR /tmp/fastcat
RUN make clean && make fastcat bamstats bamindex

3. Observed errors:

# Header file not found
fatal error: zlib-ng.h: No such file or directory

# Library linking failure
undefined reference to `hts_idx_load'

Key Configuration Issues

1. Path Mismatches

Component Expected Path Actual Path
zlib-ng zlib-ng/zlib.h /opt/zlib-ng/include/zlib.h
htslib htslib/libhts.a /opt/htslib/lib/libhts.a

2. Makefile Limitations

The current Makefile contains hardcoded relative paths:

# Current problematic lines
fastcat: ... zlib-ng/zlib.h htslib/libhts.a

# Needs modification to support absolute paths
# fastcat: ... /opt/zlib-ng/include/zlib.h /opt/htslib/lib/libhts.a

Proposed Solutions

Option 1: Symbolic Links (Temporary Fix)

# Create symlinks in project directory
mkdir -p zlib-ng/htslib
ln -s /opt/zlib-ng/v2.2.4/include/zlib-ng.h zlib-ng/zlib.h
ln -s /opt/htslib/v1.21/include/htslib htslib/htslib
ln -s /opt/zlib-ng/v2.2.4/lib/libz-ng.a zlib-ng/libz.a
ln -s /opt/htslib/v1.21/lib/libhts.a htslib/libhts.a

Option 2: Environment Variable Support (Recommended)

Add path configuration to Makefile:

# Add to Makefile
ZLIB_INCLUDE ?= /opt/zlib-ng/v2.2.4/include
HTS_INCLUDE ?= /opt/htslib/v1.21/include
ZLIB_LIB ?= /opt/zlib-ng/v2.2.4/lib
HTS_LIB ?= /opt/htslib/v1.21/lib

fastcat: ... $(ZLIB_INCLUDE)/zlib-ng.h $(HTS_INCLUDE)/htslib/hts.h

Build Environment

• OS: Ubuntu 22.04 LTS (Docker base image) • Compiler: GCC 11.4.0 • Installed Dependencies:

apt list --installed | grep -E 'autoconf|automake|libtool|build-essential'

Required Maintenance Actions

  1. Update Makefile to: • Accept absolute library paths via environment variables • Support system-wide installations • Remove hardcoded relative paths

  2. Add documentation section for:

## System-Wide Installation
```bash
# For pre-installed dependencies
export ZLIB_INCLUDE=/usr/local/zlib/include
export HTS_LIB=/usr/local/htslib/lib
make all

Temporary Workaround

For immediate testing in Docker:

# Add to Dockerfile
RUN ln -s /opt/zlib-ng/v2.2.4/include/zlib-ng.h /tmp/fastcat/zlib-ng/zlib.h
RUN ln -s /opt/htslib/v1.21/lib/libhts.a /tmp/fastcat/htslib/libhts.a

This clearly identifies:

  1. The root cause (path resolution failures)
  2. Required maintenance actions
  3. Viable short-term workarounds
  4. Standard professional formatting for maintainers
    Q&A: Build Failure - Dependency Paths Not Recognized for zlib-ng and htslib ================================================================================

Q: What is the root cause of the build failure?

A: The root cause of the build failure is the inability of the Makefile to recognize properly installed zlib-ng and htslib libraries located in /opt directories. The Makefile hardcodes relative paths (zlib-ng/zlib.h) instead of respecting absolute paths or proper library installations.

Q: What are the symptoms of the build failure?

A: The symptoms of the build failure include:

  • Header file not found: fatal error: zlib-ng.h: No such file or directory
  • Library linking failure: undefined reference to 'hts_idx_load'

Q: How do I reproduce the issue?

A: To reproduce the issue, follow these steps:

  1. Install dependencies to standard locations.
  2. Attempt to build with a clean Makefile.
  3. Observe the errors.

Q: What are the key configuration issues?

A: The key configuration issues are:

  • Path mismatches: The Makefile uses relative paths instead of absolute paths.
  • Makefile limitations: The Makefile contains hardcoded relative paths.

Q: What are the proposed solutions?

A: The proposed solutions are:

  • Option 1: Symbolic links (temporary fix)
  • Option 2: Environment variable support (recommended)

Q: What are the required maintenance actions?

A: The required maintenance actions are:

  • Update Makefile to accept absolute library paths via environment variables.
  • Support system-wide installations.
  • Remove hardcoded relative paths.

Q: What is the temporary workaround?

A: The temporary workaround is to create symbolic links in the project directory.

Q: What is the build environment?

A: The build environment is:

  • OS: Ubuntu 22.04 LTS (Docker base image)
  • Compiler: GCC 11.4.0
  • Installed dependencies:
apt list --installed | grep -E 'autoconf|automake|libtool|build-essential'

Q: What are the benefits of using environment variable support?

A: The benefits of using environment variable support are:

  • Flexibility: The Makefile can be easily modified to support different library installations.
  • Portability: The Makefile can be used on different systems without modification.
  • Maintainability: The Makefile is easier to maintain and update.

Q: What are the best practices for maintaining the Makefile?

A: The best practices for maintaining the Makefile are:

  • Use absolute paths instead of relative paths.
  • Use environment variables to configure library installations.
  • Remove hardcoded relative paths.
  • Document the Makefile and its configuration options.

By following these best practices and using environment variable support, you can ensure that your Makefile is flexible, portable, and maintainable.