[Bug]I'm Trying To Use Unsloth With Geforce 5700 Ti And Ubuntu, Flash Attn And Xformers Prevent Me

by ADMIN 99 views

Describe the Bug

We are experiencing a critical issue with the Unsloth library on Ubuntu 25.04, utilizing the Geforce 5700 Ti GPU. The problem arises when attempting to use the Unsloth library with the xformers library, which is required for certain operations. Despite setting the environment variable correctly and uninstalling the xformers library, the Unsloth library continues to rely on xformers, resulting in a fatal error.

Environment Setup

To reproduce this issue, we have the following environment setup:

  • Operating System: Ubuntu 25.04
  • Python Version: 3.11.12
  • Frameworks/Libraries:
    • absl-py==2.2.2
    • accelerate==1.6.0
    • aiohappyeyeballs==2.6.1
    • aiohttp==3.11.18
    • aiosignal==1.3.2
    • annotated-types==0.7.0
    • anyio==4.9.0
    • attrs==25.3.0
    • bitsandbytes==0.45.5
    • certifi==2025.4.26
    • charset-normalizer==3.4.1
    • cut-cross-entropy==25.1.1
    • datasets==3.5.1
    • diffusers==0.33.1
    • dill==0.3.8
    • distlib==0.3.9
    • docstring_parser==0.16
    • filelock==3.18.0
    • frozenlist==1.6.0
    • fsspec==2025.3.0
    • grpcio==1.71.0
    • h11==0.16.0
    • hf-xet==1.1.0
    • hf_transfer==0.1.9
    • httpcore==1.0.9
    • httpx==0.28.1
    • huggingface-hub==0.30.2
    • idna==3.10
    • importlib_metadata==8.7.0
    • Jinja2==3.1.6
    • Markdown==3.8
    • markdown-it-py==3.0.0
    • MarkupSafe==3.0.2
    • mdurl==0.1.2
    • mpmath==1.3.0
    • msgspec==0.19.0
    • multidict==6.4.3
    • multiprocess==0.70.16
    • networkx==3.4.2
    • ninja==1.11.1.4
    • numpy==2.2.5
    • nvidia-cublas-cu12==12.8.3.14
    • nvidia-cuda-cupti-cu12==12.8.57
    • nvidia-cuda-nvrtc-cu12==12.8.61
    • nvidia-cuda-runtime-cu12==12.8.57
    • nvidia-cudnn-cu12==9.7.1.26
    • nvidia-cufft-cu12==11.3.3.41
    • nvidia-cufile-cu12==1.13.0.11
    • nvidia-curand-cu12==10.3.9.55
    • nvidia-cusolver-cu12==11.7.2.55
    • nvidia-cusparse-cu12==12.5.7.53
    • nvidia-cusparselt-cu12==0.6.3
    • nvidia-nccl-cu12==2.26.2
    • nvidia-nvjitlink-cu12==12.8.61
    • nvidia-nvtx-cu12==12.8.55
    • ollama==0.4.8
    • packaging==25.0
    • pandas==2.2.3
    • peft==0.15.2
    • pillow==11.2.1
    • platformdirs==4.3.7
    • propcache==0.3.1
    • protobuf==3.20.3
    • psutil==7.0.0
    • pyarrow==20.0.0
    • pydantic==2.11.4
    • pydantic_core==2.33.2
    • Pygments==2.19.1
    • python-dateutil==2.9.0.post0
    • pytorch-triton==3.3.0+git96316ce5
    • pytz==2025.2
    • PyYAML==6.0.2
    • regex==2024.11.6
    • requests==2.32.3
    • rich==14.0.0
    • safetensors==0.5.3
    • sentencepiece==0.2.0
    • shtab==1.7.2
    • six==1.17.0
    • sniffio==1.3.1
    • sympy==1.14.0
    • tensorboard==2.19.0
    • tensorboard-data-server==0.7.2
    • tokenizers==0.21.1
    • torch==2.7.0+cu128
    • torchaudio==2.7.0+cu128
    • torchvision==0.22.0+cu128
    • tqdm==4.67.1
    • transformers==4.51.3
    • triton==3.3.0
    • trl==0.15.2
    • typeguard==4.4.2
    • typing-inspection==0.4.0
    • typing_extensions==4.13.2
    • tyro==0.9.19
    • tzdata==2025.2
    • unsloth==2025.4.4
    • unsloth_zoo==2025.4.4
    • urllib3==2.4.0
    • virtualenv==20.30.0
    • Werkzeug==3.1.3
    • xxhash==3.5.0
    • yarl==1.20.0
    • zipp==3.21.0

Model Details

  • Model ID: qwen3-14B
  • Model Configuration: [e.g., lora params, quantization, etc.]

Expected Behavior

We expect the Unsloth library to function correctly without relying on the xformers library, allowing us to continue fine-tuning the model.

Actual Behavior

However, when we attempt to use the Unsloth library, we encounter the following error:

Filehome/himizunoa/project/Mitsugo/.venv-llm/lib/python3.11/site-packages/unsloth/models/qwen3.py", line 138, in Qwen3Attention_fast_forward
    has_swa = isinstance(causal_mask, xformers.attn_bias.BlockDiagonalCausalMask)
                                      ^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'attn_bias'
  0%|          | 0/315 [00:00<?, ?it/s]

Additional Notes

This issue is critical for users who want to utilize the Geforce 5700 Ti GPU with the Unsloth library. Despite uninstalling the xformers library and setting the environment variable correctly, the Unsloth library continues to rely on xformers, resulting in a fatal error.

Possible Solutions

  1. Update Unsloth Library: We can try updating the Unsloth library to the latest version, which may resolve the issue.
  2. Modify Unsloth Code: We can attempt to modify the Unsloth code to remove the reliance on xformers, allowing us to use the library without xformers.
  3. Use Alternative Library: We can explore using an alternative library that does not rely on xformers, such as the Hugging Face Transformers library.

Conclusion

In conclusion, we have identified a critical issue with the Unsloth library on Ubuntu 25.04, utilizing the Geforce 5700 Ti GPU. Despite setting the environment variable correctly and uninstalling the xformers library, the Unsloth library continues to rely on xformers, resulting in a fatal error. We have proposed possible solutions to resolve this issue, including updating the Unsloth library, modifying the Unsloth code, and using an alternative library.

Q: What is the Unsloth library, and why is it causing issues with my Geforce 5700 Ti GPU?

A: The Unsloth library is a Python library designed for natural language processing (NLP) tasks, particularly for fine-tuning and adapting pre-trained language models. It relies on the xformers library for certain operations, which is causing issues with your Geforce 5700 Ti GPU.

Q: What is the xformers library, and why is it required for Unsloth?

A: The xformers library is a Python library designed for efficient and scalable transformer-based models. It provides a set of optimized functions for transformer operations, which are required for certain Unsloth operations.

Q: I've uninstalled the xformers library, but Unsloth still relies on it. What's going on?

A: This is a known issue with Unsloth, where it continues to rely on xformers even after uninstalling it. This is due to the way Unsloth is designed, and it's not a straightforward fix.

Q: Can I use an alternative library that doesn't rely on xformers?

A: Yes, you can explore using alternative libraries that don't rely on xformers, such as the Hugging Face Transformers library. However, this may require significant changes to your code and may not be compatible with all Unsloth features.

Q: How can I update Unsloth to the latest version to resolve this issue?

A: You can update Unsloth to the latest version using pip: pip install --upgrade unsloth. However, this may not resolve the issue if it's a fundamental design flaw in Unsloth.

Q: Can I modify the Unsloth code to remove the reliance on xformers?

A: Yes, you can attempt to modify the Unsloth code to remove the reliance on xformers. However, this is a complex task that requires significant expertise in Python and NLP.

Q: What are the possible solutions to this issue?

A: The possible solutions to this issue are:

  1. Update Unsloth Library: Update Unsloth to the latest version, which may resolve the issue.
  2. Modify Unsloth Code: Modify the Unsloth code to remove the reliance on xformers.
  3. Use Alternative Library: Use an alternative library that doesn't rely on xformers, such as the Hugging Face Transformers library.

Q: How can I troubleshoot this issue further?

A: To troubleshoot this issue further, you can try the following:

  1. Check Unsloth Version: Check the version of Unsloth you're using and update it to the latest version.
  2. Check xformers Version: Check the version of xformers you're using and uninstall it.
  3. Check Code: Check your code for any issues or conflicts with Unsloth or xformers.
  4. Check Documentation: Check the Unsloth and xformers documentation for any known issues or workarounds.

Q: Can I get help from the Unsloth community or developers?

A: Yes, you can reach out to the Unsloth community or developers for help. They may be able to provide guidance or assistance in resolving this issue.

Q: What are the next steps to resolve this issue?

A: The next steps to resolve this issue are:

  1. Update Unsloth Library: Update Unsloth to the latest version.
  2. Modify Unsloth Code: Modify the Unsloth code to remove the reliance on xformers.
  3. Use Alternative Library: Use an alternative library that doesn't rely on xformers.

By following these steps, you should be able to resolve the issue and use Unsloth with your Geforce 5700 Ti GPU.