[Bug]: MiniCPM3 Failed On Ascend Npu Because Of ModuleNotFoundError: No Module Named 'triton'

by ADMIN 94 views

Bug: MiniCPM3 Failed on Ascend NPU Due to ModuleNotFoundError: No module named 'triton'

The MiniCPM3 model is a powerful language model developed by the OpenBMB team. However, when trying to run the model on an Ascend NPU, users may encounter a ModuleNotFoundError: No module named 'triton' error. This issue is caused by the import of the 'triton' module and the use of the '@triton.jit' decorator in various parts of the code. In this article, we will delve into the details of this issue and provide a solution.

Below is the output of python collect_env.py, which provides information about your current environment:

The output of `python collect_env.py` 
Your output of `python collect_env.py here

The reproduction scripts for this issue are as follows:

import os
from vllm import LLM, SamplingParams
os.environ["VLLM_USE_MODELSCOPE"] = "True"
prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The capital of France is",
    "The future of AI is",
]

# Create a sampling params object.
sampling_params = SamplingParams(max_tokens=100, temperature=0.0)
# Create an LLM.
llm = LLM(model="OpenBMB/MiniCPM3-4B")

# Generate texts from the prompts.
outputs = llm.generate(prompts, sampling_params)
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

This issue is mainly caused by the import of the 'triton' module and the use of the '@triton.jit' decorator in various parts of the code. This breaks many platforms that do not support 'triton', such as the Ascend NPU and CPU.

A pull request has been made to solve this issue: https://github.com/vllm-project/vllm/pull/15099. We hope that this issue can be fixed as soon as possible.

The following issues are related to this issue:

Before submitting a new issue, please make sure you have:

  • Searched for relevant issues
  • Asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

In conclusion, the MiniCPM3 model failed on the Ascend NPU due to a ModuleNotFoundError: No module named 'triton' error. This issue is caused by the import of the 'triton' module and the use of the '@triton.jit' decorator in various parts of the code. A pull request has been made solve this issue, and we hope that it can be fixed as soon as possible.
Q&A: MiniCPM3 Failed on Ascend NPU Due to ModuleNotFoundError: No module named 'triton'

We have compiled a list of frequently asked questions related to the MiniCPM3 model failing on the Ascend NPU due to a ModuleNotFoundError: No module named 'triton' error.

Q: What is the cause of the ModuleNotFoundError: No module named 'triton' error?

A: The cause of the ModuleNotFoundError: No module named 'triton' error is the import of the 'triton' module and the use of the '@triton.jit' decorator in various parts of the code. This breaks many platforms that do not support 'triton', such as the Ascend NPU and CPU.

Q: How can I fix the ModuleNotFoundError: No module named 'triton' error?

A: A pull request has been made to solve this issue: https://github.com/vllm-project/vllm/pull/15099. We hope that this issue can be fixed as soon as possible.

Q: What are the related issues to this problem?

A: The following issues are related to this issue:

Q: How can I prevent this issue from occurring in the future?

A: To prevent this issue from occurring in the future, you can:

  • Check the documentation for the 'triton' module and the '@triton.jit' decorator to ensure that they are supported on your platform.
  • Use a different module or decorator that is supported on your platform.
  • Report any issues or bugs related to the 'triton' module or the '@triton.jit' decorator to the developers.

Q: What are the system requirements for running the MiniCPM3 model?

A: The system requirements for running the MiniCPM3 model are:

  • A compatible CPU or GPU
  • A compatible operating system (e.g. Linux, Windows)
  • A compatible Python version (e.g. Python 3.6 or later)
  • The necessary dependencies and libraries (e.g. 'triton', '@triton.jit')

Q: How can I get help with the MiniCPM3 model?

A: You can get help with the MiniCPM3 model by:

  • Checking the documentation and tutorials on the official website
  • Asking questions on the official forum or community
  • Reporting issues or bugs to the developers
  • Reaching out to the support team for assistance

In conclusion, the MiniCPM3 model failed on the Ascend NPU due to a ModuleNotFoundError: No module named 'triton' error. This issue is caused by the import of the 'triton' module and the use of the '@triton.jit' decorator in various parts of the code. A pull request has been made to solve this issue, and we hope that it can be fixed as soon as possible. We have also provided a list of frequently asked questions and answers to help you understand and this issue.