Pylsp_document_symbols Fails When The Document Imports A Namedtuple From Another Module

by ADMIN 88 views

Introduction

The pylsp_document_symbols hook in the PyLSP plugin is used to provide document symbols for a given text document. However, when a Python module imports a namedtuple type from another module, the pylsp_document_symbols hook fails with a TypeError. In this article, we will explore the issue and provide a solution.

Steps to Reproduce

To reproduce the issue, follow these steps:

Step 1: Set up a Python Workspace

Create a Python workspace with the following files a.py and b.py:

# a.py
from .b import MyNamedTuple

a_symbol = "a_symbol"

# b.py
from collections import namedtuple

MyNamedTuple = namedtuple("MyNamedTuple", ["abc"])

b_symbol = "b_symbol"

Step 2: Start PyLSP

Start PyLSP with the following command:

pylsp --verbose

Step 3: Send Messages to PyLSP

Send the following two messages to PyLSP:

{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"workspaceFolders": [{"uri": "/home/ksmai/dev/pylsp-repro-namedtuple", "name": "test"}]}}

{"jsonrpc": "2.0", "id": 2, "method": "textDocument/documentSymbol", "params": {"textDocument": {"uri": "/home/ksmai/dev/pylsp-repro-namedtuple/a.py"}}}

Step 4: Observe the Error

An error occurs, and no results are returned. The error message is:

WARNING - pylsp.config.config - Failed to load hook pylsp_document_symbols: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'
Traceback (most recent call last):
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pylsp/config/config.py", line 39, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_result.py", line 62, in from_call
    result = func()
             ^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 477, in <lambda>
    lambda: old(hook_name, hook_impls, caller_kwargs, firstresult)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pylsp/plugins/symbols.py", line 88, in pylsp_document_symbols
    if _include_def(d) and Path(document.path) == Path(d.module_path):
                                                  ^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/lib/python3.12/pathlib.py", line 1162, in __init__
    super().__init__(*args)
  File "/home/ksmai/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/lib/python3.12/pathlib.py", line 373, in __init__
    raise TypeError(
TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'

Analysis

The error occurs because the pylsp_document_symbols hook is trying to access the module_path attribute of the d object, which is None. This is because the namedtuple type is imported from another module, and the module_path attribute is not set for imported types.

Solution

To fix the issue, we need to modify the pylsp_document_symbols hook to handle the case where the module_path attribute is None. We can do this by adding a check to see if the module_path attribute is None before trying to access it.

Here is the modified code:

def pylsp_document_symbols(document, **kwargs):
    if document.module_path is None:
        return []
    # rest of the code remains the same

Conclusion

In conclusion, the pylsp_document_symbols hook fails when a Python module imports a namedtuple type from another module because the module_path attribute is None. To fix the issue, we need to modify the hook to handle this case. By adding a check to see if the module_path attribute is None, we can prevent the error from occurring.

Additional Information

The code for reproducing the issue can be found at: https://github.com/ksmai/pylsp-repro-namedtuple

Q: What is the issue with pylsp_document_symbols when a document imports a namedtuple from another module?

A: The issue is that the pylsp_document_symbols hook fails with a TypeError when a document imports a namedtuple type from another module. This is because the module_path attribute of the d object is None, and the hook tries to access it.

Q: What is the error message that is displayed when the issue occurs?

A: The error message is:

WARNING - pylsp.config.config - Failed to load hook pylsp_document_symbols: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'
Traceback (most recent call last):
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pylsp/config/config.py", line 39, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_result.py", line 62, in from_call
    result = func()
             ^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 477, in <lambda>
    lambda: old(hook_name, hook_impls, caller_kwargs, firstresult)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/dev/pylsp-repro-namedtuple/.venv/lib/python3.12/site-packages/pylsp/plugins/symbols.py", line 88, in pylsp_document_symbols
    if _include_def(d) and Path(document.path) == Path(d.module_path):
                                                  ^^^^^^^^^^^^^^^^^^^
  File "/home/ksmai/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/lib/python3.12/pathlib.py", line 1162 in __init__
    super().__init__(*args)
  File "/home/ksmai/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/lib/python3.12/pathlib.py", line 373, in __init__
    raise TypeError(
TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'

Q: How can I reproduce the issue?

A: To reproduce the issue, follow these steps:

  1. Set up a Python workspace with the following files a.py and b.py:
# a.py
from .b import MyNamedTuple

a_symbol = "a_symbol"

# b.py
from collections import namedtuple

MyNamedTuple = namedtuple("MyNamedTuple", ["abc"])

b_symbol = "b_symbol"
  1. Start PyLSP with the following command:
pylsp --verbose
  1. Send the following two messages to PyLSP:
{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"workspaceFolders": [{"uri": "/home/ksmai/dev/pylsp-repro-namedtuple", "name": "test"}]}}

{"jsonrpc": "2.0", "id": 2, "method": "textDocument/documentSymbol", "params": {"textDocument": {"uri": "/home/ksmai/dev/pylsp-repro-namedtuple/a.py"}}}

Q: What is the solution to the issue?

A: The solution to the issue is to modify the pylsp_document_symbols hook to handle the case where the module_path attribute is None. We can do this by adding a check to see if the module_path attribute is None before trying to access it.

Here is the modified code:

def pylsp_document_symbols(document, **kwargs):
    if document.module_path is None:
        return []
    # rest of the code remains the same

Q: What is the code for reproducing the issue?

A: The code for reproducing the issue can be found at: https://github.com/ksmai/pylsp-repro-namedtuple

Q: What are the Python and PyLSP versions that are affected by the issue?

A: The issue is affected by Python version 3.12.9 and PyLSP version v1.12.2.