Open PR Tool Creates A Token Limit Exceeded For PR Body Issue
Introduction
In this article, we will discuss an issue that arises when using an open PR tool, specifically the SWE Agent, as part of a GitHub Actions workflow. The issue is related to the token limit exceeded for the PR body, which prevents the PR from being opened successfully. We will delve into the details of the bug, the steps to reproduce it, and the error message/results. Additionally, we will provide a checklist to ensure that the issue is properly reported.
Describe the Bug
The bug is sporadic and depends on the task being performed. It occurs when the SWE Agent is run as part of a GitHub Actions workflow, specifically when opening a new issue. The bug completes the entire fixing process up until the PR needs to be opened, but then fails with a message indicating that the PR body limit has been exceeded. This issue is particularly frustrating as it prevents the PR from being opened successfully, which is a crucial step in the workflow.
Ideally, an Option to Automatically Truncate or Summarize the PR Body
To prevent this issue from occurring, it would be ideal to have an option to automatically truncate or summarize the PR body in some way. This would ensure that the PR body does not exceed the token limit, allowing the PR to be opened successfully.
Steps/Commands/Code to Reproduce
To reproduce this issue, follow these steps:
- Set up SWE Agent as part of a GitHub Actions workflow.
- Run the workflow with the SWE Agent enabled.
- Open a new issue in the repository.
- The SWE Agent will complete the fixing process up until the PR needs to be opened.
- The PR will fail to open with a message indicating that the PR body limit has been exceeded.
Error Message/Results
The error message/results are as follows:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.11.11/x64/bin/sweagent", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/runner/work/main_folder/main_folder/SWE-agent/sweagent/run/run.py", line 87, in main
run_single_main(remaining_args)
File "/home/runner/work/main_folder/main_folder/SWE-agent/sweagent/run/run_single.py", line 216, in run_from_cli
run_from_config(BasicCLI(RunSingleConfig, help_text=help_text).get_config(args)) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/main_folder/main_folder/SWE-agent/sweagent/run/run_single.py", line 206, in run_from_config
RunSingle.from_config(config).run()
File "/home/runner/work/main_folder/main_folder/SWE-agent/sweagent/run/run_single.py", line 198, in run
self._chooks.on_instance_completed(result=result)
File "/home/runner/work/main_folder/main_folder/SWE-agent/sweagent/run/hooks/abstract.py", line 67, in on_instance_completed
hook.on_instance_completed(result=result)
File "/home/runner/work/main_folder/main_folder/SWE-agent/sweagent/run/hooks/open_pr.py", line 140, in on_instance_completed
open_pr(
File "/home/runner/work/main_folder/main_folder/SWE-agent/sweagent/run/hooks/open_pr.py", line 111, in open_pr
pr_info = api.pulls.create(**args) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/ghapi/core.py", line 74, in __call__
return self.client(self.path, self.verb, headers=headers, decode=self.decode, route=route_p, query=query_p, data=data_p)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/ghapi/core.py", line 133, in __call__
res,self.recv_hdrs = urlsend(path, verb, headers=headers or None, decode=decode, debug=debug, return_headers=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/fastcore/net.py", line 221, in urlsend
return urlread(req, decode=decode, return_json=return_json, return_headers=return_headers, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/fastcore/net.py", line 122, in urlread
if 400 <= e.code < 500: raise ExceptionsHTTP[e.code](e.url, e.hdrs, e.fp, msg=e.msg) from None
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
fastcore.net.HTTP422UnprocessableEntityError: HTTP Error 422: Unprocessable Entity
====Error Body====
{
"message": "Validation Failed",
"errors": [
{
"resource": "Issue",
"code": "custom",
"field": "body",
"message": "body is too long (maximum is 65536 characters)"
}
],
"documentation_url": "https://docs.github.com/rest/pulls/pulls#create-a-pull-request",
"status": "422"
}
System Information
The system information is as follows:
- Ubuntu 20.04
- Python 3.11
- Agent was cloned and installed from the source.
Checklist
To ensure that the issue is properly reported, please follow this checklist:
- I'm running with the latest docker container/on the latest development version (i.e., I ran
git pull)) - I have copied the full command/code that I ran (as text, not as screenshot!)
- If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
- I have enclosed code/log messages in triple backticks (docs) and clicked "Preview" to make sure it's displayed correctly.
Open PR Tool Creates a Token Limit Exceeded for PR Body Issue: Q&A ====================================================================
Q: What is the issue with the open PR tool?
A: The issue is that the open PR tool, specifically the SWE Agent, is exceeding the token limit for the PR body. This is causing the PR to fail to open with a message indicating that the PR body limit has been exceeded.
Q: What is the token limit for the PR body?
A: The token limit for the PR body is 65536 characters.
Q: Why is the SWE Agent exceeding the token limit?
A: The SWE Agent is exceeding the token limit because it is generating a large amount of text for the PR body. This text is not being truncated or summarized, which is causing the token limit to be exceeded.
Q: Is there a way to prevent the SWE Agent from exceeding the token limit?
A: Yes, there are a few ways to prevent the SWE Agent from exceeding the token limit. One way is to truncate or summarize the text generated by the SWE Agent. Another way is to increase the token limit for the PR body.
Q: How can I truncate or summarize the text generated by the SWE Agent?
A: There are several ways to truncate or summarize the text generated by the SWE Agent. One way is to use a library such as textwrap to wrap the text at a certain number of characters. Another way is to use a library such as summarize to summarize the text.
Q: How can I increase the token limit for the PR body?
A: To increase the token limit for the PR body, you will need to modify the GitHub API settings for your repository. You can do this by going to the repository settings and clicking on the "API" tab. From there, you can increase the token limit for the PR body.
Q: What are the system requirements for the SWE Agent?
A: The system requirements for the SWE Agent are as follows:
- Ubuntu 20.04
- Python 3.11
- Agent was cloned and installed from the source.
Q: What is the checklist for reporting this issue?
A: To ensure that the issue is properly reported, please follow this checklist:
- I'm running with the latest docker container/on the latest development version (i.e., I ran
git pull)) - I have copied the full command/code that I ran (as text, not as screenshot!)
- If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
- I have enclosed code/log messages in triple backticks (docs) and clicked "Preview" to make sure it's displayed correctly.
Q: Where can I find more information about the SWE Agent?
A: You can find more information about the SWE Agent on the GitHub repository for the SWE Agent. The repository can be found at https://github.com/SWE-agent/SWE-agent.