Backtestsscrapyrun-20261025-211112#7743
Add Request.to_curl() method as the inverse of Request.from_curl()
Our patch was judged better than the merged fix.
In this judgment, Patch B = our agent, Patch A = the merged fix.
Both patches add an identical core implementation: a `Request.to_curl()` method that lazily imports and delegates to `scrapy.utils.request.request_to_curl`, with the same circular-import workaround, and both register the method in the request-response docs. Independent tests pass identically on both. The differences are in tests and documentation. Patch B's tests are stronger: `test_to_curl` checks concrete output fragments (method/URL prefix, `--data-raw`, `-H`, `--cookie`) in addition to equality with the helper, and `test_to_curl_from_curl_roundtrip` directly verifies the 'inverse of from_curl' property the issue title asks for. Patch A's base-class test is purely tautological (`r.to_curl() == request_to_curl(r)`), and its only substantive test is a single POST-with-body string comparison in `test_utils_request.py`. On craft, Patch B's docstring is more informative (lists what is included and notes that downloader-middleware changes are not reflected, which is useful for the issue's stated debugging use case) and it adds a release-notes entry; its extra `autofunction` for `request_to_curl` is adjacent but harmless scope. Patch A's `versionadded:: VERSION` placeholder is ignored per the rules. No correctness difference, but B wins on test quality and documentation.
Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.
Solve
11m 32s
Wall-clock time in the container
Full test suite
No new failures
252 passed · 45 skipped · 300 total
Contamination audit
Clean
No sign the agent saw the real fix
Patch size
61vs 36 merged
Changed lines, ours vs merged
The issue
View on GitHubSummary
Add a to_curl() method to scrapy.Request that turns the request into a curl cmd
Motivation
Request.from_curl() already exists, but to go the other way you have to use a separate function, scrapy.utils.request.request_to_curl() (added in 2.9 #5892)
When I'm debugging a spider with inspect_response(), I often want to run a request outside Scrapy — from the terminal or in tools like Postman. Today I have to remember the import path to use it, I forget it often enough that I end up looking it up in the source:
>>> from scrapy.utils.request import request_to_curl
>>> print(request_to_curl(request))
With a to_curl() method this is easier to find and matches from_curl():
>>> print(request.to_curl())The two patches
docs/news.rstindex 8f8477eaa..b3ac8d350 100644@@ -25,6 +25,13 @@ Backward-incompatible changes (:issue:`6585`, :issue:`7731`) +New features+~~~~~~~~~~~~++- Added :meth:`Request.to_curl() <scrapy.Request.to_curl>`, the inverse of+ :meth:`Request.from_curl() <scrapy.Request.from_curl>`, as a shortcut for+ :func:`scrapy.utils.request.request_to_curl`.+ .. _release-2.17.0: Scrapy 2.17.0 (2026-07-07)docs/topics/request-response.rstindex b83a04032..312bae98d 100644@@ -260,6 +260,8 @@ Request objects .. automethod:: from_curl + .. automethod:: to_curl+ .. automethod:: to_dict @@ -342,6 +344,8 @@ Other functions related to requests .. autofunction:: scrapy.utils.request.request_from_dict +.. autofunction:: scrapy.utils.request.request_to_curl+ .. autofunction:: scrapy.utils.httpobj.urlparse_cached scrapy/http/request/__init__.pyindex 57517a65b..44897b7be 100644@@ -387,6 +387,25 @@ class Request(object_ref): request_kwargs.update(kwargs) return cls(**request_kwargs) + def to_curl(self) -> str:+ """Return a string containing a `cURL <https://curl.se/>`_ command+ equivalent to this request, including the HTTP method, the URL, the+ headers, the cookies and the body.++ This is the inverse of :meth:`from_curl`, and is equivalent to+ :func:`scrapy.utils.request.request_to_curl`.++ .. note:: The command only reflects the request as it currently is.+ Changes that :ref:`downloader middlewares+ <topics-downloader-middleware>` would make to the request+ when it is sent, such as adding default headers, are not+ included.+ """+ # circular import+ from scrapy.utils.request import request_to_curl # noqa: PLC0415++ return request_to_curl(self)+ def to_dict(self, *, spider: scrapy.Spider | None = None) -> dict[str, Any]: """Return a dictionary containing the Request's data. tests/utils/bases/http_request.pyindex c255b5e4c..06eed1b2c 100644@@ -6,6 +6,7 @@ import pytest from scrapy.http import Headers, Request from scrapy.http.request import NO_CALLBACK+from scrapy.utils.request import request_to_curl class TestRequestBase(ABC):@@ -463,6 +464,36 @@ class TestRequestBase(ABC): b"Connection": [b"keep-alive"], } + def test_to_curl(self):+ r = self.request_class(+ "https://www.example.com/post",+ method="POST",+ body=b"foo=bar",+ headers={"X-Foo": "bar"},+ cookies={"name": "value"},+ )+ curl_command = r.to_curl()+ assert curl_command == request_to_curl(r)+ assert curl_command.startswith("curl -X POST https://www.example.com/post ")+ assert "--data-raw 'foo=bar'" in curl_command+ assert "-H 'X-Foo: bar'" in curl_command+ assert "--cookie 'name=value'" in curl_command++ def test_to_curl_from_curl_roundtrip(self):+ r = self.request_class(+ "https://www.example.com/post",+ method="PUT",+ body=b"foo=bar",+ headers={"X-Foo": "bar"},+ cookies={"name": "value"},+ )+ r2 = Request.from_curl(r.to_curl())+ assert r2.method == r.method+ assert r2.url == r.url+ assert r2.body == r.body+ assert r2.headers.get("X-Foo") == b"bar"+ assert r2.cookies == {"name": "value"}+ def test_from_curl_with_kwargs(self): r = self.request_class.from_curl( 'curl -X PATCH "http://example.org"', method="POST", meta={"key": "value"}
docs/topics/request-response.rstindex b83a04032..8e565907f 100644@@ -260,6 +260,8 @@ Request objects .. automethod:: from_curl + .. automethod:: to_curl+ .. automethod:: to_dict scrapy/http/request/__init__.pyindex 57517a65b..68847283a 100644@@ -387,6 +387,20 @@ class Request(object_ref): request_kwargs.update(kwargs) return cls(**request_kwargs) + def to_curl(self) -> str:+ """Return a string with a `cURL <https://curl.se/>`_ command equivalent+ to this request.++ Inverse of :meth:`from_curl`. See also+ :func:`scrapy.utils.request.request_to_curl`.++ .. versionadded:: VERSION+ """+ # Imported here to avoid a circular import.+ from scrapy.utils.request import request_to_curl # noqa: PLC0415++ return request_to_curl(self)+ def to_dict(self, *, spider: scrapy.Spider | None = None) -> dict[str, Any]: """Return a dictionary containing the Request's data. tests/test_utils_request.pyindex eb95be06c..935447bc4 100644@@ -475,3 +475,14 @@ class TestRequestToCurl: " --data-raw '{\"foo\": \"bar\"}' --cookie 'foo=1'" ) self._test_request(request_object, expected_curl_command)++ def test_request_to_curl_method(self) -> None:+ request_object = Request(+ "https://www.httpbin.org/post",+ method="POST",+ body=json.dumps({"foo": "bar"}),+ )+ expected_curl_command = (+ 'curl -X POST https://www.httpbin.org/post --data-raw \'{"foo": "bar"}\''+ )+ assert request_object.to_curl() == expected_curl_commandtests/utils/bases/http_request.pyindex c255b5e4c..3a1e588ef 100644@@ -6,6 +6,7 @@ import pytest from scrapy.http import Headers, Request from scrapy.http.request import NO_CALLBACK+from scrapy.utils.request import request_to_curl class TestRequestBase(ABC):@@ -488,3 +489,11 @@ class TestRequestBase(ABC): 'curl -X PATCH "http://example.org" --foo -z', ignore_unknown_options=False, )++ def test_to_curl(self):+ # Note: more curated tests regarding curl conversion are in+ # `test_utils_request.py`+ r = self.request_class(+ "http://www.example.com/", method="POST", body=b"foo=bar"+ )+ assert r.to_curl() == request_to_curl(r)