Skip to content

Commit

Permalink
fix: Accept string payload in the Request constructor (#697)
Browse files Browse the repository at this point in the history
- This is a follow-up to
#668 (comment).
- Let's accept `payload` as both str and bytes in the `Request.from_url`
constructor.
- The field `Request.payload` has a validator that accepts both. With
this change, we are consistent in the `from_url` constructor as well.
- We do a similar thing for the `Request.headers` as well.
  • Loading branch information
vdusek authored Nov 14, 2024
1 parent b703709 commit 19f5add
Showing 1 changed file with 8 additions and 2 deletions.
10 changes: 8 additions & 2 deletions src/crawlee/_request.py
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ def from_url(
*,
method: HttpMethod = 'GET',
headers: HttpHeaders | dict[str, str] | None = None,
payload: HttpPayload | None = None,
payload: HttpPayload | str | None = None,
label: str | None = None,
unique_key: str | None = None,
id: str | None = None,
Expand All @@ -195,6 +195,9 @@ def from_url(
if isinstance(headers, dict) or headers is None:
headers = HttpHeaders(headers or {})

if isinstance(payload, str):
payload = payload.encode()

unique_key = unique_key or compute_unique_key(
url,
method=method,
Expand Down Expand Up @@ -276,7 +279,7 @@ def from_url(
*,
method: HttpMethod = 'GET',
headers: HttpHeaders | dict[str, str] | None = None,
payload: HttpPayload | None = None,
payload: HttpPayload | str | None = None,
label: str | None = None,
unique_key: str | None = None,
id: str | None = None,
Expand Down Expand Up @@ -317,6 +320,9 @@ def from_url(
if isinstance(headers, dict) or headers is None:
headers = HttpHeaders(headers or {})

if isinstance(payload, str):
payload = payload.encode()

unique_key = unique_key or compute_unique_key(
url,
method=method,
Expand Down

0 comments on commit 19f5add

Please sign in to comment.