Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: MissingContentLength in boto3 version 1.36.1 #4398

Open
1 task
alireza-fa opened this issue Jan 17, 2025 · 6 comments
Open
1 task

fix: MissingContentLength in boto3 version 1.36.1 #4398

alireza-fa opened this issue Jan 17, 2025 · 6 comments
Assignees
@RyanFitzSimmonsAK
Labels
bug This issue is a confirmed bug. p2 This is a standard priority issue response-requested Waiting on additional information or feedback. s3

Comments

@alireza-fa
Copy link

Describe the bug

Description

I encountered an issue while using boto3 version 1.36.1 in conjunction with django-storages. When attempting to upload files to AWS S3, I received the following error:

An error occurred (MissingContentLength) when calling the PutObject operation: Unknown

Steps to Reproduce

  1. Install boto3 version 1.36.1 and django-storages.
  2. Configure Django to use S3 for file storage.
  3. Attempt to upload a file to S3.

Environment

  • boto3 version: 1.36.1
  • django-storages version: django-storages==1.14.4
  • Django version: django==5.1.2
  • Python version: python==3.11

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

The file should upload successfully without any errors.

Current Behavior

The upload fails with the MissingContentLength error.

Reproduction Steps

I found that downgrading boto3 to version 1.34.131 resolves the issue, and file uploads work as expected.

Possible Solution

No response

Additional Information/Context

No response

SDK version used

boto3 version: 1.36.1

Environment details (OS name and version, etc.)

ubuntu22.04 boto3 version: 1.36.1 - django-storages version: django-storages==1.14.4 - Django version: django==5.1.2 - Python version: python==3.11
@alireza-fa alireza-fa added bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged. labels Jan 17, 2025
@LeaveMyYard
Copy link

Downgrading to boto3==1.35.95 seem to be fixing the problem, at least as a temporary solution

@RyanFitzSimmonsAK RyanFitzSimmonsAK self-assigned this Jan 17, 2025
@RyanFitzSimmonsAK RyanFitzSimmonsAK added p2 This is a standard priority issue s3 and removed needs-triage This issue or PR still needs to be triaged. labels Jan 17, 2025
@RyanFitzSimmonsAK
Copy link
Contributor

Hi @alireza-fa, thanks for reaching out. This is related to the recent announcement of new default checksum behavior starting in v1.36.0. I suspect this is likely an issue with django-storages.

Disclaimer: The AWS SDKs and CLI are designed for usage with official AWS services. We may introduce and enable new features by default, such as these new default integrity protections prior to them being supported or handled by third-party service implementations. You can disable the new behavior with the WHEN_REQUIRED value for the request_checksum_calculation and response_checksum_validation configuration options covered in Data Integrity Protections for Amazon S3.

Are you able to reproduce this behavior using AWS S3, or with default integrity options set to when-required?

@RyanFitzSimmonsAK RyanFitzSimmonsAK added the response-requested Waiting on additional information or feedback. label Jan 17, 2025
@alireza-fa
Copy link
Author

Hi @RyanFitzSimmonsAK , thank you for your response!

I appreciate your insights regarding the new default checksum behavior. However, I want to clarify that I am not using the official AWS S3 service; instead, I am utilizing a compatible service. You raised a valid point that this issue may affect many users, and with the recent changes from AWS, it might take some time for compatible services to update accordingly.

Unfortunately, I cannot reproduce the behavior using AWS S3, as I am working with a different service. If you have any further suggestions or insights, I would be grateful to hear them!

Thank you again for your help!

singiamtel added a commit to singiamtel/alibuild that referenced this issue Jan 19, 2025
@singiamtel
Pin boto3
a9f5fe8 
Should bypass this issue on uploads boto/boto3#4398

```
An error occurred (MissingContentLength) when calling the PutObject operation: Unknown
```
@singiamtel singiamtel mentioned this issue Jan 19, 2025
Merged
@Krzysiek555
Copy link

Krzysiek555 commented Jan 20, 2025 •
edited
Loading

I can confirm that we are also facing same issue.

Django 5.1.5
django-storages 1.14.4

After upgrading boto3 to 1.36.1 (or to 1.36.2) we started getting error:

An error occurred (MissingContentLength) when calling the PutObject operation: None

When downgraded boto3 to 1.35.99 error disappears.

We are using DigitalOcean Spaces S3 compatible buckets.

fcanovai added a commit to cloudnative-pg/postgres-containers that referenced this issue Jan 20, 2025
@fcanovai
fix: pin boto3
7f1a929 
Pin boto3 version to 1.35.99.
version 1.36.0 of boto3 is affected by boto/boto3#4398.
fcanovai added a commit to cloudnative-pg/postgres-containers that referenced this issue Jan 20, 2025
@fcanovai
fix: pin boto3
bf372e4 
Pin boto3 version to 1.35.99.
version 1.36.0 of boto3 is affected by boto/boto3#4398.

Signed-off-by: Francesco Canovai <[email protected]>
fcanovai added a commit to cloudnative-pg/postgres-containers that referenced this issue Jan 20, 2025
@fcanovai
fix: pin boto3
d7b2489 
Pin boto3 version to 1.35.99.
version 1.36.0 of boto3 is affected by boto/boto3#4398.

Signed-off-by: Francesco Canovai <[email protected]>
@mnencia mnencia mentioned this issue Jan 20, 2025
Open
singiamtel added a commit to alisw/alibuild that referenced this issue Jan 20, 2025
@singiamtel
Pin boto3 (#900)
1091f6f 
Should bypass this issue on uploads boto/boto3#4398

```
An error occurred (MissingContentLength) when calling the PutObject operation: Unknown
```
fcanovai added a commit to cloudnative-pg/postgres-containers that referenced this issue Jan 20, 2025
@fcanovai
fix: pin boto3
fb9cca5 
Pin boto3 version to 1.35.99.
version 1.36.0 of boto3 is affected by boto/boto3#4398.

Closes #139

Signed-off-by: Francesco Canovai <[email protected]>
@fcanovai fcanovai mentioned this issue Jan 20, 2025
Merged
fcanovai added a commit to EnterpriseDB/docker-postgresql that referenced this issue Jan 20, 2025
@fcanovai
fix: pin boto3
c23732e 
Pin boto version to 1.35.99.
Version 1.36+ of boto3 are affected by boto/boto3#4398.

Signed-off-by: Francesco Canovai <[email protected]>
@fcanovai fcanovai mentioned this issue Jan 20, 2025
Merged
mnencia pushed a commit to cloudnative-pg/postgres-containers that referenced this issue Jan 20, 2025
@fcanovai
fix: pin boto3 (#140)
39597ef 
Pin boto3 version to 1.35.99.
version 1.36.0 of boto3 is affected by boto/boto3#4398.

Closes #139

Signed-off-by: Francesco Canovai <[email protected]>
sxd pushed a commit to EnterpriseDB/docker-postgresql that referenced this issue Jan 20, 2025
@fcanovai
fix: pin boto3 (#204)
bcd9fd3 
Pin boto version to 1.35.99.
Version 1.36+ of boto3 are affected by boto/boto3#4398.

Signed-off-by: Francesco Canovai <[email protected]>
@ryanovas
Copy link

This is also an issue with pyiceberg/s3fs

@ryanovas ryanovas mentioned this issue Jan 21, 2025
Open
fcanovai added a commit to cloudnative-pg/postgres-containers that referenced this issue Jan 21, 2025
@fcanovai
fix: pin boto3 (#140)
353be36 
Pin boto3 version to 1.35.99.
version 1.36.0 of boto3 is affected by boto/boto3#4398.

Closes #139

Signed-off-by: Francesco Canovai <[email protected]>
@djhmateer djhmateer mentioned this issue Jan 21, 2025
Closed
@derekleverenz
Copy link

derekleverenz commented Jan 21, 2025 •
edited
Loading

Same issue also occurs for us with OCI s3-compatible storage (using boto3 directly, not through another library)

singiamtel added a commit to singiamtel/ali-bot that referenced this issue Jan 22, 2025
@singiamtel
Pin boto3
24c289a 
Should bypass this issue on uploads
boto/boto3#4398

```
An error occurred (MissingContentLength) when calling the PutObject
operation: Unknown
```
@singiamtel singiamtel mentioned this issue Jan 22, 2025
Merged
@Ash-Crow Ash-Crow mentioned this issue Jan 23, 2025
Closed
openstack-mirroring pushed a commit to openstack/openstack that referenced this issue Jan 23, 2025
@sabaini @openstack-gerrit
Update git submodules
f417b44 
* Update charm-ceph-radosgw from branch 'master'
  to 41d4c74ec9697d93c4b683a401517c4aba29bebd
  - Tests: pin boto3
    
    For boto/boto3#4398
    
    Also lint updates
    
    Signed-off-by: Peter Sabaini <[email protected]>
    Change-Id: Ieb248434b482f513060ca8b453e2068d51fde9cf
openstack-mirroring pushed a commit to openstack/charm-ceph-radosgw that referenced this issue Jan 23, 2025
@sabaini
Tests: pin boto3
41d4c74 
For boto/boto3#4398

Also lint updates

Signed-off-by: Peter Sabaini <[email protected]>
Change-Id: Ieb248434b482f513060ca8b453e2068d51fde9cf
openstack-mirroring pushed a commit to openstack/charm-ceph-radosgw that referenced this issue Jan 23, 2025
@sabaini
Tests: pin boto3
e11db42 
For boto/boto3#4398

Also lint updates

Signed-off-by: Peter Sabaini <[email protected]>
Change-Id: Ieb248434b482f513060ca8b453e2068d51fde9cf
Cherry picked from 41d4c74
sxd added a commit to cloudnative-pg/postgis-containers that referenced this issue Jan 23, 2025
@sxd
fix: pin boto3
36ff350 
Pin boto3 version to 1.35.99, version 1.36.0 of boto3 is
affected by boto/boto3#4398.

Signed-off-by: Jonathan Gonzalez V. <[email protected]>
@sxd sxd mentioned this issue Jan 23, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RyanFitzSimmonsAK RyanFitzSimmonsAK

Labels
bug This issue is a confirmed bug. p2 This is a standard priority issue response-requested Waiting on additional information or feedback. s3
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@derekleverenz @ryanovas @Krzysiek555 @LeaveMyYard @RyanFitzSimmonsAK @alireza-fa