Remove Incomplete Multipart Upload files from AWS S3

actually we have one application for file storage (Dropbox) which is using the AWS s3 bucket .

we have diffrect plans for end users like Free and silver/paied depanding on the size of file.

Some time users upload the file druing the upload process its intrept due to some reason like

1 - user cancel the uploading Process in middle 2 - Network glitch between user's internet and AWS S3

In above cases if for example user try to upload 1GB file and in the middle of upload process user/he/she cancel it, in this cases 50% (0.5GB) file was already uploaded to S3. so that uploaded file is there on the s3 backet and it occoupied the space on s3 and also we have to pay for that 0.5GB file.

I want if upload process kill by end user or due to the network issue the uploaded part of file should be delete from s3 after some time or on the same time when user upload it and it was not completed/intercepted. how can i define a life cycle for S3 bucket to accomplished my requirement.

1 answer

  • answered 2018-05-16 06:50 Kush Vyas

    You can create a new rule for incomplete multipart uploads using the Console:

    1) Start by opening the console and navigating to the desired bucket

    enter image description here

    2) Then click on Properties, open up the Lifecycle section, and click on Add rule:

    enter image description here

    3) Decide on the target (the whole bucket or the prefixed subset of your choice) and then click on Configure Rule:

    enter image description here

    4) Then enable the new rule and select the desired expiration period:

    enter image description here

    5) As a best practice, we recommend that you enable this setting even if you are not sure that you are actually making use of multipart uploads. Some applications will default to the use of multipart uploads when uploading files above a particular, application-dependent, size.

    Here’s how you set up a rule to remove delete markers for expired objects that have no previous versions: enter image description here

    You can refer this AWS Blog Post