AWS S3 File Size Limits: Max Object Size, Single PUT, and Multipart Rules

The Amazon S3 file size limit specifies a maximum object size of 50 TB, with individual single PUT uploads capped at 5 GB, requiring multipart upload for larger payloads. While historical documentation and third-party tutorials frequently cite 5 TB as the ultimate ceiling, current Amazon S3 service documentation establishes that individual objects can range from 0 bytes up to a maximum of 50 TB when uploaded using the multipart upload API.

Amazon S3 imposes no restriction on the total volume of data or the total number of objects stored within a bucket. An organization can store petabytes of data distributed across billions of keys. However, every individual object upload must conform to strict payload boundaries defined by the Amazon S3 ingestion pipeline.

Understanding these boundaries prevents failed deployments and avoidable network re-transmissions. When designing ingestion pipelines for data lakes, machine learning training sets, enterprise database backups, or AI agent context repositories, engineers select the upload pattern that matches their payload dimensions.

The Single PUT Ceiling

The single PUT operation, executed through the PutObject REST API endpoint, accepts an atomic payload stream in a single HTTPS request. Amazon S3 single PUT upload operations enforce a strict payload size limit capped at 5 GB.

If an application attempts a single PUT upload for an object larger than 5 GB, the Amazon S3 edge proxy terminates the transfer and returns an HTTP 400 Bad Request status code. The response body contains the XML error code EntityTooLarge along with the message: "Your proposed upload exceeds the maximum allowed size."

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>EntityTooLarge</Code>
  <Message>Your proposed upload exceeds the maximum allowed size</Message>
  <ProposedSize>5368709121</ProposedSize>
  <MaxSizeAllowed>5368709120</MaxSizeAllowed>
  <RequestId>4YEXAMPLE7890ABC</RequestId>
  <HostId>AbCdEfGhIjKlMnOpQrStUvWxYz1234567890ExampleHostId=</HostId>
</Error>

Beyond the 5 GB boundary, relying on single PUT upload operations for payloads above 100 MB introduces operational risks. If a network socket drops, times out, or encounters packet corruption near completion of a single PUT transfer, the entire transmission fails. The client application must restart the transfer from byte zero.

Console Upload Limits Versus API Endpoints

The AWS Management Console provides an interactive browser interface for uploading files to S3 buckets, but it operates under tighter constraints than the underlying REST API. Within the browser interface, Amazon S3 enforces a practical upload limit of 160 GB per individual file.

Browser environments run within memory-constrained JavaScript sandboxes. Managing large file streams inside a web browser risks tab crashes, memory exhaustion, and session token expirations during prolonged network transfers. For large payloads, engineering teams rely on programmatic tools such as the AWS Command Line Interface (AWS CLI), official AWS SDKs, or dedicated cloud storage connectors.

• S3 Presigned URL File Size Limit: Upload Caps, Multipart Transfers & AlternativesThe Amazon S3 presigned URL file size limit is strictly 5 GB for a single HTTP PUT operation, governed by the PutObject...
• Cloudflare Upload Limits: Plan Caps, Error 413, and WorkaroundsCloudflare restricts client request bodies passing through its edge proxy to 100 MB on Free and Pro tiers, 200 MB on...
• AWS Lambda File Size Limits: Payload, Package, and Ephemeral Storage CapsAn AWS Lambda file size limit encompasses the execution constraints of AWS Lambda functions, specifically the 50MB...
• Notion File Upload Limits: Size Caps by Plan and Large-File WorkaroundsNotion limits individual file uploads to 5MB on the Free plan and provides unlimited file uploads with a 5GB maximum...
• Box File Size Limit: Upload Caps by Plan and Large-Corpus AI SearchBox documents an individual file upload limit across account plans, ranging from 250 MB on free accounts to 5 GB on...
• Google AI Studio File Upload Limits: File API Constraints and Large CorporaGoogle AI Studio and the Gemini File API enforce a 2 GB per-file upload limit and a 20 GB total project storage cap,...

More on this subject: Agent File and Document Workflows (397 guides)

The multipart upload API solves the reliability and throughput constraints of single PUT uploads. Instead of transmitting a massive dataset as a monolithic stream, multipart upload breaks the file into independent chunks that upload concurrently.

The multipart workflow executes across three distinct phases:

1. Initiation (InitiateMultipartUpload): The client application requests an upload session by targeting a bucket and key. Amazon S3 initializes the session and returns an XML payload containing an UploadId. This identifier binds all subsequent chunk transmissions to the target object.
2. Part Transmission (UploadPart): The client divides the source file into numbered chunks and uploads each part independently using UploadPart requests that reference the UploadId. Parts can upload in parallel across multiple threads or network connections. S3 stores each part and returns an HTTP response header containing an ETag checksum.
3. Completion or Abort (CompleteMultipartUpload or AbortMultipartUpload): Once all chunks upload, the client sends an XML manifest listing every part number and its corresponding ETag in ascending order. Amazon S3 validates the checksums, concatenates the parts into a single logical object, and releases the upload session. If the transfer is cancelled, the client issues an AbortMultipartUpload request to purge stored parts.

Part Sizing Constraints and Boundaries

Every multipart upload in Amazon S3 must satisfy four structural rules:

• Part Number Range: Every part requires an integer identifier between 1 and 10,000 inclusive. S3 rejects any part number outside this range.
• Minimum Part Size: Parts 1 through 9,999 must each measure at least 5 MiB (5,242,880 bytes). Submitting an intermediate part smaller than 5 MiB results in an HTTP 400 Bad Request error with code EntityTooSmall.
• Final Part Exception: The final part of an Amazon S3 multipart upload operation has no minimum size requirement, allowing smaller trailing payloads to complete the object.
• Maximum Part Size: An individual part cannot exceed 5 GiB (5,368,709,120 bytes).

Calculating Part Sizes for Large Archives

Because Amazon S3 limits every multipart upload to exactly 10,000 parts, client applications cannot use small chunk sizes for massive files. The minimum allowable part size is directly determined by the total file size divided by 10,000.

In an Amazon S3 multipart upload operation, selecting the minimum part size for large objects produces too many chunks, exceeding the 10,000-part ceiling. Teams determine chunk sizes based on payload scale:

Configuring Transfer Thresholds in the AWS CLI

The official AWS Command Line Interface abstracts low-level multipart coordination, but production environments require tuning concurrency and part sizing parameters. You can configure these thresholds in your AWS CLI configuration file (~/.aws/config) or pass them directly via environment flags:

[default]
s3 =
    max_concurrent_requests = 20
    max_queue_size = 10000
    multipart_threshold = 64MB
    multipart_chunksize = 64MB

In this configuration, files exceeding the configured threshold trigger multipart processing, uploading parts in parallel across concurrent worker threads to saturate available bandwidth without exceeding part count constraints.

Multipart uploads provide resilience against network instability. If an individual part fails due to a socket timeout, the client only re-transmits that specific part rather than restarting the entire payload. However, this failure model introduces an operational hazard: orphaned, uncompleted parts.

When a multipart upload starts, every uploaded chunk is immediately committed to persistent S3 storage. If an upload job fails, crashes, or is terminated by an automated CI/CD runner, those uploaded parts remain in the bucket indefinitely.

Uncompleted parts are invisible to standard bucket listing commands. Running aws s3 ls s3://my-bucket/ or viewing the bucket in the AWS Management Console will not show the incomplete object. Yet Amazon S3 continues to meter and bill those hidden bytes at standard S3 storage rates every month.

Inspecting and Aborting Orphaned Multipart Sessions

To identify abandoned multipart uploads that accumulate storage fees, use the S3 API directly:

aws s3api list-multipart-uploads --bucket production-archive-bucket

The command returns an array of active upload sessions:

{
    "Uploads": [
        {
            "UploadId": "EXAMPLE_UP_ID_987654321_abcdef",
            "Key": "backups/2026/quarterly-dataset.tar.gz",
            "Initiated": "2026-08-14T03:22:11.000Z",
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "infra-team",
                "ID": "abc123ownerid456"
            }
        }
    ]
}

If an upload session was initiated weeks earlier and never completed, abort the session manually using its UploadId:

aws s3api abort-multipart-upload \
    --bucket production-archive-bucket \
    --key "backups/2026/quarterly-dataset.tar.gz" \
    --upload-id "EXAMPLE_UP_ID_987654321_abcdef"

Expiring Orphaned Parts With S3 Lifecycle Rules

Relying on manual cleanup scripts across thousands of buckets creates operational debt. The standard AWS production practice is attaching an Amazon S3 Lifecycle configuration that automatically aborts incomplete multipart uploads after a specified retention window.

Create a lifecycle configuration rule that removes uncompleted uploads:

{
    "Rules": [
        {
            "ID": "AbortIncompleteMultipartUploadsAfter7Days",
            "Status": "Enabled",
            "Filter": {},
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 7
            }
        }
    ]
}

Apply this rule to your target bucket using the AWS CLI:

aws s3api put-bucket-lifecycle-configuration \
    --bucket production-archive-bucket \
    --lifecycle-configuration file://lifecycle-mpu-cleanup.json

Once active, Amazon S3 tracks the initiation timestamp of every multipart session. If a session fails to execute CompleteMultipartUpload within the defined window, the storage engine purges all associated parts and stops recurring storage charges.

ETag Checksums and Data Integrity Verification

In standard single PUT operations, the ETag HTTP response header represents the hexadecimal MD5 checksum of the object content. In multipart uploads, the ETag calculation differs.

The multipart ETag is computed through a two-tiered hashing procedure:

1. The client or S3 computes the raw binary MD5 digest of each individual part.
2. The raw binary digests are concatenated in ascending order of part numbers.
3. An MD5 checksum is computed across the concatenated binary string.
4. The resulting hex digest is appended with a hyphen and the total part count.

For example, an ETag formatted as "c9b7e12f6a890453d1e2f3a4b5c6d7e8-42" indicates that the object was assembled from 42 separate parts. Comparing this value against a local file single MD5 hash will result in an apparent mismatch. To verify data integrity, client verification tools must compute the hash using identical chunk boundaries.

Modern S3 ingestion pipelines increasingly configure additional checksum algorithms (SHA-256, CRC32, or CRC32C) during upload initiation. For multipart uploads using additional checksums, part numbers must be uploaded in strictly consecutive sequence, providing end-to-end mathematical verification of massive binary archives.

Connect Large Project Archives to AI Workspaces

Organize project files in a shared workspace where AI agents search indexed content through remote MCP instead of filling prompt windows. Monthly plans start with a 14-day free trial.

Start 14-Day Trial

Autonomous AI agents, coding assistants, and automated multi-agent pipelines interact with corporate data through programmatic tool calls rather than manual consoles. As engineering teams deploy tools like Claude Code, Cursor, Codex, OpenClaw, and custom LLM runners, these systems encounter strict boundaries when processing large files.

While Amazon S3 provides an ideal repository for massive files, passing large objects directly into AI agent workflows creates severe architectural bottlenecks.

Context Window Limits Versus Raw File Sizes

Large language models process token streams, not raw byte buffers. Commercial chat and project interfaces enforce strict file attachment thresholds. In chat interfaces, services enforce discrete per-file size ceilings and file count limits, while project workspaces require all attached documents to fit entirely within the model context window.

Context windows represent working memory, not file storage. Slicing and feeding raw multi-megabyte text into an agent prompt causes three systemic failures:

1. Context Window Overflow: Large text payloads exhaust prompt capacity, truncating document contents and preventing the model from completing its task.
2. Token Inefficiency and Cost Inflation: Pushing hundreds of thousands of raw boilerplate tokens through frontier model APIs incurs substantial operating expenses on every conversation turn.
3. Attention Degradation: LLM reasoning degrades when forced to retrieve factual needles from vast haystacks of unstructured, unparsed file content.

Why Raw S3 Bucket Traversal Stalls AI Agents

To bypass chat attachment caps, developers often provide AI agents with AWS credentials and S3 SDK tools. However, raw object storage was engineered for blob persistence rather than low-latency agent reasoning.

Amazon S3 is a flat key-value store. Directory trees are simulated by inserting forward slashes into object keys. When an agent searches an S3 bucket for relevant documentation, it must execute iterative list_objects_v2 requests, paginate through thousands of keys, download candidate files over HTTPS to local scratch disks, and extract text manually.

Downloading multi-gigabyte files to local disk to answer a specific factual query burns local I/O, consumes network bandwidth, and introduces multi-second latency spikes into agent reasoning loops.

The Large-Corpus Strategy: Decoupling Storage from Agent Retrieval

The solution for autonomous agent teams is decoupling raw object storage from the agent retrieval surface. Rather than forcing agents to download massive files or cramming attachments into prompt windows, organizations place their documents into an indexed workspace like Fast.io for Agents.

The master files remain securely preserved in cloud storage or dedicated project workspaces. When indexed in an intelligent workspace, the files are parsed and vectorized upon arrival. AI assistants connect to the workspace through the remote Model Context Protocol (MCP) server. Teams can review the Fast.io documentation for agent onboarding guidance.

When an agent needs information, it does not download the raw object. Instead, the agent invokes semantic search tools over MCP, retrieving concise, highly relevant text passages accompanied by exact document citations. This keeps the agent prompt lean, minimizes token consumption, and maintains high reasoning accuracy.

Engineering teams choose their storage layer based on access patterns, collaboration needs, and processing requirements. A complete system architecture balances raw object storage, local scratch disks, and collaborative workspaces.

Storage Paradigms for Autonomous Teams

Each storage environment fulfills a specific function in modern software and AI pipelines:

• Raw Amazon S3: The standard choice for unstructured data lakes, cold archival storage, immutable database snapshots, and direct web asset distribution. Amazon S3 object storage provides scalability and durability for objects up to 50 TB using multipart upload, but lacks built-in semantic search, human collaboration surfaces, and native MCP agent tooling.
• Local NVMe and Block Storage (EBS): Essential for high-IOPS compiler scratchpads, transient build caches, and ephemeral agent runtimes. Local storage provides sub-millisecond read/write speeds but remains isolated on a single machine, lacking multi-user visibility and shared versioning.
• Fast.io Intelligent Workspaces: The coordination layer for human teams and AI agents. Fast.io combines cloud file storage with built-in search intelligence, structured document extraction, and native agent connectivity.

Fast.io Capabilities for Modern Project Workspaces

Fast.io delivers a unified environment where humans and autonomous AI agents share the same storage and intelligence layer:

• Intelligence Mode and Built-in RAG: Once Intelligence is enabled on a workspace, incoming files are automatically parsed and indexed for hybrid search. Agents query documents using natural language through the MCP server, receiving grounded answers backed by source citations without external vector databases.
• Metadata Views for Structured Extraction: Fast.io Metadata Views turn documents into live, queryable databases. Users describe the fields they want extracted, and the platform generates a typed schema (Text, Integer, Decimal, Boolean, URL, JSON, Date & Time), populating sortable tables across PDFs, spreadsheets, and scanned documents.
• Advisory File Locks: When multiple agents collaborate in a workspace, race conditions can occur. Agents acquire, heartbeat, and release advisory per-file leases through the MCP storage_manage actions lock-acquire and lock-release, and storage action lock-status. Because locks are advisory, concurrent writes are never dropped, and version history preserves every revision.
• Cloud Import and Sync: Organizations connect existing file repositories directly. Fast.io provides Cloud Sync for Google Drive, Dropbox, Box, and OneDrive (including SharePoint libraries via OneDrive). Cloud Sync operates one-way or two-way, on a schedule or on demand. One-time cloud import is also available.
• Detailed Activity Log: The activity log records file uploads, deletions, and permission updates, ensuring transparency across organizational workflows.

Transparent Pricing and Trial Structure

Fast.io plans are structured around organizational workspaces with predictable billing. Monthly plans start with a 14-day free trial, which requires a credit card. Subscription tiers are Starter at $9.99/mo, Business at $49.99/mo, and Enterprise at $199.99/mo on Fast.io pricing.

Additional storage beyond the plan allowance is billed at 1.5 cents per GB per month, and bandwidth beyond plan limits is billed at 4 cents per GB. Credits meter AI operations, with overage billed at $10 per 100,000 credits. Developers manage workspaces programmatically using the remote MCP server or the official @vividengine/fastio-cli command-line tool.