Sending a 5 GB video file might seem simple. You select the file, click Upload, wait for the progress bar to finish, and share a link with someone. But there are several things happening in the background while the file is being uploaded.
The file has to travel across a network, reach a server, and be stored somewhere before another person can download it. The service also needs to make sure that the recipient can access the file without accessing the sender’s account. If the internet connection drops halfway through the upload, it needs a way to continue without sending the entire file again.
File transfer services use HTTP requests, storage systems, databases, encryption, and background processes to manage these tasks. But how does a website actually handle a file that’s several gigabytes in size? Does it upload everything in one request? Where does the file go after the upload finishes? And what happens when its download link expires?
In this article, we’ll look at why email attachments fail, how file transfer services are architected, and how chunked, resumable uploads let large files survive unreliable connections.
Why Email Attachments Fail: SMTP Wasn’t Designed for Large Files
Before looking at dedicated file transfer services, it helps to understand why we can’t just attach massive files to an email. Sending large files through email is fundamentally flawed because the underlying architecture was built for text, not high-resolution media.
SMTP and Attachment Size Limits
Simple Mail Transfer Protocol (SMTP) handles the transmission of email messages across the internet. Because servers have limited bandwidth and storage, email providers impose strict attachment size limits, usually capping out around 25 MB. However, these limits apply to the encoded message size, not just the original file sitting on your hard drive.
The Base64 Problem
Email attachments are commonly encoded using MIME and Base64 formats. Base64 is necessary because it converts binary data into text-compatible characters that email servers can safely route. While this solves the transmission problem, it creates a massive efficiency issue: this encoding increases the data size by approximately 33%.
You can read the foundational specifications for this in RFC 2045, which defines MIME’s Base64 transfer encoding. In practice, this means a 30 MB video file will require roughly 40 MB of space before additional message overhead is even calculated, guaranteeing it will bounce back from most email servers.
The Core Architecture: What Happens Behind a File Transfer Service?
To bypass the limitations of email, file transfer services use a specialized, decoupled architecture. Having built a few of these systems myself, I believe establishing a clear mental model of the five major components is crucial for understanding the flow of data.
The Client
The journey begins in the client, which is usually a web browser or a desktop application where the sender selects a file. Modern browsers use JavaScript’s File API to access file metadata, read the raw contents and intelligently divide the file into smaller, manageable parts before any data hits the network.
The Upload Server
Once the client starts sending data, the upload server steps in. This server receives the upload requests, authenticates the user, validates incoming data and coordinates the entire upload process. It acts as the traffic cop, ensuring that chunks of data are arriving in the correct format.
The Storage Layer
Traditional application servers are terrible at holding massive files, so the storage layer handles the actual heavy lifting. Uploaded files are typically routed directly to object storage solutions. This allows the system to scale infinitely without crashing the primary upload servers.
The Metadata Database
While the physical file goes to object storage, the application needs a way to remember what that file is. The metadata database stores critical information, including the original filename, file size, content type, the specific object storage key, a unique transfer identifier and the exact expiration timestamp.
The Download Server
When someone opens a shared link, the download server takes over. It checks the database to verify permissions and expiration dates, and if everything is valid, it retrieves the stored object from the storage layer for delivery.
Chunked and Resumable Uploads: How Large Files Survive Bad Internet Connections
Transferring a 5 GB file in one continuous network request is a disaster waiting to happen. If a connection drops at 99%, a single-request architecture forces the user to start all over again.
Breaking a Large File Into Smaller Chunks
To solve this, modern clients divide large files into smaller parts, such as 10 MB chunks. Each chunk is uploaded to the server as an independent HTTP request. The server meticulously tracks which chunks have arrived. Some advanced systems even allow multiple chunks to upload simultaneously in parallel, drastically reducing the overall waiting time.
It’s important to note that chunked uploads are an architectural pattern, which is entirely different from HTTP chunked transfer encoding (a mechanism for framing an HTTP message body).
What Happens When the Internet Disconnects?
Suppose 80% of a massive file has successfully uploaded when your Wi-Fi randomly disconnects. Without resumable uploads, that progress is lost. With resumable architecture, the client reconnects and asks the server for the upload offset, the exact number of bytes successfully received and stored. The client then seamlessly continues uploading from that point, sending only the remaining chunks instead of the entire file again.
Because progress is tracked on the server, a dropped connection costs only a short delay rather than a full restart. This is what lets file transfer services handle multi-gigabyte uploads reliably, even on unstable networks.
