How to upload big (chunked) files in Laravel without extra packages

Nov 5th 2023

If you want to upload big files using laravel you probably should get some help from a library like this one here. But if you need a very straightforward solution, one that doesn't handle edge cases, or you just want have an idea of how it could be done then this tutorial is for you.

The frontend

Fist step is to install a library to manage uploads on the frontend. For this tutorial we're going to use Filepond. It can be installed via CDN (by dropping the css and js in the view) or via npm, I've chosen the npm way using Vite so

npm install filepond

then we open up resources/js/bootstrap.js and add

import * as FilePond from 'filepond';
import 'filepond/dist/filepond.min.css';

window.FilePond = FilePond;

don't forget to @vite('resources/js/app.js') into your blade view. Now you will have a Filepond object available globally.

To simplify things we're going to put all the backend logic in the routes file using PHP closures. The only file we're going to create is a view that we will call chunked-tutorial.blade.php. So, we prepare the routes:

Route::get('chunked-tutorial', function() {
    return view('chunked-tutorial');
});
Route::post('chunked-tutorial', function(){})->name('chunked-tutorial');
Route::patch('chunked-tutorial', function(){})->name('chunked-tutorial');

then we prepare the chunked-tutorial.blade.php view:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Chunked Tutorial</title>
    @vite('resources/js/app.js')
</head>
<body>
    <h1>Chunked Tutorial</h1>
    <input
        type="file"
        id="bigfile"
        name="bigfile"
        data-route="{{ route('chunked-tutorial') }}"
        data-csrf="{{ csrf_token() }}"
    >
</body>
</html>

We don't need to have a form because the file is sent via ajax (and I'm trying to keep things simple). We need to provide a csrf token and a destination for the upload, there are many ways of doing this, I think adding a data attribute is the simplest way for this tutorial.

We now transform the input into a Filepond uploader:

<script>
    document.addEventListener('DOMContentLoaded', function() {
        const inputElement = document.getElementById('bigfile');

        // Create a FilePond instance
        const pond = FilePond.create(inputElement, {
            allowMultiple: false,
            chunkUploads: true,
            chunkSize: 100000, // in bytes
            server: {
                url: inputElement.dataset.route,
                headers: {
                    'X-CSRF-TOKEN': inputElement.dataset.csrf
                }
            }
        });
    });
</script>

with this the frontend site is complete. For this tutorials I've put a very low chunk size, but you can put a much higher one.

The backend

How does Filepond uploads files? If the file is smaller than a chunk, in our case smaller than 100kb, it will send the whole file in one go with the first POST request. If the file is bigger than a chunk, it will actually do chunked uploads, which means that it will send a first POST request to ask the backed to prepare to receive the chunks, and then it will send the file's chunks in subsequent PATCH requests.

Prepare to receive a whole file when it's smaller than a chunk

let's reach the POST route for this tutorial and change it so it will store the file when it's sent whole

Route::post('chunked-tutorial', function(Request $request){
    if($request->hasFile('bigfile')) {
        $file = $request->file('bigfile');
        $newFilename = uniqid(). '-' .$file->getClientOriginalName();
        $file->storeAs('tutorial-uploads', $newFilename);
        return $newFilename; 
    }
})->name('chunked-tutorial');

Now try to upload a file that is less than 100kb, and when we reach storage/app there should be a tutorial-uploads folder which contains said file. We then return the server's filename and this could be used by Filepond for further operations (for example allowing the user ti delet the file if he changes his mind).

Receiving a file in chunks

Now that we have managed the case in which the file is sent whole, let's finally manage the case in which the file is big and sent in chunks!

When we upload a big file the first thing that Filepond does is sending a POST request where it asks the server to prepare to receive the file. Filepond expects back an id that will be then sent in the following PATCH requests along with the file's chunks.

We're still modifying the POST route closure

Route::post('chunked-tutorial', function(Request $request){
    if($request->hasFile('bigfile')) {
        $file = $request->file('bigfile');
        $newFilename = uniqid(). '-' .$file->getClientOriginalName();
        $file->storeAs('tutorial-uploads', $newFilename);
        return $newFilename;
    }

    $fileId = uniqid().'.part';
    Storage::put('tutorial-uploads/tmp/'.$fileId, '');
    return $fileId;
})->name('chunked-tutorial');

We're using Illuminate\Support\Facades\Storage to create an empty file into the storage/app/tutorial-uploads/tmp folder. The $fileId will contain the filename of the temporary file, that will also work as identification for the subsequent Filepond's requests. We return $fileId and Filepond begins to send PATCH requests. Each PATCH request contains

the chunk
the header with
1. Upload-Length (The total size of the file being transferred)
2. Upload-Name (The name of the file),
3. Upload-Offset (The offset of the chunk being transferred, not used by us)
4. Content-Type (the content type of a patch request, set to 'application/offset+octet-stream')
the id of the file we sent back with the first post request in the query

So, knowing this we can create the functionality that puts together the chunks:

Route::patch('chunked-tutorial', function(Request $request){
    $fileId = $request->query('patch');
    Storage::append('tutorial-uploads/tmp/'.$fileId, $request->getContent(), NULL);

    if(Storage::size('tutorial-uploads/tmp/'.$fileId) == $request->header('Upload-Length')) {
        $newFilename = uniqid(). '-' . $request->header('Upload-Name');
        Storage::move('tutorial-uploads/tmp/'.$fileId, 'tutorial-uploads/'.$newFilename );
        return $newFilename;
    }
    return $fileId;
})->name('chunked-tutorial');

Let's go step by step:

$fileId = $request->query('patch'): we retrieve the file name
Storage::append('tutorial-uploads/tmp/'.$fileId, $request->getContent(), NULL): we append the content of the request, the file's chunk, in the temporary file. The NULL is needed to avoid line breaking which would ruin the file.
when the file's size on disk matches the expect whole file size (Upload-Length) we get the original filename, we add to it a new id to avoid file collision (of course this is something I'm doing for this tutorial, you may have your own method) and we move the now completed file to a new location.

If we've done everything correctly, when we upload a file bigger than 100kb, we will see it uploaded in storage/app/tutorial-uploads/

See you in the next tutorial