Uploading Files Directly to Amazon S3 Using Uppy
This guide will help you to create a user-friendly image upload feature with image processing.
Note
This example uses some Twitter Boostrap classes to make it a little bit prettier.
Requirements
This is an advanced tutorial. To follow it, you should be familiar with basic platformOS concepts, HTML, JavaScript, Liquid, GraphQL, and some topics in the Get Started section.
Steps
Uploading images directly to Amazon S3 is a four-step process:
Step 1: Create Table and image processing configuration
There will be only one property in Table used in this example (photo), but it will generate 4 different versions of the image uploaded by the user. Apart from explicitly defined versions, there is also an original version that is always there.
modules/direct_s3_upload/public/schema/uppy.yml
name: uppy
properties:
- name: photo
type: upload
options:
content_length:
lte: 2048 # maximum file size will be 2048KB
versions:
- name: large_bw # version name
output:
format: jpeg # output format
quality: 70 # output quality
resize:
width: 1600 # width of the image - in this case max width
height: 1200 # height of the image - in this case max height
fit: inside # preserve aspect ratio, do not crop the image, make sure all pixels are within viewport
without_enlargement: false # if image is smaller than width/height, do not enlarge it
manipulate:
greyscale: true # make image greyscale (similar to css "filter: grayscale(1)")
blur: 20 # blur using gaussian mask (similar to css "filter: blur(1.5rem)")
flatten: red # merge alpha transparency channel (if exists) with given color
- name: small
output:
format: webp
quality: 80
resize:
width: 640
height: 480
fit: cover
position: top # define the gravity direction or strategy when cropping the image using fit cover or contain
- name: thumb
output:
format: png
quality: 80
resize:
width: 200
fit: contain
background: "rgba(255, 255, 255, 0.5)" # if resized image is not covering whole viewport, set background color
- name: center
output:
format: jpeg
quality: 100
resize:
height: 400
width: 400
fit: cover
Step 2: Get presign data and expose it using a form
To upload a file to AWS S3 directly, you will need the server-generated data called signature. This information is from GraphQL. It takes table and property_name as arguments. Other options like versions, content_length, ACL are taken from property configuration.
modules/direct_s3_upload/public/views/pages/images.liquid
---
slug: direct-s3-upload/images
---
{% graphql data %}
mutation presign {
property_upload_presigned_url(table: "modules/direct_s3_upload/uppy", property_name: "my_upload") {
upload_url
upload_url_payload
}
}
{% endgraphql %}
{% assign data = data.property_upload_presigned_url %}
<form action="{{ data.upload_url }}"
data-s3-uppy="form"
data-s3-uppy-user-id="{{ context.current_user.id }}"
hidden>
{% for field in data.upload_url_payload %}
<input type="hidden" name="{{ field[0] }}" value='{{ field[1] }}'>
{% endfor %}
</form>
Below the form, paste the code responsible for containing Uppy.
modules/direct_s3_upload/public/views/pages/images.liquid
<div class="row">
<div class="col-5">
<div class="card">
<div class="card-body">
<div id="drag-drop-area"></div> <!-- This is where Uppy will inject itself -->
</div>
</div>
</div>
<div class="col-7">
<div class="card">
<div class="card-body">
<p>URLs to image returned by AWS (input):</p>
<small>
<ol data-s3-uppy="log"></ol>
</small>
</div>
</div>
</div>
</div>
Step 3: Configure Uppy and use presigned data
Uppy is a JavaScript widget that helps to handle uploads. With its S3 plugin on board, you can make the implementation much shorter and easier.
However, before you can use any scripts, you need to load them.
modules/direct_s3_upload/public/views/pages/images.liquid
<link href="https://transloadit.edgly.net/releases/uppy/v1.6.0/uppy.css" rel="stylesheet">
<script src="https://transloadit.edgly.net/releases/uppy/v1.6.0/uppy.js" defer></script>
<script src="https://cdn.jsdelivr.net/npm/sweetalert2@9" defer></script>
<script src="{{ 'modules/direct_s3_upload/images.js' | asset_url }}" defer></script>
This is the final images.js file, responsible for uploading the image, logging its URL, creating the record, and showing the confirmation box.
modules/direct_s3_upload/public/assets/images.js
const _form = document.querySelector('[data-s3-uppy="form"]');
const _log = document.querySelector('[data-s3-uppy="log"]');
const uppy = Uppy.Core({
autoProceed: true,
restrictions: {
maxFileSize: 2097152, // Limit size to 2 MB on the javascript side
maxNumberOfFiles: 3,
allowedFileTypes: ['image/png', 'image/jpeg', 'image/webp']
}
})
.use(Uppy.Dashboard, {
inline: true,
target: '#drag-drop-area',
note: 'Images only, up to 3 files, 2MB each',
width: '100%',
height: 350
})
.use(Uppy.DragDrop)
.use(Uppy.GoldenRetriever)
.use(Uppy.AwsS3, {
getUploadParameters() {
// 1. Get URL to post to from action attribute
const _url = _form.getAttribute('action');
// 2. Create Array from FormData object to make it easy to operate on
const _formDataArray = Array.from(new FormData(_form));
// 3. Create a JSON object from array
const _fields = _formDataArray.reduce((acc, cur) => ({ ...acc, [cur[0]]: cur[1] }), {});
// 4. Return resolved promise with Uppy. Uppy it will add file in file param as the last param
return Promise.resolve({
method: 'POST',
url: _url,
fields: _fields
});
}
});
uppy.on('upload-success', (_file, data) => {
const li = document.createElement('li');
li.textContent = data.body.location;
_log.appendChild(li);
});
uppy.on('complete', ({ failed, successful }) => {
/*
For every successfully uploaded image to S3, send request to the Instance
that will create a record with the uploaded image's URL as direct_url param.
*/
Promise.all(successful.map(({ response }) => createImage(response.body.location)))
.then(() => {
// Show confirmation box (SweetAlert2) after all the images has been created
Swal.fire({
title: 'Images uploaded',
icon: 'success',
text: 'Press refresh to display the results',
confirmButtonText: 'Refresh',
showCloseButton: true
}).then(result => {
if (!result.value) {
return;
}
// Reload page after "Refresh" button has been clicked inside Sweet Alert2
window.location.reload();
});
});
});
const createImage = imageUrl => {
// Get logged in user id
const userId = _form.dataset.s3UppyUserId;
// Create record for this user with s3 image url
return fetch('/direct-s3-upload/images/record_create', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Accept: 'application/json'
},
body: JSON.stringify({ direct_url: imageUrl, user_id: userId })
}).then(response => response.json());
};
Step 4: Create page that will create the record with the image
For fetch to succeed, you need to create a page under the /direct-s3-upload/images/record_create path. It will execute the record_create mutation with params passed in the XHR request.
modules/direct_s3_upload/public/views/pages/record_create.liquid
---
slug: direct-s3-upload/images/record_create
method: post
---
{% if context.params.user_id %}
{% graphql record, direct_url: context.params.direct_url, user_id: context.params.user_id %}
mutation record_create($direct_url: String!, $user_id: ID!) {
record_create(record: {
table: "modules/direct_s3_upload/uppy"
user_id: $user_id
properties: [{ name: "my_upload" value: $direct_url }]
}) {
id
}
}
{% endgraphql %}
{{ record | fetch: "record_create" }}
{% endif %}
Note
You might want to add an authorization policy to ensure that users can only add images to their own accounts.
Step 5: Show the results
To check the results of image processing, get some of them from the database and display them.
First you need a query to fetch records.
modules/direct_s3_upload/public/graph_queries/get_records.graphql
query get_records($per_page: Int = 5, $user_id: ID!) {
records(
per_page: $per_page
sort: { created_at: { order: DESC } }
filter: {
table: { value: "modules/direct_s3_upload/uppy" },
user_id: { value: $user_id }
}
) {
total_entries
results {
id
my_upload: property_upload(name: "my_upload") {
url
versions
}
}
}
}
Then you can use it to display images.
modules/direct_s3_upload/public/views/pages/images.liquid
<!-- Get recently uploaded images for this user, in all versions -->
{% graphql records = "modules/direct_s3_upload/get_records", user_id: context.current_user.id | dig: "records" %}
{% if records.total_entries > 0 %}
<hr>
<div class="my-2 row">
<div class="col-9">
<h3>Last 5 processed images:</h3>
</div>
</div>
<!-- Show all versions next to each other -->
{% for record in records.results %}
{% assign versions = record | dig: "my_upload", "versions" %}
<div class="card my-4">
<div class="card-body">
<div class="row">
<div class="col-sm-2">
<h4>Original</h4>
<a href="{{ record.my_upload.url }}"><img src="{{ record.my_upload.url }}" alt="Original" width="150"></a>
</div>
{% for version in versions %}
<div class="col-sm-2">
<h4>{{ version[0] }}</h4>
<a href="{{ version[1] }}"><img src="{{ version[1] }}" alt="{{ version[0] }}" width="150"></a>
</div>
{% endfor %}
</div>
</div>
</div>
{% endfor %}
{% endif %}
Important notes / troubleshooting
General
- Instead of sending an image to our server you will send a url (returned by S3 response) to the image inside the text field for given image. Image URL should look similar to
https://near-me-oregon-staging.s3-us-west-2.amazonaws.com/uploads%2Fbc50d6da-c17e-477d-b143-c65422c221fd%2Ftest.3+%282%29.png - After form submission give our server couple of seconds to pickup your image, generate size versions, process them and upload to the new S3 location so the CDN could pick it up for you and datatabase to be updated with CDN links
If you have problems that you cannot figure out, go to live example page, inspect whats being sent to S3 and our server respectively and find differences between your requests (params, headers, method, form encoding) and those that work - the difference usually is whats broken.
AWS
- Make sure your form is using POST method
- Make sure your form is sending data as
enctype="multipart/form-data" - Do not include any fields that are not listed in the
upload_url_payloadkey, otherwise AWS will return error 403 with messageInvalid according to Policy: Extra input fields: xxx - Make sure file input is sent last in the form
- Remember to set your file input name to
file
Read more about aws requirements.
Uppy
If you use Uppy and its S3 plugin as we did in this tutorial, you dont have to worry about most AWS notes.
Live example and source code
To play with a live example, create a developer account at https://examples.platform-os.com and go to the Direct S3 upload images page.
Source code can be found on GitHub.