Uploading Files Directly to Amazon S3 Using HTML Form

Last edit:  Jan 19, 2019

Contributors:  Slashek diana-lakatos lemingos pavelloz

This guide will help you use the Amazon S3 direct upload from the front-end by adding the files from form_builder that are exposed to you for each file uploader.

This guide presents a low-tech demonstration without any JavaScript. It only uses a standard HTML form to show you what exactly you need to include in the request to be able to directly upload to S3.

In most cases you will want to do the upload using AJAX, because doing it like in this guide has some limitations:

  • Standard submit redirects the user to a response page, which in this case is on AWS domain.
  • You cannot submit any data to our system using a form that is sending data to AWS.
  • The URL with uploaded file needs to be sent to our system to inform it where the file lies. Without JavaScript, this is very difficult/impossible.

Requirements

This is an advanced tutorial. To follow it, you should be familiar with basic platformOS concepts, HTML, Liquid, APIs and Form Configurations, and the topics in the Get Started section.

This guide assumes that there is a user profile called developer which is defined as such:

marketplace_builder/user_profile_types/developer.yml

name: developer
properties:
- name: bio
  type: string
- name: avatar
  type: photo
- name: banner
  type: photo
- name: cv
  type: custom_attachment

This guide uses the following form configuration that is used to update developers profile:

marketplace_builder/form_configurations/developer/update_profile.liquid


---
name: update_profile
resource: User
flash_notice: Congratulations, your profile has been updated.
redirect_to: /update_profile
fields:
  profiles:
    developer:
      properties:
        bio:
      custom_attachments:
        cv:
          file:
      custom_images:
        avatar:
          image:
        banner:
          image:
---

{% include 'update_profile/html_form' %}
{% include 'update_profile/ajax' %}
{% include 'update_profile/name_with_ajax' %}

A simple page renders the form mentioned above:

marketplace_builder/views/pages/developer/update_profile.liquid


---
slug: update_profile
---
{% include_form 'update_profile', id: context.current_user.id %}

Presigned metadata

Access all metadata required to post data to S3 through the field's s3_upload key.

Example object converted to JSON:

{
   "form_data": {
      "key": "uploads/d123dc4d-f134-41bc-b984-3abdaf89fe8d/${filename}",
      "success_action_status": "201",
      "acl": "public-read",
      "x-amz-meta-original-filename": "${filename}",
      "policy": "eyJleHBpcmF0aW9uIjoiMjAxOC0wNy0yOVQxMzowODozN1oiLCJjb25kaXRpb25zIjpbeyJidWNrZXQiOiJuZWFyLW1lLW9yZWdvbi1zdGFnaW5nIn0sWyJzdGFydHMtd2l0aCIsIiRrZXkiLCJ1cGxvYWRzL2QxMjNkYzRkLWYxMzQtNDFiYy1iOTg0LTNhYmRhZjg5ZmU4ZC8iXSx7InN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyI6IjIwMSJ9LHsiYWNsIjoicHVibGljLXJlYWQifSxbInN0YXJ0cy13aXRoIiwiJHgtYW16LW1ldGEtb3JpZ2luYWwtZmlsZW5hbWUiLCIiXSxbInN0YXJ0cy13aXRoIiwiJHgtYW16LW1ldGEtd2lkdGgiLCIiXSxbInN0YXJ0cy13aXRoIiwiJHgtYW16LW1ldGEtaGVpZ2h0IiwiIl0seyJ4LWFtei1jcmVkZW50aWFsIjoiQUtJQUpDMzdaNlhDT0NSMjQ1WUEvMjAxODA3MjkvdXMtd2VzdC0yL3MzL2F3czRfcmVxdWVzdCJ9LHsieC1hbXotYWxnb3JpdGhtIjoiQVdTNC1ITUFDLVNIQTI1NiJ9LHsieC1hbXotZGF0ZSI6IjIwMTgwNzI5VDEyMDgzN1oifV19",
      "x-amz-credential": "AKIAJC37Z6XCOCR245YA/20180729/us-west-2/s3/aws4_request",
      "x-amz-algorithm": "AWS4-HMAC-SHA256",
      "x-amz-date": "20180729T120837Z",
      "x-amz-signature": "d6763dcae2c01c87859ed9735dfea319178e548420c3773c4a3bd348a98b671d"
   },
   "direct_upload_url": "https://near-me-oregon-staging.s3-us-west-2.amazonaws.com"
}

Steps

Uploading content directly to Amazon S3 is a three-step process:

Step 1: Obtain required metadata from the server

Access all metadata required to post data to S3 through the field's s3_upload key.

Example for property of image type:


{{ form_builder.fields.profiles.expert.custom_images.photo.image.s3_upload }}

Example for property of custom attachment type:


{{ form_builder.fields.profiles.expert.custom_attachments.cv.file.s3_upload }}

Step 2: Prepare HTML form with fields including required metadata

Set up action and all required data in hidden inputs:

marketplace_builder/form_configurations/developer/update_profile.liquid


{% assign profile = form_builder.fields.profiles.developer %}
{% assign avatar = profile.custom_images.avatar %}
{% assign image_s3_upload = avatar.image.s3_upload %}

<form action="{{ image_s3_upload.direct_upload_url }}" accept-charset="UTF-8" method="post" enctype="multipart/form-data">

  {% comment %} Loop over all form_data keys and create hidden inputs in the form to send the data to S3 {% endcomment %}
  {% for input in image_s3_upload.form_data %}
    <input type="hidden" name="{{ input[0] }}" value="{{ input[1] }}" />
  {% endfor %}

  {% comment %} Obligatory metadata key, even though not used in this case {% endcomment %}

  {% comment %} Partial with  {% endcomment %}
  <h4>Upload your image</h4>
  <input name="file" type="file" />

  <button class="btn btn-primary">Submit image</button>
</form>

Step 3: Submit form with metadata + file

This uploads the file to S3 and gives you the URL in the response. Response is in XML and the URL you are interested in is named Location.

Example S3 response:

<PostResponse>
   <Location>https://near-me-oregon-staging.s3-us-west-2.amazonaws.com/uploads%2F9e8d2647-0057-43b6-b422-7aa8dda33181%2Ftest.jpg</Location>
   <Bucket>near-me-oregon-staging</Bucket>
   <Key>uploads/9e8d2647-0057-43b6-b422-7aa8dda33181/test.jpg</Key>
   <ETag>"ef6af0efd437d68085cf841d5c668b20"</ETag>
</PostResponse>

Important notes / troubleshooting

General

  • Instead of sending an image to our server you will send a url (returned by S3 response) to the image inside the text field for given image. Image URL should look similar to https://near-me-oregon-staging.s3-us-west-2.amazonaws.com/uploads%2Fbc50d6da-c17e-477d-b143-c65422c221fd%2Ftest.3+%282%29.png
  • After form submission give our server couple of seconds to pickup your image, generate size versions, compress them and upload to the new S3 location so the CDN could pick it up for you and datatabase to be updated with CDN links

If you have problems that you cannot figure out, go to live example page, inspect whats being sent to S3 and our server respectively and find differences between your requests and those that work - the difference usually is whats broken

AWS

  • Make sure your form is using POST method
  • Make sure your form is sending data as enctype="multipart/form-data"
  • Do not include any fields that are not listed in the s3_upload key, otherwise AWS will return error 403 with message Invalid according to Policy: Extra input fields: xxx
  • Make sure file input is sent last in the form
  • Remember to set your file input name to file

Read more about aws requirements.

Live example and source code

To play with a live example, create a developer account at https://examples.platform-os.com and go to the update_profile page.

Source code can be found on GitHub.

Additional resources

Questions?

We are always happy to help with any questions you may have. Check out our Help page, or contact us.