Using DigitalOcean Spaces or Amazon S3 for file storage in XF 2.x

Using DigitalOcean Spaces or Amazon S3 for file storage in XF 2.x

No permission to download
XenForo Version Compatibility

The following guide will work in either XF 2.0 or XF 2.1. You need to ensure you are using the correct version of the add-on files.

Both are included when you click the download button, please ensure you download version 2.0.x if you are using XenForo 2.0.x and please ensure you download version 2.1.x if you are using XenForo 2.1.x.

Why this guide?

Since XenForo 2.0.0 we have supported remote file storage using an abstracted file system named Flysystem. It's called an abstracted file system as it adds a layer of abstraction between the code and a file system. It means that it provides a consistent API for performing file system operations so that whether the file system is a local disk-based file system or a distributed and remotely accessible file system, our code calls the same functions and Flysystem takes care of the rest.

As useful as that is, it isn't the most obvious or straightforward thing to set up so this guide and accompanying add-on will help.

So, if you're planning to make use of the video uploads function in XF 2.1 and you're worried about increased disk space requirements, or even if you're staying with 2.0 for a while and you just need to offload your storage requirements elsewhere, this will help.


Making the required files available

Although it is possible for you to download the files and set up things like the autoloader yourself, you will probably prefer to simply download the add-on that is attached to this resource. You can install the add-on in the usual fashion.


Before you start

If you're setting this up on an existing site, you will need to manually move your existing files over. There's a section about that at the end. While you are moving the existing files, setting things up and testing, we recommend closing the forum first.


Autoloading the AWS SDK (for XenForo 2.0.x only)

The first thing to do is set up the autoloading of the AWS SDK.

Open your src/config.php file.

There are a number of different vendor packages involved and provided by the add-on attached to this resource, the following lines ensure they are all autoloaded. We are doing this here because we need the files to be available as early as possible in the request:
PHP:
\XFAws\Composer::autoloadNamespaces(\XF::app());
\XFAws\Composer::autoloadPsr4(\XF::app());
\XFAws\Composer::autoloadClassmap(\XF::app());
\XFAws\Composer::autoloadFiles(\XF::app());
Now we're ready to set up a specific implementation. Look below for the Setting up DigitalOcean Spaces guide or skip ahead for the Setting up Amazon S3 guide.


Setting up DigitalOcean Spaces

We'll cover this first as it is the most straightforward to set up. If you'd prefer to use Amazon S3 skip ahead to the Setting up Amazon S3 section below.
  1. Go to the DigitalOcean Cloud page and sign up or log in.
  2. At this point, if you're new to DigitalOcean, you may need to set up billing.
  3. You will now be able to create a new project.
  4. Click the "Start using Spaces" link.
  5. Choose your datacenter region (I have chosen Amsterdam).
  6. Leave "Restrict File Listing" selected.
  7. Choose a unique name (I have chosen "xftest")
  8. Click "Create a space"
Now the space is created, you should have an endpoint URL, similar to: https://xftest.ams3.digitaloceanspaces.com. Note this down for later.

Now we need to create some API credentials. To do this:
  1. Click "Manage" in the left sidebar.
  2. Click "API".
  3. In the "Spaces access keys" section click "Generate New Key".
  4. Type a name for the key (Again, I have chosen "xftest") and save.
This will give you a key and a secret. Note them down.

Configuring XF to use DigitalOcean Spaces

We now need to configure XF to use DigitalOcean Spaces for file storage. We'll start with what usually goes into the data directory first. This generally includes attachment thumbnails and avatars.

Open your src/config.php file.

First we need to configure the Amazon S3 client (the DigitalOcean Spaces API is compatible with the Amazon AWS SDK).

We will do this using a closure so that we can reuse the same code and we only have to type it out once:
PHP:
$s3 = function()
{
   return new \Aws\S3\S3Client([
      'credentials' => [
         'key' => 'ABC',
         'secret' => '123'
      ],
      'region' => 'ams3',
      'version' => 'latest',
      'endpoint' => 'https://ams3.digitaloceanspaces.com'
   ]);
};
Note that the key and secret are what you noted down after setting up the "Spaces access key" earlier. The region can be inferred from the endpoint URL you noted down earlier. It's the part after the first . in the URL, in my case it is ams3. The endpoint is the same endpoint URL minus the unique name you chose.

Next we need to set up the actual Flysystem adapter to use the S3 client:
PHP:
$config['fsAdapters']['data'] = function() use($s3)
{
   return new \League\Flysystem\AwsS3v3\AwsS3Adapter($s3(), 'xftest', 'data');
};
Finally, we need to ensure that avatar and attachment thumbnail URLs are prepended with the correct URL. This requires the endpoint URL you noted down earlier, again:
PHP:
$config['externalDataUrl'] = function($externalPath, $canonical)
{
   return 'https://xftest.ams3.digitaloceanspaces.com/data/' . $externalPath;
};
At this point, everything should be working in terms of new uploads. Don't be alarmed if you notice that avatars and thumbnails are missing; if you have existing files, they will need to be moved over manually which we'll go through later.

First, we need to test that the configuration works. Simply go and upload a new avatar. The avatar will now be stored and served remotely!

If you check your DigitalOcean Spaces account now, you should see that new folders have been created containing your new avatar:

cloud-digitalocean-com_spaces_xftest_i-3f9b71-path-data-2favatars-2fo-2f0-2f-png.187981
Author
dinhphucv
Downloads
2
First release
Last update
Rating
0,00 star(s) 0 ratings

More resources from dinhphucv

Top