How to Mount S3 Bucket on CentOS and Ubuntu using S3FS

Posted on February 1st, 2017

In this tutorial we can check how to mount S3 bucket on your InterServer VPS or Dedicated Server.

S3FS is a FUSE (File System in User Space) will mount Amazon S3 as a local file system. S3FS has an ability to manipulate Amazon S3 bucket in many useful ways. If you wish to access your Amazon S3 bucket without mounting it on your server, you can use s3cmd command line utility to manage S3 bucket.

 

What is an Amazon S3 bucket?

Amazon S3 is a cloud based web service interface that you can used to store and retrieve any amount of data. To upload your data, first you need to create an S3 bucket in one of the Amazon regions.

 

Creating a Bucket

S3 provides an API for creating and managing buckets. You can create a maximum of 100 buckets from your AWS console. When you create a bucket, you need to provide a name and AWS region where you want to create the bucket. In each bucket, you can store any number of objects. You can use your AWS account root credentials to create a bucket, but it is not recommended. Instead  just create an IAM user and add full permission to that user on S3 bucket. You can access your S3 bucket from your Amazon S3 console.

Please follow the below steps to mount s3 bucket on your server.

 

1) Remove Existing Packages

Before installing any package, first you need to check if you have any existing fuse or S3FS on your server. If it is already existing, then remove it from your server to avoid further conflicts. Use the following command to check if you have any existing fuse or S3FS on your server

CentOS users:

$ yum remove fuse fuse-s3fs

Ubuntu Users:

$ apt-get remove fuse

 

2) Install Packages

Install all dependency packages for fuse and s3cmd using the below command.

CentOS users:

$ yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap

Ubuntu Users:

$ apt-get install build-essential libcurl4-openssl-dev libxml2-dev mime-support

 

3) Download and Compile Fuse

Move to /usr/src then download and compile fuse source code. After compiling, add fuse to kernel. In my case the latest version of fuse is fuse-3.0.0

$ cd /usr/src/

$ wget https://github.com/libfuse/libfuse/releases/download/fuse-3.0.0/fuse-3.0.0.tar.gz

$ tar xzf fuse-3.0.0.tar.gz

$ cd fuse-3.0.0

$ ./configure –prefix=/usr/local

$ make && make install

$ export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

$ ldconfig

$ modprobe fuse

 

4) Download and compile S3FS

Navigate to /usr/src, Download and compile s3fs source code.

$ git clone https://github.com/s3fs-fuse/s3fs-fuse.git

$ cd s3fs-fuse

$ ./autogen.sh

$ ./configure

$ make

$ make install

 

5) Setup Access Key

Both access key and secret key of your s3 AWS account is required for configuring S3FS. Replace the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with your actual key values.

$ vi /etc/passwd-s3fs

AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY

Make sure that the file has proper permission.

$ chmod 600 /etc/passwd-s3fs

 

6) Mount S3 Bucket

You can run the below command to mount s3fs.

$ s3fs mybucket /path/to/mountpoint -o passwd_file=/etc/passwd-s3fs

You can also mount the s3 bucket on boot by following below commands.

$ mkdir /tmp/cache

$ mkdir /path/to/mountpoint

$ chmod 777 /tmp/cache /path/to/mountpoint

$ vi /etc/fstab

s3fs#<mybucket> /path/to/mountpoint fuse allow_other,use_cache=/tmp/cache,uid=userid,gid=groupid 0 0

$ mount -a

 

Congratulations you have successfully mounted s3 bucket on your server.

 

If you need any further assistance please contact our support department.

 

 

18 Responses to “How to Mount S3 Bucket on CentOS and Ubuntu using S3FS”

  1. ylluminate says:

    s3fs `./configure` fails every time on CentOS 7 with:

    configure: error: Package requirements (fuse >= 2.8.4 libcurl >= 7.0 libxml-2.0 >= 2.6) were not met:

    No package ‘fuse’ found

    Consider adjusting the PKG_CONFIG_PATH environment variable if you
    installed software in a non-standard prefix.

    Alternatively, you may set the environment variables common_lib_checking_CFLAGS
    and common_lib_checking_LIBS to avoid the need to call pkg-config.
    See the pkg-config man page for more details.

    • Jithin says:

      You are getting that error because package requirement is not met. The required packages and the required versions are mentioned in the error report and you have to install the right versions to make this work. Since this is considered as third-party our support team may not be able to do this for you. But if you have tried everything and if nothing is working, we can take a look : https://www.interserver.net/resolve

    • Johnny says:

      If you already installed fuse, try to install “fuse-devel”.
      #> yum install fuse-devel -y
      and run it again.
      This is the way I fixed on my CentOS 7, For your reference.

  2. ylluminate says:

    @Jithin & @Johnny please note this ticket for details on the failure:
    https://github.com/s3fs-fuse/s3fs-fuse/issues/536

  3. RP says:

    Does anyone know how to make this accessible by ALL users on the linux Server?

    Right not when I run the s3fs command as root.

    s3fs #BUCKETNAME /mnt/AWSS3 -o passwd_file=/etc/passwd-s3fs -o allow_other,use_cache=/root/cache

    OR

    s3fs bbcorp-qa /mnt/AWSS3 -o passwd_file=/etc/passwd-s3fs -o allow_other, uid=1000, umask=077, gid=1000, use_cache=/root/cache

    The full directory structure is accessible by the root user ONLY, any other user that trys to access a file or directory get the following error:

    ls -al /mnt/AWSS3/user
    ls: cannot open directory /mnt/AWSS3/user: Operation not permitted
    cat /mnt/AWSS3/mytest.txt
    cat: /mnt/AWSS3/mytest.txt: Permission denied

    Is there a switch that can help me with the access to ALL users or a specific one?

    • Jithin says:

      Actually -o uid=1000,umask=077,gid=1000,… will do it. Make sure you provide correct uid and gid of the user that you want to allow access. ( Run “id -u username” to get the uid and “id -g username” to get uid. Replace “username” with the appropriate user’s username )

  4. Scouser says:

    Hi,

    Unfortunately, it doesn’t work for me

    # mkdir /tmp/s3
    # id alex
    uid=1000(alex) gid=1001(alex) groups=1001(alex)
    # s3fs my-bucket /tmp/s3 -o passwd_file=/etc/passwd-s3fs,uid=1000,gid=1001,umask=077
    # su – alex
    $ ls -las /tmp/s3
    ls: cannot access ‘/tmp/s3’: Permission denied

    # la -ld /tmp/s3
    drwx—— 1 alex alex 0 Jan 1 1970 /tmp/s3
    #mount
    s3fs on /tmp/s3 type fuse.s3fs (rw,nosuid,nodev,relatime,user_id=0,group_id=0)

    Please advise

    • Jithin says:

      Try full permission for the mount point.

      • Jlabuelo says:

        Hi there
        I have tried to give full permission to the mount point with Chmod 777 -R /mnt/uploadS3, but I get also the “Input/output error”.
        My problem is that I am able to write and copy files in the mount point with the “root” user, but not for example with other users like “tomcat”.
        Any advice??

  5. james says:

    Unable to create files: I have compiled and installed libfuse 2.9.7 and latest build of S3fs on centos 6.9. I am able to mount S3 bucket a.b.com on /home/backup. No errors reported in logs. But I am unable to create any folder on my Linux server. Also folders in bucket visible on S3 console is not present on my linux server mount point folder. I am also unable to create new folder or file on linux server mounted folder.
    Command used is s3fs a.b.com /home/backup -o passwd_file=/root/.passwd-s3fs,curldbg -o use_path_request_style -o allow_other

    Error is: mkdir: cannot create directory `file’: File exists

  6. Nicolas says:

    Mounting the my S3 bucket via the cli “s3fs mybucket /path/to/mountpoint -o passwd_file=/etc/passwd-s3fs” works fine
    but putting the following statement in the /etc/fstab:
    s3fs#s3-ftp-energiscloud /mnt/s3mnt fuse allow_other,use_cache=/tmp/cache,uid=500,gid=500 0 0

    gives the following error when executing mount:
    mount: wrong fs type, bad option, bad superblock on s3fs#s3-ftp-energiscloud,
    missing codepage or helper program, or other error

    In some cases useful info is found in syslog – try
    dmesg | tail or so.

    Can someone help me ? thanks

    • DWY says:

      I’m having the same issue, the syntax for fstab doesn’t work for me or is missing a lot of parameters like the password file, API URL, etc. or is none of that needed for fstab?

  7. Andrew McGregor says:

    Here are the commands I used on Amazon Linux:

    cd /usr/src/
    wget https://github.com/libfuse/libfuse/releases/download/fuse-3.0.0/fuse-3.0.0.tar.gz
    tar xzf fuse-3.0.0.tar.gz
    cd fuse-3.0.0
    ./configure –prefix=/usr/local
    make && make install
    export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
    ldconfig
    modprobe fuse
    cd ..
    git clone https://github.com/s3fs-fuse/s3fs-fuse.git
    cd s3fs-fuse/
    yum install automake
    ./autogen.sh
    ./configure
    yum install fuse-devel
    ./configure
    make
    make install

  8. veuncent says:

    Thanks for the great walkthrough!
    The dependencies for s3fs have changed a little in the meantime: https://github.com/s3fs-fuse/s3fs-fuse

    For RHEL/CentOS:

    sudo yum install automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel

    For Ubuntu:

    sudo apt-get install automake autotools-dev fuse g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config

  9. Jeremy says:

    s3fs is available via yum now, by the way.

    yum search s3fs
    yum install s3fs-fuse.x86_64

  10. suresh says:

    In my ubuntu instance using s3sf mounting s3 bucket successfully, but when I am getting objects it through 403 forbidden error. I am trying through command for permission denied, but it not working. How to solve 403 error.

Leave a Reply