Create a bucket
Log into your console and go to the amazon s3 tool and click
create bucket.
Give it a unique name and click next
For me I am going to leave the defaults here and click next
The default settings are good for me on this page as
well. I want my aws user to have
read/write to this bucket, even though I am not going to use that user to mount
the S3 bucket. I also want to make the
bucket private so “Do not grant public read access” . Click Next
Click Create Bucket!
There is the bucket!
Create a user with credentials
Now I want to create a user with credentials who has
permissions to read/write to this new bucket.
From the AWS console open up the IAM tool and click on users
Click add user.
Enter in a user name, select Programmatic Access and click
Next Permissions.
Select Attach existing policies and search for s3.
Click on AmazonS3FullAccess and click on JSON
This JSON policy
{
"Version":
"2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "*"
}
]
}
|
Is close to what I want.
It gives read/write access, but to all my s3 buckets. I want to tweak this slightly to limit it to
a single bucket.
Here is a policy that will do what we want.
{
"Version":
"2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": ["arn:aws:s3:::a-test-bucket-124568d"]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": ["arn:aws:s3:::a-test-bucket-124568d/*"]
}
]
}
|
Click on Create Policy
Click on JSON and paste the policy in.
Click Review Policy
Give it a name and click Create Policy
The policy has been created!
From the create user page select “Customer Managed” from the
pull down menu.
Then click Refresh
Select the policy you just made and click Next: Review
Click Create User.
Record your Access Key ID and show your secret access key.
In my case
Access Key: AKIAJETFXNV4NYVV64EA
Secret: Gcdpi2aETNuLsBxB1DKKU9g44qhpaAl6Eoviqreo
(don’t worry I am deleting this bucket and user after
writing this how to)
Click Close you are done with this part.
Install s3fs
On Ubuntu 16.04 install the following tools
> sudo apt-get
install automake autotools-dev \
fuse g++ git libcurl4-gnutls-dev libfuse-dev \
libssl-dev libxml2-dev make pkg-config
|
Download the s3fs tool from github https://github.com/s3fs-fuse/s3fs-fuse
[1] and run the following to install it.
> git clone
https://github.com/s3fs-fuse/s3fs-fuse.git
> cd s3fs-fuse
> ./autogen.sh
> ./configure
> make
> sudo make
install
|
Create the /etc/passwd-s3fs file
> sudo vi
/etc/passwd-s3fs
|
And place the ACCESS KEY ID : Secret Access KEY ID
So in my case I would put
AKIAJETFXNV4NYVV64EA:Gcdpi2aETNuLsBxB1DKKU9g44qhpaAl6Eoviqreo
|
Save it now chmod it to 640
> sudo chmod
640 /etc/passwd-s3fs
|
Mount it!
Create a mount point and mount it!
> sudo mkdir
-p /s3/bucket-test
> sudo s3fs -o
allow_other a-test-bucket-124568d /s3/bucket-test
|
Its mounted now let me write a file to it.
Let me use /dev/urandom to create a 100MiB file with random
data in it in a /tmp folder
> cd /tmp
> dd if=/dev/urandom
of=random.txt count=1048576 bs=100
|
Now copy it over to the s3 bucket
> cp
/tmp/random.txt /s3/bucket-test/
|
Which fails because only root can write to it at the moment…
Let me use sudo
> sudo cp
/tmp/random.txt /s3/bucket-test/
|
OK that worked…
But how do I copy files over if I am not root?
I think I have an issue with /etc/fuse.conf file…
> sudo vi
/etc/fuse.conf
|
Uncomment out the user_allow_other line
Unmount the s3 drive and remount it.
> sudo umount
/s3/bucket-test
> sudo s3fs -o
allow_other a-test-bucket-124568d /s3/bucket-test
|
Now try and copy a file over.
> cp
/tmp/random.txt /s3/bucket-test/random2.txt
|
Worked
Mount using /etc/fstab
First let me unmount the s3 bucket
> sudo umount
/s3/bucket-test
|
Open and edit /etc/fstab
> sudo vi
/etc/fstab
|
And append the following line to the bottom of the file
s3fs#a-test-bucket-124568d
/s3/bucket-test fuse retries=5,allow_other,url=https://s3.amazonaws.com 0 0
|
Here you can see the bucket and the mount point.
Now mount it
> sudo mount
/s3/bucket-test
|
Run a quick test
> cp
/tmp/random.txt /s3/bucket-test/random3.txt
|
That worked!
Also I do not think I need the user_allow_other in the
fuse.conf file
> sudo vi
/etc/fuse.conf
|
Comment out the
user_allow_other line
I am just going to reboot at this point and see if it works
with the new fuse.conf file and also if it automounts
> sudo reboot
now
|
Wahoo it is all working J
So that’s how you mount an S3 bucket as a drive in Ubuntu
16.04
References
[1] s3fs github repo
This was very helpful to me. Thank you!
ReplyDeleteCan you do a tutorial on how to do this on a CentOS based machine?
I am running into some dependency issues trying to do this on a RHEL based EC2 instance.
for CentOS
Deletehttps://cloudkul.com/blog/mounting-s3-bucket-linux-ec2-instance/
for centOS
ReplyDeletehttps://cloudkul.com/blog/mounting-s3-bucket-linux-ec2-instance/
when I tried to mount it returns with no error but mounting did not happen. can you advice what would be the issue? I am trying to mount it on ubuntu 18.02 instance
ReplyDeleteI had the same issue, the _netdev mount option solved it. Perhaps this is worth a try ?
DeleteThis was an awesome how-to! Thank you!
ReplyDeleteThank you for this one ! I just connected my Wasabi S3 Storage. Works in the same way, just the URL is different.
ReplyDeleteExample for an s3fs command:
sudo s3fs bucket mountpoint -o passwd_file=/etc/passwd-s3fs -o url=https://s3.wasabisys.com
-o allow_other
It still have some issues (auto mount upon reboot does not work yet - no idea why) though - but will proceed with fingers crossed.
... and now even the mount problems are gone: the _netdev mount option solved the problem.
DeleteThis was super helpful to me! Thank you!!!
ReplyDeleteGreat Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.
ReplyDeleteAWS Online Training
IEEE Final Year projects Project Centers in India are consistently sought after. Final Year Students Projects take a shot at them to improve their aptitudes, while specialists like the enjoyment in interfering with innovation. For experts, it's an alternate ball game through and through. Smaller than expected IEEE Final Year project centers ground for all fragments of CSE & IT engineers hoping to assemble. Final Year Projects for CSE It gives you tips and rules that is progressively critical to consider while choosing any final year project point.
DeleteJavaScript Online Training in India
JavaScript Training in India
The Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training
great tutorial, worked perfectly.
ReplyDeleteGreat blog thanks for sharing Leaders in the branding business - Adhuntt Media is now creating a buzz among marketing circles in Chennai. Global standard content creation, SEO and Web Development are the pillars of our brand building tactics. Through smart strategies and customer analysis, we can find the perfect audience following for you right now through Facebook and Instagram marketing.
ReplyDeletedigital marketing company in chennai
seo service in chennai
web designing company in chennai
social media marketing company in chennai
Thank you for sharing .The data that you provided in the blog is informative and effective.
ReplyDeleteweb designing training in bangalore
web designing courses in bangalore
web designing classes in bangalore
web designing training institute in bangalore
web designing course syllabus
best web designing training
web designing training centers
Very useful blog thanks for sharing IndPac India the German technology Packaging and sealing machines in India is the leading manufacturer and exporter of Packing Machines in India.
ReplyDeleteI have been reading for the past two days about your blogs and topics, still on fetching! Wondering about your words on each line was massively effective.
ReplyDeletephp online training in chennai
php programming center in chennai
php class in chennnai
php certification course
php developer training institution chennai
php training in chennnai
php mysql course in chennai
php institute in chennnai
php course in chennnai
php training with placement in chennnai
php developer course
Nice post. Thanks for sharing! I want people to know just how good this information is in your article. It’s interesting content and Great work.
ReplyDeleteappium online training
appium training centres in chennai
best appium training institute in chennnai
apppium course
mobile appium in chennnai
mobile training in chennnai
appium training institute in chennnai
ReplyDeleteThanks for your extraordinary blog. Your idea for this was so brilliant. This would provide people with an excellent tally resource from someone who has experienced such issues. You would be coming at the subject from a different angle and people would appreciate your honesty and frankness. Good luck for your next blog!
Tally ERP 9 Training
tally classes
Tally Training institute in Chennai
Tally course in Chennai
seo training classes
seo training course
seo training institute in chennai
seo training institutes
seo courses in chennai
seo institutes in chennai
seo classes in chennai
seo training center in chennai
Resources like the one you mentioned here will be very useful to me ! I will post a link to this page on my blog. I am sure my visitors will find that very useful
ReplyDeleteAWS training in chennai | AWS training in annanagar | AWS training in omr | AWS training in porur | AWS training in tambaram | AWS training in velachery
If only it were that simple: archaeology involves collaboration and working with teams, and every excavation is a project, so management skills are critical to success. nice page
ReplyDeleteAi & Artificial Intelligence Course in Chennai
PHP Training in Chennai
Ethical Hacking Course in Chennai Blue Prism Training in Chennai
UiPath Training in Chennai
The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. machine learning projects for final year In case you will succeed, you have to begin building machine learning projects in the near future.
ReplyDeleteProjects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.
Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.
https://digitalweekday.com/
ReplyDeletehttps://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
ReplyDeletehttps://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
https://digitalweekday.com/
Thanks for your extraordinary blog. Your idea for this was so brilliant. This would provide people with an excellent tally resource from someone who has experienced such issues.
ReplyDeleteAWS training in Chennai
AWS Online Training in Chennai
AWS training in Bangalore
AWS training in Hyderabad
AWS training in Coimbatore
AWS training
AWS online training
I was following your blog regularly and this one is very interesting and knowledge attaining. Great effort ahead. you can also reach us for
ReplyDeleteweb design company in chennai
web development company in chennai
website designers in chennai
website designing company in chennai
web design in chennai
website development company in chennai