Overview
In June of 2012 Amazon AWS introduced Roles with the IAM
(Identity and Access Management) tool.
You can see their blog post and a short video covering its user here http://aws.typepad.com/aws/2012/06/iam-roles-for-ec2-instances-simplified-secure-access-to-aws-service-apis-from-ec2.html
[1]
In a nutshell
1. Create
a Role within IAM
2. Set
that roles policies
3. Create
an EC2 instance with this role assigned to it
(must do at creation cannot do after the fact)
That is it, now your EC2 machine has access to metadata
that allows it to access this role and its permissions. In theory it could work with things like
s3cmd or s3fs but as of this writing I do not think those tools have been
updated to take advantage of this feature.
The Amazon provided SDKs can take advantage of this, so that is what I
will cover here.
Create IAM Role
Sign into the AWS web console
Go to http://aws.amazon.com/
Click on My Account/Console à AWS management Console
Sign in with your amazon account email and password
Click on IAM
Click on Roles then Create New Role.
Give it a name, I am
calling mine ec2viewer as I am going to use it to get some metrics on my EC2
instances. Click continue.
Select Policy Generator and click Select.
Choose Amazon CloudWatch and select “All Actions”
Click on Add Statement
Select Amazon EC2,
and checkbox all the Describe actions
Click Create Role
Now this role is available to assign to an ec2 while you are
creating it.
Select the role you just created. Select the Summary Tab. Then copy the Instance Profile ARN. This contains your account number, so you
may notice that I covered mine up.
arn:aws:iam::XXXXXXXXXXXX:instance-profile/ec2viewer
You will use this when creating an ec2 instance via the
command line.
Create EC2 instance with role
You can get some more command line information about roles
and ec2 from
I prefer using the command line tools when creating ec2
instances so to following along with this one.
The following assumes you have set up your machine to do ec2 command
line tools.
In this case I am creating an Ubuntu 12.04 LTS in the
us-east region
> ec2-run-instances ami-9c78c0f5 -b
/dev/sda1=:8:true -k my-keypair
-t t1.micro -g default -p
arn:aws:iam::XXXXXXXXXXXX:instance-profile/ec2viewer
--availability-zone us-east-1a
|
(use your own keypair)
Now log into your Ubuntu 12.04 EC2 instance
In my case it’s at ec2-184-72-175-14.compute-1.amazonaws.com
> ssh
-i .ec2/my-keypair.pem ubuntu@ec2-184-72-175-14.compute-1.amazonaws.com
|
(again this assumes you have your keypair in the given
location)
Confirm that the machine does have this role assigned to
it. Each machine can query certain
information about itself via 169.254.169.254 internally. A good spot to learn more about this is http://www.practicalclouds.com/content/guide/accessing-aws-information-within-your-instance
[3]
You could use CURL but I use GET perl tools for this.
To install GET command line on Ubuntu 12.04 run the
following command
> sudo
apt-get install libwww-perl
|
If you wanted to see what security groups this machine is
assigned to run the following command.
> GET
http://169.254.169.254/latest/meta-data/security-groups; echo
> GET
http://169.254.169.254/latest/meta-data/security-groups; echo
|
Here you can see that it has the default group and its AMI is ami-9c78c0f5.
To confirm that it has a role assigned to it run this command
> GET
http://169.254.169.254/latest/meta-data/iam/security-credentials/; echo
|
Here you can see it returns an ec2viewer.
Run this
> GET
http://169.254.169.254/latest/meta-data/iam/security-credentials/ec2viewer;
echo
|
This returns security credentials.
Java SDK setup
For this Java AWS SDK example I am just going to create a
simple program, not even using an IDE, just to show how it works.
But first I need to install the latest version of Java on
Ubuntu.
Installing Java 7
First to install JAVA 1.7 on Ubuntu…
Go to
And click on JDK Download
Click on the “Accept License” Button then click on the
version you want to download.
In my case I chose jdk-7u9-linux-x64.tar.gz
It used to be that you could then copy the link and use a wget
from the command line to get the file, but no longer. So manually
download the version you want.
Scp over to your instance.
> scp
-i ~/.ec2/my-keypair.pem jdk-7u9-linux-x64.tar.gz
ubuntu@ec2-54-242-202-63.compute-1.amazonaws.com:
|
Untar it
> tar
xvf jdk-7u9-linux-x64.tar.gz
|
Move JDK folder to /usr/lib
> sudo
mkdir /usr/lib/jvm
> sudo
mv jdk1.7.0_09 /usr/lib/jvm/jdk1.7.0_09
|
Make a link
> sudo
ln -fs /usr/lib/jvm/jdk1.7.0_09/bin/java /usr/bin/java
|
Now if you run
> java
-version
|
You should get the correct response
Make a link
> sudo
ln -fs /usr/lib/jvm/jdk1.7.0_09/bin/javac /usr/bin/javac
|
Now if you run
> javac
-version
|
> sudo
ln -fs /usr/lib/jvm/jdk1.7.0_09/bin/jar /usr/bin/jar
|
Add JAVA_HOME variable in .bashrc
> cd
> sudo
vi .bashrc
|
Add the following to the bottom
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_09
|
Run the following
command
> source
.bashrc
|
Download AWS Java SDK
The AWS Java SDK can be found at http://aws.amazon.com/sdkforjava/
[4]
Run the following wget command from the ec2 instance.
> wget
http://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip
|
Unzip it
> unzip
aws-java-sdk.zip
|
Move all the third party jar files for convenience.
> cd
> cd aws-java-sdk-1.3.24/third-party/
> find . -iname "*.jar" | xargs -i mv
{} .
|
The Code
There were several sources I used to help me create this
code. The samples given in the SDK
download are one of them I also use.
http://aws.amazon.com/documentation/sdkforjava/
[5] and http://heshans.blogspot.com/2012/03/monitor-cpu-utilization-of-amazon-ec2.html
[6] which has a very good example that set me in the right direction.
First run the following command
> cd
> mkdir
mycode
> cd
mycode
> mkdir
dist
> mkdir
–p com/_10x13/aws/cloudwatch
> vi
com/_10x13/aws/cloudwatch/CloudWatchTest.java
|
Here is the code, you will need to adjust it to your
environment.
package
com._10x13.aws.cloudwatch;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.Date;
import java.util.Comparator;
import
java.util.Collections;
import
com.amazonaws.services.cloudwatch.AmazonCloudWatchClient;
import
com.amazonaws.services.cloudwatch.model.Metric;
import
com.amazonaws.services.cloudwatch.model.GetMetricStatisticsRequest;
import
com.amazonaws.services.ec2.AmazonEC2Client;
import
com.amazonaws.services.ec2.model.DescribeInstancesResult;
import
com.amazonaws.services.ec2.model.Reservation;
import
com.amazonaws.services.ec2.model.Instance;
import
com.amazonaws.services.ec2.model.Tag;
import
com.amazonaws.services.cloudwatch.model.Dimension;
import
com.amazonaws.services.cloudwatch.model.GetMetricStatisticsResult;
import
com.amazonaws.services.cloudwatch.model.Datapoint;
public class CloudWatchTest
{
public CloudWatchTest(){
AmazonCloudWatchClient cwc = new
AmazonCloudWatchClient();
AmazonEC2Client ec2 = new
AmazonEC2Client();
Instance ec2JoomlaInstance = null;
//This will print out a list of
current metrics available to you
//per virtual device out there
for(Metric met :
cwc.listMetrics().getMetrics()) {
System.out.println(met);
}
System.out.println();
//The following will get the instance
id for an ec2 server I have named
//Joomla then get a List of
CPUUtilization data over the last 24 hours
//order it by timestamp then print it
out.
DescribeInstancesResult di =
ec2.describeInstances();
List<Reservation> reservations
= di.getReservations();
Set<Instance> instances = new
HashSet<Instance>();
for (Reservation reservation :
reservations) {
instances.addAll(reservation.getInstances());
}
for(Instance instance : instances) {
for(Tag tag : instance.getTags())
{
if(tag.withKey("Name").getValue().equals("Joomla")){
//This is the instance I
want
ec2JoomlaInstance =
instance;
}
}
}
long offsetInMilliseconds =
1000*3600*24; //24 hours of data:
GetMetricStatisticsRequest request =
new GetMetricStatisticsRequest()
.withStartTime(new Date(new
Date().getTime() - offsetInMilliseconds))
.withNamespace("AWS/EC2")
.withPeriod(60 * 15) //15 minute fidelity
.withDimensions(new
Dimension().withName("InstanceId")
.withValue(ec2JoomlaInstance.getInstanceId()))
.withMetricName("CPUUtilization")
.withStatistics("Average",
"Maximum")
.withEndTime(new Date());
GetMetricStatisticsResult
metricResults = cwc.getMetricStatistics(request);
List<Datapoint> dataList =
metricResults.getDatapoints();
Collections.sort(dataList, new
CustomComparator());
for(Datapoint data : dataList){
System.out.println(data);
}
}
public static void main(String...args) {
new CloudWatchTest();
}
//Private class used to sort list based
on timestamp
private class CustomComparator implements
Comparator<Datapoint> {
public int compare(Datapoint dat1,
Datapoint dat2) {
return
dat1.getTimestamp().compareTo(dat2.getTimestamp());
}
}
}
|
Now to compile and run use the following commands.
> javac -cp
~/aws-java-sdk-1.3.24/lib/aws-java-sdk-1.3.24.jar -d dist
./com/_10x13/aws/cloudwatch/CloudWatchTest.java
> jar cfv aws_10x13.jar -C dist/ .
> java -cp
aws_10x13.jar:/home/ubuntu/aws-java-sdk-1.3.24/lib/aws-java-sdk-1.3.24.jar:/home/ubuntu/aws-java-sdk-1.3.24/third-party/*
com._10x13.aws.cloudwatch.CloudWatchTest
|
The output of the code will have two sections
The first shows metrics you can get information on .
The second shows 24 hours of CPU utilization for a specific
named server
Here is a blow by blow on the code.
public CloudWatchTest(){
AmazonCloudWatchClient cwc =
new
AmazonCloudWatchClient();
AmazonEC2Client ec2 = new
AmazonEC2Client();
Instance ec2JoomlaInstance = null;
//This will print out a list of
current metrics available to you
//per virtual device out there
for(Metric met :
cwc.listMetrics().getMetrics()) {
System.out.println(met);
}
|
This part simply writes out the list of metrics you have
access to. So if you have 3 running EC2
instances you will see 3 CPUUtilizations you can read one for each
instance. It will also list metrics you
can read on volumes, etc. Fairly
simple, I only made this list so I could better understand what was available
to me.
I am going to split
the next section up into a few different parts. The purpose of it was to list the last 24
hours of CPU Utilization of a server I have called “Joomla” in my ec2 system.
By Name I mean the
tag you can set within the web Console.
DescribeInstancesResult di =
ec2.describeInstances();
List<Reservation> reservations
= di.getReservations();
Set<Instance> instances = new
HashSet<Instance>();
for (Reservation reservation :
reservations) {
instances.addAll(reservation.getInstances());
}
|
This part finds all
the instances on ec2 and adds them to a SET.
for(Instance instance : instances) {
for(Tag tag : instance.getTags())
{
if(tag.withKey("Name").getValue().equals("Joomla")){
//This is the instance I
want
ec2JoomlaInstance =
instance;
}
}
}
|
This part looks
through all the instances and finds the instance that has been named “Joomla”
long offsetInMilliseconds =
1000*3600*24; //24 hours of data:
GetMetricStatisticsRequest request =
new
GetMetricStatisticsRequest()
.withStartTime(new Date(new
Date().getTime()
- offsetInMilliseconds))
.withNamespace("AWS/EC2")
.withPeriod(60 * 15) //15 minute fidelity
.withDimensions(new Dimension()
.withName("InstanceId")
.withValue(ec2JoomlaInstance.getInstanceId()))
.withMetricName("CPUUtilization")
.withStatistics("Average", "Maximum")
.withEndTime(new Date());
GetMetricStatisticsResult
metricResults =
cwc.getMetricStatistics(request);
|
This part sets up
the request. We have a start time set 24
hours ago and end time of now, We are
looking at EC2, The period of time (fidelity)
is 15 minutes. We are only looking at a
single instance. It only looks for the
Metric CPUUtilization.
List<Datapoint> dataList =
metricResults.getDatapoints();
Collections.sort(dataList, new
CustomComparator());
for(Datapoint data : dataList){
System.out.println(data);
}
|
The bit retrieves
the list as a List. This list is
unordered, so I ordered it by its timestamp.
And there you have
it. Hope this helps someone out.
References
[1] IAM roles for EC2 instances –
Simplified Secure Access to AWS service APIs from EC2
Visited 11/2012
[2] AWS Identity and
Access Management
Visited 11/2012
[3] Accessing AWS
information from within your instance
Visited 11/2012
[4] AWS SDK for Java
Visited 11/2012
[5] AWS SDK for Java
Documentation
Visited 11/2012
[6] Monitor CPU Utilization of a
Amazon EC2 instance using Amazon CloudWatch
Visited 11/2012
No comments:
Post a Comment