sys·ad·min·ol·o·gy [sis-ad-mih-nol-uh-jee]


  1. The scientific study of system administration and related phenomena.

Thursday, 24 January 2013

Amazon Cloud: How Security Groups Work

Continuing my last post on elements of the Amazon Cloud, this time we look at Amazon EC2 Instance security groups. At first glance, they can appear quite easy to understand, but if you have had to manage instances of any number, they can soon get unmanageable!

No doubt you have already drawn the conclusion that an Amazon security group is similar to a firewall. If so, you are on the right track. Imagine a single iptables configuration file, that is read by many hosts, but with some caveats:

Amazon security groups: trip hazards

  1. An instance can belong to one or many security groups. While this might seem confusing at first, it's actually quite useful. We'll see how later in this post.
  2. Security groups do not automatically trust other instances in your AWS account. So If instance A is in security group A and instance B in security group B, there is no trust implied between instance A and instance B. Rule must be explicitly defined to allow traffic to flow.
  3. Instances in the same security group do not trust each other either. Again, rules need to be explicitly defined to allow communication. This is an especially important point if you are doing any kind of replication that uses the network. E.g. MySQL or a distributed file system.
  4. Once an instance has been assigned a security group or groups, they cannot be changed while the instance is running. This means you must choose your security groups wisely when you create the instance. Otherwise, you will have to shut down your instance to change its security group membership
  5. You CAN change the rules of the security while the instances that belong to it, are up and running.
  6. You CAN include other security groups in security group rules. E.g. any member of security group A can access port 22 on any member of security group B. Again, this comes in handy later on.
    • You may have realised, that given that nested security groups are allowed, then it should be possible to negate gotcha 2 by including a security in itself, and saying (for example) that all ports are accessible. Logically, all instances belonging to that security group should trust each other to communicate on all ports. However, this does not work. It doesn't throw up and error in the web interface, instances just refuse to talk to each other... so, back to the drawing board! :D

One other thing with Amazon EC2 security groups

So as you've probably gathered by now, the rules we assign to the security group must specifically allow traffic that we want to flow between it's members. We could just allow everything to communicate with everything (source ip of on ports 0-65535) but that would not be secure, so instead, in order for Instance A to talk to Instance B on port 22, we need to define two rules:
  1. A rule that allows communication from A to B on port 22
  2. A rule that allows communication from B to A on port 22
Pretty simple so far. However, now lets consider that unless you have an Elastic IP assigned to your instance, then the IP address of either instance can, and most probably will change should the instance ever crash, or be shut down.

Clearly, when that happens you will have to go through your security groups, and change the relevant IP's accordingly. This might not sound a big job, but scale out by a few instances here and there, and it soon becomes a very difficult task to manage!

So what's the good news?

Essentially, a little planning and care over what security groups are defined, and an instances membership at time of creation can save a lot of headache later on. I have put together this strategy based on the guidelines I found in this book: Cloud Application Architectures By George Reese, and my own experience.

The scenario

Let's say we have two web servers: web1 and web2 and two database servers db1 and db2. Let's assume that the two web servers are load balanced, and the database nodes replicate data between themselves. Essentially, we have a two tier architecture:

  1. The application layer: The world needs to access web1 and web2 on port 80, and the sysadmins need access on port 22 
  2. The database layer: Only the web servers need access to the db1 and db2 on port 3306 (We're using MySQL it would seem) and the sysadmins need access on port 22
So it should be fairly obvious, to start with we should have three basic security groups:
  1. SG-T-WEB: Security Group Tier Web will allow communication on port 80
  2. SG-T-DB: Security Group Tier Database will allow communication on port 3306 from any member of SG-T-WEB
  3. SG-T-ADMIN: Security Group Tier Admin will allow communication on port 22, ideally from a single IP from where sysadmin is usually carried out.
I should point out that I have chosen the above naming convention. I'm not saying it's the best, but it's a convention, which is better than non at all! Feel free to suggest your own using the comments section.

When each instance type is created, we add them to the relevant tier group, AND to the SG-T-ADMIN group as well, so they are accessible via port 22. There is one other group we need to add instances to however. But before that, lets answer:

How do we solve the problem of changing IP addresses and having to manually edit security groups?

As I mentioned earlier, unless your instance has an Elastic IP assigned to it. Then the address of your instance could change. Using Elastic IPs is an option, but Amazon ration them to five per account, and the Internet is running out of version 4 addresses, so it seems prudent to find an alternative solution.
For the database tier, obviously, it is vital that you use Elastic IP's (you don't want to have to reconfigure your application every time the IP address of the database node(s) changes) but the web servers don't need them. We can use nested security groups to help us out here.

So continuing our example, lets say we want to add another application node called web3. Wouldn't it be great if we could add something to the SG-T-WEB security group that would stay the same even when the IP address of the application node(s) change? Well, we can!

Put each node in your infrastructure into its own dedicated security group. So in addition to the security groups defined above, we also have:
  1. SG-I-WEB1: Security Group Instance web1
  2. SG-I-WEB2: Security Group Instance web2
  3. SG-I-WEB3: Security Group Instance web3: for our new web node
  4. SG-I-DB1: Security Group Instance db1
  5. SG-I-DB2: Security Group Instance db2
We can then define a rule in SG-T-WEB that says any member of SG-I-WEB3 is accessible via port 80. Should web3 ever go down, or be shut down, we can bring it back up without having to edit any security groups.

This means that when you create each instance, you must add it to at least three security groups:
  1. The security group for the tier the instance belongs to (SG-T-WEB or SG-T-DB)
  2. The security group allowing admin access (SG-T-ADMIN)
  3. A security group reserved ONLY for the new instance itself. (SG-I-NEWINSTANCE)


The above strategy helped me reduce the admin headache of changing IP addresses after outages or downed instances. Do you have a tactic for reducing security group admin? Share it in the comments section!

Until next time... Nano! Nano! (#morkandmindy not #friendlierAlternativeToVi)

1 comment:

  1. I really appreciate information shared above. It’s of great help. If someone want to learn Online (Virtual) instructor lead live training in AWS Big Data, kindly contact us
    MaxMunus Offer World Class Virtual Instructor led training on AWS Big Data. We have industry expert trainer. We provide Training Material and Software Support. MaxMunus has successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar, Saudi Arabia, Bangladesh, Bahrain and UAE etc.

    For Free Demo Contact us:
    Name : Arunkumar U
    Email :
    Skype id: training_maxmunus
    Contact No.-+91-9738507310
    Company Website –