Creating a Key Pair to share between AWS EC2 Instances

By Semir H.


If you find this tutorial useful, please consider making a donation to support future work like this:

Thank you :)

Sign into Amazon Web Services Console. Go to the Amazon EC2 tab.
Pick your desired Region as this is where the key will be stored.
Locate the “Key Pairs” link under Networking & Security section and click on it.
Click on “Create Key Pair” button on the top to start the process.
Give the Key Pair a name. E.g. APAC-Keys-2011. Click on “Create”.

The Key Pair will be created and you will be prompted to save the private portion (a .pem file) on your computer/device. Please do so and make sure you keep it private,

You can now use this Key Pair for your new Instances. Please pick the “Choose from your existing Key Pairs” at the “Key Pairs” option when creating an Instance and select your pre-made Key Pair. Example below:

Connecting to AWS Linux (Ubuntu) Instance from Windows using PuTTY and the SSH protocol

By Semir H.

This part continues from the previous “NAME” part where we launched an Instance. In this part I will show you how to use PuTTY to connect to your running Linux (Ubuntu) Instance. In order for PuTTY to access your Instance over the SSH protocol we opened up the SSH TCP port (22) by using a Security Group which is essentially a Firewall.

Creating a PuTTY Private Key:

We must first create a PuTTY Private Key using the .PEM key generated beforehand. For this you need to get puttygen.
Run puttygen to start the process. Click on “File” and on “Load private key”.

Browse to the private key file (.pem) you saved earlier and load it. You should get a success notice like the one below. Click on OK.

You may want to edit the key comment and put a key passphrase before we generate a PPK file. I recommend using a key passhprase but please do not forget it or your resulting PPK will be of no use. Leave the type of key as SSH-2 RSA and leave the number of bits as 1024. Once you’ve made desired comments and put in the phrase click on “Save private key”. Give it a meaningful name and save as .ppk.

Connecting to AWS Linux Instances:

We must first find out the public DNS of the Instance we wish to connect to. To do this go to your AWS Management Console, go to Amazon EC2 tab, Instances, click on your Instance and look down under “Description”. You should see an address, next to Public DNS, similar to this: ec2-122-248-203-235.ap-southeast-1.compute.amazonaws.com

Next, start PuTTY, put the DNS entry into the Host Name field and then browse down to the SSH Category and to the Auth section under SSH. This is where you add your private key (PPK) for later authentication. Go back to the Session category, make sure the DNS entry is still there, give this session a name (e.g. Test AWS Ubunt) and click on Save.
See below for an example:

Now it’s a simple matter of loading the saved session. First time you connect you will get a security alert saying that the server’s host key is unknown to you. Please select Yes to continue if you think it is safe to do so (in this case it is). You will get this with any new server the first time. You will also get this if you turn your Instance off and then on again (because the DNS and server hostname will change). The private key will not change so that can be re-used to connect.
You will be prompted to put in a username to log in. For this particular AMI it is ‘ubuntu’. Once you put that in you will get prompted to put the PPK passphrase (if you set one). If all goes well you should be logged into your Ubuntu Linux Amazon AWS Instance. Yay :)

Troubleshooting:

If you’re having issues then please revisit the steps above. Please make sure you’re using the correct DNS entry and the correct private key. You can check which private key the Instance is using by looking at the Instance Details section (same one where DNS info is located). Please be aware that the public DNS string will change if you shut the server down and start it again. This can be addressed using Elastic IPs but we’ll cover that in another tutorial some other time.

Launching a Linux (Ubuntu) Instance on Amazon Web Services EC2 - A Beginner's Guide

By Semir H.

To create an AWS account please see How To Sign Up.

Log into your AWS account and go to the AWS Management Console.
Go to the Amazon EC2 tab. Make sure you’ve picked the desired Region (see this article for more detail).

Click on the ‘Launch Instance” button to start the process.

I’ll be creating a micro Instance of Ubuntu 11.04 64-bit server from one of the Community Amazon Machine Images (AMI). Community AMIs are pre-configured operating systems available for anyone to create Instances from.

Choose an AMI:

Click on Community AMIs and find your desired AMI. In my case it will be Ubuntu Natty Daily with AMI ID = ami-48aad21a. Click on “Select” next to it to continue.

Instance Details:

You can leave Instance Details at defaults. I am choosing a Micro Instance and I do not care about the Availability Zone for now. Click on “Continue”.

Leave the Advanced Instance Options at defaults an click on “Continue”.

Leave the Tags part empty for now. We’ll cover this in another tutorial. Click on “Continue”.

 
Create a Key Pair:

We will  need to create public/private key pairs to later access our Linux (Ubuntu) Instance.
You can create these per Instance or you can create them beforehand and use for future Instances. We’ll create one now for testing purposes. I’ll cover creating some for re-use later.

Enter a name for your key pair and click on the “Create and Download you Key Pair” link. Example:

 You will get prompted to open/save the resulting .pem file. Please do so.

Configure Firewall:

The next part configuring a security group which is basically a firewall.
Since I want to open ports for SSH (TCP 22) and HTTP (TCP 80) I will choose to create a new group and add those rules in at this stage. You can come back to this later to open more ports or close some.
So pick “Create a new Security Group”, give it a name and a description and add the desired rules. Some are pre-defined (e.g. SSH and HTTP) so you need to just pick them from the “Create a new rule” drop-down and add them. I’m leaving the source at default. Once done click on “Continue”.
See here for an example:

Review:

This is the final stage before launching. It gives you the opportunity to review your choices and edit them if you wish to. If you’re happy with them then click on “Launch”.
You will get a message that the Instance is being launched.

To view the progress of the launch you go back to the AWS Management Console and go to the Amazon EC2 tab. Click on Instances in the menu on the left. You should see your instance and the various properties it has as well as the Status. In no time it will turn to “running”. Here’s an example of this view:

That’s it. You can now connect to your running Instance. I’ll cover how to do that in my next tutorial.

How To KILL a stuck Virtual Machine (ESX 3.5)

Found this useful the other day - twice.

The first time a VM would not live migrate from one host to another. I stopped it and migrated but then it would not start, giving me the "Could not power on VM: No swap file. Failed to power on VM"
Using the below instruction I managed to kill the process that was still making it seem like the VM was running fine.

The second time I was taking a snapshot of a VM and it just did not work. Timed out AND made the VM freeze. I tried restarting and turning off the VM but it would not as there was "another process running" which I guess was the failed snapshot. Again, I used the below commands to kill the process (the VM) and was then able to start the VM.


ps -auxwww | grep -i


Find the PID and kill -9 pid

or

vm-support -x - to list vmids

vm-support -X to kill it

How To Install Applications on Microsoft Terminal Servers Correctly

This article assumes that the Microsoft Terminal Services Licensing and other TS features are already configured. It concentrates only on new Terminal Servers and how to deploy Applications on those new Servers.

Many people new to Microsoft Terminal Services will not know how to correctly install various applications in order for them to function properly in a Terminal Services environment. The basic steps are:

1. Install Microsoft Windows Server (2000/2003/2008).
2. Do all the Security and Updates Patching.
3. Install Microsoft Terminal Services in Application Mode.
4. Enter the "Install" mode.
5. Install Applications.
6. Enter the "Execute" mode.
7. Join the Server to the Domain and the rest of the Terminal Servers.

I won't go into details for most of these steps.

To use the Terminal Server in Application Mode then you must have Microsoft Terminal Services Licensing service installed. If you are joining a new TS Server to the existing TS environment then you already most likely have this. Then you just have to install the Terminal Services role on the new server.

The next step is to go into the "Install Mode". This is done as follows:

• Open a command window (cmd).
• Type "change user /install" (without the quotes) and press ENTER.

Next you install the desired applications (e.g. Microsoft Office 2003 Enterprise, Crystal Reports, Cisco admin applications, etc)

Next you switch into the "Execute Mode". This is done as follows:

• Open a command window (cmd).
• Type "change user /execute" (without the quotes) and press ENTER.

IF you have installed the applications before installing Microsoft Terminal Services role then you will have to uninstall and install again following the steps outlined above. Hopefully this will help some of you avoid re-installing the applications by doing it correctly the first time.

For more info please see "When you toggle Terminal Services to Application Server mode, some programs may stop working" from Microsoft Help and Support.

iSCSI Storage Connectivity Best Practice with VMware

I am quite impressed with VMware I must say. We use it in production at work and now I cannot imagine my professional life without it.
In this post I will briefly describe what your ideal cheap iSCSI Storage setup should look like.

Normally, production environments of many companies will use full versions of VMware ESX (e.g. ESX 3.5 - commonly;y know as VI3 Infrastructure or the new vSphere - ESX 4.0) with a SAN (Storage Area Network) where the Virtual Machines will live. In most cases the connection between the ESX boxes and the SAN is via Fiber Channel (FC) (see here for more info). This is usually very expensive.

Many companies resort to using iSCSI for their test environments or even production environments. You can learn more about iSCSI here.

Both ESX and ESXi versions support Software iSCSI Adapters. You are free to buy iSCSI HBAs but I found that the Software adapter is quite enough for all my testing needs. Money saved.

The diagram bellow shows what you ideal iSCSI network setup should be (click on it for the bigger version).


This setup requires that each ESX (ESXi) box has (at least) two NICs that will carry the data over iSCSI. Each NIC will hook into a separate Network Switches. So you will need two Network Switches too. The idea is that if one of the ESX (ESXi) NICs fails you will still have connectivity over the other NIC. And if one of the Network Switches fails you should still have connectivity to the iSCSI Storage Device.
So the single point of failure in this example is the iSCSI Storage Device (this post will not deal with that).

You can have iSCSI working with just one network sw and one NIC per ESX box but that does not give you redundancy. The above setup is good enough to use in production. I would recommend the 1 SW + 1NIC/ESX for TEST environments only.

I will probably write a more detailed tutorial on how to do the above setup with VMware ESX (ESXi) in the future.

That's it for now :)

Securing Websites with Pound (reverse HTTP proxy & SSL wrapper)

This is a short article on how to use Pound as a Reverse SSL Proxy.

Problem/Requirement:

You have a few websites in your DMZ or even on the LAN. You want them to be accessible from to the outside (Internet). Some of these websites are web based applications. You also have 1 external IP.
You want to serve all these websites from the one external IP and you want most of them to be secured via SSL (HTTPS).
On top of that you have different web servers serving the pages (e.g. Apache, IIS).

The diagram below depicts a typical inefficient web server setup (click on image to see larger version):

Requirements Summary:

• Use just one external IP.
• Use one SSL certificate (for websites under the same domain/sub domains).
  
• Secure websites REGARDLESS of the web server technology used to serve those websites.
• Have the ability to pick which websites get SSL encryption based on the source of the request (e.g. if from Internet then use SSL, if from corporate LAN then do not encrypt).

Solution:

Use a free application called Pound.

What is Pound?

Pound is a reverse HTTP proxy, load balancer, and SSL wrapper. It proxies client HTTPS requests to HTTP backend servers, distributes the requests among several servers while keeping sessions, supports HTTP/1.1 requests even if the backend server(s) are HTTP/1.0, and sanitizes requests.
It runs on Linux.
You can find more info @ http://www.apsis.ch/pound/

How Will it Work?

The diagram below shows where we want to be (click on image to see larger version):


Basically we want the Pound server to be the first point of contact from the outside. This way we reduce the number of "holes" we have to open in the firewall(s) and put another layer of security between the Internet and our web servers.
Pound meets all of our previously stated requirements:

• We can use just one External IP to server all our websites.
  
• Using a wildcard certificate (e.g. *.testdomain.test) we can encrypt all the web sites under the same domain/sub domains with just that one cert.
  
• Because we are doing the SSL encryption with Pound, it no longer matters what the other web servers inside your network use to serve pages. They can be Apache, IIS, Tomcat, etc. You can leave them ALL at the default port 80 (or 8080 for some). Pound can talk to them on pretty much any available port (even 443).
• We can make pound encrypt all requests that come from the Internet but NOT do so if the requests come from the LAN. Some companies choose to do that because it makes things faster. Not using SSL/HTTPS internaly can be done in two ways. One way is through Pound (so all the internal websites would point to the same internal Pound IP) and the other way is to not involve Pound at all. People wold just directly access the individual web servers. An advantage of going via Pound is that you can track access from one location and perhaps do custom redirects when needed (e.g. if a web server is down for maintenance you can have pound redirect to a notification page somewhere else).

Of course, there is one problem with using Pound for evertynig. It becomes a single point of failure. I suppose that can be remedied by using another Pound server to load balance or in another way (e.g. run as Virtual Machine - which works good. Let VMware worry about high availability, etc)

Next we will examine how some of the above is actually implemented:

Please Note:
This tutorial will not examine how the SSL certificates are acquired. The only thing you have to make sure is that they are Linux compatible. Pound runs on Linux. I usually use a self signed cert but if you generate one via a trusted CA then use the Apache/Linux instructions.

Let's assume you want to Encrypt web access for two of your internal web servers IF the requests are from the Internet. If they are from the LAN then do not encrypt.

Domain = testdomain.test
Internal Web Srv 1 = 192.168.7.5 (DMZ)
Internal Web Srv 2 = 192.168.0.10 (LAN)

The Pound server should always be put in the DMZ. Ideally all of your "exposed" servers should be but that is sometimes not an option.

Pound Server = 192.168.7.50 (DMZ)

First we define a HTTP (port 80) listening directive. Under that we put in our two web servers and since we want to encrypt them (if the requests is from the Internet and is on port 80 (HTTP) we will redirect to HTTPS:

ListenHTTP
Address 192.168.7.50
Port    80
xHTTP   2
Service
HeadRequire "Host: .*web1.testdomain.test.*" 
Redirect "https://web1.testdomain.test"
End
Service
HeadRequire "Host: .*web2.testdomain.test.*"
Redirect "https://web2.testdomain.test"
End
End

Next we define a HTTPS directive. This will be the one that will accept the redirects from above AND direct HTTPS requests (no redirect required in that case). This is also where we define which certificate to use and which back servers to contact.

ListenHTTPS
Address 192.168.7.50
Port    443
 Cert    "/path/to/cert.pem"
AddHeader "X-Forwarded-Proto: https"
HeadRemove "X-Forwarded-Proto"
Ciphers "ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL"
xHTTP   1

Service
HeadRequire "Host: .*web1.testdomain.test.*"
BackEnd
Address 192.168.7.5
Port 80
End
End

Service
HeadRequire "Host: .*web2.testdomain.test.*"
BackEnd
Address 192.168.0.10
Port 80
End
End
End

There is also a recommended "catch-all" deirective that goes to the end. This will be what Pound will serve if nothing above is matched. Example:

Service
BackEnd
Address 192.168.0.10
Port    80
End
Session
Type    BASIC
TTL     300
End
End

Access from the Internet:

Your external DNS will point web1 and web2 URLs to the same external IP. Your firewall (I use shorewall) will redirect traffic that comes on that IP over port 80 and 443 to the Pound server in the DMZ. Your internal DNS will point the above URLs to the DMZ IP of the Pound server (192.168.7.50).
So when the requests come from outside on HTTP they first hit the HTTP directive which just redirect to HTTPS. Internal DNS kicks in and the traffic is redirected back to Pound but this time over HTTPS (443). This is when the HTTPS directive kicks in. If you asked for web1 then request is directed internally to 192.168.7.5 over port 80 (HTTP). The answer is returned to the Pound server over port 80 (not encrypted) and then the traffic is encrypted to goioutside (from Pound to the Internet).
If the request from the outside is over HTTPS then the HTTPS directive is used straight away. Answers are returned to the Internet over HTTPS of course.

Access from the Inside:

The easy way is NOT to use pound if you don't want encryption.

If you want internal encryption then you just have to point the internal DNS to the Pound IP.

If you do not want encryption but still want to go through Pound then you need to give the Pound another internal IP (either another NIC or add an IP to the current NIC) and then create HTTP directives that forward traffic to the individual web servers. Of course, your Internal DNS will then have to point to that new Pound IP.
What happens if people have bookmarks that have HTTPS in them?
You can create HTTPS directives that redirect to HTTP. Effectivelly you do the opposite of what you did for the Internet traffic. Slightly more work but not the end of the world :)

Pound is capable of more complex setups than this and I may describe some of those in the future.