How to increase disk space on existing AWS EC2 Linux (Ubuntu) Instance without losing data

By Semir H.

Let’s say you have a Linux Amazon Web Services (AWS) EC2 Instance up and running and you start running out of space. I’ll show you how to quickly clone its disk (EBS volume) onto a bigger disk (also EBS), replace the smaller disk with the bigger one and boot off the bigger (new) disk. No need for any third party tools like Acronis or Ghost or even rsync.


If you find this tutorial useful, please consider making a donation to support future work like this:

Thank you :)

Create Snapshot of the original disk:

First thing we’ll do is to create a Snapshot of the original disk so that we can then create a new, bigger volume out of that Snapshot. There are a couple of things we have to note down first: the Instance number and then the volume attached to that Instance. The easiest way to fine the Instance number is in the Instances part of the AWS Management Console. It will start with “i” and be under the “Instance” column. Now navigate to the Volumes section (Elastic Block Store section) and locate the volume that is attached to your Instance. You can see the Instance number under the “Attachment Information” column. The first half of that string will be the Instance number. Please see the images below for an example.

Instance (in this case number is i-1920b74c):

Please also note which Zone your Instance is in. This will be important for later. In my case the Zone is “ap-southeast-1b” in the Singapore Region. The Zone can be seen by selecting the Instance and looking under its Description tab (bottom).

Volume (note the Instance number):

As you’ll notice in the examples above, the size of the original Volume (EBS Disk) is 8 GB. I’ll increase that to 25 GB.

Creating a Snapshot

To create a snapshot of the Volume you simply right-click on it and click on Create Snapshot. Or you can select the Volume and click on the Create Snapshot button at the top.

Please note that I am creating a Snapshot of the running virtual machine. If you have a database on it or some other application that has a lot of transactions happening then it would be best to turn the virtual machine off (schedule some downtime) and do the Snapshot.

Give the Snapshot a meaningful description and a name. Click on “Yes, Create”.

Once done, you will be able to see your Snapshot in the Snapshots part of the Elastic Block Store area. See example below. Please note the Name, Description and Capacity (original 8 GB).

Creating a new (clone) Volume

Next, we’ll create a Volume out of the Snapshot. Simply right-click on the Snapshot and select “Create Volume from Snapshot” from the resulting menu or you can use the button at the top.

Put in the desired Size of the new Volume and make sure the Availability Zone coresponds to the AZ that the original Instance is in. Click on “Yes, Create”.

If you navigate to the Volumes section you will see the new 25 GB volume there. You’ll notice that it is not in use.

Attach new Volume to existing Instance:

And now for the fun part. We will stop the Instance, detach the original 8 GB Volume, attach the new 25 GB Volume and make sure we are able to boot. Please note the new Volume will have the data up to the point when the Snapshot was taken. If the original machine had more data put onto it since the Snapshot then that will have to be dealt with. This is beyond the scope of this tutorial.

Stop the Instance (if it is running).

Go to the EBS Volumes section, select the original Volume, right click on it and select “Detach Volume”. You can also use the Detach Volume button. When prompted, please select “Yes, Detach”.

To attach the new Volume right-click on it and select “Attach Volume”.

Make sure you select the correct Instance. In the example below I am also modifying the Device to be the same as the original one. The example is for a Linux (Ubuntu) Instance. This is important. Click on “Yes, Attach”.

The Status should change to “in-use”.

Go back to your Instance and start it up. Log back into your Linux Instance and run the following command (for ext3 file system):

    resize2fs /dev/xvda1

Please note that the device is not called sda1 but xvda1. The device was renamed by the Kernel.
After it is done you should be able to see the full 25 GB. use the “du -h” command. Example:

Cleanup:

Once you’ve made sure all works as it should and that all your data is fine on the new Volume, you can remove the Snapshot and the old Volume if you do not intend to use them anymore. Go to the relevant AWS Management Console sections and remove (delete) them.

Replace a lost Key Pair an existing AWS EC2 Instance uses

By Semir H.

This tutorial will show you how to use a new Key Pair with an existing Linux instance.

If you find this tutorial useful, please consider making a donation to support future work like this:

Thank you :)

Problem:

You have an existing EBS (root device) based Instance with data on it that needs to be saved. The original Key Pair has been lost so you can’t log into the Linux Instance. You need to get to the data on the virtual machine.

Solution:

We’ll clone the running Instance and re-deploy it but with a new Key Pair. We can either use a pre-made Pair or create a Pair while we’re re-deploying. My example uses Ubuntu but should apply to other flavours as well.

Creating an Image:

Log into your Amazon Web Services (AWS) Management Console, go to Amazon EC2, pick the Region where the Instance in question is running and click on the Instances link. Right click on the Instance in question and select “Create Image (EBS AMI)” from the resulting menu.

Give it a Image Name and a Description and click on “Create This Image” button. Example:

The process will now begin.  Close the notification window.

Once the AMI is created (won’t take long) you should see it in the "Images" - "AMIs" part of your Amazon EC2 AWS Management Console. Please make sure you are still in the correct Region. Example:

Creating new Instance out of the new AMI:

Go to Images AMIs, locate your recently created AMI, right-click on it and select “Launch Instance” from the resulting menu.

Follow the prompts to finish creating the Instance (see my previous post) but make sure you select the correct Key Pair (one you create earlier, not the lost one) in the “Create Key Pair” section. Example:

All other settings should be the same as for the original Instance.
After a little bit of time your new Instance should be up and running.
Here’s an example below. Please note the different Key Pairs.

You can now connect to it, using your new private key, and confirm that all your data is still there.
Once you confirm no data is missing you can stop the old Instance and eventuality terminate it (after making absolutely sure you will never need anything from it).

Cleaning Up:

You can now de-register the AMI (unless you want to use later again).

You should also go to EBS - Snapshots and delete the Snapshot of the disk that was created when the AMI was made (unless you intend to use it for something in the future).

Creating a Key Pair to share between AWS EC2 Instances

By Semir H.


If you find this tutorial useful, please consider making a donation to support future work like this:

Thank you :)

Sign into Amazon Web Services Console. Go to the Amazon EC2 tab.
Pick your desired Region as this is where the key will be stored.
Locate the “Key Pairs” link under Networking & Security section and click on it.
Click on “Create Key Pair” button on the top to start the process.
Give the Key Pair a name. E.g. APAC-Keys-2011. Click on “Create”.

The Key Pair will be created and you will be prompted to save the private portion (a .pem file) on your computer/device. Please do so and make sure you keep it private,

You can now use this Key Pair for your new Instances. Please pick the “Choose from your existing Key Pairs” at the “Key Pairs” option when creating an Instance and select your pre-made Key Pair. Example below:

Connecting to AWS Linux (Ubuntu) Instance from Windows using PuTTY and the SSH protocol

By Semir H.

This part continues from the previous “NAME” part where we launched an Instance. In this part I will show you how to use PuTTY to connect to your running Linux (Ubuntu) Instance. In order for PuTTY to access your Instance over the SSH protocol we opened up the SSH TCP port (22) by using a Security Group which is essentially a Firewall.

Creating a PuTTY Private Key:

We must first create a PuTTY Private Key using the .PEM key generated beforehand. For this you need to get puttygen.
Run puttygen to start the process. Click on “File” and on “Load private key”.

Browse to the private key file (.pem) you saved earlier and load it. You should get a success notice like the one below. Click on OK.

You may want to edit the key comment and put a key passphrase before we generate a PPK file. I recommend using a key passhprase but please do not forget it or your resulting PPK will be of no use. Leave the type of key as SSH-2 RSA and leave the number of bits as 1024. Once you’ve made desired comments and put in the phrase click on “Save private key”. Give it a meaningful name and save as .ppk.

Connecting to AWS Linux Instances:

We must first find out the public DNS of the Instance we wish to connect to. To do this go to your AWS Management Console, go to Amazon EC2 tab, Instances, click on your Instance and look down under “Description”. You should see an address, next to Public DNS, similar to this: ec2-122-248-203-235.ap-southeast-1.compute.amazonaws.com

Next, start PuTTY, put the DNS entry into the Host Name field and then browse down to the SSH Category and to the Auth section under SSH. This is where you add your private key (PPK) for later authentication. Go back to the Session category, make sure the DNS entry is still there, give this session a name (e.g. Test AWS Ubunt) and click on Save.
See below for an example:

Now it’s a simple matter of loading the saved session. First time you connect you will get a security alert saying that the server’s host key is unknown to you. Please select Yes to continue if you think it is safe to do so (in this case it is). You will get this with any new server the first time. You will also get this if you turn your Instance off and then on again (because the DNS and server hostname will change). The private key will not change so that can be re-used to connect.
You will be prompted to put in a username to log in. For this particular AMI it is ‘ubuntu’. Once you put that in you will get prompted to put the PPK passphrase (if you set one). If all goes well you should be logged into your Ubuntu Linux Amazon AWS Instance. Yay :)

Troubleshooting:

If you’re having issues then please revisit the steps above. Please make sure you’re using the correct DNS entry and the correct private key. You can check which private key the Instance is using by looking at the Instance Details section (same one where DNS info is located). Please be aware that the public DNS string will change if you shut the server down and start it again. This can be addressed using Elastic IPs but we’ll cover that in another tutorial some other time.

Launching a Linux (Ubuntu) Instance on Amazon Web Services EC2 - A Beginner's Guide

By Semir H.

To create an AWS account please see How To Sign Up.

Log into your AWS account and go to the AWS Management Console.
Go to the Amazon EC2 tab. Make sure you’ve picked the desired Region (see this article for more detail).

Click on the ‘Launch Instance” button to start the process.

I’ll be creating a micro Instance of Ubuntu 11.04 64-bit server from one of the Community Amazon Machine Images (AMI). Community AMIs are pre-configured operating systems available for anyone to create Instances from.

Choose an AMI:

Click on Community AMIs and find your desired AMI. In my case it will be Ubuntu Natty Daily with AMI ID = ami-48aad21a. Click on “Select” next to it to continue.

Instance Details:

You can leave Instance Details at defaults. I am choosing a Micro Instance and I do not care about the Availability Zone for now. Click on “Continue”.

Leave the Advanced Instance Options at defaults an click on “Continue”.

Leave the Tags part empty for now. We’ll cover this in another tutorial. Click on “Continue”.

 
Create a Key Pair:

We will  need to create public/private key pairs to later access our Linux (Ubuntu) Instance.
You can create these per Instance or you can create them beforehand and use for future Instances. We’ll create one now for testing purposes. I’ll cover creating some for re-use later.

Enter a name for your key pair and click on the “Create and Download you Key Pair” link. Example:

 You will get prompted to open/save the resulting .pem file. Please do so.

Configure Firewall:

The next part configuring a security group which is basically a firewall.
Since I want to open ports for SSH (TCP 22) and HTTP (TCP 80) I will choose to create a new group and add those rules in at this stage. You can come back to this later to open more ports or close some.
So pick “Create a new Security Group”, give it a name and a description and add the desired rules. Some are pre-defined (e.g. SSH and HTTP) so you need to just pick them from the “Create a new rule” drop-down and add them. I’m leaving the source at default. Once done click on “Continue”.
See here for an example:

Review:

This is the final stage before launching. It gives you the opportunity to review your choices and edit them if you wish to. If you’re happy with them then click on “Launch”.
You will get a message that the Instance is being launched.

To view the progress of the launch you go back to the AWS Management Console and go to the Amazon EC2 tab. Click on Instances in the menu on the left. You should see your instance and the various properties it has as well as the Status. In no time it will turn to “running”. Here’s an example of this view:

That’s it. You can now connect to your running Instance. I’ll cover how to do that in my next tutorial.