Resizing my Ubuntu Server AWS Boot Disk

AKA: Building a Bigger GeoSandbox

(Note: This article has been updated to make it clear that expanded EBS volumes will = additional charges from AWS. Something that is not clearly stated in the AWS documentation.)
If you’ve been reading my last few blog posts, you know I’ve been experimenting with various Ubuntu server configurations using Amazon Web Services (AWS) to serve web-maps and spatial data. As my procedures have evolved, the micro-instances I started working with have outgrown their usefulness. Lately, I’ve been testing GeoWebCache, and seeing how that works with GeoServer and the rest of the OpenGeo Suite. As anyone who’s ever delved into the map-tile world knows, tile caches start gobbling up disk space pretty quick once you start seeding layers at larger scales. I had to figure out a way to expand my storage space if I wanted to really test out GeoWebCache’s capabilities without bringing my server to its knees.
The Ubuntu AMIs I’ve been using all start out with an 8GB EBS volume as the root drive with an additional instance-store volume that can be used for “ephemeral” storage. That “ephemeral” storage means, whatever is in there is lost every time the instance is stopped. Supposedly, a reboot will not clear out the ephemeral storage, but a stop and then start, will. There are procedures you can set up to save whatever is in the ephemeral instance-store volume before you stop it, but I was looking for something a bit easier.
A medium instance AMI includes a 400GB instance-store volume, but it still starts out with the same 8GB root drive that a micro instance has. So, what to do? How do I expand that 8GB disk so I can save more data without losing it every time I stop the system?
A little searching led to a couple of articles that described what I wanted to do. As usual, though, I ran into a couple of glitches. So, for my future reference and in case it might be of some help to others, the following paragraphs describe my procedure.
The two articles this post was compiled/aggregated/paraphrased from are:

• The ‘Ask Ubuntu Stack Exchange’ question and answer: Not enough disk space ‘/’ in AWS instance
• The ‘Alestic Ubuntu on Amazon EC2’ blog post: Resizing the Root Disk on a Running EBS Boot EC2 Instance.

The standard “Out of the Box” Ubuntu AMI disk configuration

First, connect to the server using WinSCP, SecPanel, or some other means as described in one of my previous posts. Then open a terminal (or PuTTY) window, and enter:
df -h
You should see something like this:

The first line (/dev/xvda1) is the EBS root disk, and it shows 8.0 GB, with about 3.1 GB being used. The last line (/dev/xvdb) is the instance-store “ephemeral” space that’s wiped clean on every stop.

Note: The Ubuntu AMIs use “xvda1” and “xvdb” as device identifiers for the attached disks and storage space, while the AWS console uses “sda1” and “sdb”. In this case, “xvda1” equals “sda1”. Keep this in mind as you’re navigating back and forth between the two.

Step One: Shut It Down

First, look in the AWS console, and make a note of what availability zone your server is running in. You will need to know this later on. The one I’m working on is in “us-east-1d”. Then, using the AWS console stop the EC2 instance (Do not terminate it, or you will wind up rebuilding your server from scratch). Then move to the “Volumes” window, choose the 8GB volume that’s attached to your server, and under the “More…” drop-down button, choose “Detach Volume”. It will take some time for the detach action to complete.

Step Two: Make A Copy

Next, with the same volume chosen, and using the same “More…” button, create a “Snapshot” of the volume. I recommend you give this (and all your volumes) descriptive names so they’re easier to keep track of.

Step Three: Make It Bigger

Once the snapshot is done processing, it will show up in the “Snapshot” window. Again, giving the snapshot a name tag helps tremendously with organization. Choose this snapshot, and then click on the “Create Volume” button.

In the Create Volume dialog, enter the size you want the new root disk to be. Here, I’ve entered 100 GB, but I could enter anything up to the nearly 400GB of storage space I have left in my Medium Instance. Also in this dialog, choose the availability zone to create the volume in. Remember earlier in this post when I said to note the availability zone your server is running in? This is where that little piece of information comes into play. You MUST use the same availability zone for this new, larger volume as your original server volume used. Click the “Yes, Create” button, and a new larger volume will be placed in your list of volumes.

Step Four: We Can Rebuild It

Next, attach the new larger EBS volume to the original Ubuntu server instance. Go back to the Volume window, choose the newly created larger volume, click the “More…” button, and choose “Attach Volume”.

In this dialog box, make sure the correct instance is showing in the “Instance” drop-down. In the “Device” text box, make sure the device is listed as it is shown above. It should be “/dev/sda1”. Note: This will not be the default when the dialog opens. You must change it!
Clicking on the “Yes, Attach” button will begin the attachment process, which will take a minute or two to complete. Once it’s done, you can spin up the server with the new root drive and test it out.

Step Five: Start It Up Again

Choose the server, and under “Instance Actions”, choose “Start”. Once started, connect to the server using your preferred client. Open a terminal or PuTTY window, and once again enter:
df -h
You should now see something like this:

Notice the differences from the first df command. Now the root disk (/dev/xvda1) will show a size of 99GB, or whatever size you might have made your new volume.

More Room To Play

Now I can adjust my root disk size to suit the task at hand. I can store more spatial data in my GeoServer data directory, and seed my map tiles down to ever larger scales. Knowing how to shuffle and adjust these volumes opens up a slew of other possibilities, too. I can imagine setting up a separate volume just to hold spatial data and/or tiles, and using that to copy or move that data back and forth between test servers.
Be mindful though, this extra space is not free. The larger EBS volume does not replace the space on the ephemeral instance-store volume, it is an addition to it. There will be additional charges to your AWS account for the larger EBS volume based on it’s size. This fact is not made clear in the AWS documentation. So, I recommend you increase the size of the EBS root disk as much as you need, but no more.
Oh the possibilities…