Creating an Ubuntu image on Google Compute Engine the right way

I’ve recently started playing with the Google Compute engine as part of me volunteering in The Public Knowledge Workshop here in Israel.
Being used to working with Ubuntu on Amazon EC2, while launching my first instance on GCE I was surprised to find out they don’t have Ubuntu images.
Googling revealed this guide, which seems way too complicated for doing a simple task – making Ubuntu cloud images run on GCE.

Doing some more Googling, apparently the GCE image format is simply a tar.gz file containing a disk.raw image.
Reading a bit more, I discovered you need to modify the image a bit for it to play nicely on GCE, mostly installing some Google daemons & tools inside it.

So, the first step would be to launch a Debian instance for creating the image. It’d require access to the Cloud Storage service for uploading the tar.gz file, and to the Compute service for creating the actual image.
I’m not going to get into the details of how to setup your account and configure the Cloud SDK on your computer. Google it 🙂

$ gcloud compute \
    --project "<project>" \
    instances create "ubuntu-image-creator" \
    --zone "<zone>" \
    --machine-type "n1-standard-1" \
    --network "default" \
    --metadata "sshKeys=<username>:<public-ssh-key>" \
    --maintenance-policy "MIGRATE" \
    --scopes "https://www.googleapis.com/auth/compute" \
    "https://www.googleapis.com/auth/devstorage.full_control" \
    --image "https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/debian-7-wheezy-v20140814"

After this is done, SSH to your newly create instance:

$ gcloud --project "<project>" compute ssh --zone "<zone>" ubuntu-image-creator

Inside the new instance, download the latest Ubuntu 14.04 (Trusty) image:

$ wget https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img

Because the image format is QCOW2, we’ll start by converting it to RAW format.

$ sudo apt-get update -y -q
$ sudo apt-get install -y -q qemu
$ qemu-img convert -f qcow2 -O raw trusty-server-cloudimg-amd64-disk1.img disk.raw

Now we have to mount it and install the Google tools inside it.
This image contains more than just a filesystem. It’s an entire disk.
To mount the root partition we’ll have to tell where the actual filesystem begins. Here’s how:

$ /sbin/fdisk -l disk.raw

Disk disk.raw: 2361 MB, 2361393152 bytes
4 heads, 32 sectors/track, 36032 cylinders, total 4612096 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00089410

   Device Boot      Start         End      Blocks   Id  System
disk.raw1   *        2048     4194303     2096128   83  Linux

As you can see, the partition begins in sector 2048. Each sector is 512 bytes, so the partition begins in the 1048576th byte. Lets mount it, copy our resolv.conf and chroot ourselves in!

$ sudo mkdir /mnt/loop
$ sudo mount -o loop,offset=1048576 disk.raw /mnt/loop
$ sudo rm -f /mnt/loop/etc/resolv.conf
$ sudo cp /etc/resolv.conf /mnt/loop/etc/resolv.conf
$ sudo chroot /mnt/loop /bin/bash

Next, lets install the Google tools (again, according to the manual):

# wget https://github.com/GoogleCloudPlatform/compute-image-packages/releases/download/1.1.6/python-gcimagebundle_1.1.6-1_all.deb
# wget https://github.com/GoogleCloudPlatform/compute-image-packages/releases/download/1.1.6/google-compute-daemon_1.1.6-1_all.deb
# wget https://github.com/GoogleCloudPlatform/compute-image-packages/releases/download/1.1.6/google-startup-scripts_1.1.6-1_all.deb
# apt-get install kpartx # python-gcimagebundle requirement
# dpkg -i python-gcimagebundle_1.1.6-1_all.deb
# dpkg -i google-compute-daemon_1.1.6-1_all.deb
# dpkg -i google-startup-scripts_1.1.6-1_all.deb

Now, lets get out of the chroot (type exit or press <CTRL>+d) and pack our image!

# exit
$ sudo rm -f /mnt/loop/resolv.conf
$ sudo umount /mnt/loop
$ tar cvfz trusty-server-cloudimg-amd64.tar.gz disk.raw

Our last step is creating the actual image:

$ gsutil mb gs://hasadna-images/ # change this to something else
$ gsutil cp trusty-server-cloudimg-amd64.tar.gz gs://hasadna-images/
$ gcloud compute images create trusty-server-cloudimg-amd64 \
    --source-uri gs://hasadna-images/trusty-server-cloudimg-amd64.tar.gz

Lets launch a new instance with the newly created image!

$ gcloud compute \
    --project "<project>" \
    instances create "ubuntu-instance" \
    --zone "<zone>" \
    --machine-type "n1-standard-1" \
    --network "default" \
    --metadata "sshKeys=<username>:<public-ssh-key>" \
    --maintenance-policy "MIGRATE" \
    --image "https://www.googleapis.com/compute/v1/projects/<project>/global/images/trusty-server-cloudimg-amd64" \
    --no-scopes

SSH to the newly created Ubuntu instance:

$ gcloud --project "<project>" compute ssh --zone "<zone>" ubuntu-instance

And check for the OS version:

omrib@ubuntu-instance:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 14.04.1 LTS
Release:	14.04
Codename:	trusty