flypig.co.uk

Personal Blog

RSS feed Click the icon for the blog RSS feed.

Blog

5 most recent items

24 Dec 2016 : You are old, Acer Laptop #
"You are old, Acer Laptop" this blog-writer wrote,
"And your battery has become rather shite;
And yet you incessantly compile all this code –
Do you think, at your age, it is right?"

"In my youth," Acer Laptop replied to the man,
"I feared it might injure my core;
But now that I'm perfectly sure I have none
Why, I do it much more then before."

"You are old," said the man, "As I mentioned before,
And have grown most uncommonly hot;
Yet you render in Blender in HD or more –
Pray, don't you think that's rather a lot?"

"In my youth," said the Acer, as he wiggled his lid,
"I kept all my ports very supple
By the use of this app—installed for a quid—
Allow me to sell you a couple?"

"You are old," said the man, "And your threading's too weak
For anything tougher than BASIC;
Yet you ran Java 5 with its memory leak –
Pray, how do you manage to face it?"

"In my youth," said the laptop, "I took a huge risk,
And argued emacs over vim;
And the muscular strength which it gave my hard disk,
Has lasted through thick and through thin."

"You are old," said the man, "one can only surmise
That your circuits are falling apart;
Yet you balanced a bintree of astonishing size—
What made you so awfully smart?"

"I have answered three questions, now leave me alone,"
Said the Acer; "It's true I'm not brand new!
Do you think I'm like Siri on a new-fangled phone?
Be off, or I'll have to unfriend you!"


My current laptop is getting a bit long-in-the-tooth. It's an Acer Aspire S7 which Joanna and I bought cheap a couple of years ago as an ex-display machine. It's a thin, light ultrabook that's worked really well with Linux and still feels powerful enough to use as my main development machine. Amongst it's excellent qualities, the only two negatives have been a rather loud fan, and a less-than-perfect keyboard.

Still, it's getting a bit worn-out now and I've used it so much some of the keys have worn through to the backlight. I've also noticed some very appealing ultrabook releases recently, including the new Acer Swift 7 and the Asus Zenbook 3. Both of these hit important milestones, with the Swift being less than 1cm think, and the Zenbook coming in at under 1kg in weight. Impressive stuff.

My rather bruised keyboard

With these two releases having piqued my interest, and with my current machine due for renewal, it seemed like a good time to reassess the ultrabook landscape and figure out whether I can justify getting a new machine.

Most manufacturers now offer some impressive ultrabook designs. HP has its Elitebook and Spectre  ranges, Apple's MacBook Pro now falls firmly within the category, Dell has the XPS devices and Razer is a newcomer to the ultrabook party with its new Blade Stealth laptop. They all seem to have received decent reviews and there's clearly been some design-love spent on them all.

However, they are also all expensive machines (around the £1000 mark). I'm going to use this as my main work machine for the next couple of years, during which time it'll get daily use, so I have no qualms about spending a lot on a good laptop. On the other hand, if I make a bad decision it'll be an expensive mistake. Given this, it's only sensible I should spend some time considering the various options and try to make a decision not just based on instinct, but on the hard specs for each machine.

There are plenty of reviews online which there's no need for me to duplicate; however I have some particular requirements and preferences, so this analysis is based firmly on these.

My requirements are for a thin, light laptop that's got a really good screen (the larger and higher the resolution the better). When I say thin, I mean ideally 1cm or thinner. By light, I mean as close to 1kg as possible. By good screen, it should be at least a 13in screen with better-than-FHD resolution (given FHD is what my current laptop supports). Any new machine must be better than my current laptop by a significant margin. My current laptop is still perfectly usable, and I'm happy with the size, weight, processing speed and resolution; but it doesn't make sense to get a new machine if it's not going to be a noticeable upgrade.

I've been single-booting Linux for many years now, and plan to do the same with whichever laptop I get next. That means the Windows/macOS distinction is irrelevant for me: they'll get wiped off as the first thing I do with the machine either way.

Before starting this task I was certain I'd end up getting the Acer Swift 7. Based on the copy I'd read, it's the thinnest 13.3in laptop you can buy and looks quite attractive to me (apart from the horrible 'rose gold' colour; ugh). If this didn't work out, I thought the numbers would point in the direction of an Apple device, given almost everyone I know in the Computer Lab uses an Apple laptop (there must be something in that, right?). After carefully working through the specs, I've been really surprised by the results.

The MacBook Pro appears to be decent in most areas, but in fact is worse than the best of its competitors in almost all respects. Since I don't want to run macOS, the only thing in its favour is the attractive design. The MacBook Air is really showing its age now, and is even beaten by the MacBook Pro on everything except price.

The Swift 7 is thin, but turns out to be a really poor choice. That just goes to show how unreliable my gut instinct is and I'm glad I didn't buy it without looking at the alternatives. It's running an M-class processor with no touchscreen or keyboard backlight. The port selection is average and in practice its only strengths are the thin chassis and fanless design. Both are nice features, but the result of the package is hardly an upgrade over my existing machine.

The Razer Blade Stealth was originally down as my alternative choice. It has a gloriously high-resolution (3840 * 2160) screen, and personally I love the multi-coloured keyboard lighting. Some might say it's a just gimmick, and I could never justify a purchase because of it (especially bearing in mind it almost certainly won't work properly on Linux), but I still think it's glorious. Unfortunately the Stealth turns out to have a small screen size and suffers problems running Linux. Both are show-stoppers for me.

The Zenbook also looks really appealing, with its incredibly lightness. Unfortunately, like the Stealth it suffers from a smaller screen size and Linux problems. Too bad.

I kept the Spectre in for comparison, but I could never have gone for it given it's horrific aesthetics. I admit, I'm shallow. Nevertheless, it turns out it doesn't offer enough of an upgrade over my existing system anyway (same resolution, worse dimensions and weight).

The unequivocal standout winner is the Dell XPS. In some ways I'm sad about this, as in my mind I associate Dell with being the height of box-shifting PC dullness. Dell's aggressive product placement really puts me off. The machine itself doesn't have a particularly spectacular design. Yet there's no denying the numbers, and the screen really does appear to be way-ahead of the competition, with its unusually thin bezel, high-resolution and decent size. I was tempted by the 15 in version, given its discreet graphics, but the size and weight just nudge outside the area I feel is acceptable for me.

That leaves only the XPS 13 standing. To top everything else off, Dell is the only company to officially support Linux (Ubuntu) on its machines, which it deserves credit for. I'm not sure whether I'll end up getting a new laptop at all, but if I do I'd want it to be this.

Scroll past the pictures to see my full 'analysis' of the different laptops.
Acer Aspire S7 Acer Swift 7 Hp Spectre 13t
Acer Aspire S7 Acer Swift 7 HP Spectre 13t
Razer Blade Stealth Apple MacBook Pro Apple MacBook Air
Razer Blade Stealth MacBook Pro MacBook Air
Asus Zenbook 3 Dell XPS 13 Dell XPS 15
Asus Zenbook 3 UX390UA Dell XPS 13 Dell XPS 15


Colour coding:
Best
Better
The same as my current Acer Aspire S7
Worse
Show-stopper
 
 

Acer Aspire S7

Acer Swift 7

HP Spectre 13t (or 13-v151nr)

Razer Blade Stealth

MacBook Pro

CPU

1.9GHz Intel Core i5-3517U

1.2GHz Intel Core i5-7Y54

2.5GHz Intel Core i7-6500U

2.7GHz Intel Core i7-7500U

3.3GHz Intel Core i7

RAM (max)

4GB, 1600MHz DDR3

8GB, LPDDR3

8GB, LPDDR3

16GB, 1866MHz LPDDR3

16GB, 2133MHz LPDDR3

NVM (max)

256GB SSD

256GB SSD

512GB SSD

1TB SSD

1TB SSD

GPU

Intel HD4000

Intel HD615

Intel HD620

Intel HD620

Intel HD550

Resolution

1920x1080

1920x1080

1920x1080

2560x1440 or 3840x2160

2560x1600

Screen size (in)

13.3

13.3

13.3

12.5

13.3

Battery (hours)

6

9

9.75

9

10

Height (mm)

11.9

9.98

10.4

13.1

14.9

Width (mm)

323

324.6

325

321

304.1

Depth (mm)

223

229.6

229.4

206

212.4

Weight (kg)

1.3

1.13

1.16

1.29

1.37

Touchscreen

Yes

No

No

Yes

No

Fan

Yes

No

Yes

Yes

Yes

Backlit keyboard

Yes

No

Yes

Yes

Yes

Ports

USB2*2, 3.5mm, HDMI, SD, AC

USB3*2, 3.5mm

USB3*3, 3.5mm

USB3*2, 3.5mm, HDMI, AC

USB3*2, 3.5mm

Aesthetics

         

Linux compat

Good

Reportedly works OK

Unknown

Works with glitches (e.g. WiFi)

Currently flaky (will improve)

Price (£)

 

1000

1035

1250

1450

Price spec

 

8GB, 256GB

8GB, 256GB

16GB, 256GB

8GB, 256GB

Notes

Has been perfect, apart from the poor keyboard

Underpowered, and not big enough upgrade to be worthwhile

Ugly, ugly, ugly

Really tempting, good value, but small screen size is a problem

Quite big and heavy. Decent, but the Dell XPS 13 is better in every respect

           
 

Acer Aspire S7

MacBook Air

Asus Zenbook 3 UX390UA

Dell XPS 13

Dell XPS 15

CPU

1.9GHz Intel Core i5-3517U

2.2GHz Intel Core i7

Intel Core i7-7500U

3.1GHz Intel Core i5-7200U

3.5GHz Intel Core i7-6700HQ

RAM (max)

4GB, 1600MHz DDR3

8GB, 1600MHz LPDDR3

16GB, 2133MHz LPDDR3

8GB, 1866MHz LPDDR3

32GB, 2133MHz DDR4

NVM (max)

256GB SSD

512GB SSD

512GB SSD

256GB SSD

512GB SSD

GPU

Intel HD4000

Intel HD6000

Intel HD620

Intel HD620

NVIDIA GeForce GTX 960M

Resolution

1920x1080

1440x900

1920x1080

3200x1800

3840x2160

Screen size (in)

13.3

13.3

12.5

13.3

15.6

Battery (hours)

6

12

9

18

17

Height (mm)

11.9

17

11.9

15

17

Width (mm)

323

325

296

304

357

Depth (mm)

223

227

191

200

235

Weight (kg)

1.3

1.35

0.91

1.29

1.78

Touchscreen

Yes

No

No

Yes

Yes

Fan

Yes

Yes

Yes

Yes

Yes?

Backlit keyboard

Yes

Yes

Yes

Yes

Yes

Ports

USB2*2, 3.5mm, HDMI, SD, AC

USB3*2, 3.5mm, TB, SD, AC

USB3, 3.5mm

USB3*3, 3.5mm, SD, AC

USB3*3, 3.5mm, SD, HDMI, AC

Aesthetics

         

Linux compat

Good

Works

Works but volume, FP, HDMI issues

Officially supported

Reported to work well

Price (£)

 

1100

1500

1250

1749

Price spec

 

8GB, 256GB

16GB, 521GB

8GB, 256GB

16GB, 512GB

Notes

Has been perfect, apart from the poor keyboard

The low resolution being worse than my current laptop, as well as being thick, rules this out

Thin and really light, makes it really appealing, but the small screen size is a problem

Relatively thick and heavy, but the screen is really great

Just a bit too big and heavy to be viable

 
Comment
9 Dec 2016 : Cracking PwdHash #
On Wednesday Graham Rymer and I presented our work on cracking PwdHash at the Passwords 2016 conference. It's the first time I've done a joint presentation, which made for a new experience. It was also a very enjoyable one, especially having the chance to work with such a knowledgeable co-author.

The work we did allowed us to search for the original master passwords that people use with PwdHash. Passwords which are used to generate the more complex site-specific passwords given to websites, and which may then have been exposed by recent password leaks in hashed form. We were surprised, both by the number of master passwords we were able to find, and the speed with which hashcat was able to eat its way through the leaked hashes.

Running on an Amazon EC2 instance, we were able to work through the SHA1-hashed LinkedIn.com leak by generating 40 million hashes per second. In total we were able to recover 75 master passwords from the leak, as well as further master passwords from the Stratfor.com and Rootkit.com leaks.

Feel free to download the paper and presentation slides, or watch the video captured during the conference (unfortunately there's only audio with no video for the first segment).

Here are a few of the master passwords Graham was able to recover from the password leaks.
 
Domain Leaked hash Password
Stratfor e9c0873319ec03157f3fbc81566ddaa5 frogdog
Rootkit 2261bac1dfe3edeac939552c0ca88f35 zugang
Rootkit 43679e624737a28e9093e33934c7440d ub2357
Rootkit dd70307400e1c910c714c66cda138434 erpland
LinkedIn 508c2195f51a6e70ce33c2919531909736426c6a 5tgb6yhn
LinkedIn ed92efc65521fe5074d65897da554d0a629f9dc7 Superman1938
LinkedIn 5a9e7cc189fa6cf1dac2489c5b81c28a3eca8b72 Fru1tc4k3
LinkedIn ba1c6d86860c1b0fa552cdb9602fdc9440d912d4 meideprac01
LinkedIn fd08064094c29979ce0e1c751b090adaab1f7c34 jose0849
LinkedIn 5264d95e1dd41fcc1b60841dd3d9a37689e217f7 linkedin

I'll leave it as an exercise for the reader to decide whether these are sensible master passwords or not.
Comment
16 Oct 2016 : Fixing snap apps with relocatable DATADIRS #

It's luminous, not florescent

Recently I've been exploring how to create snaps of some of my applications. Snap is the new 'universal packaging format' that Canonical is hoping will become the default way to deliver apps on Linux. The idea is to package up an app with all of its dependencies (everything needed apart from ubuntu-core), then have the app deployed in a read-only container. The snap creator gets to set what are essentially a set of permissions for their app, with the default preventing it from doing any damage (either to itself or others). However, it's quite possible to give enough permissions to allow a snap to do bad stuff, so we still have to trust the developers of the snap (or spend your life reading through source-code to check for yourself). If you want to know more about how snaps work, the material out there is surprisingly limited right now. Most of the good stuff - and happily it turns out to be excellent - can be found at snapcraft.io.

Predictably the first thing I tried out was creating a snap for functy, which means you can now install it just be typing 'snap install functy' on Yakkety Yak. If your application already uses one of the conventional build systems like cmake or autotools, creating a snap is pretty straightforward. If it's a command-line app, just specifying a few details in a yaml file may well be enough. Here's an example for a fictional utility called useless, which you can get hold of from GitLab if you're interested (the code isn't fictional, but the utility is!).

The snapcraft file for this looks like this.
 
name: useless
version: 0.0.1
summary: A poem transformation program
description:
  Has a very limited purpose. It's mostly an arbitrary example of code.
confinement: strict

apps:
  useless:
    command: useless
    plugs: []

parts:
  useless:
    plugin: autotools
    source: https://gitlab.com/flypig/useless.git
    build-packages:
      - pkg-config
      - libpcre2-dev
    stage-packages:
      - libpcre2-8-0
    after: []

This just specifies the build system (plugin), some general description, the repository for the code (source), a list of build and runtime dependencies (build-packages and stage-packages respectively) and the command to actually run the utility (command).

This really is all you need. To test it just copy the lot into a file called snapcraft.yaml, then enter this command while in the same directory.
 
snapcraft cleanbuild

And a snap is born.

This will create a file called useless_0.0.1_amd64.snap which you can install just fine. When you try to execute it things will go wrong though: you'll get some output like this.
 
flypig@Owen:~/Documents/useless/snap$ snap install --force-dangerous useless_0.0.1_amd64.snap

useless 0.0.1 installed
flypig@Owen:~/Documents/useless/snap$ useless
Opening poem file: /share/useless/dong.txt

Couldn't open file: /share/useless/dong.txt

The dong.txt file contains the Edward Lear poem "The Dong With a Luminous Nose". It's a great poem, and the utility needs it to execute properly. This file can be found in the assets folder, installed to the $(datadir)/@PACKAGE@ folder as specified in assets/Makefile.am:
 
uselessdir = $(datadir)/@PACKAGE@
useless_DATA = dong.txt COPYING
EXTRA_DIST = $(useless_DATA)

In practice the file will end up being installed somewhere like /usr/local/share/useless/dong.txt depending on your distribution. One of the nice things about using autotools is that neither the developer not the user needs to know exactly where in advance. Instead the developer can set a compile-time define that autotools will fill and embed in the app at compile time. Take a look inside src/Makefile.am:
 
bin_PROGRAMS = ../useless
___useless_SOURCES = useless.c

___useless_LDADD = -lm @USELESS_LIBS@

___useless_CPPFLAGS = -DUSELESSDIR=\"$(datadir)/@PACKAGE@\" -Wall @USELESS_CFLAGS@

Here we can see the important part which sets the USELESSDIR macro define. Prefixing this in front of a filename string literal will ensure our data gets loaded from the correct place, like this (from useless.c)
 
char * filename = USELESSDIR "/dong.txt";

If we were to package this up as a deb or rpm package this would work fine. The application and its data get stored in the same place and the useless app can find the data files it needs at runtime

Snappy does things differently. The files are managed in different ways at build-time and run-time, and the $(datadir) variable can't point to two different places depending on the context. As a result the wrong path gets baked into the executable and when you run the snap it complains just like we saw above. The snapcraft developers have a bug registered against the snapcraft package explaining this. Creating a generalised solution may not be straightforward, since many packages - just like functy - have been created on the assumption the build and run-time paths will be the same.

One solution is to allow the data directory location to be optionally specified at runtime as a command-line parameter. This is the approach I settled on for functy. If you want to snap an application that also has this problem, it may be worth considering something similar.

The first change needed is to add a suitable command line argument (if you're packaging someone else's application, check first in case there already is one; it could save you a lot of time!). The useless app didn't previously support any command line arguments, so I augmented it with some argp magic. Here's the diff for doing this. There's a fair bit of scaffolding required, but once in, adding or changing the command line arguments in the future becomes far easier.

The one part of this that isn't quite boilerplate is the following generate_data_path function.
 
char * generate_data_path (char const * leaf, arguments const * args) {
    char * result = NULL;
    int length;

    if (leaf) {
        length = snprintf(NULL, 0, "%s/%s", args->datadir, leaf);
        result = malloc(length + 2);
        snprintf(result, length + 2, "%s/%s", args->datadir, leaf);
    }

    return result;
}

This takes the leafname of the data file to load and patches together the full pathname using the path provided at the command line. It's simple stuff, the only catch is to remember to free the memory this function allocates after it's been called.

For functy I'm using GTK, so I use a combination of GOptions for command line parsing and GString for the string manipulation. The latter in particular makes for much cleaner and safe code, and helps simplify the memory management of this generate_data_path function.

Now we can execute the app and load in the dong.txt file from any location we choose.
 
useless --datadir=~/Documents/Development/Projects/useless/assets

There's one final step, which is to update the snapcraft file so that this gets added automatically when the snap-installed app is run. The only change now is set the executed command as follows.
 
    command: useless --datadir="${SNAP}/share/useless"

Here's the full, updated, snapcraft file.
 
name: useless
version: 0.0.1
summary: A poem transformation program
description:
  Has a very limited purpose. It's mostly an arbitrary example of code.
confinement: strict

apps:
  useless:
    command: useless --datadir="${SNAP}/share/useless"
    plugs: []

parts:
  useless:
    plugin: autotools
    source: https://gitlab.com/flypig/useless.git
    build-packages:
      - pkg-config
      - libpcre2-dev
    stage-packages:
      - libpcre2-8-0
    after: []

And that's it! Install the snap package, execute it (just by typing 'useless') and the utility will run and find it's needed dong.txt file.

There's definitely a sieve involved
Comment
2 Oct 2016 : Server time-travel, upgrading four years in nine days #
Years of accumulation has left me with a haphazard collection of networked computers at home, from a (still fully working) 2002 Iyonix through to the ultrabook used as my daily development machine. There are six serious machines connected to the network, if you don't count the penumbra of occasional and IoT devices (smart TV, playstation, Raspberry Pis, retired laptops, A9).

All of this is ably managed by Constantia, my home server. As it explains on her webpage, Constantia's a small 9 Watt fanless server I bought in 2011, designed to be powered using solar panels in places like Africa with more sun than infrastructure (and not in a Transcendence kind of way).

Constantia's physical presence

Although Constantia's been doing a phenomenal job, until recently she was stuck running Ubuntu 12.04 LTS Precise Pangolin. Since there's no fancy graphics card and 12.04 was the last version of Ubuntu to support Unity 2D, I've been reticent to upgrade. A previous upgrade from 8.04 to 10.04 many years ago - when Constantia inhabited a different body - caused me a lot of display trouble, so she has form in this regard.

Precise is due to fall out of support next year, and I'd already started having to bolt on a myriad PPAs to keep all the services pumped up to the latest versions. So during my summer break I decided to allocate some time to performing the upgrade, giving me the scope to fix any problems that might arise in the process.

This journey, which started on 19th September, finished today, a full two weeks later.

As expected, the biggest issue was Unity, although the surprise was that it ran at all. Unity has a software graphics-rendering fallback using LLVMPipe, which was actually bearable to use, at least for the small amount of configuration needed to get a replacement desktop environment up and running. After some research comparing XFCE, LXDE and Gnome classic (the official fallback option) I decided to go for XFCE: lightweight but also mature and likely to be supported for the foreseeable future. Having been running it for a couple of weeks, I'm impressed by how polished it is, although it's not quite up there with Unity in terms of tight integration.

The XFCE desktop running on Constantia with beautiful NASA background

There were also problems with some of the cloud services I have installed. MediaWiki has evaporated, but I was hardly using it anyway. The PPA-cruft needed to support ownCloud, which I use a lot, has been building up all over the place. Happily these have now been stripped back to the standard repos, which makes me feel much more comfortable. Gitolite, bind, SVN and the rest all transferred with only minor incident.

The biggest and most exciting change is that I've switched my server backups from USB external storage to Amazon AWS S3 (client-side encrypted, of course). Following a couple of excellent tutorials on configuring deja-dup to use S3 by Juan Domenech, and on S3 IAM permissions by Max Goodman, got things up and running.

But even with these great tutorials, it was a bit of a nail-bighting experience. My first attempt to back things up took five days continuous uploading to reach less than 50% before I decided to reconfigure. I've now got it down to a full backup in four days. By the end of it, I feared I might have to re-mortgage to pay the Amazon fees.

So, how much does it cost to upload and store 46 GiB? As it turns out, not so much: $1.06. I'm willing to pay that each month for effective off-site backup.

The upgrade of Constantia also triggered some other life-refactoring, including the moving of my software from SourceForge to GitLab, but that's a story for another time.

After all this, the good news is that Constantia is now fully operational an up-to-date with Ubuntu 16.04 LTS Xenial Xerus. This should get her all the way through to 2021. Kudos to the folks at Aleutia for creating a machine up to the task, and to Ubuntu for the unexpectedly smooth upgrade process.

The bad news is that nextCloud is now waxing as ownCloud wanes. It doesn't seem to yet be the right time to switch, but that time is approaching rapidly. At that point, I'll need another holiday.
 
Comment
18 Jul 2016 : Using a CDN for regrexitmap.eu #
This weekend I've been playing around with Amazon's CloudFront CDN. I've been setting up a new site, www.regrexitmap.eu, and although I'm not expecting it to be heavily used, the site is bandwidth-heavy and entirely static on the server-side, so a good candidate for deployment via CDN. For those unfamiliar with the term, CDN stands for Content Delivery Network, able to push the content of a website out to multiple servers across the world. This moves the content closer to the end-users, in theory reducing latency and making the site feel more responsive.

There are other benefits of using a CDN. Because the site is served from multiple locations it also makes it less susceptible to denial of service attacks. Since I work in security, there's been a lot of discussion in my research group about DoS attacks and I recently saw a fascinating talk by Virgil Gligor on the subject (the paper's not yet out, but Ross Anderson has written up a convenient summary).

The availability that DoS attempts to undermine offers a wholly different dynamic from the confidentiality, integrity and authenticity that I'm more familiar with. These four together make up the CIAA 'triad' (traditionally just CIA, but authenticity is often added as another important facet of information security). Tackling DoS feels much more practical than the often cryptographic approaches used in the other three areas. An attacker can scale up their denial of service by sending from multiple sources (for example using a botnet), while a CDN redresses the balance by serving from multiple sources, so there's an elegant symmetry to it.

In addition to all of that, CloudFront looks to be pretty cheap, at least compared to spinning up an EC2 instance to serve the site. That makes it both educational and practical. What's not to like?

Amazon makes it exceptionally easy to serve a static site from an S3 bucket. Simply create a new bucket, upload the files using the Web interface and select the option to serve it as a site.

S3 bucket

The only catch is that you also have to apply a suitable policy to the bucket to make it public. Why Amazon doesn't provide a simpler way of doing this is beyond me, but there are plenty of how-tos on the Web to plug the gap.

S3 bucket policy

Driving a website from S3 offers serviceable, but not great, performance. A lot of sites do this, and already in May 2013 netcraft identified 24.7 thousand hostnames running an entire site served directly from S3 (with many more serving part of the site from S3). It's surely much higher now.

Once a site's been set up on S3, hosting it via CloudFront is preposterously straightforward. Create a new distribution, set the origin to the S3 bucket and use the new cloudfront.net address.

S3 distribution origin settings

The default CloudFront domains aren't exactly user-friendly. This is fine if they're only used to serve static content in the background (such as the images for a site, just as the retail Amazon site does), but an end-user-facing URL needs a bit more finesse. Happily it's straightforward to set up a CNAME to alias the cloudfront subdomain. Doing this ensures Amazon can continue to manage the DNS entry it points to, including which location to serve the content from. So I spent £2.39 on the regrexitmap.eu domain and am now fully finessed.

Finally I have three different domain names all pointing to the same content.
The process, which is in theory very straightforward, was in practice somewhat glitchy. The bucket policy I've already mentioned above. The part that caused me most frustration was in getting the domain name to work. Initially the S3 bucket served redirects to the content (why? Not sure). This was picked up by CloudFront, which happily continued to serve the redirects even after I'd changed the content. The result was that visiting the CloudFront URL (or the rerexitmap.eu domain) redirected to S3, changing the URL in the process, even though the correct content was served. It took several frustrating hours before I realised I had to invalidate the material through the CloudFront Web interface before all of the edge servers would be updated. Things now seem to update immediately without the need for human intervention; it's not entirely clear what changed, but it certainly hindered progress before I realised.

The whole episode took about a day's work and next time it should be considerably shorter. The cost of running via CloudFront and S3 is a good deal less than the cost of running even the most meagre EC2 instance. Whether it gives better performance is questionable.

Comparing basic S3 access with the equivalent CloudFronted access gives a 25% speed-up when accessed from the UK. However, to put this in context, serving the same material from my basic fasthosts web server results in a further 10% speed-up on top of the CloudFront increase.

Loading times for S3
Loading times accessing the site on S3 (2.54s total).

Loading times for CloudFront
Loading times accessing the site via CloudFront (1.92s total).

Loading times for fasthosts
Loading times accessing the site on fasthosts.co.uk (1.75s total).

If I'm honest, I was expecting CloudFront to be faster. On the other hand this is checking only from the UK where my fasthosts server is based. The results across the world are somewhat more complex, as you can see for yourself from the table below.

Ping times for the three access methods from across the world (all times in ms, from dotcom).
Location S3 CloudFront Fasthosts
Amsterdam, Netherlands 16 9 12
London, UK 13 19 9
Paris, France 14 10 26
Frankfurt, Germany 26 7 23
Copenhagen, Denmark 32 18 33
Warsaw, Poland 48 26 38
Tel-Aviv, Israel 79 58 72
VA, USA 88 93 86
NY, USA 76 105 87
Amazon-US, East 80 99 100
Montreal, Canada 92 100 92
MN, USA 106 114 106
FL, USA 107 118 109
TX, USA 114 117 138
CO, USA 129 118 138
Mumbai, India 142 124 130
WA, USA 135 144 136
CA, USA 141 149 137
South Africa 157 165 155
CA, USA (IPv6) 278 149 230
Tokyo, Japan 243 229 231
Buenos Aires, Argentina 263 260 224
Beijing, China 260 253 249
Hong Kong, China 283 293 287
Sydney, AU 298 351 334
Brisbane, AU 313 343 331
Shanghai, China 332 419 369

We can render this data as a graph to try to make it more comprehensible. It helps a bit, but not much. In the graph, a steeper line is better, so CloudFront does well at the start and mid-table, but also has the site with the longest ping time overall. The lines jostle for the top spot, from which it's reasonable to conclude they're all giving pretty similar performance in the aggregate.

Pings times cummulative over location

In conclusion, apart from the unexpected redirects, setting up CloudFront was really straightforward and the result is a pretty decent and cheap website serving platform. While I'm not in a position to compare with other CDN services, I'd certainly use CloudFront again even without the added incentive of wanting to know more about it.

I'm now looking in to adding an SSL cert to the site. Again Amazon have made it really straightforward to do, but the trickiest part is figuring out the cost implications. The site doesn't accept any user data and SSL would only benefit the integrity of the site (which, for this site, is of arguable benefit), so I'd only be doing it for the experience. If I do, I'll post up my experiences here.
Comment