Daily Archives: December 8, 2008

The only technology trend you need to be aware of.

This link was passed around at work today. It discusses one of the oft-touted aspects of cloud computing which is auto-scaling dynamically where the host platform is “aware” of the stress on the application and can dynamically allocate additional resources (often in the shape of more virtual machines) to handle the load.

As George Reese (the author of the article) quite rightly points out you can remove some of the need for this auto-scaling capability by doing decent capacity planning. A laudable goal and I agree whole heartedly.

Everyone’s dream is to wake up each morning and find their web-sites unavailable because their application crashed because the transaction logs on the database server ran out of disk space writing all those credit card authorisation numbers – but it is just that, a dream. When most sites get “slashdotted” its because someone has posted Photoshop pictures of Paris Hilton being spotted at an open source convention hanging out with Richard Stallman.

Features, Benefits and Trade-offs

What George Reese is pointing out is that for the majority of applications auto-scaling won’t be a lot of use to you. The reason for this is that unless the platform intrinsically knows how your application is stressing, adding a server might not lead to much more than you paying more for your hosting.

For most applications, auto-scaling is just another feature shown as a bullet point by the hosting provider.

On the other hand, there are classes of applications (like public video encoding sites) which could benefit from the additional processing capacity. These applications can really benefit from the combination of EC2 and S3 where a user uploads a video in its native format and it dispatches to the next available server to process the video. When a server is idle for more than a few minutes it simply shuts itself down (maintain a minimum pool unused resources in reserve ready to jump to action immediate).

For these applications auto-scaling is a benefit – and its all about knowing the difference between a feature and a benefit.

In reality you would probably be best served adding end-user features to your application or coming up with a standard server image which you can manually provision so you reach the middle ground between auto-scaling and “oh crap, my site is going to be offline for three days until I can get another server configured”.

I’ve always wanted to write a Fork bomb for EC2 though :)

Technology Trends

Stepping back for a moment, the whole discussion does make me smile just a little bit. The industry is undoubtedly on a bit of a “cloud”-kick at the moment. It is so typical of IT to have bandwagons that every jumps on, the natural clustering behaviour which draws us together behind visionaries and leaders is part of what makes us human.

I don’t have a problem with that, it is in fact a fairly natural reaction to a number of forces that have been in play for quite some time in IT and its fair the say that we are truly changing the way that we provision and pay for computing services (maybe IBM was right?). It is all a bit cyclic though and if I was to design technology trends as a sequential workflow it would look something like this:

  1. Someone invents a technology.
  2. Years pass because everyone is too busy looking at the hot new couple Paris and Richard.
  3. Someone uses technology (see step 1) to do something interesting.
  4. Cult is formed around this interesting use of existing technology (see SOA, REST, and now cloud-computing).
  5. Cult manages to convince industry journalists to write about it (it was easy because Paris and Richard weren’t interesting anymore).
  6. “Enterprises” adopt it but apply their own special spin which either misses the point or renders the approach useless.
  7. A bright spark observes that “some people don’t get it” and blogs about it, starting the wave of negativity.
  8. Early adopters jump off the wave and look to the next big thing hoping that no-one remembers they were a member of the technology cult.
  9. A few years pass.
  10. People still use the technology because it was useful “some of the time”, just not “all of the time”.
  11. Goto 1

Right now I think that cloud-computing is somewhere between 4-6, I’m hoping that this wave combined with a renewed since of fiscal responsibility wipes away some of the “old think” around IT services delivery before the cult is exposed.