08 October 2008

Backing stuff up on Amazon's S3


Amazon Simple Storage Service can in one word be described as great, but that's still an understatement. You pay for what you use and it's still cheap.
I needed a good place to backup stuff from my new servers. Finding a good backup solution can be:
- hard
- costly
- time consuming
- frustrating

When considering the cost of a decent backup hard drive like 300GB WD VelociRaptor (282,15 Euros = 384,767955 U.S. dollars at the moment), power consumption, noise, unpredictable home electricity shortages and the fact that you have to buy at least four of those to be an effective backup solution, you realize that S3 is much much cheaper. My current backup costs at S3 are less than $13 a month. Now that's cheap.

That's where S3 shines. You register an account within minutes and you're ready to go. All you have to do now is to get some software (windows,mac,Linux) or write a simple Perl/Ruby script that automates your backups(Linux/Unix servers). There are also a lot of good existing scripts out there so you only need to modify them to suit your needs.

I've decided for a mixed method. Since there are a lot of files on the servers, I would pay more then necessary because of the $0.01 per 1,000 PUT, POST, or LIST requests. My servers use cPanel control panel and it has a good method of backing stuff up on the server. Every user account is packed and compressed in one file, so at the end of the day I don't have millions of files (PUT, POST, or LIST queries) but only let's say 100 - and that saves valuable money. cPanel makes backups and sends them over to my home server. Backups go into a folder which is configured to be shared with SAMBA throughout the windows network. What I did next was that I used Jungle Disk - a windows S3 client to automate my backups to S3.

The reason why I went through all these jumps are next:

- I saved my servers from bandwidth outside my country/continent
- Servers don't have to encrypt anything, so that's saved server CPU time
- My customers don't feel a thing, since the backups are done fast
- Everything I send to S3 from my home computers is encrypted

To be on the safe side I also save backups on my home servers throughout multiple hard drives, you never know...

2 comments:

Unknown said...

interesting post - have a look at smestorage.com and the free windows file explorer also - you might want to use this combination with Jungle

Salko said...

thanks Jim, I'll give it a try