Today is World Backup Day – a friendly reminder to backup and check restores
worldbackupday.comA friendly reminder to everyone. RAID is not backup: http://serverfault.com/questions/2888/why-is-raid-not-a-back...
For your home, consider an offline service such automatically backing up to a NAS as well as an online service such as Crashplan or Backblaze.
Crashplan or backblaze are very user-friendly, but rather expensive, too. As a cheaper alternative, you can use Amazon Glacier. It costs mere pennies a month. ($0.18 in my case, where I would spend over $5.00 a month at crashplan/backblaze). However, AWS will charge you a lot more if you ever need to get your data back in a hurry. As a last resort solution (for instance my house burns down and my off-site external harddisk gets stolen) it's excellent, though.
Crashplan and backblaze are not rather expensive, they're cheap. Five bucks a month and you don't have to do any complicated calculations when you want your data restored.
... and at $5/month, if you're a photographer with many terabytes of data, it is 1/10 the cost of amazon glacier, last time I checked!
Okay, "expensive" might be stretching it a bit... But in my specific case it's a lot cheaper. If you have several TB of data/photos and/or want an impeccable UI, by all means, go for one of these!
With how the new AWS S3/Glacier storage works, you no longer have to do any calculations for restoring from AWS either
I considered the same thing. But when you are backing up several terabytes of photos and home movies collected over many years the cost of storing them on Glacier isn't as cheap as using the unlimited service of Crashplan or Backblaze ($10 a month vs $5-6 month). Furthermore, Crashplan and Backblaze don't charge you to retrieve your data unlike Glacier, which is where it gets very expensive.
Absolutely. After messing around with various combinations of rsync/ssh/unison/s3/glacier/etc for a few years, I bit the bullet and paid for Crashplan. It's cheap - the family plan is ~150 USD/year for up to ten computers, unlimited.
It's a lifesaver. I've installed it on my parents/wifes/siblings computers and the piece of mind it gives me is immense - when I get the inevitable phone calls about the damn computer deleting that important file, I can either tell them how to recover it, or do it myself.
I'd like to use the Crashplan Family option to cover everyone but since it's all shared and there's seemingly no way to silo the data per-computer, that's a total non-starter. Which is a shame because I'd love to give them my cash.
Just get an individual account for that purpose?
Doesn't really solve the problem of backing up multiple machines spread over multiple sites for multiple people though.
Yea. Backblaze is pretty rad. I work there. We just got polo shirts today!
Has anyone created a backup app that interfaces with glacier?
Arq works quite nicely with Glacier: http://www.haystacksoftware.com/blog/2012/11/arq-cloud-backu...
And they have a special discount for Arq 4 today.
https://store.haystacksoftware.com/?product=arq4&c=YAH4KD2AT...
I'd been thinking about buying Arq for the last month or so, thank you for posting this link, it made buying Arq a no brainer.
How come it isn't advertised anywhere on their site or blog? I checked for one this morning!
Just on Mac OS X, alas.
Does anyone have an open implementation of the glacier api? I'd like to add support for it to my backup tool, but I don't want to have to take on the financial risk of a large bill from Amazon if I happen to mis-use the api. So I'd like to try it out on a local server first, to project what the cost would be after a few months of use.
Amazon has official, open source SDKs in most major languages that cover all of the AWS service APIs.
Actually, I meant is there an open project where I can put something on my own server, to provide the same API that glacier uses? That way I can development / test my code against it instead of testing against Amazon directly.
Did you mean
Today is a made-up day to drive traffic to
our highly prized Sponsors and Offers page
That also used to be one Reddit's feel-good
community projects until guy in charge filed
for copyright and grabbed it all to himself.
Check the site. No sponsors/offers this year and we've taken steps to avoid what happened last time - our brand/logo is released under a loose Creative Commons Attribution-NonCommercial-NoDerivs, and the content on the site is pretty much all Creative Commons.
We've got the Attribution/NonCommercial stuff in the logo license to protect against sites implying that we approve of what they're selling. We just want businesses using the logo FOR PROFIT to state that we don't endorse or promote their product. Everything else is free reign!
We definitely never intended to trademark/restrict the use of the term. We just got a bit over-enthusiastic with the legal side of things - now we're trying to put trust in the community instead (which, admittedly, should have happened in the first place).
If none of this makes sense, I'd be happy to explain it further
I think in this case it is more important they remind people of an important issue. Most people do not have any sort of backups.
It looks like the WBD website got rid of the 'deals' page this year.
If you're not a fan of the trademark on World Backup Day™, we have another trademark-free event called Backup Week. :)
https://backupweek.com/the-road-to-backup-week
World Backup Day™ does not endorse and is not associated with Backup Awareness Week
You are exactly the same, except you call it the "Deals $" page.
Please also remember that a backup is only as good as the restore process. If you've never had cause to test that your restore process works as intended then you're still vulnerable.
PS: I admit that while I keep backups, I've not once checked that I can actually restore (but I know I should). I suspect many people are the same.
This is why I make bootable backup copies of my laptop's internal hard drive to external USB drives. Right after the backup is done, I test it by booting from the USB drive to make sure everything worked. (I use Carbon Copy Cloner. I have no affiliation with them other than liking their product.)
I had a conversation the other day with a bunch of wedding photographers and it’s amazing how much money they spend on a yearly basis on backup. They all shoot raw and they end up with something like 50-60GB of data per event. And that’s only for photographers, videographers are on a completely different scale.
Now, their number one problem isn’t storage per se, since cost per TB is constantly dropping, but uploading data to the cloud. It takes them anything from days to a full week to upload data.
So here’s my $64.000 question. Why hasn’t anyone tried to disrupt this industry? How about setting up a cloud backup company that sends you a portable disk where you write your data and then send it back. From my understanding money isn’t an issue, they would gladly pay tens of dollars on a monthly fee if someone could handle all their backup issues for them.
Crash Plan already offers this: http://support.code42.com/CrashPlan/Latest/Backup/Seeded_Bac...
As I believe many larger backup services do nowadays.
At least some large cloud storage providers offer it. For example, AWS has Import/Export[0] and Google has Offline Disk Import for Google Cloud Storage[1].
Oh nice, I didn't know that. Thanks a million :)
Lets talk about something exiting for a change, what is your personal backup policy? Here is mine:
- I backup my online accounts every two weeks. Google has full export including GMail: https://support.google.com/plus/answer/1045788?hl=en . For some websites I use scripts to do website mirroring.
- I make full phone backup every two weeks
- I have incremental daily backups inside computer. So I have daily snapshots of my data for past 5 months.
- I have two USB harddrives, I make full backup 2x a week, every time on different USB hdd.
- online store where I backup every week (slow internet connection)
- and there is a physical off-site backup I do once a month
I have a terrible backup policy, where everybody's computer is backed up every monday morning on a local server using rsnapshot. I'm thinking regularly of find some decent online backup service which is amenable to receiving VM snapshots regularly before getting distracted and forgetting about it.
Do you do anything to check against corruption (e.g. bit flipping)? I've been experimenting with md5deep recently to accomplish this, but it's a little time-consuming. I'd be very interested in hearing about more efficient ways of dealing with this problem.
No, except daily snapshots. If I would find main version corrupted I can always recover old version a few month back. I
I found bit flipping problematic only when transferring larger amount of data over network.
If you are not checking it, how do you know your drives are not flipping bits?
Just use ZFS. It works.
But, as with the old adage "RAID is not backup", don't forget that ZFS is not a backup either.
ZFS has bugs like any other codebase, and metadata corruption will make your whole pool unrecoverable. RAID-Z and block checksums won't save you from that.
I speak from personal experience with ZFS. Backups are important, no matter what filesystem you use.
ZFS is a good answer to the question "Do you do anything to check against corruption?"
If you're doing differential backups on external USB disks (generally of dubious reliability), you definately want some integrity checking. Bitrot happens!
I'm using ZFS on my home server. The backups of my main system are created with the ubuntu desktop utility and written to the server via NFS. This means I have incremental backups and the backups themselves are protected from a hardware failure using raidz. A problem with this approach is the possibility to rm -rf /mnt/backup - that's why I only start and mount the server when I need it. It's not running 24/7.
Using Windows? Try ExactFile
Nothing. I don't have any digital assets I care about.
I find that surprising. Even if you don't have anything unrecoverable, like personal photos and such, just having to reinstall and reconfigure everything from scratch is more than enough reason for me to backup stuff up. Especially since I know I won't remember half of it.
I actually did a wipe and clean install of my computer a few months ago. I couldn't think of a single thing to keep so I didn't and the only thing I've reinstalled since then is Chrome.
What's a backup?
Funny timing. Had a near-panic last night where I almost had to tell my wife I lost the last 2 years worth of photos because backups are stale. Fortunately, they were all found on an old laptop, so disaster averted.
I spent last night looking into off-site backup solutions, and haven't yet found one I would be willing to pay for. They all seem to require strange client software to work. Is there anything out there where I can set up an account and simply have cron rsync everything over every week? Why do I need a Java GUI (Crashplan) or some 3rd party python script (Glacier) to do a task that doesn't change week to week?
Not sure if you consider Tarsnap to be a "strange client"
I can't afford 10 more terabytes of space to create a backup right now. Guess I'll just pick 50gb of non video content to back up.
Also what is the basis for the claim that 33% of people have never backed up anything? It seems like it would be far higher than that, but I'm not really sure what they consider having done a backup. Copying a picture off your camera to your computer might be considered a backup to them.
Not to mention, "29% of all disasters are caused by accidents". What on earth are the rest caused by? I'm sure a few are caused by malice, but I strongly doubt that it's 61%.
I read the icon to mean 29% of data loss is caused by human error, as opposed to hardware failure, etc. How that's measured, I have no idea (the citation is "Safeware, 2001" for what it's worth).
What about the other 10%? =)
My personal backup plan for family photos and non-work stuff:
Keep 2 copies on 2 external HD's. 1 gets backed up twice a year and put into a safe deposit box at the bank. One gets backed up once a month and put stored in filing cabinet in our house. I will also make backups onto DVD occasionally.
In addition to these local backups, I run a daily job to backup to an FTP & dropbox.
Where are the offers? Show us the offers!
Edit: SpiderOak has unlimited storage: https://spideroak.com/blog/20140327085145-spideroak-offers-u...
Here are some offers: https://backupweek.com/deals
Edit for completeness: CrashPlan https://www.code42.com/store/#/?d=GET_BACKUP_WBD2014
I wish that they included options for Linux too.
What Linux backup tools do people use? Currently I'm making do with a bash script that's just a wrapper around rsync, but it'd be nice to have something a little more powerful, especially with the option to back up things to glacier, etc.
I use tarsnap for all my backups, but that's a service as well as a tool.
This is my daily crontab: http://pastie.org/8982530
Same here. Just did a full backup of my VPS:
Deduplication ftw.Total size Compressed size All archives 3.7 GB 1.5 GBThat's just showing the compression (deflate). Deduplication is why "(unique data)" line which you didn't paste will be lower.
CrashPlan. Works well on all major OSes but it does use a lot of memory since it's Java. You can use it to backup to their own online service or you can do peer-to-peer between systems with CP installed.
rsync.net lets you do things like this:
"dd of=db_dump"mysqldump -u mysql db | ssh 1234@usw-s001.rsync.net \and this:
and this:ssh user@rsync.net s3cmd get s3://rsynctest/mscdex.exe
... and has had a "HN readers" discount for new users for many years now.ssh user@rsync.net "git clone git://github.com/freebsd/freebsd.git freebsd"Yeah, would have loved to have something for all platforms... Haven't used Linux in a long time so I'll admit I'm unfamiliar with it all. Did try to do some research but couldn't find a solution that would cover all distros.
Next year, with more time, I'll try and add something there!
> but couldn't find a solution that would cover all distros.
For your purposes, it would be fine to post something that works on Ubuntu as long as you can verify that it's not Ubuntu-only (if it's open source and works on at least two distros, chances are it works on any distro)[0].
[0] Heck, even most things that are made for Ubuntu work on other distros with minimal modification - it's just a matter of convenience.
Updated the site a couple of hours ago with Ubuntu instructions, thanks to our German translator
rsnapshot (http://www.rsnapshot.org/) which is a wrapper around rsync.
Backups go to a local machine with a three drive linux software raid-5 array.
Occasional offsite sync to another remote Linux box.
There are a lot:
http://skeptu.com/online-backup-service+has-client:linux-app...
I'm using spideroak. The Linux client is actually functional.
duplicity to S3 here. I'd like to eventually figure out how to point duplicity to Glacier. And of course it'd be nice if GPG supported some more modern cyphers and modes, but it's Good Enough(tm).
I don't think you can point it directly at Glacier (if I'm wrong someone please let me know) but what you can do is to set "Lifecycle" options on your S3 bucket.
I have a folder in one of my buckets, for example, that I call "archive". I put in there anything I want to store on Glacier long-time. With my lifecycle settings, objects in that folder are automatically moved to Glacier after one day.
backuppc, incremental backups, de-duplication, supports Windows/Linux with smb/ssh/nfs etc
Not sure why but this site makes my GPU and a couple of CPU cores working at a full load. I use Chrome 34.0.x.
Hm, really? No such issues here. Could it the transform-3D SVG animation on page load?
I am not sure. I am on Linux and nvidia binary drivers. I have seen a fair share of problems, especially webgl based pages eating away CPU, or new tabs eating CPU cycles until they become active for the first time.
This is the first time though I believe that I saw a site eating GPU cycles and this is why I mentioned it.
Self-hosted, privacy-friendly alternative: BTSync. Enough said.
What a gorgeous site