nas - Could server's support of RSync speed up backup process?

08
2014-07

LA_

I have Synology 210j NAS, which supports rsync. Currently this feature is disabled and I use the following code to backup data there from my Mac:

mkdir /Volumes/BackupTemp
mount -t smbfs //login:[email protected]/backup /Volumes/BackupTemp

rsync -archive -backup --stats -h --exclude='.metadata/' --exclude='.Trash/' --exclude='*.pyc' /Users/me/Dev/ /Volumes/BackupTemp/MacBook/Work/

cp /Users/me/KeePass_db.kdbx  /Volumes/BackupTemp/MacBook/

There is Enable network backup service option on the NAS (which supports rsync). I don't understand - if I enable this option, will it help me somehow to speed up back up process? What is the difference when this option is enabled and when it is not?

Answers

Sami Kuhmonen

When you use rsync like you showed, all file transfers are going through the file share and the NAS knows nothing about rsync being used. If you enable the rsync protocol, then for example checking for file changes, checksums etc can be run on the NAS and not on the your computer. So instead of reading the file from the NAS and doing checks, only simple "yes, this file has changed" and/or checksums are sent over the wire. So it will be a lot faster, since less data has to be sent over.

You can easily test this by enabling the rsync server option and changing the rsync command line to use rsync protocol and not a file share.

grawity

Yes, it most likely would.

When using SMB (which is really -t cifs for the last decade), rsync works the same way as with local files. If it sees that a large file differs in modification times, it has to read the entire file over SMB to determine which pieces to update.

On the other hand, if it can talk to another rsync on the other device, it can use an efficient protocol to determine what exactly it should update and send the needed blocks of data.

That said, I'm not sure if the option is needed, if it enables the "bare rsync" mode. Running rsync over SSH would be more secure, and it does not need explicit activation, just a special path:

rsync -avz /Users/me/ 192.168.1.2:/backup/MacBook/

Related Answers

David Spillett

In the past I've used bqbackup and found their service to be quick and reliable, there are other similar services out there too such as rsync.net. Services such as these are specifically geared around providing rsync and similar services for backups.

I currently "host" my own backups though, as I have enough resource dotted around such that a server in one location can hold backups for others and vice-versa. If you want to try DIY, you could no doubt find a VPS somewhere that gives the space you need within your budget if you want to manage it yourself. Search places like WHT's VPS offers area for a selection of possible providers then hunt for reviews of them to make sure they aren't fly-by-night operations. Another thing to note when using rsync is that if you are syncing a large directory structure (large being in terms of the number of files, not their over-all size) rsync can consume a fair chunk of memory, so don't get a VPS that has too little RAM (though for most cases 128Mb will be plenty you don't do nothing (or very little) else with the VM, for example you just install the minimum Debian/Stable with rsync+ssh).

Peter Rowell

DIY really isn't that hard. I have Comcast at home and speedtest.net claims I average around 8-12mbps. Once an hour a cron job rsnapshots (rsync w/multi-day recovery capabilities) all of my commercial website customers to a wimpy little Linux box in my garage with a buttload of storage. 95% of the time there are no changes and it takes about 35 seconds total. When there are changes it normally takes longer for rsync to find them than to actually do the download.

Once a day I rsync those changes to another local server. Belt and suspenders, don't you know?

I sleep better knowing I can rebuild a customer's site in a matter of an hour or so.

fencepost

JungleDisk won't do it directly with rsync, but their software does provide the same basic functionality of backing up only changed portions of files. If you've looked at it in the past this used to be called JungleDisk Plus and was an added-cost service beyond the base software license; they now just charge a slightly higher cost per month with no upfront cost. They have a $2/month plan that's backup-only and targeted at basic users (Grandma) and a $3/month plan that includes network mapped drives and other features.

Since you're probably in Europe based on your pricing, it's worth noting that JungleDisk offers 3 options for file storage (all include the first 5GB "free") - Amazon S3 in the US for $0.15/GB/month, Amazon S3 Europe for $0.18/GB/month and Rackspace US for $0.15/GB/Month. Both Amazon options charge for bandwidth, Rackspace does not. If you're very price sensitive, Rackspace US would be the way to go, otherwise Amazon Europe may be faster for you.

With Rackspace storage, you'd be paying around $9.75/month total; with Amazon EU you'd be paying $12-13/month depending on how much data you were moving each week and your initial upload would cost around $5. Subtract around $1.50 for Amazon US.

JungleDisk also runs on Windows/Linux/Mac.

muncherelli

I'm using s3rsync.com to rsync to my s3 bucket. It work excellent. The price is very competitive.

Dado

tolomea

GoGrid supports RSYNC and charges $0.15 per gig per month with no transfer charges.

http://wiki.gogrid.com/wiki/index.php/Cloud_Storage:Cloud_Storage

Home

nas - Could server's support of RSync speed up backup process?