unix - rsync and file permissions - linux

04
2013-09
  • Fred Snertz

    I'm trying to use rsync to copy a set of files from one system to another. I'm running the command as a normal user (not root). On the remote system, the files are owned by apache and when copied they are obviously owned by the local account (fred).

    My problem is that every time I run the rsync command, all files are re-synched even though they haven't changed. I think the issue is that rsync sees the file owners are different and my local user doesn't have the ability to change ownership to apache, but I'm not including the -a or -o options so I thought this would not be checked. If I run the command as root, the files come over owned by apache and do not come a second time if I run the command again. However I can't run this as root for other reasons. Here is the command:

    /usr/bin/rsync --recursive --rsh=/usr/bin/ssh --rsync-path=/usr/bin/rsync --verbose [email protected]:/src/dir/ /local/dir
    
  • Answers
  • chrishiestand

    Here's the answer to your problem:

    -c, --checksum
          This changes the way rsync checks if the files have been changed and are in need of a  transfer.   Without  this  option,
          rsync  uses  a "quick check" that (by default) checks if each file's size and time of last modification match between the
          sender and receiver.  This option changes this to compare a 128-bit checksum for each file  that  has  a  matching  size.
          Generating  the  checksums  means  that both sides will expend a lot of disk I/O reading all the data in the files in the
          transfer (and this is prior to any reading that will be done to transfer changed files), so this  can  slow  things  down
          significantly.
    
          The  sending  side  generates  its checksums while it is doing the file-system scan that builds the list of the available
          files.  The receiver generates its checksums when it is scanning for changed files, and will checksum any file  that  has
          the  same  size  as the corresponding sender's file:  files with either a changed size or a changed checksum are selected
          for transfer.
    
          Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by  checking
          a  whole-file  checksum  that is generated as the file is transferred, but that automatic after-the-transfer verification
          has nothing to do with this option's before-the-transfer "Does this file need to be updated?" check.
    
          For protocol 30 and beyond (first supported in 3.0.0), the checksum used is MD5.  For older protocols, the checksum  used
          is MD4.
    

    So run:

    /usr/bin/rsync -c --recursive --rsh=/usr/bin/ssh --rsync-path=/usr/bin/rsync --verbose [email protected]:/src/dir/ /local/dir
    

    Note there may be a time+disk churn tradeoff by using this option. Personally, I'd probably just sync the file's mtimes too:

    /usr/bin/rsync -t --recursive --rsh=/usr/bin/ssh --rsync-path=/usr/bin/rsync --verbose [email protected]:/src/dir/ /local/dir
    

  • Related Question

    rsync file permissions on windows
  • avguchenko

    I have an rsync service that syncs files from remote machine to a machine that drops them on a network drive.

    I need the copied files to take on the native permissions of the destination folder.

    The sync process runs fine, but after it is finished, I cannot access some of the folders -- Permission Denied.

    I am logged in as domain admin; it won't allow me to modify any permissions on said folders, either. What gives?

    run command:

    rsync.exe  -v -rlt -z --delete "src_path" "dst_path"
    

  • Related Answers
  • avguchenko

    (from http://www.samba.org/ftp/rsync/rsync.html)

    In summary: to give destination files (both old and new) the source permissions, use --perms.

    To give new files the destination-default permissions (while leaving existing files unchanged), make sure that the --perms option is off and use --chmod=ugo=rwX (which ensures that all non-masked bits get enabled).

    If you'd care to make this latter behavior easier to type, you could define a popt alias for it, such as putting this line in the file ~/.popt (the following defines the -Z option, and includes --no-g to use the default group of the destination dir):

        rsync alias -Z --no-p --no-g --chmod=ugo=rwX
    
  • miking

    Cygwin's "posix" security has caused me lots of problems with Windows NTFS file permissions - even using --no-perms with rsync.

    I found that newly-created files/folders don't properly inherit default permissions, but every file/folder ends up with lots of <not inherited> entries in the Windows file/folder Advnanced security tab. (And this problem is not just rsync-related).

    I found this related post and this link both very helpful in how to resolve these problems using the noacl option in cygwin's /etc/fstab file. The downside of this solution is that cygwin loses the ability to set file/folder permissions, but in many cases this is not important.

    (Googling this topic you'll probably find references to setting the CYGWIN=NONTSEC environment variable, but this is for cygwin v1.5 and doesn't work in cygwin v1.7 onwards.)

  • Wernight

    On Windows with DeltaCopy I could make it work with:

    rsync --perms --chmod=a=rw,Da+x ...
    

    It worked even with --recursive