hard drive - Windows 7 disk properties dialog reports more used space than total size of files on the disk

07
2014-07
  • tigrou

    I have two hard drives (same size, same partitions, same model) that have been synchronized using FreeFileSync. They are used to store data, not the system. After sync, the drives should report same free and used disk space but it is not the case.

    First drive (used for storage) reports 781 GB of used disk space, while second drive (backup) report only 778 GB. To get that information, I right clicked on the disk then choose properties.

    First, I was thinking that synchronising tool was not doing is job (by excluding some files) but using other file synchronisation tools (like SyncToy or DirSyncPro) give me same result (no difference in files reported justify a delta of 2 GB).

    Also: if I go to drive root folder, select everything (all files and folders) and right click then choose properties, total reported files size is 778 GB on both drives. If i look closer at the numbers, there is a small difference of a few megabytes between the two drives however.

    Notes :

    • Computer have been restarted several times.

    • Recycle bin have been emptied on both drives (if I go to $RECYCLE.BIN folder, both are almost empty (only a few KB).

    • Shadow copy is disabled. vssadmin list shadows doesn't report anything.

    • Windows Explorer folder settings have been modified to display system files and hidden files.

    • I have run "Error checking" tool (with "Automatically fix file system" enabled) on both disks.

    Why is there a difference in used and free space between the two drives? How can I solve this?

    • Answers
  • haimg

    Alternate data streams

    Your source files might have NTFS Alternate data streams. If your backup/synchronization software is not aware of them, they won't be copied to the target. There are many tools that can display them, and since Windows Vista you can do dir /r, which displays not only files but their alternate data streams as well.

    Different filesystem cluster size

    Disk space taken up by a file is rounded up, as a file takes a whole number of filesystem clusters. E.g. if your file's size is 3000 bytes, and filesystem cluster size is 4K, then the file will take 4K exactly. If your two filesystems use different cluster size, then amount of disk space wasted in this manner will be different.

    Other rarely-used NTFS features

    NTFS has several other features which may or may not be supported by your backup software:

    • Compression. If compressed files are uncompressed during copying, they will take more space.
    • Sparse files: Special care must be taken when copying these, otherwise they will "explode", e.g. take much more space and not be sparese anymore.
    • Reparse points, hard links, and many other features that can confuse your backup software.

    Conslusion

    Don't rely on filesystem totals. Use some kind of auditing to make sure your files are copied properly. For example, you may try md5deep to calculate hashes and compare files.

  • mprill

    This issue is connected to the backward compatibility of Windows Vista and later. In those systems some folders that had "known" names in Windows XP are mapped to the new locations using NTFS Directory Junction Points.

    An example of such mappnig is: \Documents and Settings\<username>\Application Data being mapped to \Users\<username>\AppData\Roaming.

    It is possible that you have some application data in your %appdata% that is counted twice by Windows' disk space usage tools. Or perhaps there are other Junction Points or file symbolic links that are affecting this measurement.

    To check if the backup drive contains the Junction Points use dir /A:L /S in it's root and compare it to what it returns on the source drive. If they are the same then your problem lies elsewhere.

    If they are not the same then in order to copy your data correctly you have to use a program that understands NTFS Directory Junction Points and can copy them as well, one of such programs is Microsoft Robocopy or plain xcopy.

    Another possibility are files in use, on Windows only the VSS can copy files in use.

    I'd personally use Robocopy with these options:

    • /COPYALL - COPY ALL file info
    • /B - copy files in Backup mode
    • /SEC - copy files with SECurity
    • /MIR - MIRror a directory tree
    • /R:n - number of Retries
    • /W:n - Wait time between retries
    • /LOG - Output log file
    • /NFL - No file logging
    • /NDL - No dir logging

    ROBOCOPY source destination /COPYALL /B /SEC /MIR /R:0 /W:0 /LOG:log.txt /NFL /NDL

    The wait timers are defined so that files in use won't stall your copy process.

    You can also use xcopy with /B which Copies the Symbolic Link itself versus the target of the link.

    But generally performing a copy of an on-line file system, that is in use by currently running OS is a tricky matter if you don't want to use VSS.


    • Related Question

    windows 7 - What is the difference between "used space", "size" and "size on disk"?
  • Tim

    There is only one partition for both system files and personal data, which is C:. The other two partitions are for boot and for recovery.

    Now I have two different reports on the size of C: on my Windows 7:

    1. by clicking the property of C:, it shows 59.4 GB for "used space";
    2. by selecting all contents (I have setup to view all hidden files and directories) under C: and viewing their property, it shows 46.9 GB for "size" and 47.6 GB for "size on disk".

    Here are my questions:

    1. I was wondering why the two reports are not the same, and where the difference comes?
    2. in the second way, why "size" and "size on disk" are different? What do they mean actually? What is the difference between them and "used space" in the first way?

    Thanks in advance!


    • Related Answers
  • Community
    1. There's some files and folders that you don't have access to, even as Administrator. System Volume Information is one such folder.
    2. You can't directly see some items that take up space, like the MFT and USN Journal.
    3. On Vista & 7, the Windows directory is densely hardlinked. Explorer is unable to handle this and will count files with n hardlinks n times.
    4. Explorer doesn't take NTFS Alternate Data Streams into account.
    5. Files locked for exclusive use won't be counted either.

    **No matter what you do, you're not going to be able to select all the files/folders on the drive and make the number match the drive's properties.

    As for "size" vs "size on disk", there's several things that make the two differ. "Slack space", as mentioned by the others are the most common. But files stored with NTFS compression may take up less space on disk, affecting Explorer's display. Sparse files, offline files, and other special attributes can also cause the 2 to differ significantly.

    **You can use a linux based os or a live cd to see all the hidden folders. For Windows 7 one of those folders is called ProgramData and it accounts for the big difference in size reports.

  • Rhys Gibson
    1. Are you sure you selected everything? No hidden files in the root of C:\ that you missed? e.g. your page file or hibernate file?
    2. Size refers to the sum of the total bytes in the file whereas size on disk refers to how much space they are taking up on the disk. File sizes are aligned to multiples of 4K boundaries (it's faster), so a 1 byte file still takes up 4K on the disk.
  • Mikel

    For the totals to come close, you also need to uncheck "Hide protected operating system files (Recommended)".

    Get there from Windows Explorer by pressing Alt+T, then clicking on Folder Options, then the View tab.

    Otherwise, selecting all files is not selecting pagefile.sys, which is typically about 4 GB.

    Size and size on disk are different because of how hard drives and file systems work.

    Every file uses a certain number of blocks. Typically a block would be about 4 KB. So when you save a file that's 1 KB, it actually uses up 4 KB of disk space. A 5 KB file uses 8 KB, etc.

    Update

    Found a thread on MSDN called NTFS Misreports Free Space?.

    It suggests running chkdsk to see a more detailed breakdown. Note that you must run it as Administrator.

  • Joel Coehoorn

    One other item you're missing is that you don't have permissions to see inside folders for others users by default, even as administrator. Even if you're the only real user of the system, there's likely a few other folders in the c:\users folder that belong to system accounts of various persuasions.

  • Kevin

    Are you referring to the difference in the size when you look at disk properties? - For example, my Hard Disk shows my Used Space at 47,575,797,760 bytes (44.3 GB).

    The reason for this is because 1 GB is not actually 1 billion bytes, it's 2^30 bytes. It's all built to support binary, so 1 KB is actually 2^10 bytes, 1 MB is 2^20 bytes, and 1 GB is 2^30 bytes --> which is why 1 GB RAM = 2^10 MB, which is 1024 MB.

    So in my case, 47,575,797,760 bytes = (47,575,797,760)/(2^30) = 44.3084144..., which rounds to 44.3 GB, as indicated.

    It can be confusing because we recognize, by common definition that Giga means 10^9, but for binary systems, Giga means 2^30.

    So, when you buy a "100 GB HD", what you're actually buying is 100 billion bytes, which is recognized by the computer as 93.1 GB, as (100,000,000,000)/(2^30) ~ 93.1 GB.

    There's nothing wrong and it's not because of hidden files or different size clusters.