windows - Can we store unlimited data in a Disk

07
2014-07
  • Shashank Sharma

    I was wondering that can we store unlimited data in a disk. So I just started researching that how can we do that. Well at first it seems quite inappropriate question but when I come to know about this then it make me to think that can i really put infinite data.


    EXAMPLE

    Here is an example suppose in Windows I just made one new .txt file and rather than storing data in the notepad what if I store my data on Name place means where file name are written.

    Like -

    Suppose i put some word abcdefghijklmnopqrstuvwxyz and i was able to put it 8 times means

    26 * 8 - 1 = 207 words I can put (-1 as Z was not coming)

    It means 1 thing is sure that i can put 207 words which is quite usefull. But if I try to see the properties I get size of file is 0 bytes it means I can put Infinite data with this type. enter image description here

    So my question is, Is my Hypothesis is correct or not because as per normal people they can easily get fool by this explanation but I want to know the real reason behind this if it is wrong.

    Note: This is ubuntu skin for windows xp so my operating system is windows xp professional.

  • Answers
  • Frank Thomas

    No, storing data as filenames will not work. Disks store bits, and there is a finite number of bits per disk. Data must use some of those bits, so there can never ever be infinite capacity on the disk while these two facts remain true (and they aren't changing any time this decade).

    Filenames are stored by your file system separately from your file data, so the file appears to be 0KB, but the filename itself is 207B long at least.

    Names are stored in your filesystem's metadata tables (the specifics depend on your exact filesystem, but the $MFT is the NTFS table). These tables have a fixed size, so they can name only so many files per directory, and per disk. Additionally when a filename is very long, it usually uses more then one slot in the name table.

    In the case you have described, you would be putting all your data in the name table, and would fill it up very quickly. you would be able to store far less data in the name table than you would in files on the disk, that used short names. You idea only looks like it works when dealing with a small number of files.

  • Jan Doggen

    No, there is such a thing as the Shannon entropy, which establishes the limits to possible data compression. There is a certain amount of information in any message that you cannot reduce by any means when coding/compressing/storing the data. So you will always fill up any medium, i.e. you can never store unlimited data anywhere.

    For more information:

    Even if you extend your thinking to any physical space (i.e. not necessarily a hard disk) there is a maximum theoretical data density.


  • Related Question

    windows - Why does the /winsxs folder grow so large, and can it be made smaller?
  • Jeff Atwood

    A Vista virtual machine I use only has a 10 GB virtual hard disk -- and I'm worried about it running out of space over time.

    I used TreeSize to check for the culprits..

    enter image description here

    And the primary culprit is that /winsxs or "Windows Side-By-Side" folder.

    enter image description here

    What is the Windows Side By Side folder? Well, that's explained pretty well here.

    All of the components in the operating system are found in the WinSxS folder – in fact we call this location the component store. Each component has a unique name that includes the version, language, and processor architecture that it was built for. The WinSxS folder is the only location that the component is found on the system, all other instances of the files that you see on the system are “projected” by hard linking from the component store. Let me repeat that last point – there is only one instance (or full data copy) of each version of each file in the OS, and that instance is located in the WinSxS folder. So looked at from that perspective, the WinSxS folder is really the entirety of the whole OS, referred to as a "flat" in down-level operating systems. This also accounts for why you will no longer be prompted for media when running operations such as System File Checker (SFC), or when installing additional features and roles.

    That explains why the folder starts off big, but not why it gets larger over time – the answer to that question is servicing. In previous versions of Windows the atomic unit of servicing was the file, in Windows Vista it’s the component. When we update a particular binary we release a new version of the whole component, and that new version is stored alongside the original one in the component store. The higher version of the component is projected onto the system, but the older version in the store isn’t touched. The reason for that is the third part of why the component store gets so large.

    Lots of other people complaining about the size of the /winsxs folder, too:

    A coworker recently received his every two year work laptop refresh and was in the middle of loading his new laptop with Vista. For some reason, his hard drive C: was partitioned to only 25GB and the rest of the space reserved for a much larger drive D:.

    Anyway, after loading many Windows Updates, he was left with hardly any drive space left on C:, and for some reason, the WinSxS directory had ballooned to over 8GB.

    While I knew what the WinSxS (Windows Side-by-Side) directory was for, I wasn't quite sure why that directory would explode in size so fast with normal operating system updates.

    I don't really care about having all these duplicate files for safety, but I do care about the VM exploding due to an out of disk space error. So.. can /winsxs be safely pared down in any way? It doesn't seem that it can be..

    Administrators should not, for any reason, take it upon themselves to clean out the /winsxs directory - doing so may prevent Windows Update and MSI from functioning properly afterwards. Preventing accidental deletion from the directory is accomplished by putting a strong security descriptor on the directory that inherits to its children.

    If this /winsxs folder gets any larger, I'm going to have to re-image this virtual machine.. which I really don't want to do!


  • Related Answers
  • Peter Mortensen

    There is a nice command that cleans up after a Windows 7 SP1 installation (it saved me around 3 GB):

    DISM /online /cleanup-Image /spsuperseded
    
  • TomA

    If you have Vista and have SP2 installed, you can clean up old system files with the new Service Pack Clean-up Tool.

    1. Click Start>All Programs>Accessories>Command Prompt or click Start>Run and type cmd to open Command Prompt Windows
    2. Execute the command “Compcln.exe”. The path is “c:\Windows\System32 \compcln.exe”.
    3. Users will be prompted a question whether to keep Vista SP2 permanently in the system.
    4. Type “Y” and press enter, the system will start performing the windows components clean.
  • Peter Mortensen

    These duplicates and backups are extremely annoying when having limited disk space. Being in that place, the only way I found that would not jeopardize the OS stability was to generously use the "Compress contents to save disk space" setting. By compressing all those backup directories under the Windows directory, I was able to reduce the directory size from 6.5 GB to 5 GB, with no recognizeable side effects. Using it on some of your stuff outside the Window directory would stretch your 10 GB even more.

  • Sathya

    TL;DR

    winsxs actually contains hard links (which are kind of like 'shortcuts level 2') -- it does not take up as much additional data as you think. (You would not gain as much space as you think if you delete it.)


    winsxs is not a problem

    ...it's a solution! (Well, mostly.)

    A summary, "Marked As Answer by Ronnie Vernon, MVP, Moderator", says here:

    Wow, long discussion about hardlinks and winsxs. Anyway here are some summarized points of what I understood about how all of it works.

    To start by answering the OP's question with the perfect answer posted above by Andre.Ziegler, you don't touch winsxs. Maybe some of the files can be trimmed like the mp3's, avi's, etc but the rest of the system files should be left alone. I too am using Link Shell Extension as posted above by Ronnie Vernon, easy way to see which are hard linked (red arrow) and which are simple files.

    Before getting here I saw in a lot of places people asking for the cleanup tool after installing SP1 for Windows 7 like vsp1cln. There is no dedicated tool, it all got integrated into the Disk Cleanup service. One note that I would make here is that even if the option says it will remove a few hundreds of MB after I cleaned up my drive the service removed around 3GB of excess files, so it varies from system to system. Funny, now I have even more space than I had before installing SP1. Nice one, MS.

    Regarding the hard links, I got amused by the confusion created around them. I'm not a linux geek, and I usually use only soft links... they're enough for me, but people didn't get how the hard ones work and how size is counted.

    A group of selected files, links or not, will always be counted as a sum. This means that if I take a 10MB file, and make 2 hard links, the selection of all three of them will have 30MB and this is the correct size, because unlike the soft links which are related to the file's original Master File Table entry and have 0 bytes, hard links are related to the data itself, have their own entry in MFT but point to the same data address, obviously having as size the size of the actual data, same times, attributes and so on. Why, below.

    How much is left on the drive? Simple: exactly as much as the drive's own properties reports. Don't go selecting all the files, hidden and non hidden on C and making comparisons, or using tools that do that automatically because it won't show the actual HDD usage. Why? Because: 1. that operation makes a list of all the selected files including hard links as separate files, takes each of them and adds the sizes up, the drives properties counts the actual stored data 2. not all the files will be counted, folders protected from traversing and files from reading will not be counted and the System Volume Information usually has very large files protected just in this manner 3. that number is useful only in cases of file transfer, if the files, links, etc need to be copied to DVD or NAS, that's how much they would occupy, and the links will not be preserved since they are NTFS specific features, not available or available in different forms under ISO, UDF, NFS and other file systems, but they will be cloned.

    Make a test... copy a 10MB file, a.exe, from a different drive and notice how the free space drops by the 10MB. Make a hard link (mklink /H b.exe a.exe). Now these two files selected will "use" 20MB in the selection properties, but actually only 10MB verified at the drive's properties, which will have the same free space as it did/does after the file copy. Delete one, either one, the free space will remain the same. Now delete the last one and the space will increase by 10MB. Was there any harm done? NO.

    What is winsxs? A much better dllcache. Where with the dllcache in XP we had copies of files that actually added to the size usage since they were cloned, this actually improves usage but not total protection (against viruses). If a rogue uninstaller thinks it has to remove some runtime libs from System32, it would only delete the links, but not the data, then the links would be recreated while keeping data space usage at lowest possible, better than archiving winsxs in cabs and having the cabs along side the original files. Also winsxs bring serialization, compartmentalizing files with the same name but different version, for the dev's sake, which is a good thing / bad thing, it makes them lazy to code properly but it helps when they don't want to code anymore.

    I did this to help anyone that lands here out of the same reasons I did... excessive "usage of space" by winsxs after SP1. And I hope it was helpful.

  • Ben Lings

    You could use VSP1CLN.EXE to make SP1 permanent, which according to this page will free up superseded-by-SP1 components.

  • magicandre1981

    In Windows 8 you can reduce the size of the WinSxS folder by removing Windows Features:

    DISM.exe /Online /Disable-Feature /Featurename:<name> /Remove
    

    The important thing is the /Remove parameter which is new to Windows 8.

    The most space in WinSxS is used the WindowsUpdates which you install over the time. In Windows 8 you can detect and remove Updates which are replaced by newer updates (like newer cumulative IE Updates) be running this command:

    dism.exe /online /cleanup-image /startcomponentcleanup
    

    or running the disk cleanup wizard and select "Windows Update Cleanup"

    enter image description here

    enter image description here

    enter image description here

    Also make sure you've installed the servicing stack update http://support.microsoft.com/kb/2821895.

    This updates run an in deep cleanup of WinSxS and compresses all DLLs/Exe files, from WinSxS which are not currently in use, into smaller delta file:

    enter image description here

    Look at the different file sizes.

    Windows 8.1 adds new DISM commands. The /startcomponentcleanup gets a new parameter /ResetBase which makes your last Update permanent and removes all older files to save more space.

    You can also run a new command dism /online /cleanup-image /analyzecomponentstore to analyze the WinSxS folder:

    enter image description here

    It tells you now when you should run the cleanup to save space.

  • Mark Henderson

    Good news! There is now an officially sanctioned method, from Microsoft, for Windows 7 SP1, to clean out your WinSXS folder.

    1. Install HotFix 2852386 - you can either download manually, or wait for it to come through your Windows Update
    2. This adds a new option to the Disk Cleanup Wizard (Under Clean Up System Files) that you can enable for "Windows Update Cleanup". Tick this option.
    3. Click OK

    Caveat: You may be unable to un-install certain Windows Updates after running the cleanup of your WinSxS folder.

  • Jeff Atwood

    Looks like there is finally a way to do this in Windows 8.1 service pack 1 and Windows Server 2012 R2.

    Component store analysis tool:

    Dism.exe /Online /Cleanup-image /AnalyzeComponentStore

    At the end of the scan, the user gets a report of the results (actual size of the winsxs folder) like this:

    analyzecomponentstore results

    Component store cleanup:

    Today, Component Store Cleanup must be triggered manually by an end-user, either by running DISM, or by using the Disk Cleanup Wizard. In order to make Component Store Cleanup more useful for the average end-user, it will be added into a maintenance task, automatically saving disk space for end-users. To enable this, a change will be made to allow uninstallation of superseded inbox drivers without requiring a reboot (today, all driver installs/uninstalls done by CBS require a reboot).

    1. Dism.exe /online /Cleanup-Image /StartComponentCleanup

    2. Disk Cleanup tool, Clean up system files button.

      disk cleanup tool

    3. schtasks.exe /Run /TN "\Microsoft\Windows\Servicing\StartComponentCleanup"

  • Jeff Atwood

    There is a program that can reduce the size of Winsxs, use at your own risk

    About the program

    Now this post we written to introduce the first ever Third party application to help you with the problem if WinSXS problem, and it is called WinSXSLite. It is one Batch file written by Christian Bering Boegh and available for download at their Google Code site.

    Before running the WinSXSLite, we personally advise that you take a full system backup of your windows partition by using any of the backup utility. I use CloneZilla for this because it is free and Open Source. Once Backup is taken you can run.

    WinSXSLite is split into 2 phases, the first phase will search the files which are identical in size with the files stored in WinSXS folder and as per program can be removed. This is required because with a lot of Files backed up by MS Windows are actually not changing but stored as backup in WinSXS folder and resulting in huge size of Folder.

    Second Phase will replace all version of each and every file stored in WinSXS folder and will create hard link to highest version of file in program location or other location. So that if any other program look for File it can be found and at the same time There is no actual file but only Directory entries for the file.

    This way we can remove the files after making hard links and Still MS windows will think that it has the file backed up in WinSXS folder. Only disadvantage of this process will be that you can restore back your System files, but that you anyway don’t want if you need space recovered, because for having restoration utility to work you must have back up copy.

    Program Download

  • Jeff Atwood

    You can remove those files safely. If you ever need them again it will just require you to install a few more things.

    Now on to the fix!

    • Open Windows Explorer and navigate to C:\Windows\System32. Look for the file "vsp1cln.exe."
    • Right click this file and select the 'Run As Administrator' option.
    • The Vista Service Pack 1 Cleanup Tool will remove all of the redundant files that it has replaced.

    The amount of disk space you gain will depend on the system, what programs are installed, etc.