hard drive - Place entire folder tree into DDR's cache?
2013-08
IDLacrosseplayer
I was playing Starcraft 2 on my machine with 32GB RAM. I was thinking on how the game only ever maxes out at ~2GB, but it still has to load the different single player maps.
I see the memory structure as "the base game framework + current level goes to working memory" which would include all the music, sprites, etc. When I go to load another level, I get several cache hits from the framework, then several misses as the application goes to fetch the new level from my HDD.
My question is if anyone knows if there is a way to cache the entire game folder, as it only is 13GB, thereby making any and all accesses to any game-related resource a RAM-Cache hit and thereby reducing load times?
I would think if it were all in RAM (PC3-12800), the theoretical bandwidth is ~12GByte/sec, which would make all loading near instantaneous, no?
G Koe
Why don't you try installing it in a ram drive, or moving the files there? Short of reverse engineering the program, a ram drive is probably the best way to go.
Here's more superuser information on ramdisks: Is there any open-source RAMDisk software for Windows?
mvp
Your operating system is already doing exactly what you wanted - it actively caches in RAM all data that was read from your hard drive in most recently used fashion.
For any OS that worth its salt, really "free" memory should be really small amount - just few megabytes. All remaining memory is devoted to caching disk data. If some program requests big memory block using malloc or GlobalAlloc, non-important data cached for read is thrown away to satisfy this memory allocation request.
One interesting property about this caching is that even if you have 32-bit application which cannot allocate more than 4GB of RAM, files accessed by this application will be still cached by your 64-bit OS, even if total size of all files is well beyond 4GB.
In other words, if you have 32GB RAM, and you start playing your game which weighs in at 13GB, Windows will try to actively cache all 13GB in RAM. However, it does not mean that it will stay in that cache forever, because most likely data is highly compressed, and as Starcraft needs to unpack this data in memory to be able to manipulate it and create your gaming world representation, it might try to allocate a lot of memory from Windows, which may force it to throw away some of that cached content (but even then, in least recently used manner).
On a practical note, probably easiest way to speed up load times is to use SSD drive. Modern SSD drives typically have read speed around 500MB/s - around 5x faster than typical hard drive. What's better, SSD read speed does not take a huge nose dive on reading really small files (for standard hard drive, read speed could be 100x slower for small files).
htalvitie
I want to tell the operating system to stop using my hard disk and only use RAM as a cache instead.
Optimally, I want to tell Windows to:
a) pre-cache executable images and necessary bootstrap data into RAM, and b) buffer all disk writes to RAM.
Is this possible?
Background:
My laptop hard disk is loud and I want to put it to sleep when I know I don't need it (e.g., "hdparm -Y").
A typical use case would be booting Win 7 only to make a Remote Desktop Connection to a remote system, thus using the laptop only as a dumb terminal. There's no need to have the HD powered on, since everything's happening in memory, LAN etc.
I have tried basically everything to shut down disk activity when the system is idle: killed nearly all processes, used Process Monitor in advanced mode to try to find which process is responsible for performing disk activity, but not succeeded.
The closest I was able to reach my goal was with XP, when I installed a RAM overlay driver made by Microsoft and designed for Windows Embedded testing. That actually worked: all disk operations were intercepted and this prevented physical HD activity.
Unfortunately, the WinCE RAM overlay driver was very hard to use and I managed to corrupt the disk at the end because the order in which I was supposed to unload and flush things was very particular. :-(
I have now installed a RAM disk (RamDisk Plus) on my Win7 box and tried to install portable apps on it, but I don't think that even installing a user profile on the RAM disk, putting all temp files there etc. would be enough: There's some disk activity originating from the Windows kernel level, that ultimately wants to write to a physical disk thus waking up the hard drive.
There are several other use cases for this need: keeping the system cool, running the laptop with less power, prolonging the lifespan of both physical and solide-state drives, etc.
Suggestions?
quack quixote
I don't believe you'll be able to accomplish this. Windows is designed around the concept that the system partition is always available, and your experiment with the RAM overlay shows it's not designed for what you have in mind.
Instead, change the system partition. Use a non-hard-drive boot partition -- a LiveCD or USB key. You can certainly disable Windows access to non-system partitions, so moving the system partition off of hard drives will let you power down any attached hard drives.
Update:
After reviewing the materials on EWF, yes, this is possible.
I still think this is a case of not-what-it's-designed-to-do, but...
The EWF you used in WinXP came from Embedded XP. The Win-7 analogue is Windows Embedded Standard 2009, and does include the Enhanced Write Filter. I don't know if you'll be able to shoehorn EWF into Win-7, but you'll have better luck with the modern version than the old Embedded XP version.
It looks like you want "EWF RAM Reg mode", which places your EWF configuration data in the system Registry, and doesn't require a separate partition on the drive. The other modes (EWF Disk and EWF RAM modes) at minimum require a separate partition to operate.
You will want to review the techdoc on EWF Performance Considerations, particularly disabling last-access timestamps on NTFS, disabling System Restore, and changing the size of the pagefile (although it seems preferable to disable it, if possible).
Consider designing your own run-time image as your OS base. By doing so you can eliminate components of Win-7 that may be problematic in an embedded, static configuration (which is essentially what you're describing). You may also want to use a checked build, if available, at least until you get things working as expected.
Even this is, at best, educated guesswork. This method seems to be what a LiveCD uses to accomplish its magic, so you might have more luck looking into those. Even if a LiveCD is not your end-goal, the methods they use to build a system image may assist your endeavors. Best of luck!
progrmr
If money is no object, you could replace the noisy HDD with an SSD. I have a MacBook with SSD and it is totally silent (and faster).
Simon
Re your loud hard disk... have you tried turning on AAM (Automatic Acoustic Management) on the hard drive?
http://en.wikipedia.org/wiki/Automatic%5Facoustic%5Fmanagement