hard drive - Why is my /dev/random so slow when using dd?

06
2014-04
  • Mikey

    I am trying to semi-securely erase a bunch of hard drives. The following is working at 20-50Mb/s

    dd if=/dev/zero of=/dev/sda
    

    But

    dd if=/dev/random of=/dev/sda 
    

    seems not to work. Also when I type

    dd if=/dev/random of=stdout
    

    It only gives me a few bytes regardless of what I pass it for bs= and count=

    Am I using /dev/random wrong? What other info should I look for to move this troubleshooting forward? Is there some other way to do this with a script or something like

    makeMyLifeEasy | dd if=stdin of=/dev/sda
    

    Or something like that...

  • Answers
  • Keith Thompson

    Both /dev/random and /dev/urandom use an "entropy pool". When the pool runs out, /dev/random waits for it to refill, which requires monitoring system behavior (keyboard input, mouse movement, etc.), whereas /dev/urandom will continue to give you pseudo-random data. /dev/random is theoretically higher quality, but /dev/urandom is almost certainly good enough for your purposes. (But even /dev/urandom is likely be slower than some other methods. A faster, but lower quality, generator is probably good enough for erasing hard drives. It's not clear that an attacker would gain any advantage from knowing the sequence that's going to be generated, or that random numbers are better for this purpose than a sequence like 0, 1, 2, 3, 4, ....)

    Quoting the random(4) man page:

    If you are unsure about whether you should use /dev/random or /dev/urandom, then probably you want to use the latter. As a general rule, /dev/urandom should be used for everything except long-lived GPG/SSL/SSH keys.

    EDIT : The following is a digression on "true" random numbers vs. pseudo-random numbers. If all you're interested in is a practical answer to the question, you can stop reading now.

    I've seem claims (including in other answers here) that /dev/random implements a "true" random number generator, as opposed to a pseudo-random number generator (PRNG). For example, the Wikipedia article makes such a claim. I don't believe that's correct. There's some discussion of it here which refers to hardware random number generators, but I see no evidence that /dev/random typically uses such a device, or that typical computers even have such a device. They differ from PRNGs like the C rand() function in that they're not deterministic, since they harvest entropy from sources that are practically unpredictable.

    I'd say there are three classes of "random" number generators:

    1. Deterministic PRNGs, like C's rand() function, which use an algorithm to generate repeatable sequences that have (more or less) the statistical properties of a truly random sequence. These can be good enough for games (given a good way of seeding them), but not for cryptography.

    2. Generators like /dev/random and /dev/urandom that harvest entropy from some practically unpredictable source like I/O activity (this is why pounding on the keyboard or moving the mouse can cause /dev/random to produce more data). It's not clear (to me) whether these satisfy the definition of a PRNG (I've seen definitions that say a PRNG is deterministic), but neither are they true random number generators.

    3. Hardware random number generators that are physically unpredictable even with complete knowledge of their initial state, and that additionally use mathematical techniques to ensure the right statistical properties.

  • Sathya

    /dev/random is a source of true entropy, truly random bytes. As such, it needs a source of randomness. You can 'use up' the randomness by reading from it. It will give you all the randomness that it has, then block until it gets more. You're probably just sitting there waiting, and the machine gets very little new randomness, and it just waits.

    /dev/random for truly random crypto, high quality randomness. As such it's is overkill for a disk drive overwrite. Writing from /dev/zero a few times is fine. Or you can write from /dev/urandom, which won't block and gives pseudo-random numbers when it runs out of true randomness.

  • Sachin Divekar

    In Linux /dev/random is a special file which serves high quality pseudo random numbers. This implementation collects entropy from events originating from the keyboard, mouse, disk and system interrupts.(refer this document) So when there are no such events, the entropy pool is empty, reads from /dev/random will block until additional environmental noise is gathered. This explains your problem. This explains your problem. To fill the entropy pool you can press keys on keyboard.

    On the other note a truly random number generator uses Hardware random number generator which generates random numbers from physical processes.These processes include microscopic phenomena that generate a low-level, statistically random "noise" signal, such as thermal noise or the photoelectric effect or other physical phenomena. These processes are, in theory, completely unpredictable, and the theory's assertions of unpredictability are subject to experimental test.

    A hardware random number generator typically consists of a transducer to convert some aspect of the physical phenomena to an electrical signal, an amplifier and other electronic circuitry to increase the amplitude of the random fluctuations to a macroscopic level, and some type of analog to digital converter to convert the output into a digital number, often a simple binary digit 0 or 1. By repeatedly sampling the randomly varying signal, a series of random numbers is obtained.

    The Hardware Random Number Generator gathers environmental noise from device drivers and other sources into an entropy pool. From this entropy pool random numbers are created. When read, the /dev/random device will only return random bytes within the estimated number of bits of noise in the entropy pool. This explains your problem.

    Some implementations of Hardware RNG are explained in kernel doc and information on a device.

    A counterpart to /dev/random is /dev/urandom ("unlocked"/non-blocking random source) which reuses the internal pool to produce more pseudo-random bits. This means that the call will not block, but the output may contain less entropy than the corresponding read from /dev/random.

    So if your intent is not to generate CSPRNG(Cryptographically secure pseudorandom number generator), you should use /dev/urandom.

  • Randy Orrison

    Without answering your question -- there are complete answers here already -- you could also check out Darik's Boot and Nuke a.k.a. DBAN which is a live-CD drive wiper.

  • pixelbeat

    Just use the shred command that comes with coreutils. It uses random data in an efficient way. dd is a low level too that's probably a bit too low level for this task. shred will efficiently skip unwritable portions of the device for example.


  • Related Question

    linux - Harddisk formatting with dd return error
  • usajbalt

    I tried to format my harddisk (160 GB) with the following command

    dd   if=/dev/zero   of=/dev/sda
    

    After some 3 hours, following error came up:

    dd: writing to '/dev/sda' : No space left on device 
    312581809+0 records in 
    312581808+0 records out 
    160041885696 bytes (160 GB) copied, 10708.3 s, 14.9 MB/s
    

    Any idea what went wrong?


  • Related Answers
  • Journeyman Geek

    well, if you wanted to wipe a disk 'shred' is the proper command. You'd then want to create partitions with fdisk, and use some variety of mk__fs (mke2fs etc) or parted to make a file system in the partition.

  • Hugh Allen

    /dev/zero is in principle infinite. /dev/sda is not.