Unix delete with find: Delete all files that are listed in a file

06
2014-04
  • Jonathan

    In my root directory, I have a couple folders named something like AA, BB, CC etc., each containing files in the format AA1001.txt, BB1002.txt etc. In my root folder, I also have a file all_to_delete which has a bunch of file names separated with newlines, thus looking something like this:

    AA1004.txt

    BB3004.txt

    BB3005.txt ...

    I now want to go through all subdirs in my root directory and delete all files that match the given filename. Until now, I have tried something like:

    while read line; do find . -type f -name $line -exec rm -f {} \;; done Though, this cannot work as already while read line; do find . -type f -name $line; done does not match any file (as the find gives its output as ./AA/AA1001.txt ...)

    Do you guys have a solution for me?

  • Answers
  • terdon

    Given a file with filenames, the easiest thing to do would be to read it line by line, and pass it to find. However, this will result in a separate instance of find for each file name and can become very slow for large lists of files and many files in a directory tree.

    Instead, I would do something like this:

    find . -type f -name "*txt" | grep -wFf to_delete.txt | xargs -I{} rm '{}'
    

    The trick is to give grep your file as a list of patterns to search for (-f). The -F makes sure your filenames are treated as strings and not regular expressions, that way your file names ca contain special characters like * or [ or |. You then pass to xargs and use quoted '{}', otherwise it fails on white space, to delete the files.

    NOTE: This assumes that your file names are all unique, that one name cannot be contained in another. For example, that you don't have files called foo and foobar. If you do, given a pattern foo, this will delete both files. To avoid this use:

    while IFS= read -r line; do find . -name "$line" -delete; done < to_delete.txt 
    

    From man find:

       -delete
              Delete files; true if removal succeeded. 
    
  • terdon

    How literal is your description of your problem?  Are the directory names all two characters?  Will file BB3004.txt always be in directory BB?  If yes, then you don’t need find; just extract the directory name from the first two characters of the file name:

    while read -r line
    do
        dir=$(expr "$line" : '\(..\)')
        echo rm "$dir/$line"
        rm "$dir/$line"
    done < all_to_delete
    
  • Luis Nardella

    There are many different ways to accomplish that, but here's how I'd do it:

    Created a test dir with a few files in it:

    % ls ./teste
    lala  lele  lolo  lulu
    

    Created a file listing the ones I'd like to delete:

    % cat to_delete.txt 
    lele
    lolo
    lulu
    

    Here I loop trough each line of the 'to_delete.txt' file passing the file name to the find command and then finally removing them:

    % while read filename; do find ./teste -name ${filename} -print0 | xargs -0 rm -vf; done < to_delete.txt 
    removed `./teste/lele'
    removed `./teste/lolo'
    removed `./teste/lulu'
    

    Done:

    % ls ./teste                                                                                      
    lala
    

  • Related Question

    command line - List all content of all files in the directory UNIX
  • PaN1C_Showt1Me

    How to list all files-content in a directory?

    something like ls -la | cat.


  • Related Answers
  • Studer

    Use the following command, recursive :

    find /path/to/folder -type f -exec cat {} \;

    Non-recursive version (due to popular pression) :

    find /path/to/folder -maxdepth 1 -type f -exec cat {} \;

  • samueldr

    The following command:

    find ./ -type f -exec cat {} \;
    

    would find only files (-type f) from the current folder (./).

    Studer's answer is good, and excluding directories is a good idea because it is an undefined behavior between unices, read grawity's comment. Here are two known behaviors :

    • cat on Linux will throw an error message when trying to cat a directory cat: ./folder: Is a directory).
    • cat on FreeBSD will dump the raw directory, as stored on-disk.

    If you need more information about the command or something more about it, please reply and I will annotate more/help you.

    Edit: As John T pointed out, this command will go into every sub-directories. If you need only to cat files from the current directory, you would need -maxdepth 1, thus giving:

    find ./ -maxdepth 1 -type f -exec cat {} \;
    

    The -maxdepth n option can also be used to limit it to an n amount of sub-directories, 1 being the current directory, 2 being the current directory and its direct descendants, and so on.