linux - Shell script bash: Moving file iterate based on month

06
2014-04
  • Ivan Herlambang

    I have very little knowledge about shell scripts, but unfortunately I have to write one. I want to ask about bash script iteration moving files, I need to move log files sorted by month which will be executed by cronjob. The plan was to move mtime +30 (1 month before) files into several folders and the cronjob will be executed daily e.g:

    BEFORE

    /home/Work/LogFiles/20131200012.log
    /home/Work/LogFiles/thisLogIsDifferent.log
    /home/Work/LogFiles/20120322222.log 
    /home/Work/LogFiles/20140100011.log
    /home/Work/LogFiles/thisLogIsDifferent2.log
    

    AFTER

    /home/Work/LogFiles/thisLogIsDifferent.log
    /home/Work/LogFiles/thisLogIsDifferent2.log
    /home/Work/LogFiles/2013/DEC/20131200012.log
    /home/Work/LogFiles/2012/MAR/20120322222.log 
    /home/Work/LogFiles/2014/JAN/20140100011.log
    

    which I haven't get any clue the methods I had to use. So here's my awful shell script:

    BASE_DIR=/home/Work/LogFiles
    REPORT_DIR_YEAR=$BASE_DIR/`date +%Y`
    REPORT_DIR=$REPORT_DIR_YEAR/`date +%b`
    
    NOW=$(date +"%Y%m")
    
    if ! [ -d $REPORT_DIR_YEAR ]; then
        mkdir $REPORT_DIR_YEAR
    
        if ! [ -d $REPORT_DIR ]; then
            mkdir $REPORT_DIR
        fi
    fi
    
    #THIS PART NEED TO BE RE-ARRANGED
    #What I expect is not date=NOW; BUT SOME KIND LIKE date %m-1? but I still don't have any ideas about modify date function.
    
    for file in find $BASE_DIR -maxdepth 1 -type f -mtime +30 -name '*$NOW*'
    do
    
     month=$(ls -l $file | awk '{ print $6 }')
        case "$month" in
          "Jan") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Feb") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Mar") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Apr") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "May") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Jun") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Jul") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Aug") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Sep") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Oct") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Nov") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
          "Dec") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
              *) echo " Do nothing " ;;
        esac
    
    done
    

    And yes, the case $month doesn't work with the previous for loop $file. Why? I don't know. I just copy from various sources, forums, examples in for loop, and yet it doesn't work.

  • Answers
  • terdon

    First of all, it is never a good idea to parse the output of ls since it can lead to all sorts of problems. A better way to get the age of a file is stat. For example:

    $ ls -l 20120322222.log 
    -rw-r--r-- 1 terdon terdon 0 Jan  1  2012 20120322222.log
    $ stat -c %y 20120322222.log 
    2012-01-01 00:00:00.000000000 +0100
    

    So, now we know how to get the age of the file,the question is how to convert that to a three letter month name. The easiest is to use date:

     $ date -d "2012-01-01" +"%b"
    Jan
    

    Combining the two commands gives:

    $ date -d "$(stat -c %y 20120322222.log)" +"%b"
    Jan
    

    So,with this in mind, you can write your script as:

    #!/usr/bin/env bash
    BASE_DIR=/home/Work/LogFiles
    
    
    ## Find those files that are older than a month
    find "$BASE_DIR" -maxdepth 1 -mtime +30 -type f -name "20*" | 
     while IFS= read -r file; do
        ## Get the file's modification year
        year=$(date -d "$(stat -c %y $file)" +%Y)
        ## Get the file's modification month
        month=$(date -d "$(stat -c %y $file)" +%b)
    
        ## Create the directories if they don't exist. The -p flag
        ## makes 'mkdir' create the parent directories as needed so
        ## you don't need to create $year explicitly.
        [[ ! -d "$BASE_DIR/$year/$month" ]] && mkdir -p "$BASE_DIR/$year/$month"; 
    
        ## Move the file
        mv "$file" "$BASE_DIR/$year/$month"
    done
    

    The script above assumes that you want to get real modification date of the files, not parse the name. If you want to parse the name instead let me know and I'll modify accordingly.

  • Biapy

    here is my solution:

    #!/bin/bash
    
    BASE_DIR="${1}"
    
    if [ -z "${BASE_DIR}" ]; then
      BASE_DIR="/home/Work/LogFiles"
    fi
    
    if [ ! -d "${BASE_DIR}" ]; then
      echo "Error: '${BASE_DIR}' does not exists." >2
      exit 1
    fi
    
    declare -a MONTH_NAMES
    MONTH_NAMES[1]='JAN'
    MONTH_NAMES[2]='FEB'
    MONTH_NAMES[3]='MAR'
    MONTH_NAMES[4]='APR'
    MONTH_NAMES[5]='MAY'
    MONTH_NAMES[6]='JUN'
    MONTH_NAMES[7]='JUL'
    MONTH_NAMES[8]='AUG'
    MONTH_NAMES[9]='SEP'
    MONTH_NAMES[10]='OCT'
    MONTH_NAMES[11]='NOV'
    MONTH_NAMES[12]='DEC'
    
    
    find "${BASE_DIR}" -maxdepth 1 -mtime +30 -type f -name '*.log' \
      | grep -e '/[0-9]*.log$' \
      | while read FILE; do
    
      FILENAME="$(basename "${FILE}")"
    
      FILE_YEAR="$(echo "${FILENAME}" | cut --bytes=1-4)"
      FILE_MONTH="$(echo "${FILENAME}" | cut --bytes=5-6)"
    
      FILE_MONTH_NAME="${MONTH_NAMES[${FILE_MONTH}]}"
    
      REPORT_DIR="${BASE_DIR}/${FILE_YEAR}/${FILE_MONTH_NAME}"
      test -d "${REPORT_DIR}" || mkdir -p "${REPORT_DIR}"
    
      mv "${FILE}" "${REPORT_DIR}"
    
      echo "'${FILENAME}' moved to '${REPORT_DIR}'"
    done
    

  • Related Question

    Bash Shell Scripting - How to iterate through directories, and copy and rename files?
  • Cypher

    I have a directory setup as follows:

    /hosted/partner1/logo.png
    /hosted/partner2/logo.png
    /hosted/partner3/logo.png
    /hosted/partner4/logo.png
    /hosted/partner5/logo.png
    ..etc.
    

    I want to write a script that can COPY those files to a different location, with a different file name, like this:

    /partners/partner1.png
    /partners/partner2.png
    /partners/partner3.png
    ..etc.
    

    Any ideas? I'm not so great with shell scripting and there are a lot of files that I need to migrate to a single directory...


  • Related Answers
  • Dennis Williamson
    find /hosted -maxdepth 1 -name "partner*" -type d | while read -r dir
    do
        cp "${dir}/logo.png" "/partners/$(basename ${dir}).png"
    done
    

    Or

    find /hosted -maxdepth 1 -name "partner*" -type d | while read -r dir
    do
        cp "${dir}/logo.png" "/partners/${dir##*/}.png"
    done