linux - Shell script bash: Moving file iterate based on month
2014-04
I have very little knowledge about shell scripts, but unfortunately I have to write one. I want to ask about bash script iteration moving files, I need to move log files sorted by month which will be executed by cronjob. The plan was to move mtime +30 (1 month before) files into several folders and the cronjob will be executed daily e.g:
BEFORE
/home/Work/LogFiles/20131200012.log
/home/Work/LogFiles/thisLogIsDifferent.log
/home/Work/LogFiles/20120322222.log
/home/Work/LogFiles/20140100011.log
/home/Work/LogFiles/thisLogIsDifferent2.log
AFTER
/home/Work/LogFiles/thisLogIsDifferent.log
/home/Work/LogFiles/thisLogIsDifferent2.log
/home/Work/LogFiles/2013/DEC/20131200012.log
/home/Work/LogFiles/2012/MAR/20120322222.log
/home/Work/LogFiles/2014/JAN/20140100011.log
which I haven't get any clue the methods I had to use. So here's my awful shell script:
BASE_DIR=/home/Work/LogFiles
REPORT_DIR_YEAR=$BASE_DIR/`date +%Y`
REPORT_DIR=$REPORT_DIR_YEAR/`date +%b`
NOW=$(date +"%Y%m")
if ! [ -d $REPORT_DIR_YEAR ]; then
mkdir $REPORT_DIR_YEAR
if ! [ -d $REPORT_DIR ]; then
mkdir $REPORT_DIR
fi
fi
#THIS PART NEED TO BE RE-ARRANGED
#What I expect is not date=NOW; BUT SOME KIND LIKE date %m-1? but I still don't have any ideas about modify date function.
for file in find $BASE_DIR -maxdepth 1 -type f -mtime +30 -name '*$NOW*'
do
month=$(ls -l $file | awk '{ print $6 }')
case "$month" in
"Jan") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Feb") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Mar") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Apr") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"May") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Jun") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Jul") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Aug") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Sep") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Oct") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Nov") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
"Dec") mv $file $REPORT_DIR_YEAR/$month/$file echo "$file moved to $REPORT_DIR/$file";;
*) echo " Do nothing " ;;
esac
done
And yes, the case $month
doesn't work with the previous for loop $file
. Why? I don't know. I just copy from various sources, forums, examples in for loop, and yet it doesn't work.
First of all, it is never a good idea to parse the output of ls
since it can lead to all sorts of problems. A better way to get the age of a file is stat
. For example:
$ ls -l 20120322222.log
-rw-r--r-- 1 terdon terdon 0 Jan 1 2012 20120322222.log
$ stat -c %y 20120322222.log
2012-01-01 00:00:00.000000000 +0100
So, now we know how to get the age of the file,the question is how to convert that to a three letter month name. The easiest is to use date
:
$ date -d "2012-01-01" +"%b"
Jan
Combining the two commands gives:
$ date -d "$(stat -c %y 20120322222.log)" +"%b"
Jan
So,with this in mind, you can write your script as:
#!/usr/bin/env bash
BASE_DIR=/home/Work/LogFiles
## Find those files that are older than a month
find "$BASE_DIR" -maxdepth 1 -mtime +30 -type f -name "20*" |
while IFS= read -r file; do
## Get the file's modification year
year=$(date -d "$(stat -c %y $file)" +%Y)
## Get the file's modification month
month=$(date -d "$(stat -c %y $file)" +%b)
## Create the directories if they don't exist. The -p flag
## makes 'mkdir' create the parent directories as needed so
## you don't need to create $year explicitly.
[[ ! -d "$BASE_DIR/$year/$month" ]] && mkdir -p "$BASE_DIR/$year/$month";
## Move the file
mv "$file" "$BASE_DIR/$year/$month"
done
The script above assumes that you want to get real modification date of the files, not parse the name. If you want to parse the name instead let me know and I'll modify accordingly.
here is my solution:
#!/bin/bash
BASE_DIR="${1}"
if [ -z "${BASE_DIR}" ]; then
BASE_DIR="/home/Work/LogFiles"
fi
if [ ! -d "${BASE_DIR}" ]; then
echo "Error: '${BASE_DIR}' does not exists." >2
exit 1
fi
declare -a MONTH_NAMES
MONTH_NAMES[1]='JAN'
MONTH_NAMES[2]='FEB'
MONTH_NAMES[3]='MAR'
MONTH_NAMES[4]='APR'
MONTH_NAMES[5]='MAY'
MONTH_NAMES[6]='JUN'
MONTH_NAMES[7]='JUL'
MONTH_NAMES[8]='AUG'
MONTH_NAMES[9]='SEP'
MONTH_NAMES[10]='OCT'
MONTH_NAMES[11]='NOV'
MONTH_NAMES[12]='DEC'
find "${BASE_DIR}" -maxdepth 1 -mtime +30 -type f -name '*.log' \
| grep -e '/[0-9]*.log$' \
| while read FILE; do
FILENAME="$(basename "${FILE}")"
FILE_YEAR="$(echo "${FILENAME}" | cut --bytes=1-4)"
FILE_MONTH="$(echo "${FILENAME}" | cut --bytes=5-6)"
FILE_MONTH_NAME="${MONTH_NAMES[${FILE_MONTH}]}"
REPORT_DIR="${BASE_DIR}/${FILE_YEAR}/${FILE_MONTH_NAME}"
test -d "${REPORT_DIR}" || mkdir -p "${REPORT_DIR}"
mv "${FILE}" "${REPORT_DIR}"
echo "'${FILENAME}' moved to '${REPORT_DIR}'"
done
I have a directory setup as follows:
/hosted/partner1/logo.png
/hosted/partner2/logo.png
/hosted/partner3/logo.png
/hosted/partner4/logo.png
/hosted/partner5/logo.png
..etc.
I want to write a script that can COPY those files to a different location, with a different file name, like this:
/partners/partner1.png
/partners/partner2.png
/partners/partner3.png
..etc.
Any ideas? I'm not so great with shell scripting and there are a lot of files that I need to migrate to a single directory...
find /hosted -maxdepth 1 -name "partner*" -type d | while read -r dir
do
cp "${dir}/logo.png" "/partners/$(basename ${dir}).png"
done
Or
find /hosted -maxdepth 1 -name "partner*" -type d | while read -r dir
do
cp "${dir}/logo.png" "/partners/${dir##*/}.png"
done