linux - Automate submission and review

08
2014-07
  • FSUresearch

    I need help writing a script in BASH. I must submit a large number of jobs to an hpc. Because of the large number of jobs, I am trying to write a script that can automate the review of past jobs and then submit more. The reviewal of the jobs involves going into each individual output file and looking for a number over 80000 and then backing up and submitting ten more jobs. These ten new jobs must be the next ten numerically after the last file. If the number is less than 80000, I would like the script to delete it and check the file before it and so on. This must be done for a number of files within 144 subdirectories... As you can imagine, it does take a long time to do by hand. All of my attempts have been met with nothing but errors and blank screens. I need help writing the command to apply to more than one file system. Manually the process goes cd ./cat1-A/cat1-A-1 && du. I then cd into the highest numbered restart01 file and then i vi charmm.out. I check this file for a number anywhere near the bottom of the file that is over 80000. If that file has the number, I back up to cat1-A-1 and restart 7 17 1 backfill, if the last file I checked was restart06. If restart06 did not have the number I would back up to cat1-A-1 and go to restart 05 and check that one. I would use dclean 4 6 if those files werent acceptable. I need to do this for a file structure that goes like cat1-A with the subdirectory being cat1-A-1 and the next would be cat1-B/cat1-B-1 for letters A-L. The next twelve would be cat2-A/cat2-A-1 and cat2-B/cat2-B-1 and so on... At this point I have, deleted my previous attempts and I am just trying to start from scratch.

  • Answers
    Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

    Related Question

    ubuntu - Bash: automate backup from FTP to local linux machine
  • user66638

    I am looking for a bash script, which can download all files from FTP and zip or tar it locally with date in the file name. In simple words: I have at home an Ubuntu machine and I want to backup my website (FTP) locally.


  • Related Answers
  • kubanczyk

    Just out of the top of my head:

    #!/bin/sh
    mysite=your.site.com
    rm -rf "/tmp/$mysite"
    wget -m "user/pass@ftp://$mysite" && tar cvf "$mysite.`date +%Y-%m-%d`.tar" /tmp/$mysite