Topic awaiting preservation: My global search & destroy shell script

Ah....the joys of shell scripting. I needed to replace a few thousand email addresses with ascii characters because our mailto: links are being scraped. So I write myself a handy search and replace script using sed.

I test it on my local box and it works great! I upload it to my intranet webserver and run a few tests....thumbs up all the way around. Then I upload the script to the main production server and run it a few times in a test directory, because you can just never be too sure. Flawless.... my script does exactly what it should because I am such a clever chap.

Well.....
Then I run the script from a high level directory on the production server. It takes a few minutes to crank it's way through the job because there are a lot of files. The end result? I managed to single handedly wipe out the contents of a major production server.

Thank God for tape back-up. It took two hours to restore 106,0000 files, but at least my ass is out of the fire.

What a day.....

you just made up all that about the testing didn't you?! You can tell us, your boss is nowhere to be seen

Happy days man.

I hate when I have one of those days.
Is there any chance we can see the script, or at least the string search and replace part ?

Here is the trouble maker. Actually two scripts-

replace.sh

#!/bin/bash

# This script is a batch-replace tool for changing the value of all 

# occurances of a given string. It is strongly suggested that a back up copy of this directory

# and all subdirectories be made be for the execution of this script, as all changes 

#are final and  are not ' undo-able '.

#The script 'goReplace.sh' needs to be present

# in the same directory for this to run.

#

echo "This script is a batch-replace tool for changing the value of all  occurances of a given string. It is strongly suggested that a back up copy of this directory and all subdirectories be made before the execution of this script, as all changes are final and  are not ' undo-able '."

echo " "

echo "enter string to be replaced:"

read oldS

OLDS=$oldS

export OLDS

echo "enter replacement string"

read newS

NEWS=$newS

export NEWS





     

    find ../ -type d -exec ./goReplace.sh {} \;

and then we havegoReplace.sh

#this file is called by 'replace.sh'

cd $1

for file in *.txt;do

 

  sed s/$OLDS/$NEWS/ $file>fout

  mv fout $file

done



for file in *.html;do

  

  sed s/$OLDS/$NEWS/ $file>fout

  mv fout $file

done



for file in *.htm;do

  

  sed s/$OLDS/$NEWS/ $file>fout

  mv fout $file

done



for file in *.shtml;do

  

  sed s/$OLDS/$NEWS/ $file>fout

  mv fout $file

done



for file in *.php;do

 

  sed s/$OLDS/$NEWS/ $file>fout

  mv fout $file

done



rm \*.php 

rm \*.htm 

rm \*.html 

rm \*.shtml 

rm \*.txt 



clear



echo "All occurrences of $OLDS have been replaced with $NEWS."

echo " "

echo "That was kinda fun....Can we do it again?"

echo " "

My guess is the for loops were eating up too much memory. It works fine on directories and subdirectories with 20 or 30 files, but maybe a few hundred at at time was biting off more than it could chew. Right about now I feel like that pretty much descibes my style of Systems Administration...

I keep thinking of that line from the old movie Airplane- "Looks like I picked the wrong week to quit shooting heroin".

/* Sure, go ahead and code in your fancy IDE. Just remember: it's all fun and games until someone puts an $i out */

(Edited by norm on 09-25-2004 06:56)