Topic awaiting preservation: Coding nightmare - Shell scripts deleted randomly

'lo all.

This is the most bizarre thing I have ever seen.
I have an application comprised of 40'000 lines of code.

Of which, 19 files perform "rm" operations on specific directories or subdirectories.
So far, so good.

If the globals used by the script are not defined, all rm operations will still try to find a specific file or directory - there is nothing as a rm $path* command,
it's all rm $path"/explicitsubpath"* at worst.

0 rmdir commands inside the code.
There are a bunch of "mv" commands that, again, act on specific subpaths or even files.

..Now, every once in a while, some of the scripts, in two of the application folders, disappear.
That's right, they just DISAPPEAR.

So to assist with the analysis, I have requested a comprehensive log of file deletions occuring on those folder.
Anybody has any idea what could be going on? It's the weirdest thing I have ever seen, simply put.

even money on $path being overwritten somewhere, or some rarely used
code path being rm %path"/explicitsubpath/ "*.

I'd replace every rm by a 'log & rm' combination
(clone line, echo rm ... &>/tmp/rmlog),
then dig through that.

Option c) would of course be to remove the write permission on said scrips^^

Thanks, but I've not been specific enough I think.

1) Permissions on the scripts are 744 : only the owner can delete them.
2) All rm lines are "log & rm" already, and nothing in the logs says anything about... er, anything
3) Explicitsubpath is never one of the paths containing script : say those are named "/scriptpath" instead. Explicitsubpath is NEVER the name of one of the
folders containing actual script - and I am here with the full listing of rm commands inside the code.

The applications works against a monstruous database : I have access to all the shell script and outer sql stored procs, BUT I have no access to
the internal stored procs (I have requested it).

Stored procs alone cannot do the nasty job of deleting anything, but they could call some third party snippet which tries to cleanup stuff.

...It is driving me nuts : anything you see outside of the scope of the program that I might have missed?

option c) would involve removing write permission from the owner... and seeing what breaks


well, I'd take the fine comb and look for anything calling libc's unlink function - since those database
calls (hopefully) aren't done in 'shell script'... what language are your shell scripts anyhow?


option d) hard link removed files, try to restore them after each and every run.
Then prey no one ever asks you why you're doing htat...

optien e) - buy new harddrive, and run the whole app under strace ./run_my_app >/mnt/new_harddisk/log_file_that_will_be_big
Then use the big shovel to find out what's going on.

We decided to go for c with the sysadmin in charge of the server.

No unlink in the scripts (it's plain vanilla sh).
The database system is Sybase, and it's a several GB - if not TB - huge database.

Sybase simply provides a little query tool - but the inner stored procedures, those that are inside the database, were never handed to me in first place.

Option e : could do, but due to the security restrictions that apply to a company of av. 600'000 employees, by the time I can install this,
we will all be on quantum computers

Hopefully, option c and checking the root crons, and the inner stored procedures, should be enough.

...You know, in terms of "process management", if the issue is outside of the code that was handed and documented to me, I bear 0 responsibility,
but I *hate* having done a fine job to see surprises of this kind. Oh well...