Python: script to make multiple bash scripts

I have a file, called list.txt, which is a list of names of files:


I want to build a python script that will make one file for each of these filenames. More precisely, I need it to print some text, incorporate the name of the file and save it to a unique .sh file. My script is as follows:

import os

with open('list2.txt','r') as f:    
    lines =' ')

for l in lines:
    print "#!/bin/bash\n#BSUB -J",l+".sh","\n#BSUB -o /scratch/DBC/user/"+l+".sh.out\n#BSUB -e /scratch/DBC/user/"+l+".sh.err\n#BSUB -n 1\n#BSUB -q normal\n#BSUB -P DBCDOBZAK\n#BSUB -W 168:00\n"
    print "cd /scratch/DBC/user\n"
    print 'grep "input"',l+" > result."+l+".txt"

    with open('script{}.sh'.format(l), 'w') as output:

I have a few issues:

  • The output files just contain the name of the file - not the content I have printed.

To be clear my output files (I should have 3) should look like this:

#BSUB -o /scratch/DBC/user/
#BSUB -e /scratch/DBC/user/
#BSUB -n 1
#BSUB -q normal
#BSUB -W 168:00

cd /scratch/DBC/user
grep "input" input3 > result.input3.txt

I am very new to Python so I am assuming there could be much simpler ways of printing and inserting my text. Any help would be much appreciated.

UPDATE I have now made the following script, which nearly works.

import os

with open('list.txt','r') as f:
    lines ='\n')

for l in lines:
    header = "#!/bin/bash \n#BSUB -J \n#BSUB -o /scratch/DBC/user/ \n#BSUB -e /scratch/DBC/user/ \n#BSUB -n 1 \n#BSUB -q normal \n#BSUB -P DBCDOBZAK \n#BSUB -W 168:00\n"%(l,l,l)
    script = "cd /scratch/DBC/user\n"
    script2 = 'grep "input" %s > result.%s.txt\n'%(l,l)
    all= "\n".join([header,script,script2])

    with open('script_{}.sh'.format(l), 'w') as output:

The problem I still have is that this creates 4 scripts, not 3 as I expected:,, and script_sh. This last one, script_sh just has the printed text but nothing where the "input" text would be. I think this is because my list.txt file has a "\n" at the end of it? However, I looked and there really isn't. Is there a way around this? Maybe I can use some kind of length function?

1 answer

  • answered 2017-10-11 10:44 FrankBr

    So, answering in order:

    1) Can you detail this issue? You count 4 txt files bu you got just 3 different scripts generated by your code?

    2) Sure, you need to create a var, not just using the print statement 3) Just change permissions

    So, to summurize, I'd use this approach:

    import os
    for i, file in enumerate(os.listdir("/Users/user/Desktop/Folder")):
      if "input" in file:
        with open(file) as f:
            lines = f.readlines()
            for l in lines:
                data = ""
                data += "#!/bin/bash\n#BSUB -J",l+".sh","\n#BSUB -o /scratch/DBC/user/"+l+".sh.out\n#BSUB -e /scratch/DBC/user/"+l+".sh.err\n#BSUB -n 1\n#BSUB -q normal\n#BSUB -P DBCDOBZAK\n#BSUB -W 168:00\n"
                data += "cd /scratch/DBC/user\n"
                data += 'grep "input"'+l+" > result."+l+".txt"
                with open(''%i, 'w') as output:
                os.chmod("'%i", 700)

    By the way, my code it's just a guess. I think you should be more explicit when stating what's your issue. I didn't understood what you wanna achieve.