Page 1 of 1

Batch submission on stampede VIA OpenSeesMP

Posted: Tue Aug 30, 2016 11:34 pm
by evgin
I am new to stampede !

I have borrowed this script from one of Frank's presentations !

When I am running this on stampede through batch submission I am getting this error :

/home1/00477/tg457427/bin/OpenSeesMP: error while loading shared libraries: libpetsc.so.3.6: cannot open shared object file: No such file or director

So I guess the main problem is how to load the OpenSeesMP module ? If you please take a look at the script down below you will see how I am doing that !

set pid [getPID]
set np [getNP]
if {$np != 2} exit
model BasicBuilder -ndm 2 -ndf 2
uniaxialMaterial Elastic 1 3000
if {$pid == 0} {
node1 0.0 0.0
node 4 72.0 96.0
fix 1 1 1
element truss 1 1 4 10.0 1
pattern Plain 1 "Linear" {
load 4 100 -50
}
} else {
node 2 144.0 0.0
node 3 168.0 0.0
node 4 72.0 96.0
fix 2 1 1
fix 3 1 1
element truss 2 2 4 5.0 1
element truss 3 3 4 5.0 1
}


#create the recorder
recorder Node -file node4.out.$pid -node 4 -dof 1 2 disp

#create the analysis
constraints Transformation
numberer ParallelPlain
system Mumps
test NormDispIncr 1.0e-6 6 2
algorithm Newton
integrator LoadControl 0.1
analysis Static
#perform the analysis
analyze 10
# print to screen node 4
print node 4

######################

In order to be able to batch submit on stampede

I have a script as follows which is in a separate file named batchjob.txt:

#!/bin/bash
#SBATCH -J IDA # job name
#SBATCH -o IDA.o%j # output and error file name (%j expands to jobID)
#SBATCH -n 32 # total number of mpi tasks requested
#SBATCH -p development # queue (partition) -- normal, development, etc.
#SBATCH -t 01:30:00 # run time (hh:mm:ss) - 1.5 hours
#SBATCH --mail-user=af@tacc.utexas.edu
#SBATCH --mail-type=begin # email me when the job starts
#SBATCH --mail-type=end # email me when the job finishes


ibrun -np 16 /home1/00477/tg457427/bin/OpenSeesMP /home1/04103/rg/truss/truss.tcl

###############

In order to batch summit this on stampede I use this command on the login node :

sbatch batchjob.txt


The problem is that I am not getting any results while the job is submitted properly .

Any comments ?

Thanks
ER.

Re: Batch submission on stampede VIA OpenSeesMP

Posted: Wed Aug 31, 2016 9:56 am
by fmk
you need to include the commnad to load the petsc module in yout batch file

Re: Batch submission on stampede VIA OpenSeesMP

Posted: Wed Aug 31, 2016 11:07 am
by evgin
Thanks Frank !

To those whom would be using OpenSeesMP on stampede , you may use something like this as your batch script to be executed using "sbatch" command on the login node (scratch preferably)

#!/bin/bash
#SBATCH -J TRUSS # job name
#SBATCH -o TRUSS.o%j # output and error file name (%j expands to jobID)
#SBATCH -n 32 # total number of mpi tasks requested
#SBATCH -p development # queue (partition) -- normal, development, etc.
#SBATCH -t 01:30:00 # run time (hh:mm:ss) - 1.5 hours
#SBATCH --mail-user=rg@tacc.utexas.edu
#SBATCH --mail-type=begin # email me when the job starts
#SBATCH --mail-type=end # email me when the job finishes

module load petsc/3.6

ibrun -np 2 /home1/00477/tg457427/bin/OpenSeesMP /scratch/04103/rg/truss/truss.tcl