OpenSeesMP Slower than sequential analyses

[arashs]

Hi all,

I'm trying to run a opensees bridge model on local linux cluster server. This bridge model code is developed for parametric studies as the ultimate goal is the reliability analysis, thus the same script will run with new set of parameters each time. I use 3d contact elements in the model I need to figure out the best penalty values both for contact elements and also static analysis. To do so, an initial value is assigned to these parameters ( penalty parameters), script is run, new values is assigned and the analysis will be repeated. Here is sub.sh command to run on the sever:

#!/bin/bash
export PATH=/usr/local/opensees/20150226/trunk/bin:$PATH
export PATH=/usr/local/openmpi/1.6.3/gcc446/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/openmpi/1.6.3/gcc446/lib:$LD_LIBRARY_PATH
mpirun -np $NSLOTS OpenSeesMP MultiRun.tcl -par RunCounter runnumber.txt

where runnumber.txt is the txt file which contains the file name in which the parameter values are saved. and here is my TCL script to repeat the analysis (MulriRun.tcl):

set address ""
append address "Run" $RunCounter "/" "Run" $RunCounter ".txt"
puts $address
set fp [open $address r]
set file_data [read $fp]
close $fp

# Process data file
set data [split $file_data "\n"]

set counter 1

foreach line $data {
set dataline($counter) $line
puts "line $counter"
puts $dataline($counter)
incr counter
puts " "

}

source sequence.tcl

Now here is the problem:

1) Although the code works on the server, it is much slower that the sequential analysis on my PC! As the code shows, I'm not decomposing the domain between cores, rather, assigning each Run with new set of parameter values to a processor. 50 intel Xeon processors is being used.

2) As i compare the first period of the bridge, for some values of penalty, obtained values on the server are different than those obtained on the PC!

Any comments or hints will be appreciated!

[fmk]

1) possibilities:
a) you are not actually showing the tcl script part so i can make sure it is actually running in parallel and not just executing the same code on the different cores ..
b) if it is indeed running in parallel, if you are memory limited or file i/o limited, just running in parallel can degrade the performance. these days it is not the processors crunching the numbers that is the weak link, but writing data to the mechanically spinning disks if you are page faulting heavily or writing data to files.

2, can happen if system poorly conditioned as the 2 executables are not actually the same. The different compilers and compiler flags cause result in different machine level instructions to be created and when you run them you will see different answers between executables for the same scripts. These differences are typically small, but can become large if numerical issues exist.

[arashs]

Thanks for the hints. About the difference between periods on different computers, it only happens for some models with really small penalty values. As for the parallel processing, I couldn't find the problem however I'm manually assigning each run a processor and it works. I wish I could copy the whole script here but it is just too long to copy.