Need help on Using OpenSeesSP

[frank_lv]

The day I put on this new topic is 25 December 2010, so first of all MERRY ChRISTMAS all of you !!!


I have some questions about opensees. I download OpenSees source code (we used version 2.1.0 ) and compile the OpenSeesSP successfully on redhat el5 os. MPI we use OpenMPI-1.3.3, and I installed the petSC 2.2.1. This OpenSeesSP can run with example model in TeraGridExamples/Example1.tcl which resides in OpenSees source code directory EXAMPLE. But it can not run with our test model. Serial OpenSees can run with our test model successfully. Below I put some error ouputs:


OpenSees -- Open System For Earthquake Engineering Simulation
Pacific Earthquake Engineering Research Center -- 2.1.0
(c) Copyright 1999,2000 The Regents of the University of California
All Rights Reserved
(Copyright and Disclaimer @ http://www.berkeley.edu/OpenSees/copyright.html)
Slave Process Running 1
Slave Process Running 2
Master Process Running OpenSees Interpreter 0

Number of Stories in Y: 20 Number of bays in X: 10 Number of bays in Z: 3
goGravity

WARNING ArrayOfTaggedObjects::addComponent() - component with tag already exists, not adding component with tag: 0
WARNING: LoadPattern::addElementalLoad() - load could not be added
Domain::addNodalLoad() - no pattern with tag101in the model, not adding the ele loadBeam3dUniformLoad - Reference load
Transverse (y): 0
Transverse (z): 0
Axial (x): -0.0632812
Element : 20102

WARNING ArrayOfTaggedObjects::addComponent() - component with tag already exists, not adding component with tag: 0
WARNING: LoadPattern::addElementalLoad() - load could not be added
Domain::addNodalLoad() - no pattern with tag101in the model, not adding the ele loadBeam3dUniformLoad - Reference load
Transverse (y): 0
Transverse (z): 0
Axial (x): -0.0632812
Element : 20103
.
.
.
( losts of same warnings )
.
.
.
WARNING ArrayOfTaggedObjects::addComponent() - component with tag already exists, not adding component with tag: 0
WARNING: LoadPattern::addElementalLoad() - load could not be added
Domain::addNodalLoad() - no pattern with tag101in the model, not adding the ele loadBeam3dUniformLoad - Reference load
Transverse (y): -0.0375
Transverse (z): 0
Axial (x): 0
Element : 2.21031e+06

WARNING ArrayOfTaggedObjects::addComponent() - component with tag already exists, not adding component with tag: 0
WARNING: LoadPattern::addElementalLoad() - load could not be added
Domain::addNodalLoad() - no pattern with tag101in the model, not adding the ele loadBeam3dUniformLoad - Reference load
Transverse (y): -0.0375
Transverse (z): 0
Axial (x): 0
Element : 2.21031e+06

[localhost:29917] *** Process received signal ***
[localhost:29917] Signal: Segmentation fault (11)
[localhost:29917] Signal code: Address not mapped (1)
[localhost:29917] Failing at address: 0x7e
[localhost:29917] [ 0] [0xe04440]
[localhost:29917] [ 1] /OpenSeesSP(_ZN11LoadPattern9applyLoadEd+0x79) [0x83f1e19]
[localhost:29917] [ 2] /OpenSeesSP(_ZN6Domain9applyLoadEd+0xaf) [0x81a1aff]
[localhost:29917] [ 3] /OpenSeesSP(_ZN17PartitionedDomain9applyLoadEd+0x26) [0x81a9e86]
[localhost:29917] [ 4] /OpenSeesSP(_ZN13AnalysisModel15applyLoadDomainEd+0x2f) [0x821c17f]
[localhost:29917] [ 5] /OpenSeesSP(_ZN11LoadControl7newStepEv+0x91) [0x81f4351]
[localhost:29917] [ 6] /OpenSeesSP(_ZN14StaticAnalysis7analyzeEi+0x53) [0x81e0a03]
[localhost:29917] [ 7] /OpenSeesSP(_Z12analyzeModelPvP10Tcl_InterpiPPKc+0x89) [0x817bcf9]
[localhost:29917] [ 8] /usr/local/tcl8.5/lib/libtcl8.5.so(TclInvokeStringCommand+0x76) [0x67e326]
[localhost:29917] [ 9] /usr/local/tcl8.5/lib/libtcl8.5.so [0x67f558]
[localhost:29917] [10] /usr/local/tcl8.5/lib/libtcl8.5.so [0x680152]
[localhost:29917] [11] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_EvalEx+0x3a) [0x68048a]
[localhost:29917] [12] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_FSEvalFileEx+0x266) [0x6eecf6]
[localhost:29917] [13] /OpenSeesSP(_Z13OPS_SourceCmdPvP10Tcl_InterpiPKP7Tcl_Obj+0x9b) [0x817c3bb]
[localhost:29917] [14] /usr/local/tcl8.5/lib/libtcl8.5.so [0x67f558]
[localhost:29917] [15] /usr/local/tcl8.5/lib/libtcl8.5.so [0x680152]
[localhost:29917] [16] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_EvalEx+0x3a) [0x68048a]
[localhost:29917] [17] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_FSEvalFileEx+0x266) [0x6eecf6]
[localhost:29917] [18] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_FSEvalFile+0x2c) [0x6eee7c]
[localhost:29917] [19] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_EvalFile+0x41) [0x6eeed1]
[localhost:29917] [20] /OpenSeesSP(_Z9g3TclMainiPPcPFiP10Tcl_InterpEii+0x2ac) [0x816c08c]
[localhost:29917] [21] /OpenSeesSP(main+0x3e1) [0x816cc21]
[localhost:29917] [22] /lib/libc.so.6(__libc_start_main+0xdc) [0xe1ae9c]
[localhost:29917] [23] /OpenSeesSP [0x816bd01]
[localhost:29917] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 29917 on node localhost.localdomain exited on signal 11 (Segmentation fault).


I have googled them, but got nothing helpful. Then I checked OpenSees source code, I found those warnings come from slave processes, when they receive ElementalLoads. In this model the ElementalLoads are Beam3dUniformLoad, and in sendSelf() / recvSelf() function, do not send or receive Beam3dUniformLoad Object Tag, Below are some code fragments:


int Beam3dUniformLoad::sendSelf(int commitTag, Channel &theChannel)
{
int dbTag = this->getDbTag();

static Vector vectData(4);
vectData(0) = wx;
vectData(1) = wy;
vectData(2) = wz;
vectData(3) = eleTag;

int result = theChannel.sendVector(dbTag, commitTag, vectData);
if (result < 0) {
opserr << "Beam3dUniformLoad::sendSelf - failed to send data\n";
return result;
}

return 0;
}

int Beam3dUniformLoad::recvSelf(int commitTag, Channel &theChannel,
FEM_ObjectBroker &theBroker)
{
int dbTag = this->getDbTag();

static Vector vectData(4);

int result = theChannel.recvVector(dbTag, commitTag, vectData);
if (result < 0) {
opserr << "Beam3dUniformLoad::sendSelf - failed to send data\n";
return result;
}

wx = vectData(0);;
wy = vectData(1);;
wz = vectData(2);;
eleTag = (int)vectData(3);

return 0;
}




As you can see, both of two functions do not use the “theTag” attribute, which I guess uniquely identified each Object. I check the ForBeamColumn3d element class implementations, and in sendSelf()/recvSelf(), both use getTag()/setTag() function to get/set the “theTag” attribute. So I modified the source of Beam3dUniformLoad.cpp in sendSelf()/recvSelf(), add “theTag”.

Then I run OpenSeesSP with our test model again, and those warnings were disappeared, but still got segment fault. Below are error outputs:

OpenSees -- Open System For Earthquake Engineering Simulation
Pacific Earthquake Engineering Research Center -- 2.1.0
(c) Copyright 1999,2000 The Regents of the University of California
All Rights Reserved
(Copyright and Disclaimer @ http://www.berkeley.edu/OpenSees/copyright.html)
Slave Process Running 1
Slave Process Running 2
Master Process Running OpenSees Interpreter 0

Number of Stories in Y: 20 Number of bays in X: 10 Number of bays in Z: 3
goGravity

[localhost:29917] *** Process received signal ***
[localhost:29917] Signal: Segmentation fault (11)
[localhost:29917] Signal code: Address not mapped (1)
[localhost:29917] Failing at address: 0x7e
[localhost:29917] [ 0] [0xe04440]
[localhost:29917] [ 1] /OpenSeesSP(_ZN11LoadPattern9applyLoadEd+0x79) [0x83f1e19]
[localhost:29917] [ 2] /OpenSeesSP(_ZN6Domain9applyLoadEd+0xaf) [0x81a1aff]
[localhost:29917] [ 3] /OpenSeesSP(_ZN17PartitionedDomain9applyLoadEd+0x26) [0x81a9e86]
[localhost:29917] [ 4] /OpenSeesSP(_ZN13AnalysisModel15applyLoadDomainEd+0x2f) [0x821c17f]
[localhost:29917] [ 5] /OpenSeesSP(_ZN11LoadControl7newStepEv+0x91) [0x81f4351]
[localhost:29917] [ 6] /OpenSeesSP(_ZN14StaticAnalysis7analyzeEi+0x53) [0x81e0a03]
[localhost:29917] [ 7] /OpenSeesSP(_Z12analyzeModelPvP10Tcl_InterpiPPKc+0x89) [0x817bcf9]
[localhost:29917] [ 8] /usr/local/tcl8.5/lib/libtcl8.5.so(TclInvokeStringCommand+0x76) [0x67e326]
[localhost:29917] [ 9] /usr/local/tcl8.5/lib/libtcl8.5.so [0x67f558]
[localhost:29917] [10] /usr/local/tcl8.5/lib/libtcl8.5.so [0x680152]
[localhost:29917] [11] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_EvalEx+0x3a) [0x68048a]
[localhost:29917] [12] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_FSEvalFileEx+0x266) [0x6eecf6]
[localhost:29917] [13] /OpenSeesSP(_Z13OPS_SourceCmdPvP10Tcl_InterpiPKP7Tcl_Obj+0x9b) [0x817c3bb]
[localhost:29917] [14] /usr/local/tcl8.5/lib/libtcl8.5.so [0x67f558]
[localhost:29917] [15] /usr/local/tcl8.5/lib/libtcl8.5.so [0x680152]
[localhost:29917] [16] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_EvalEx+0x3a) [0x68048a]
[localhost:29917] [17] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_FSEvalFileEx+0x266) [0x6eecf6]
[localhost:29917] [18] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_FSEvalFile+0x2c) [0x6eee7c]
[localhost:29917] [19] /usr/local/tcl8.5/lib/libtcl8.5.so(Tcl_EvalFile+0x41) [0x6eeed1]
[localhost:29917] [20] /OpenSeesSP(_Z9g3TclMainiPPcPFiP10Tcl_InterpEii+0x2ac) [0x816c08c]
[localhost:29917] [21] /OpenSeesSP(main+0x3e1) [0x816cc21]
[localhost:29917] [22] /lib/libc.so.6(__libc_start_main+0xdc) [0xe1ae9c]
[localhost:29917] [23] /OpenSeesSP [0x816bd01]
[localhost:29917] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 29917 on node localhost.localdomain exited on signal 11 (Segmentation fault).

I used gdb to debug the process with rank 0, I found the segment fault error arise in ElementalLoad::applyLoad() when process 0 try to static analysis. The reason is that in model partition phase, when elements were added to subdomain, they were removed from main Domain at the same time. However, ElementalLoads were added to subdomain, were not removed from Domain, this cause some ElementalLoads had no elements associated with. So In the process 0, when static analysis begin, ElementalLoad::applyLoad() were invoked, element pointers in some ElementalLoads were point invalid memory address, and segment fault came up. The following code fragment is add ElementalLoads to subdomain in file DomainPartitioner.cpp:
ElementalLoadIter &theLoads = theLoadPattern->getElementalLoads();
ElementalLoad *theLoad;
while ((theLoad = theLoads()) != 0) {
int loadEleTag = theLoad->getElementTag();
// find subdomain with node and add it .. break if find as internal node
SubdomainIter &theSubdomains = myDomain->getSubdomains();
Subdomain *theSub;
bool added = false;
while (((theSub = theSubdomains()) != 0) && (added == false)) {
bool res = theSub->hasElement(loadEleTag);
if (res == true)
{
// opserr << "PartitionedDomain::addLoadPattern(LoadPattern *loadPattern) SUB " << theSub->getTag() << *load;
int res = theSub->addElementalLoad(theLoad, loadPatternTag);
if (res < 0)
opserr << "DomainPartitioner::partition() - failed to add ElementalLoad\n";
added = true;
}
}
}

So I modified DomainPartitioner.cpp and ElementalLoads.cpp to remove ElementalLoads which were added into subdomain, and run OpenSeesSP with our test model, and segment fault did not show up again. Unfortunately, there still were errors, the outputs are following:

Master Process Running OpenSees Interpreter 0


OpenSees -- Open System For Earthquake Engineering Simulation
Pacific Earthquake Engineering Research Center -- 2.1.0

(c) Copyright 1999,2000 The Regents of the University of California
All Rights Reserved
(Copyright and Disclaimer @ http://www.berkeley.edu/OpenSees/copyright.html)


Slave Process Running 2
Slave Process Running 1
Number of Stories in Y: 20 Number of bays in X: 10 Number of bays in Z: 3
goGravity
WARNING BandGenLinLapackSolver::solve() -LAPACK routine returned 2462
WARNING NewtonRaphson::solveCurrentStep() -the LinearSysOfEqn failed in solve()
WARNING NewtonRaphson::solveCurrentStep() -the LinearSysOfEqn failed in solve()
StaticDomainDecompositionAnalysis::analyze() - the Algorithm failed with domain at load factor 0.1
StaticAnalysis::analyze() - the Algorithm failed at iteration: 0 with domain at load factor 0.1
WARNING NewtonRaphson::solveCurrentStep() -the LinearSysOfEqn failed in solve()
StaticDomainDecompositionAnalysis::analyze() - the Algorithm failed with domain at load factor 0.1
OpenSees > analyze failed, returned: -3 error flag
Model Built

I can’t figure out what is going on.



So, the question are :

a) What I modified are correct ? Or I totally misunderstand the whole OpenSees analysis procedure?
b) I want to attach our test model scripts, but I don't how.


please help

[fmk]

you are going to have to send me the script.

fmckenna ATTTTTTTTTT ce dotttt berkeley DOTDOTDOT edu

[frank_lv]

i have send you our test model scripts

[frank_lv]

fmk wrote:
> you are going to have to send me the script.
>
> fmckenna ATTTTTTTTTT ce dotttt berkeley DOTDOTDOT edu



hi fmk, did you have received our test model scripts? i have send them to you at fmckenna@ce.berkeley.edu

thanks for your reply

[frank_lv]

hi fmk , it has been a while since I put this topic , is there anthing new to the problem that i had ?

[fmk]

i thought i emailed you back that your script was failing in the solver when i ran it sequentially .. the parallel case was just showing the same problem

[frank_lv]

I am very sorry for that i did not notice your email.


But , i am pretty sure that it can run sequentially. I ran the model using OpenSees sequentially successfully today, again just for sure.

[frank_lv]

By the way, The element load partition problem that i found, is it a real problem or it is just my mistake?

[frank_lv]

hi fmk, sorry for bother you again. could send me you send me your Makefile.def used to build sequential OpenSees. right now i am very confused that sequentially you can not run it. please let me know you computing environment, too. thanks again! My email address: lvhai@emails.bjut.edu.cn

[charleswi]

If any problem remain further please contact with me .mrahman.5410@gmail.com

[johnwise]

Frank, I've had the same problem. Have you tried it on a newer System, I found that, that was my solution.

[Kevinstan]

So I modified DomainPartitioner.cpp and ElementalLoads.cpp to remove ElementalLoads which were added into subdomain, and run OpenSeesSP with our test model, and segment fault did not show up again. Unfortunately, there still were errors, the outputs are following: We should agree with this one..

[rakibme]

Could send me you send me your Makefile.def used to build sequential OpenSees. right now i am very confused that sequentially you can not run it. please let me know you computing environment, too. thanks again!