Error in Analyzing SFSI in OpenSeesSP

[fbenedetti]

Hi,

I'm modelling a 16 story shear wall reinforced concrete building, with mat foundation and soil for the analysis of the soil-foundation-structure interaction phenomenon. The building is modelled using bar elements, but the floor slabs and the mat is modelled with shells. The soil domain is formed with hexaedral solid elements. The model is almost full non linear and has around 2.5 million DOF´s.

When I run the gravity analysis for the whole model (I haven't tried the dynamic analysis yet) using OpenSeesSP (v.2.4.0.10) on the neeshub machines, I get this error message:

OpenSeesSP:58733 terminated with signal 11 at PC=2b33b1840b5a SP=7fff413f825. Backtrace:
/apps/rhel6/openmpi-1.4.4/64/intel-12.0.084/lib/libmpi.so.0(MPI_Comm_size+0x5a)[0x2b33b1840b5a]
OpenSeesSP(Cblacs_pinfo+0x93)[0x14cbca3]
OpenSeesSP(blacs_gridmap_+0x64)[0x14c95b4]
OpenSeesSP(blacs_gridinit_+0x9e)[0x14c946e]
OpenSeesSP(dmumps_164_+0x1b2)[0x132e1f2]
OpenSeesSP(dmumps_26_+0x8978)[0x12e9258]
OpenSeesSP(dmumps_+0xc56)[0x13abd96]
OpenSeesSP(dmumps_f77_+0xea9)[0x12b8b49]
OpenSeesSP(dmumps_c+0xca6)[0x1295186]
OpenSeesSP(_ZN19MumpsParallelSolver7setSIzeEv+0x227)[0x12941e7]
OpenSeesSP(_ZN16MumpsParallelSOE7setSizeER5Graph+0x7cb)[0x1292bdb]
OpenSeesSP(_ZN13XmlFilesStreamDEOv+0x19c)[0x9cc59c]
OpenSeesSP(_ZN12NodeRecorderDEOv+0x41)[0x8692b1]
OpenSeesSP(_ZN6Domain15removeRecordersEv+0x33)[0x77cd83]
OpenSeesSP(_ZN14ActorSubdomain3runEv+0x22b)[0x5e921b]
OpenSeesSP(_ZN13MachineBroker9runActorsEv+0xa5)[0x85ed35]
OpenSeesSP(main+0xce)[0x5b8d3e]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x3e7941ecdd]
OpenSeesSP[0x5b8509]
------------------------------------------------------------------------------------------
mpirun has exited due to process rank 10 PID 18132 on
node carter -c003.rcac.purdue.edu exiting without calling "finalize". This may
have caused other processes in the aplication to be
terminated by signals sent by mpirun (as reported here)
---------------------------------------------------------------------------------------------
files does not exit

If I separate the structural model and the soil model, an then I run only the structural part, I get this message:

Error WARNING MumpsParallelSolver::solve(void)- Error -9 returned in substitution dmumps()
WARNING NewtonRaphson::solveCurrentStep() -the LinearSysOfEqn failed in solve()
StaticDomainDecompositionAnalysis::analyze() - the Algorithm failed at iteration: 0 with domain at load factor 1
OpenSees > analyze failed, returned: -3 error flag
OpenSeesSP:18130 terminated with signal 11 at PC=9cc59c SP=7fff33411aa0. Backtrace:
OpenSeesSP(_ZN13XmlFilesStreamDEOv+0x19c)[0x9cc59c]
OpenSeesSP(_ZN12NodeRecorderDEOv+0x41)[0x8692b1]
OpenSeesSP(_ZN6Domain15removeRecordersEv+0x33)[0x77cd83]
OpenSeesSP(_ZN14ActorSubdomain3runEv+0x22b)[0x5e921b]
OpenSeesSP(_ZN13MachineBroker9runActorsEv+0xa5)[0x85ed35]
OpenSeesSP(main+0xce)[0x5b8d3e]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x3e7941ecdd]
OpenSeesSP[0x5b8509]
------------------------------------------------------------------------------------------
mpirun has exited due to process rank 10 PID 18132 on
node carter -c003.rcac.purdue.edu exiting without calling "finalize". This may
have caused other processes in the aplication to be
terminated by signals sent by mpirun (as reported here)
---------------------------------------------------------------------------------------------
files does not exit

Using an i7 desktop machine (OpenSeesSP v.2.4.2), I get:
Error WARNING MumpsParallelSolver::solve(void)- Error -10 returned in substitution dmumps()
cause: Matrix is Singular Numericaly.

This is driving me insane. I have tried lots of things, but I can't find the solution. I really need help.

I would really appreciate your valuable help.

Thanks.

Best regards.

[fmk]

mumps -10 is an indication that the matrix is singular. i have updated the code to reflect the error coe meanings:

switch(info) {
case -5:
opserr << " out of memory allocation error\n";
case -6:
opserr << " cause: Matrix is Singular in Structure: check your model\n";
case -7:
opserr << " out of memory allocation error\n";
case -8:
opserr << "Work array too small; use -ICNTL14 option, the default is -ICNTL 20 make 20 larger\n\
";
case -9:
opserr << "Work array too small; use -ICNTL14 option, the default is -ICNTL 20 make 20 larger\n\
";
case -10:
opserr << " cause: Matrix is Singular Numerically\n";
default:

[fbenedetti]

Thanks a lot Mr. McKenna. I guess the problem was the mesh that I use on the soil domain. First I used a very irregular hexahedral mesh, but then I changed to a mesh with degenerated tetahedral stdBrick elements and it works quite better.

The problem now is that when I use a very refined mesh (almost 40.000 nodes and 200.000 elements for the 80m x 80m x 40m sized soil domain) instead of a very coarse mesh (which works), I get segmentation fault when running the model using OpenSeesSP on carter and local machine on Neeshub (independently the number of processors). When running the model on a desktop i7 with win7, then I get:

job aborted:
rank: node: exit code [:error code]
0: PCname: -1073741819: process 0 exited without calling finalize
1: PCname: 123
2: PCname: 123
3: PCname: 123
4: PCname: 123
5: PCname: 123
6: PCname: 123
7: PCname: 123

The errors and the segmentation fault occur after one succesful analysis step of a two steps gravity analysis with transient integrator and updateMaterialStage for the soil material.

I apreciate your support.

[fmk]

lets work on the windows problem first. can you pin-point where the problem lies, i.e. if during the analyze command, or after. if after what command does it fail in .. is it before or during the next analyze commmand.

[brag006]

I get this same segmentation fault after a two step gravity analysis and updateMaterialStage. Sometimes I get this error:

WARNING: updateMaterialStage - MaterialStageParameter::setDomain() - no effect with material tag

and depending on the number of proc the number of errors will increase.

Any thoughts?

[fmk]

that could happen if there is no material of that type in one of the subdomains.

do you get a seg fault if the update material stage command is not there?

[brag006]

In some examples there is an error and others there isn't. There's also an error in using the update parameter command it says "Domain::addParameter - parameter with tag 0already exists in model". Not sure what the problem there is exactly.

[fmk]

is it one input wih different no.s of processors or different models?

i am going to need to see the scripts. the smallest example that fails is best.

[brag006]

I am using the following example:
http://opensees.berkeley.edu/wiki/index ... oil_Column

it is one model and I am trying to run it on multiple processors using OpenSeesSP.

[fmk]

should be fixed. though for some reason ICNTL14 must be set to 80 or so for the last analyze.