When I use the Sparse solver in ANSYS I often get what seem to be cryptic messages about in core and out of core memory. Is there a way to interpret these messages and the effect that changing memory settings has on them?
|
To illustrate we'll use the 3 scenarios shown below which are of the same model running on an HP UX 64 bit os, dual processor, 8 gb of ram. All 3 jobs were submitted remotely from Workbench running on a 32 bit windows system. The first scenario used the default `m and `db settings sent by workbench. -M 610 -DB 560 (these are the defaults that workbench sent) Mem available = 1020 Mem required for in core = 3663 Opt mem for out of core = 669 Min mem for out of core = 64 Page file used (586 mb), # of r/w ops = 8270 Elapsed time = 1885 sec (CP time = 1957 sec) "Based on the above information, it was assumed that ANSYS was telling me that it was only going to be able to get 1020 MB of TOTAL memory for this job, and since it required 3663 MB to run in-core, that it would not run in core. Customers often interpret this message as ANSYS not being able to recognize that the machine has 8 gb of ram. How should this message be interpreted?" Here memory available really means available to the solver (which, in this case is tiny - 50 Mbytes ). This is the WB memory default based on their experience. It is NOT what I recommend for a machine that has plenty of memory. So, looking at the solver requirement for 670 Mbytes to run in optimal out of core I'd use something like -m 1500 -db 256 for this run. That is more than enogh to comfortably run this job in optimal out-of-core mode. The 1020 Mbyte size come from various memory heuristic formulas we apply that are too involved to explain here. But, note that in almost all cases we will chose a memory setting that is "comfortably" above the optimal out-of-core setting. BTW, applications codes like ANSYS do not know how much physical memory is on the system. There are no general calls that give this information. So, it isn't like they didn't recognize memory available. What we do instead is to use heuristics and customer feedback to determine the size of memory block that we allocate for the sparse solver. The sparse solver operates in one contiguous memory block so whatever size we chose it has to be one block not a collection of smaller blocks like is the case for the pcg solver. "Since the above scenario told us that the solver needed 3663MB to run in core, we decided to ask for that, plus some, via `m which leads to the next scenario:" -M 5000 `DB 450 Memory available = 1338 Mem req`d for in core = 3664 Opt mem for out of core = 669 Min mem for out of core = 64 Page file used during solution (605 mb), # of read/write ops = 332,211 Elapsed time = 1892 seconds (CP time = 2002 sec) What happened here is that we gave us more memory but still not enough to run incore. So we basically get the same memory required numbers but since there is more available at the start we are able to keep more - we actually shrink the solver memory back to 2X optimal OOC when that memory is available at the time the solver is called. The rest is available for Ansys to use ( of course it doesn't need it in this case so it just sits there tying up system memory ). The mystery here is why is the page file so active. It's not from the solver and 450 Mbytes should be plenty for the db space to avoid excessive I/O to this file. The way to solve the page file problem however is just increase db space to -db 610. "We now decided to try the following scenario:" -M 5500 `DB 2 Mem available = 4599 Mem req`d for in core = 4217 Opt mem for out of core = 669 Min mem for out of core = 618 Page file used (605 mb), # of r/w ops = 19,960,146 Elapsed time = 2581 sec (CP time = 2819 sec) "We finally got enough memory to run incore but the solution slows down! " BUT, look what they did to run incore - reduced db to tiny. That's a killer and 19 million I/O operations to the page file is a killer. We DID recognize the memory this time but note that the real incore memory requirement is actually 4217 Mbytes, not the 3668 reported above. This is because of an error in the way the boeing sparse solver calculates incore memory when the job is actually running in out-of-core mode. It happens because an incore run requires the input marix to be in memory as well as the factored matrix but in out-of-core mode the size of the input matrix is not added into the routine that does this printout. This should be fixed in 11.0 or at least much closer. Don't force incore sparse solver runs. That is, if the incore memory fills up the available memory then it's better to run in a comfortable out-of-core mode and leave the extra memory free for the I/O buffer cache. When ANSYS reaches the sparse solver code we look at the largest memory block available, the number of dofs in the model and we also apply a couple heuristic formulas. In order to get an incore run there has to be plenty of memory available (as determined by your heuristic formulas and code logic ). When this user set -m 5000 -db 450 that meant ANSYS scratch memory starts at 4550 - should be plenty. BUT, some of that memory is in use before the sparse solver starts up so it is not ALL available for the sparse solver work array. Hence, it probably went through initial preprocessing trying for incore but as soon as it determined that at least 4217 Mbytes was not available it reverts to optimal OOC mode. My guess here is they just missed the cutoff. Here is a simple strategy that will almost always give very good performance on a 64-bit machine. Don't worry about incore versus out-of-core. Instead use the rule of thumb that ANSYS needs 1 Gbyte per million dofs and then add in the db space. IF you have a job where you can run incore comfortably then by all means do it. That estimate is 10 Gbytes per million dofs. On a Winx64 system I've see incore runs that take 6 Gbytes do very well when t |
||
|