SYMPTOM:
The CFX Solver fails at the start of the run with an error similar to:
/ansys/v110/CFX/bin/linux-amd64/solver-hpmpi.exe was interrupted by signal SEGV (11)
when a high-speed interconnect (such as Infiniband or Myrinet) is used for a parallel run.
(Note: HPMPI has been renamed Platform MPI starting at ANSYS R14.0)
PROBLEM:
CFX Release 11.0 includes HP MPI 2.2.5 for the use of parallel jobs using the HP MPI start methods. However, this version of HP MPI does not support some of the newer hardware/firmware that is now available for high-speed interconnects. In particular:
- HP-MPI 2.2.5 works with OFED 1.0 and 1.1, but not newer versions of OFED.
- HP-MPI 2.2.5.1 is the first release which supports ConnectX.
|
WORKAROUND: If the latest version of HP MPI (currently 2.3) is downloaded from the HP website and installed, then CFX Release 11.0 can pick this up and use it instead of HP 2.2.5. The instructions are provided below. 1) Download the latest version of HP MPI, which can be obtained from <a target=_blank href="http://www.hp.com/go/mpi">http://www.hp.com/go/mpi</a>http://www.hp.com/go/mpi by following the "Download" link. Install it on all the machines to be used for the parallel run. (Note that CFX 12.0 includes HP MPI 2.3 so if you already have this installed there is no need to install another copy.) 2) Set up CFX 11.0 to point to this newer version of HP MPI. You need to set the variable "CFX5_HPMPI_DIR" to point to the installation directory of the newer version. By default it points to <install_directory>/v110/CFX/tools/hpmpi-2.2-5/Linux" (for the Linux operating system). To change this you need to add the line CFX5_HPMPI_DIR=<new install dir> into either your personal cfx5rc file, or the system-wide file. (See the online help "ANSYS CFX Introduction | Customizing ANSYS CFX | ANSYS CFX Resource Configuration Files" for details on how to do this.) 3) Disable the "-prot" option on the HP MPI start methods. The "-prot" option can be used to get a printout of the selected interconnects and CFX 11.0 uses it on the mpirun command line, but using this with HP MPI 2.3 canlead to memory corruption. This should be removed from the HP MPI start methods by editing (or taking a local copy of) <CFXROOT>/etc/start-methods.ccl and changing the line Start Command = %{hpmpidir}/bin/mpirun -prot -np %{partitions} %{executable} %{arguments} (for HP MPI Local Parallel) or Start Command = %{hpmpidir}/bin/mpirun -prot -f %{req:appfile} (for HP MPI Distributed Parallel), to remove the "-prot" option. Note that you do NOT need to acquire a separate license for HP MPI if you are only using it to run CFX. For details go to <a target=_blank href="http://www.hp.com/go/mpi">http://www.hp.com/go/mpi</a>http://www.hp.com/go/mpi and follow the link to HP-MPI FAQ. FIXED IN: ANSYS CFX Release 12.0. |
||
|