Infiniband FAQ
- Q: What is Infiniband?
- A: Infiniband is a high bandwidth low latency network commonly found on clusters.
- Q: How fast is Infiniband? What is its latency?
- A: Infiniband is a 10Gbps network, all Infiniband at the CAC is nonblocking (no single connection is ever starved of the entire 10Gbps link. The latency of Infiniband is about one order of magnitude less than Ethernet. This puts Infiniband at less than 10 usec latency, since many codes are very sensitive to latency, this can offer a large performance gain.
- Q: Do I need to recompile my code to use Infiniband?
- A: If you compiled your code with our default MPI library on nyx, you do not need to recompile.
- Q: How do I compile my code to use Infiniband?
- A: Compile your code using our standard MPI compilers (mpicc, mpif90, etc.)
- Q: I compiled my code for Infiniband can I run it on TCP (Ethernet) only nodes?
- A: Yes! This is the benefit of the MPI library that we provide. Code compiled for Infiniband will also have the ability to run on Ethernet without recompiling. If mpirun finds that some of the nodes given to you do not have Infiniband then those nodes will use Ethernet. The nodes with Infiniband will use Infiniband.
- Q: What advantages does Infiniband give me?
- A: Infiniband allows bandwidth and latency sensitive codes to scale farther. For a example a code that scales to 12 cpus on Ethernet might get to 24 on Infiniband. If a code does not communicate often (like Monti Carlo codes) Infiniband will not improve your code's performance.


