History of InfiniBand for Dummies
Version 3
InfiniBand for Dummies
Getting Started Tutorial for InfiniBand and Linux
These are mostly my personal notes, culled from various holes on the Internet. I wish there was a basic "Noob's Guide to Networking beyond 1000BaseT (i.e. ye good ol' 1 Gigabit RJ-45 Jack.)
Some of this stuff may not be accurate.. updates appreciated.
Some of this stuff may not be accurate.. updates appreciated.
Hardware
Mellanox is the king of InfiniBand, though they are selling more Ethernet equipment than InfiniBand these days.- Gotcha ConnectX-2 Cards suck because they do not support SR-IOV (which is required for KVM, see below)
- ConnectX-3 Cards are great!
- Gotcha ConnectX-3 dual port cards cannot be individually assigned
- Lowend servers / chipsets (Think Dell R4X0 and below) cannot do SR-IOV may not work properly with the Mellanox cards
Software
- The Linux kernel has built in InfiniBand support. On CentOS, you can do this: yum groupinstall "Infiniband Support"
- Gotcha Linux KVM virtualization only supports Ethernet bridging, thus you MUST use SR-IOV if you want InfiniBand in your Guest servers.