Security and access to the cluster were being handled by network department's switches and routers so the Iptables firewall was really not necessary for security purposes.
The software that was to be run on this cluster had a ton of ports that needed to be opened to the outside, and typically the software company recommended keeping firewalls disabled for simplicity. (My requests for a list of ports to open were met with a lot of hemming and hawing, so I just dropped it.)
I experimented for a bit, and found some suggestions on the interwebs. The most helpful find was from Alex Atkinson on StackExchange: http://superuser.com/a/634471
This solution was implemented on RHEL 6.6: A wide-open firewall that only does masquarade of IPs.
1. vi /etc/sysctl.conf
2. change or add net.ipv4.ip_forward = 1
3. sysctl -p /etc/sysctl.conf
4. service iptables start
5. service iptables save
6. cp /etc/sysconfig/iptables /root/iptables.20151125.1308-dah
7. (A bunch of commands to build up a new rule set
iptables -F
iptables -t nat -F
iptables -A INPUT -i lo -j ACCEPT -m comment --comment "Allow all loopback traffic"
iptables -A INPUT ! -i lo -d 127.0.0.0/8 -j REJECT -m comment --comment "Drop all traffic to 127 that does not use lo"
iptables -A OUTPUT -j ACCEPT -m comment --comment "Accept all outgoing"
iptables -A INPUT -j ACCEPT -m comment --comment "Accept all incoming"
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT -m comment --comment "Allow all incoming on established connections"
iptables -t nat -A POSTROUTING -o bond5 -j MASQUERADE -m comment --comment "Masquarade traffice headed out bond5"
iptables -A FORWARD -j ACCEPT -m comment --comment "Accept all forwarding"
8. service iptables save
9. The resulting iptables file
#Generated by iptables-save v1.X.X on Wed Nov 25 13:09:31 2015
*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A POSTROUTING -o bond1 -m comment --comment "Masquarade traffic headed out bond1" -j MASQUERADE
COMMIT
# Completed on Wed Nov 25 13:09:31 2015
# Generated by iptables-save v1.X.X on Wed Nov 25 13:09:31 2015
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -m comment --comment "Allow all loopback traffic" -j ACCEPT
-A INPUT -d 127.0.0.0/8 ! -i lo -m comment --comment "Drop all traffic to 127 that does not use lo"
-A INPUT -m comment --comment "Accept all incoming" -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -m comment --comment "Allow all incoming on established connections" -j ACCEPT
-A FORWARD -m comment --comment "Accept all forwarding" -j ACCEPT
-A OUTPUT -m comment --comment "Accept all outgoing" -j ACCEPT
COMMIT
# Completed on Wed Nov 25 13:09:31 2015
10. Set default route on all the compute nodes to the bond5 interface on the head node.
DONE!
