Had a strange error on a SQL 2008 cluster the other day,

The OS was Windows 2008 R2

We kept getting messages that the cluster node was offline because the Quorum was unavailable.   This made little sense as both nodes in this cluster were online and the Quorum disk was available.  We could ping across the heartbeat, everything looked fine except for these errors. 

After a little research we determined that a new version of Symantec Endpoint Security had been pushed to these servers.  Even with the new version of endpoint security, we could establish communication across all networks between the 2 nodes so we were a little stumped.  Eventually we ran across a policy that was being enforced from the Symantec central management server/policy/whatever its called!

As it turns out, Symantec endpoint security by default blocks all IPV6 traffic.  If you’re like me, I didn’t even realize that a windows 2008 cluster would use IPV6 for the heartbeat communication.  After disabling the rules that were preventing IPV6 traffic everything returned to normal.

So, the moral of all this is nothing new… NEVER trust anything new getting pushed to your servers..