Remove Disconnected & Evicted Windows Server Node from Cluster

When a Windows Server machine which was joined to a cluster is disconnected or evicted from a failover cluster by accident, failure or because the now standalone computer is no longer connected to the network, the Windows Server will still work fine and function normally just like a normal server.

However, the node is still attempting to connect and form a cluster, as the cluster configuration is not removed from the Windows Server itself. As the result, Windows Logs is populating which numerous stop and critical errors and warnings from System and FailoverClustering sources. For example:

Event ID 1573 from System: Node ‘windows-server’ failed to form a cluster. This was because the witness was not accessible. Please ensure that the witness resource is online and available.

Event ID 2050 from Microsoft-Windows-FailoverClustering/Diagnostic: [CORE] Node windows-server attempted to bring online the witness resource at time 2015/01/01-12:30:30.555 with result ERROR_RESOURCE_FAILED(5038) Failed to bring quorum resource 99999ee8-a1a1-111c-bb7e-b10888aaa1a8 online, status 5038

Event ID 1560 from System: Node ‘windows-server’ failed to establish a communication session while joining the cluster. This was due to an authentication failure. Please verify that the nodes are running compatible versions of the cluster service software.

Event ID 2050 from Microsoft-Windows-FailoverClustering/Diagnostic: cxl::ConnectWorker::operator (): HrError(0x000004d4)’ because of ‘[SV] Authentication or Authorization Failed’

Event ID 2050 from Microsoft-Windows-FailoverClustering/Diagnostic: mscs::ListenerWorker::operator (): HrError(0x8009030c)’ because of ‘[SV] Authentication or Authorization Failed’

Event ID 2051 from Microsoft-Windows-FailoverClustering/Diagnostic: mscs_security::BaseSecurityContext::DoAuthenticate_static: Closed(1236)’ because of ‘channel to remote endpoint 192.168.1.240:~3343~ has failed with status ERROR_SUCCESS(0)’

Event ID 2050 from Microsoft-Windows-FailoverClustering/Diagnostic: [CHANNEL 192.168.1.1:~3343~] failure, status ERROR_SUCCESS(0)

Event ID 2051 from Microsoft-Windows-FailoverClustering/Diagnostic: mscs_security::BaseSecurityContext::DoAuthenticate_static: Closed(1236)’ because of ‘channel to remote endpoint 192.168.1.1:~3343~ has failed with status ERROR_SUCCESS(0)’

And in Failover Cluster Manager, it’s not possible to connect to the cluster in order to remove the node from clustering. To make matter worse, it’s impossible to remove the Failover Cluster Feature through Remove Roles and Features Wizard (in Windows Server 2012 / R2) or Remove Roles Wizard (in Windows Server 2008 / R2) because the server is still firmly believe it’s a vital part of a cluster.

Resolution

To forcefully remove the failover clustering configuration from an evicted cluster node which is no longer connected to to cluster.

1. Open a Windows PowerShell as administrator.
2. Run the following commands, where the first command to import module is optional as it should automatically available when clustering is enabled on the node.

   Import-Module FailoverClusters
   Clear-ClusterNode

   

   Clear-ClusterNode stops the Cluster Service service and clears the cluster configuration from a node that was evicted from a failover cluster. When the node is cleared from clustering configuration, the following log is shown in Even Viewer:

   Event ID 4621: This node was successfully removed from the cluster.

   

3. Remove the Failover Clustering feature by going to Server Manager, then click on Manage -> Remove Roles and Features (in Windows Server 2012 / R2) or Features -> Remove Features (in Windows Server 2008 / R2).

   

   Uncheck the Failover Clustering in Features section. If promoted, choose whether to remove features that requires Failover Clustering, i.e. management tools. Click on Remove when done.

4. Once removal is completed, restart the server to complete the Failover Clustering feature removal process.