Availability as a quality attribute: failure recovery tactics

Failure recovery tactics aim to minimize downtime and quickly restore the system to normal operation. Let’s consider the two main groups of approaches:

Preparation and repair tactics:
🔹 Redundant spare – duplicate components replace the primary one in case of failure.
🔹 Rollback – reverting to the last stable state using checkpoints.
🔹 Exception handling – detecting and responding to errors without stopping the system.
🔹 Software upgrade – the ability to modify the code without interrupting system operation.

Possible subtypes:
🔹 Function patch – modifying a specific function without affecting other parts of the code.
🔹 Class patch – updating object-oriented structures by adding or modifying methods.
🔹 Hitless ISSU – full system upgrade without service interruption or loss of availability.
🔹 Retry – reattempting an operation if the failure was temporary.
🔹 Ignore faulty behavior – disregarding non-critical errors.
🔹 Graceful degradation – maintaining key functions even in case of partial failure.
🔹 Reconfiguration – adaptive resource allocation in case of a failure.

Reintroduction tactics:
🔹 Shadow – testing a component before reintroducing it into the system.
🔹 State resynchronization – maintaining data consistency between active and backup nodes.
🔹 Escalating restart – adaptive system recovery to minimize failure impact, involving multiple restart levels depending on error complexity.
🔹 Nonstop forwarding – data transmission via the last confirmed routes even during control component failures.

System resilience = fast recovery! A fault-tolerant system is not one without issues, but one that can quickly overcome them. By using these tactics, you create a stable IT environment that withstands any challenges!