Reliability and resilience: expect failure and organise recovery

A Mars mission cannot honestly promise that no component will fail. It must instead show that credible faults can be detected, isolated well enough, tolerated in a degraded state and repaired with available resources. This module separates reliability, availability, redundancy, maintainability and resilience, then connects them through practical analysis methods.

1. Reliability, availability and resilience ask different questions

Reliability concerns successful function over stated time and conditions. Availability also reflects restoration after failure. Resilience asks whether an acceptable mission can continue through disturbance, degradation and recovery. A system may fail often yet be quickly repairable, or fail rarely but catastrophically.

Reliability is the probability that an item performs its required function for a stated time under stated conditions. Availability asks whether the function is ready when demanded, so repair time also matters. Resilience goes further: can the system preserve an essential service, reconfigure and then return toward a sustainable state after the event? A highly reliable component can still create poor resilience if its rare failure is unrecoverable. Conversely, a component that fails more often may deliver excellent service if detection is fast, access is easy and degraded operation is acceptable. These three terms therefore need separate requirements and separate evidence.

2. MTBF is not a component death clock

MTBF is Mean Time Between Failures, a population statistic under defined use. In a simple exponential model, constant failure rate λ gives R(t)=e^−λt. The model ignores infant mortality and wear-out; its assumptions must be stated.

MTBF, mean time between failures, is a population or model statistic, not the predicted death time of one component. Under a simplified exponential model with constant failure rate λ, reliability is R(t)=e^(−λt) and MTBF=1/λ. An MTBF of 10,000 hours does not mean the item will work for 10,000 hours and then fail. It describes an event rate assumed for a population. On Mars, a constant-rate model may be poor when wear, radiation, dust or thermal cycling dominate. The number should always be accompanied by the test environment, population, confidence and mechanisms that the model does not represent.

3. Availability exposes repair time

A simple intrinsic model is A = MTBF/(MTBF+MTTR), where MTTR is Mean Time To Repair. With MTBF 1,000 h and MTTR 10 h, availability is about 0.990. Reducing repair time increases availability without changing failure rate.

A common intrinsic-availability approximation is A = MTBF/(MTBF + MTTR). With MTBF = 1,000 h and MTTR = 10 h, A ≈ 1,000/1,010 = 0.990, or 99.0%. Reducing MTTR to 2 h raises it to about 99.8% without changing failure frequency. The calculation shows why access, diagnostics, tools and post-repair testability can matter as much as component reliability. But the formula assumes repair is possible and does not automatically include spare logistics, EVA waiting time, common-cause events or maintenance queues. Engineers must know what an availability number leaves out.

4. FMEA and FMECA walk through failure modes

A Failure Modes and Effects Analysis asks how a function can fail, its local and system effects, and how the failure is detected. FMECA adds criticality assessment. The value is the disciplined search for single failures, critical items and missing detection or maintenance provisions.

FMEA walks through functions or components and asks how each can fail, what the local and system effects are, how the failure is detected and what action limits the consequence. FMECA adds criticality. The value is not the spreadsheet itself but the discovery of hidden interfaces and uncovered modes. A row labelled ‘bad sensor’ should state whether the fault is detectable, whether software can believe it and which actuator could then be commanded incorrectly. For a Mars habitat, FMEA also needs maintenance, software, human action and post-repair states; otherwise it describes only pristine hardware rather than the system that will exist after years of operation.

5. Fault trees start from the unwanted event

Fault Tree Analysis begins with an event such as loss of pressure and works backward through combinations that can cause it. AND and OR logic can structure the argument, but the physical dependencies still matter. Drawing two branches separately does not prove statistical independence.

A fault tree starts from an unwanted top event and works backward to combinations of causes. An OR gate means any one cause is sufficient; an AND gate requires a combination. This is especially useful for exposing common causes. Two pumps may look redundant, but if loss of one shared power feed is enough to lose flow, the common branch dominates the tree. Probabilities can be added when independence assumptions are credible, but the primary value is logical: showing which combinations actually remove the service and where physical, electrical or software separation changes the outcome.

6. Redundancy only works when causes are sufficiently independent

Two pumps on one bus, two computers carrying the same software defect or two sensors exposed to the same contamination can fail together. Review power, cooling, code, connectors, calibration, environment and operator procedure for common paths.

Redundancy only helps when paths are sufficiently independent. Two computers in one enclosure may share power, temperature, connector, software and configuration error. One common event can therefore remove both. Diversity may be introduced through hardware, software, sensing principle or physical location, but it creates its own verification and maintenance cost. The review question is not ‘how many units?’ but ‘which causes can still remove them together?’ For a settlement, repairability also matters: two non-repairable units may be less resilient than one robust unit supported by replaceable modules, test equipment and a bypass.

7. FDIR means detection, isolation and recovery

Detection recognises abnormal evidence. Isolation narrows the likely fault. Recovery chooses a safe or degraded configuration. Required reaction time varies: a slow leak permits analysis; loss of attitude control may demand automatic action in seconds.

FDIR means Fault Detection, Isolation and Recovery: recognise that behaviour has left the expected domain, isolate the cause or at least the affected zone, and recover a safe function. Each step can fail. A threshold that is too sensitive creates false alarms; isolation that is too aggressive can disconnect the healthy channel; automatic recovery can reintroduce the fault. Evidence should therefore be graded. One threshold may request independent confirmation, a second enter a degraded mode, and later verification may permit return to service. The objective is not to automate every decision but to keep the state understandable and controllable throughout the anomaly.

8. Safe mode is a stable survival configuration

Safe mode preserves the functions that prevent further deterioration: power, thermal control, attitude, communications and sometimes life support. It must remain viable long enough to diagnose and repair. A “safe” state that drains a finite resource too quickly is only a short transient.

Safe mode is a stable configuration designed to preserve resources and prevent escalation. It does not mean ‘turn everything off’. A spacecraft must still generate power, control critical temperatures, maintain an attitude compatible with arrays and communications, and protect the crew. Safe mode therefore has its own power, sensing, actuation and duration requirements. If it depends on the component that just failed, it is not a genuine refuge. Validation injects faults and checks that the system can reach this state with the resources actually left, then confirms that a path exists for diagnosis and recovery.

9. Maintainability lives in physical access and testability

Isolation valves, test points, modular connectors and access can reduce restoration time dramatically. A supposedly redundant box that requires two days of disassembly may still create unacceptable downtime.

Maintainability is designed before failure. Access time, panel mass, connectors, working volume, tools, test points, electrical isolation and EVA requirements can dominate MTTR. A component that is easy to replace on a bench may be almost unreachable once installed behind two fluid lines. Design reviews should therefore rehearse the intervention with representative tools and constraints, then include reconfiguration and verification time. On Mars, a locally repaired item can also create a new configuration that must be recorded. Resilience depends on proving return to service, not merely on completing the mechanical replacement.

10. Failure scenario: recovery creates a new common dependency

A first failure causes two services to be transferred onto the same backup bus. A later backup-bus failure now removes both. The common cause was created by reconfiguration. Resilience analysis therefore needs degraded configurations as well as the nominal architecture.

Two independent failures can become a common failure through procedure. If the crew loads the same bad configuration file into two replacement controllers, hardware duplication no longer protects the function. An ambiguous checklist can likewise lead two operators to isolate the wrong valve. Human and organisational reliability therefore belong in system analysis. Critical procedures need observable criteria, independent confirmation for irreversible actions where practical, and controlled version management. Lessons learned should update the authoritative document without erasing history so that a corrected error does not reappear months later on another system.

11. Mini-project: build a resilience case

1. Select a vital function.
2. Define its top unwanted event.
3. List five failure modes.
4. Identify common causes.
5. Describe detection, degraded service and repair.
6. Calculate one simple availability metric.
7. Add a second fault during degraded operation.

The case is useful when a reader can see what remains possible after failure and how full service returns.

Sources and references

Verified primary supplement: NASA Systems Engineering Handbook