2.3 Evaluation of the failures
In the failure evaluation phase, as a general rule, only one line in the FMEA form is evaluated at a time. If, for example, several causes of failure are listed in separate lines that are linked to one failure, only these failures must be considered individually. However, if the different causes of a failure are summarised in one line, the causes of the failure are evaluated together.
In general, the following aspects of failures are evaluated:
These three failure characteristics are assigned numerical values, which are used to calculate the risk priority number (RPN) by multiplying the three values together (see figure 5).
A modified FMEA is also possible, in which the three evaluation criteria O, S and D are assigned a non-numerical classification such as "low", "medium" and "high". chapter 6 Application example of a modified (simplified) FMEA provides an example of this.
It is essential that the three evaluations are performed independently of each other. For example, evaluation of the severity of a failure must not also include its probability of occurrence or detection. If the consequences of a failure lead, for example, to a S = 8, this number should not be reduced only because the failure only occurs once a year, for example.
The possible value ranges (scope of the possible numerical values) of the individual failure characteristics must be established specifically by the company. Recently, a certain standard has also developed within the pharmaceutical industry by which the values from 1-10 are permitted for each failure characteristic. However, it may be useful to restrict these value ranges for certain FMEAs (e.g. only permit values from 1-5). When determining the possible value ranges, it is always important to develop evaluation guidelines before an FMEA is executed, so that the significance of the individual numerical values is defined (e.g. what does a probability of occurrence of 3 mean? (example: "expected failure frequency ³0.05% and <0.1%") or a probability of detection of 6 (example: "The failure will probably be discovered (100% visual control is in place)").
Probability of occurrence (O)
For a risk, it is of high importance to determine how often a failure occurs or can occur. The more frequently a failure occurs, the higher the risk. This means, for example, that O = 1 might stand for a rare occurrence and O = 10 a very frequent occurrence. The probability of occurrence is generally determined by the cause of failure.
Figure 6 shows an example of an evaluation guide for the probability of occurrence.
In the example shown in figure 6, we can also directly see the problems that often occur when estimating probability of occurrence. In order to use this type of table, well-founded historical operating data must be available for a particular process or facility. If this historical operating data is not available, it can often be beneficial to reduce the level of detail by grouping together individual evaluations (see figure 7).
If it is also not possible to classify the probability of occurrence in this general evaluation, since no historical operating data regarding the probability of occurrence is available, this should initially be graded O = 10 (worst case). As new information becomes available, this figure can be reduced again.
Failure severity (S)
The severity of a failure is an essential feature for this assessment. The severity of the failure is generally determined by the consequences of the failure. It should be clarified in advance whether the failure severity only affects the "end consumer" (patient), or whether the failure severity for the next "customer" should be considered (internally, e.g. the packaging area may be a customer of the tabletting area). The latter is usually the option since it increases the stability of the processes and minimises the business risks which should be preferred (e.g. as a result of unusable products that need to be destroyed). The numerical value increases from 1-10 with increasing severity (see figure 8).
When creating a guide for the evaluation of failure severity, it is useful to take into account the effects of a failure in terms of staff, the environment and possibly data integrity issues.
Probability of detection (D)
When determining risk, it is important to know whether a failure is detected or will be noticed from the customer or the pharmacist.
The better the failure can be detected, the lower the risk. The numerical value thus decreases from 10 to 1, the higher the probability of detection is. This would mean that D = 1 is a value that, for example, can only be achieved if a fully automatic 100% test is integrated in the process or production process flow. D = 10 means that a failure is not detected (see figure 9). The probability of detection is generally determined by the cause of failure.
When evaluating the probability of detection, the test measures in particular that are already planned or designed at the time the FMEA is executed must be taken into account. If this is the case, they must be documented in the FMEA form together with the evaluation.
For example, in a tablet packaging process, if the completeness and integrity of tablets in blisters is constantly monitored by a control camera, this should be included in the evaluation. In this example, the probability that empty blisters will be detected increases, (proper functioning of the camera must be proven during qualification of the equipment), and hence the risk priority number decreases.
2.4 Evaluation of the risk priority number (RPN)
The risk priority number RPN is calculated by multiplying the values O, S and D.
Using the evaluation numbers described above, this results in RPN between 1 and 1000. In addition to the RPN, the company's own risk tolerance is also highly significant. This is the level at which the company sets the limits for determining measures. The company needs to clarify which RPN represents the critical level above which risk-reducing measures need to be implemented. You need to establish whether this should be just one limit or more, for example, two (see figure 10).
If the set limits are exceeded, the FMEA team needs to define measures that lead to a reduction of the RPN.
The top priority is assigned to measures for preventing or reducing the occurrence of the cause of failure. If this is not possible or not possible to a satisfactory level, suitable measures must be implemented to increase the probability of detection of the failure (e.g. through the use of electronic test systems).
All planned measures, including the planned implementation dates and responsibilities are documented in the FMEA form.
As a result of the planned measures, the RPN must be re-evaluated. This result is known as the potential RPN, meaning the RPN that can be expected on completion/implementation of the planned measures (see figure 11).
Some examples of suitable measures for reducing the probability of occurrence and increasing the probability of detection are listed below:
If the potential RPN is, even after definition of measures, still higher than the set limit, a decision is required at this stage, while involving the management, on whether the process or facility should be implemented or retained with the remaining risk.
Within larger projects, individual measures may often need to be adapted and re-evaluated. In these cases, the FMEA needs to be revised and new versions of the FMEA prepared.
In order to guarantee that the planned measures will be implemented, a designated member of the FMEA team should control/monitor the deadlines and implementation of all measures.
3 Further parameters for assessment of the overall process
Additional parameters that can be derived from an FMEA include the average RPN and the overall potential of a process.
The average RPN (see figure 12) is a parameter for evaluating the overall process. It can be used to provide a direct comparison between individual processes in terms of anticipated risks.
The overall potential (see figure 13) specifies the amount by which the average RPN is expected to be reduced after implementation of all planned measures.
4 Implementation of FMEA in the company
The implementation of a risk analysis in accordance with the FMEA method requires meticulous planning. Figure 14 shows an example checklist in which all important points for a successful implementation are listed in chronological order. The timeframe in which the individual points are to be processed is a project planning task for the implementation of an FMEA project.
5 Advantages and disadvantages of an FMEA
The FMEA as a methodology offers many advantages which mean that the FMEA has ultimately prevailed in many industry sectors in preference to other methods of risk assessment. The advantages of an FMEA include the following:
Quantitative evaluation of risks
The FMEA can provide a quantitative evaluation of risks. In the form displayed here, this results in a risk value (risk priority number, RPN) of between 1 and 1000, whereby the risk increases as the numerical value increases. The team or company itself can define at which level the tolerable risk and thus a limit is set (see figure 10). In general, this is a risk value between 80 and 125. The risk priority number (RPN) can be used to compare risks between processes, and also between different technical systems. This enables the available resources to be concentrated on the most risk-prone processes/systems and residual risk to be minimised.
Ranking of risks is possible
The determination of an RPN for each risk enables the creation of ranking lists. This can also considerably simplify the process of prioritisation. The decision regarding which aspects should be treated with priority is documented in a traceable form. For projects with a limited budget, this provides an efficient instrument for efficiently allocating available resources to critical risks.
Qualification aspects are defined at an early stage
Since the detection of failures (probability of detection) is an evaluation criterion within the FMEA, the test points IQ, OQ and PQ can be determined at a very early stage. This is of great benefit when planning resources for qualification.
Different levels of assessment
Depending on the progress of the project, the FMEA can be performed in more or less detail. This enables the critical processes/facilities to be classified at very early stage and then subjected to a more detailed assessment later in the process.
Interdisciplinary approach due to team members
The composition of the team who carries out the FMEA is a critical factor for the success of the FMEA. The cooperation of team members with different areas of knowledge (interdisciplinary team) enables the optimal result to be achieved. In order to ensure this, the team must work together towards the result, and the subsequent process owner (e.g. head of operation or laboratory), quality assurance, engineering and information processing departments must always be represented in the team.
Traceability of decisions
Each risk is evaluated and documented in the form in terms of probability of occurrence, severity of the failure and probability of detection, the route by which a particular risk assessment has been reached can subsequently be easily traced. When undergoing inspection by the authorities, it is also very easy to describe why some processes are considered in slightly less detail (or not at all) during the validation process while other processes have been assessed very intensively.
Evaluation guide for subsequent process/product/system changes
If, after the implementation of processes, products, systems, etc., these are subsequently changed, the FMEA created during the implementation can provide the ideal basis for the assessment of risk associated with the change process. The change itself should then also be evaluated in the FMEA. If applicable risk-minimising measures should be met and implemented.
Comprehensive method, also suitable for non-GMP risks
As explained in the previous sections, the FMEA is not only suitable for the evaluation of GMP risks. It can also be used for the evaluation of nearly all other types of risk (e.g. market supply risks, safety risks, environmental risks).
5.2 Disadvantages of an FMEA
In summary, the disadvantages of an FMEA include the following:
However, the first three disadvantages above can be said to apply to almost all other methods of risk management, and should therefore not be used alone as a deciding factor for not implementing the FMEA methodology in a company.
Training requirements before execution
Before an FMEA is executed, participants must be trained in the application of the methodology. This is indispensable, since the persons involved will otherwise be working with different perspectives and prerequisites within the group. This can lead to endless discussions and dissatisfaction of team members during FMEA meetings.
The company should therefore hold an FMEA training course in which the team members are familiarised with the method and safety is improved for the execution of an FMEA. When all members of an FMEA team are aware of what they are trying to achieve, FMEA meetings can be executed more efficiently. A training course should last for between 0.5 and 1.5 days, depending on the existing level of knowledge.
In addition to the FMEA training courses, one member of an FMEA team should be trained as a moderator. During the meeting, the moderator has the task of guiding the team towards a result. This doesn't mean that the moderator comes up with the solution, but rather that he helps the team to achieve the objective. The moderator must therefore be able to know and explain the method very well, and must always keep the objective of the FMEA in his sights. In the case of disputes within the team, the moderator can often reach a consensus.
If the FMEA is documented by using a software program, the moderator should also be responsible for this task, so that the rest of the team members can concentrate on recording the risks, their evaluation, and risk-minimising measures.
Implementation takes time
The implementation of an FMEA in the company should not be rushed. Staff first have to familiarise themselves with the method and practice it over an extended period of time before they are able to form a realistic picture of the effectiveness of the method. This process requires time, which is generally in short supply within a company.
If the implementation phase is too short, this often leads to dissatisfaction on the part of executives and staff. The former are disappointed with the results and the latter are over-burdened with expectations. The implementation phase can be shorter if the staff and the teams are provided with appropriate support, e.g. in the form of coaching or training courses.
The availability of resources must be guaranteed
In order to execute FMEA meetings efficiently, all members must be able to participate in the discussion. If this is not possible, loss of efficiency will inevitably result and the outcome will suffer due to a lack of resource availability. The FMEA meetings must therefore be prepared on time and all team members must be aware of the priority.
Quasi-objectivity of the RPN
Since the RPN (risk priority number) is a numerical value, one could incorrectly assume that it represents an objective evaluation of the circumstances. The RPN is, however, only a measure of the degree of significance of a risk. It is not down to whether the RPN is 25 or 35, but more whether the value is 25 or 120. It is always important to remember that the RPN is the result of a subjective assessment by the team. The fact that the opinion forming process is converted to a numerical value does not make it any more objective, but still partially remains a subjective estimation. This "subjectivity factor" can be reduced with the help of evaluation guides (see chapter 2.3 Evaluation of the failures), which are designed to facilitate the objective assessment process.
However, the fact remains that different teams will always produce slightly different results. This fact must be known to all involved, in order to avoid misinterpretations.
6 Application example of a modified (simplified) FMEA
The following is a description of how a modified (simplified) FMEA can be executed. Here the three evaluation criteria of "probability of occurrence", "severity", and "probability of detection" are not assigned the numerical values from 1 to 10, but instead are assigned numerical values from 1 to 3 or "low", "medium", and "high". This can be useful if a more detailed classification of the evaluation criteria into numerical values from 1 to 10 is not possible or not practical, perhaps due to a lack of experience with the process.
It is important to remember, however, that reducing the possible evaluation levels does not always mean that the project team will come to an agreement more quickly on a particular grade. In fact, it is often more difficult to agree on one grade out of "low", "medium" or "high" than to find a compromise between "9" and "7", (e.g. 8), since the latter difference is perceived as less significant.
The following is an example of possibilities for implementing an FMEA in the risk analysis of a computerised system.
The severity of the failure (S) is assigned the following values:
The probability of detection (D) is assigned the following values (in inverse sequence, since a low probability of detection increases the RPN more than a high probability of detection):
It is also possible to represent the potential higher risk (to the product or to the patient) of GxP-relevant functions in the system by assigning the GxP risks a double weighting compared to the non-GxP risks (see figure 15, "GxP-relevant?" column).
The "factor" value results from multiplication of the "severity of the failure" value and the "GxP-relevant" value (No = 1; Yes = 2). The RPN then results from multiplying the probability of detection and the "factor". The probability of occurrence of a failure in the computerised system is taken into account when defining the test scope (see figure 17)
Figure 15 shows that in this example, a potential high severity of failure in GxP-relevant functions with a low probability of detection leads to an RPN of 18, while a potential medium severity of failure for non-GxP-relevant functions with a low probability of detection leads to an RPN of 6 respectively (etc.).
For the determination of measures (e.g. the required test scope for computerised systems), the determined RPNs (which may lie between 1 and 18) can be grouped into four risk categories (see figure 16).
The evaluation of probability of detection (O) of a failure in a computerised system can be included, for example, by distinguishing between "standard functions" and "additional functions developed for the customer" (custom developments). This reflects the fact that failures occur significantly less frequently in standard functions than in custom developments, due to a wide distribution and greater experience with the standard functions. The test scope for custom developments should therefore be higher than for standard functions.
The assignment of risk categories shown in figure 17 for the required test scope demonstrates how a (modified) FMEA can be applied to control (and justifiably reduce) the required workload for computer validation.