Operation of computerised systems

9.F.1 System description

It is a requirement of Annex 11 of the EU-GMP Guideline to create a system description. This is most useful if integrated into the validation documentation.

9.F.2 User training

As emphasised in chapter 2.9 of the EU-GMP Guideline for Good Manufacturing Practice for Pharmaceuticals (see chapter C.4 Part I Basic Requirements for Medicinal Products), training is also extremely important for computerised systems. In addition to a thorough introductory training course, more advanced training should also be carried out. This should aim to increase knowledge of the computerised system and discuss more efficient use of the system.

Most computer faults can be attributed to incorrect input or poor data quality. In this case, all employees, and not only those directly affected, should be retrained. In addition to training courses, however, improvements to user requirements or improvements in the system should also be considered.

Training courses are also often based on instructions that are explained to the users.

9.F.3 Standard operating procedures (SOPs)

The standard operating procedure for users should be as detailed as necessary to allow execution of the required work. To do this, it is helpful to illustrate the explanations using printed screenshots. Screenshots can be very easy to create, simply by pressing the PRTSC key and then clicking "Insert" in the text processing application.

For more details on the topic of SOPs, refer to chapter 15.D Standard operating procedures (SOPs). The following SOPs should be provided for users of computerised systems.

For the validation and development of computerised systems:

1. The validation of GMP-relevant computerised systems (see chapter 9.E Validation of computerised systems)

2. Risk analysis (see chapter 9.D Risk analysis  and system classification)

3. The compilation of user requirements, functional and system specifications (see chapter 9.E.3 Specifications (user requirements/technical specification) for hardware and software)

4. Planning and execution of tests (see chapter 9.E.4 Unit, integration and acceptance tests)

5. Retirement of computerised systems (see chapter 9.F.9 Retirement of computerised systems)

For the operation of computerised systems:

1. Training for handling GMP-relevant computerised systems (see chapter 9.F.2 User training)

2. Authorised access and access protection (see chapter chapter 9.F.4 Authorised access and security (virus protection))

3. Change management (see chapter 9.F.7 Change management and error reporting)

4. Data backup and archiving (see chapter 9.F.5 Data backup and archiving)

5. Error handling - one SOP per system (e.g. materials management system, laboratory information management system) (see chapter 9.F.7.2 Error reporting)

6. System failure, error handling and recommissioning of a computerised system (see chapter 9.F.6 Contingency plans and data recovery)

7. Periodic review (see chapter 9.F.8 Periodic review)

9.F.4 Authorised access and security (virus protection)

9.F.4.1 Authorised access

When granting authorised access, the following aspects must be determined:

• Which prerequisites must be fulfilled before a user is permitted access to an application
• How the person is informed of the authorised access
• What the person should do if they can no longer access the system
• What the application operator should do if there is evidence of unauthorised access attempts, and
• How the different business scenarios regarding authorised access are documented.

Normally, each person should have an individual password; however, there are still some applications for which this is not possible. In this case, users have to log on indiscriminately using a group password. For example, if the control system for a tablet press only offers the three users line worker, manager and engineer, the use of group passwords is unavoidable. In this case it is important that full traceability remains possible for documentation purposes. Furthermore, there are some pieces of equipment and controls for which no passwords and individual identification codes are used, or for which measures of this type would be somewhat excessive, e.g. washing machines, pH meters, balances, and so on.

Prerequisites for authorised access

In modern applications, different authorisation levels are defined, at which different commands are available, depending on the level of authorisation. It is important that the user has participated in the appropriate training course before the authorisation is assigned. In accordance with their job description and education, the user must also be capable of performing the tasks assigned to them.

Information on authorised access

The user should be informed of an access authorisation in a way that ensures this information does not fall into unauthorised hands. If they know the user, the system administrator can inform the user of a password over the telephone. The information can also be passed on by e-mail, as long as it can be ensured that the message is not forwarded automatically. Certain e-mail systems have the option to mark messages as "private" to prevent them from being forwarded to deputies. However, the safest method is always to send the password by land mail, although the information that the letter contains a password should obviously remain discrete.

Loss of authorised access

Passwords can sometimes be forgotten. To make life easier for the user and to facilitate easier administration of passwords, if possible only one access system should be used and therefore only one password. Forgotten passwords must be reported. The user must then be informed of their reset password via an established procedure.

Abuse of authorised access

The most common abuse occurs if a user has forgotten their access code. In this case, the user account is locked following a defined number of attempts. Most systems permit two incorrect password attempts before an account is locked, however, some systems allow 10 or 15 incorrect password entries. This does not have a significant influence on security, but reduces the likelihood that a user can intentionally lock another user's account. It is also more likely that password sabotage of this type will be detected.

Documentation of authorised access

The documentation of password assignment is subject to the same prerequisites as all other documentation in the regulated environment: Traceability. It must always be clear which user has been granted or has lost which authorisations at any point in time. Users are often administrated in tabular format, if a database is not used for this purpose.

9.F.4.2 Security

Nowadays, the main challenge in terms of security is virus protection. Other tasks, such as storing the documentation in fire-proof premises and the relocation of backup files for storage at outside locations are also important, but these can be realised relatively easily and are not as time-consuming.

Important questions for the implementation of virus protection are:

• Should virus protection be installed on a validated system or not?
• Does the system have to be revalidated after installation?
• Does the same apply for virus scanners?
• What factors have to be taken into account for the virus scan?

Certain operating environments are particularly susceptible to virus attacks, although they are widely used, especially since manufacturers often supply security updates. All vulnerable systems that are integrated in a network should be equipped with virus protection. Older models, particularly those without a graphical user interface, tend to be more immune to virus attacks. Standalone systems that are not part of a network are also only rarely subjected to virus attacks. If the disk drives and open interfaces (USB) of this type of system are also locked, even the most persistent of viruses will not be able to attack.

All types of virus protection are associated with hidden dangers:

• Production data may be deleted because it coincidentally matches the signature of a virus.
• Virus protection interferes with certain interfaces and ports required for communication with the machine.
• Virus protection suppresses certain programs that are important for correct functioning.
• The virus scan consumes considerable system resources, meaning that measurement results can no longer be processed in sufficient real time.

The risk of these dangers is usually very low. In addition, less and less time is available for installing virus protection before the first exploits or viruses appear. In some cases, even on the same day that a security breach is noticed, a virus appears that exploits this deficiency. These are known as zero-day exploits. Most system operators therefore implement virus protection, since it is better to implement an uncertain low risk at a particular time than to leave the doors open to an uncertain risk at any time with unknown consequences.

No validation plans are compiled for virus protection. The only validation activity is often a positive function test. Otherwise, the virus protection and related updates are recorded in the log book or log file just as a normal maintenance activity in a traditional mechanical system.

Virus protection must be continually updated. Users should be informed of the updates. Users usually notice unusual system behaviour during their daily work in the system. The virus scan should not be performed during production time, but should be executed during breaks, clean-up or maintenance periods.

9.F.5 Data backup and archiving

These activities are normally performed by the system operator, although for standalone systems in production or in the laboratory, they can also be executed by the user or system owner.

9.F.5.1 Data backup

Backups provide protection against data loss in the case of failures. A backup consists of the restoration, i.e. recovery of a) data, b) programs and c) the correct settings for the previous configuration (configuration management). In order to avoid overlaps and gaps in the backup, it is important to know which backup method was used when, and by whom, for all systems. The backup measures should also be checked periodically to ensure that they are functioning correctly. The execution of a backup should be documented.

The following points should be defined:

• Time of the backup: Backups are normally performed overnight. In some systems, users must ensure that all files to be backed up are closed, which means that the backup should be preceded by a check.
• Frequency of backups: Depending on the criticality of the data and the system stability, more frequent backups should be considered, although normally it is sufficient to back up once per day.
• Type of the backup: Backup systems usually differentiate between:
• Full backup: All data, programs and system parameters are backed up. If data is subsequently restored, the complete system status can be restored. However, a full backup requires a large amount of memory capacity and is time-consuming, which means that it is only practical to carry out full backups on a regular basis for smaller systems. Nowadays, a common form of a full backup is to take a disk image, in which the whole contents of the hard disk are transferred to a different hard disk, sector by sector.
• Data backup: All data is saved. Backing up all the data can also be a relatively time-consuming process. The data can consist of files, configuration data, database extracts, or similar.
• Incremental backup: In this case, only data that has been changed is saved. This backup technology is the fastest, although it can become tricky if data has to be restored.
• Normally, a combination of incremental backup and data backup or full backup is used. An incremental backup is performed during the week, with a full backup at the weekend.
• Storage media: Depending on the storage medium used, data may have to be copied. For magnetic storage media, it is recommended to recopy data every two years, since the quality of magnetic information decreases over time. Optical storage media must be stored carefully to ensure that harmful solvent fumes do not impair their quality. Short-term storage can be handled with the use of magnetic media, while optical media are more suitable for long-term storage.
• Review mechanisms and time intervals: It must be verified on a regular basis that the data backup has functioned correctly. At periodic intervals, it must be checked that the old data is still legible. In addition, a recovery exercise should be used to demonstrate that data recovery processes also work in practice as well as in theory.
• Procedure in the event of an error: Appropriate measures should be established depending on the backup software used and the possible errors. In particular, these also include the repeat of unsuccessful backups and the information communicated to the system owner, quality assurance and users in the event of a failed backup.
• Storage location of the storage media: The storage may differ depending on the type of backup. Incremental backups are stored directly in the same location, while long-term backups are usually stored externally. Archived data that is normally in an archive format that has been subject to few changes can also be stored externally .
  For very sensitive study data, redundant data storage should be considered from the beginning, in which the data is recorded and stored independently in two different locations.
• Retention period: This depends on the type of backup and should be as long as the period in which the affected data may still have to be restored. For daily data, for example, it may be necessary to store this for a period such as 30 days before it is overwritten, in order to enable recovery of this data in case it is not noticed to be missing until several days later.

9.F.5.2 Archiving

Archiving is used to describe the retention of data that is currently no longer required at a secure location, from where it can be retrieved if necessary.

While there is currently no standard solution available for the archiving of electronic records, certain principles should still be complied with. The simplest and most secure method is still archiving in paper form, due to hundreds of years of experience with this method. However, many years of knowledge and experience still cannot prevent errors from occurring, for example the acid degradation of chlorine-bleached paper. Only a few years of experience have been gained with electronic data archiving, which means that no old and tested practices are established.

The system owner defines the retention period, access, and the type of archiving in collaboration with quality assurance and the system operator. In the GMP area, the required retention period differs according to national specifications. From a product liability perspective (EU Directive 85/374/EC, Art. 10 and 11), a suitable retention period of fifteen years is recommended. However, in the area of blood preparations, there is an obligation to retain data for 30 years. This period extends far beyond the product liability recommendation of 10 years following the last occasion on which a product was placed on the market.

9.F.6 Contingency plans and data recovery

This term is used to describe the recovery of a system (disaster recovery) and the alternative plan (contingency planning) in accordance with the GAMP Guide 4. The existence of a contingency plan (business continuity) is also often a requirement for inspection by the authorities and audits.

Contingency plans describe how to proceed in the event of a system failure and how the system can subsequently be recovered. During the risk assessment, the importance of a system is also determined (see chapter 9.D Risk analysis  and system classification).

The following table (figure 9.F-1) shows an example of this type of contingency plan. In addition to the points listed in the table, it is important that the user and the system owner are also informed and that all events are documented. The system operator is normally responsible for contingency planning and repairs.

9.F.7 Change management and error reporting

9.F.7.1 Change management

A distinction is usually made between planned and unplanned changes (= deviations), whereby an unplanned change may be a corrective action that has to be executed very quickly. For changes that are implemented as corrective measures, it is therefore possible to seek approval afterwards ( chapter 19.C Change control and chapter 11.K Deviations).

For certain system categories and groups, it can be useful to establish separate standard operating procedures for change management, since responsibilities can then be customised to suit the relevant system. An SOP for change management is expected to address the following points:

Change request - to be completed and signed by the applicant:

• Identification of the systems: Name, version, location, department
• Description of the change: Reason for the change, planned or unplanned

Analysis of the change - to be compiled by the system operator and approved by the system owner and quality assurance:

• Risk evaluation, GxP relevance, costs, time investment, business case (how expensive is the change and what are the benefits)
• Measures: None, training, function tests, validation according to IQ, OQ, test plan no. or other measures

Execution of the change - to be compiled by the system operator and released by the system owner and quality assurance:

• Comments, documentation, tests, adjusted documents (SOPs, specifications, test documentation, etc.)

9.F.7.2 Error reporting

All computerised systems have minor or more serious faults. It is usually only a matter of time until these appear and are detected. The older a computerised system is, the fewer faults it has.

Since errors can appear repeatedly, it is important that they are handled in an organised manner. This can result in the creation of a separate fault reporting system or - more usefully from a practical perspective - the fault reporting can be integrated with change management and the error failure investigation procedure (see chapter 1.C.2.2 Processing of changes and deviations).

The following shows an example of a fault report (figure 9.F-2).

Investigation of faults

For the investigation of errors, it depends in which stage of the software life cycle the error has occurred.

Errors that occur during design, programming and testing are normally treated using the investigation methods specified in the software manufacturer's programming and design guidelines.

Errors that are detected during operation should initiate an investigation in the same way as other problems. This is described in detail in chapter 11.K Deviations.

It is important that for each fault and each investigation, the affected product or the affected study are assessed.

9.F.8 Periodic review

In the periodic review, which is executed with a defined frequency - e.g. every three years - the system operator and the user should check that all documentation is up-to-date and accurate. Since many small changes can accumulate over time, it may be necessary to create one or more new documents.

The system operator reviews the following:

• Changes to the validation plan and specification documents
• Changes to the objectives and system scope
• Up-to-dateness of the baseline, i.e. do the version and configuration still correspond to the documentation, or have so many changes of this type been made that a new baseline needs to be defined?
• Whether a tendency towards declining system performance and long-term planning require that certain data is stored externally
• Whether all change requests are updated and stored in an organised fashion
• Whether user and usage profiles are still correct

The user reviews the following:

• Whether the SOPs are up-to-date
• Whether department names are still correct

Quality assurance checks:

• That all investigations have been completed
• Whether a supplier audit is required

At the end of the process, the Periodic Review Report is signed by all participants and by the system owner.

9.F.9 Retirement of computerised systems

Retirement can either be handled as a change, or according to a separate SOP. The important points are as follows:

1. Definition of responsibilities for retirement - proposals are suggested in the individual steps.

2. The system owner, or the person or organisation who sponsors the system, defines the time of shutdown in agreement with the operator and the user.

3. The system owner informs the users of the impending shutdown.

4. The system owner arranges for the system to be checked for data that requires archiving and data that may have to be migrated. This not only includes data that has been stored in the system throughout its operational life, but also records of the system itself, such as validation documentation and test reports.

5. The system operator arranges for the termination of maintenance contracts.

6. The system operator shuts down the system. Which individual steps are involved in the shutdown process depends on the application. For a simple spreadsheet, it is sufficient to delete this from the working data medium and transfer it to an archive data medium. For chromatographic data systems, in addition to data migration and tests, it may also be necessary to deinstall hardware and software components.

7. The system owner invalidates the user operating instructions.

8. The system operator invalidates the system operating instructions.

R.D. McDowall: Chromatography Data Systems V: Data Migration and System Retirement LC.GC Europe, January 2000.