Follow the Audit Trail Breadcrumbs Audit Trail Reviews Crucial for Maintaining Data Integrity

Data integrity is a hot topic for the U.S. FDA and other global regulatory agencies. Two crucial aspects, in particular, have been cited by regulators: audit trails and audit trail reviews.

Both are used to confirm the correctness of data. Per 21 CFR Part 11, controls for electronic systems should include: “secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes shall not obscure previously recorded information.” Without this, it is not possible to reconstruct the sequence of events documenting the who, what, when and why.

Yet these regulations do not provide specifics on what information should be in an audit trail or define how to perform an audit trail review. As a result, there is significant confusion around how to conduct an audit trail review. It is not uncommon to see minimalistic audit trail reviews that lack appropriate information, such as where the audit trail is located. It is commonly thought that every piece of data needs to be reviewed. Some companies resist conducting audit trail reviews as it is believed to take up too much time during routine laboratory operations.

Yet companies do not realize that often what is needed for an audit trail is already covered as part of existing data review/approval.

Another common thought is that one audit trail review process can be performed for all instruments. But instruments vary in terms of analyst and audit trail capabilities; therefore, one set of criteria cannot be applied generically across the board. Also, the characteristics of an audit trail differ among instruments. Thus, audit trail reviews for each instrument need to have specifics in terms of what to review and where to find the audit trail.

Step-by-Step Audit Trail Review Process

It is helpful to understand the FDA perspective. In general, the audit trail review needs to:

• Be part of the routine data review/approval process
• Look for abnormalities or inconsistencies with the generation and/or processing of data
• Be completed prior to final approval or release of data
• Be conducted by someone independent who knows the instrument

Both U.S. and EU regulations state it is acceptable to use a risk assessment when defining the audit trail review based off of the potential effect on product quality, safety and record integrity. The audit trail review process requires the following steps:

• Determining the available information in the instrument’s software and/or other utility that maintains the instrument data to ALCOA+ standards
• Defining the critical information
• “Critical information” is defined as that which establishes the who, what, when, and why in the audit trail
• Defining the audit trail review based on the defined critical information

As part of an audit trail review, the user, time and date of an event or action must always be verified. To define additional critical information, some key questions about the extent to which analysts can alter parameters, methods and data should be asked.

• If an established nonmodifiable method exists, was it verified that the correct method was chosen?
• Was the parameter choice confirmed for situations when analysts create or modify such parameters?
• Were parameters applied correctly during integration of highperformance liquid chromatography chromatograms?
• Does the test match what is defined in the SOP?
• Were parameters applied correctly during integration of high-performance liquid chromatography chromatograms? T his part of the audit trail review includes confirming that samples are used appropriately, e.g., samples are not used for conditioning the column prior to the run.
• Does an analyst have the ability to modify or delete records? If so, then the modifications and deletions need to be reviewed along with an accompanying reason and confirmation that no suspicious patterns are present. If the analyst cannot do any of the above, the only item to review in addition to user, time and date is that all records have been reported (i.e., there are no duplicate or trial/unofficial records).

Once the necessary information is identified, it must be found in the audit trail. Some instruments may not have what would be considered a typical audit trail—in these cases, the necessary information is often found in unexpected places. For instruments with controlling software that do not have any intrinsic audit trail capability, a secondary program to capture the who, what, when and why of actions can be added to the instruments. Some instruments create a nonmodifiable file. If these files are saved directly onto a locked server where they cannot be deleted or moved and the clock is locked, the audit trail review consists of verifying the user creation of files with the specified date and time along with confirmation that all records have been reported.

All audit trail reviews must include a statement outlining who performed the review and the date it occurred. There must also be a statement that no issues were found. To avoid additional forms, a statement can be added to the instrument SOP noting that that the reviewer has performed the audit trail review per the SOP and no discrepancies were found. In an R&D setting, this may be acceptable, but for manufacturing, a separate form as part of batch record may be a better option. Either way, the form must be based on what is reviewed in the audit trail for the given instrument.

To ensure a proper audit trail review, analysts who generate data must have duties separate from administrators who can establish accounts and transfer/delete data when appropriate without an audit trail. If analysts can modify or delete files without documenting the change, this must be addressed before establishing an audit trail review for the instrument. Defining the data needed as part of the audit trail review during validation, establishes the necessary data controls to maintain ALCOA+ expectations.

A few final points. Audit trails and audit trail reviews apply to analytical instruments and manufacturing equipment. As each instrument is unique, a helpful option is to place the audit trail review process within the instrument SOP rather than have one overall audit trail review procedure. The steps outlined above apply to an audit trail review during data review which is typically performed each time data is released/approved. There is also an expectation to perform an audit trail review at the system level that evaluates modification of locked methods and administrator modifications, such as user access and privilege modifications, data deletion and archiving. This can be done periodically (e.g., annually) rather than for each dataset.

The audit trail review is the last check to prevent unsuitable product from being released. As a real-world example showing the importance of an audit trail review, a company a few years back received many lack of effect complaints on a lot of product. It was determined that the lot was made half as potent compared to its label claim due to an unintended error.

Why was this not caught during release testing? When reviewing the audit trail for the release test data, it was found that the analyst tested the lot and the result came out half as potent. The analyst retested the lot where it passed, and the lot was released based on this second passing result. The first result was not reported or investigated. It was only after the numerous lack of effect complaints that the error was found. In this case, the audit trail showed the duplicate test, but no audit trail review was performed during data review. As a result, patients received subpotent product. Yet an audit trail review would have identified the extra test result and a subsequent investigation would have found the manufacturing error prior to releasing the lot.

With the increased focus on data integrity and common observations for audit trails and audit trail reviews, it is imperative to have robust audit trails and audit trail reviews. After all, the audit trail review is the last step to ensure safe and effective products are released.