How to Measure Quality Management Maturity

In January 2021, the PDA Quality Metrics and Culture Task Forces were combined and renamed as the PDA Quality Management Maturity (QMM) Task Force. Our members continue to explore the relationship between quality culture, quality metrics, and quality systems and how they collectively drive QMM. This refocusing also aligns PDA with the U.S. FDA’s recent discussions and pilot programs for assessing QMM. The QMM Task Force believes its body of research, gathered over several years and confirmed with additional research by the University of St. Gallen Institute for Technology and Enterprise Management (ITEM), positions PDA well to support industry and FDA in efforts to advance the development of QMM. This article provides a brief overview of the earlier research and highlights the most recent work of the Task Force in evaluating which quality system elements have the greatest impact on the assessment of QMM.

Background

PDA published its “Quality Culture Survey Report” in the PDA JPST in 2015 (1) and presented the findings at the “Joint PDA FDA Quality Metrics Conference” that same year. In addition, PDA established a research collaboration with the University of St. Gallen. In 2018, a joint venture between the University of St. Gallen and members of the PDA Task Force used the PDA survey results and the St. Gallen’s OPEX database to publish a second paper. This analysis further confirmed a “positive and significant correlation between quality (culture) behavior of a production site’s employees and quality (system) maturity, which represents the maturity of the quality system in place” (2).

PDA conducted further analysis of their early data to determine which of the quality (system) elements (previously referred to as metrics) had the most significant correlation and impact on quality (culture) behavior. PDA’s work led to the development of the Quality Culture Assessment Tool, which identifies five maturity levels across each of 21 maturity elements within pharmaceutical manufacturing operating systems. PDA launched its “Quality Culture Assessment Model” in 2017 and, thus far, has trained more than 100 FDA CDER and ORA staff members as well as MHRA Investigators on how to do it. In addition, more 30 companies have used the PDA Model and Survey to assess Quality Culture at a total of 50 different manufacturing sites, and have returned their data to PDA for use in the industry benchmarking analysis.

Table 1 Outline of PDA Quality Culture Assessment Model

St. Gallen has an extensive database of OPEX survey results from pharmaceutical manufacturing sites. Applying the PDA hypothesis to this dataset confirmed the conclusion that increasing the implementation of mature quality system attributes leads to more positive observed behaviors. St. Gallen’s conducted additional research for FDA that resulted in three FDA Quality Metrics Research reports being issued between July 2017 and December 2019. In the quality culture section of the final report, St. Gallen identifies a “top ten” collection of attributes most strongly differentiating sites based on their quality maturity score (see Table 2) (3).

Comparison and Analysis of Maturity Models

At the start of 2021, the first order of business for the PDA QMM Task Force was to see if the most critical elements to establishing and maintaining a mature, efficient quality management system could be identified. Using this information, the task force would determine if there were any updates needed to the current PDA model. They began by comparing PDA’s 21 mature quality culture elements with St. Gallen’s top ten quality maturity attributes and the QMM attributes identified by FDA to provide more visibility and support for using quality maturity as a basis for risk-based inspections.

Half of the St. Gallen–FDA top ten clearly align with PDA’s 21 mature quality elements as shown in Table 3. The overlap between the PDA’s and the University of St. Gallen’s research (Items A–E) represents key areas for QMM assessments as they identify mature quality behaviors both from a quality culture perspective and an OPEX perspective. Interestingly, the St. Gallen attributes consistently align with the higher end of the PDA maturity levels, at Level 3 or above, for the corresponding elements.

Five of St. Gallen’s top ten attributes did not directly correlate to PDA’s mature quality elements. While there was some overlap between the PDA and St. Gallen models with the second five attributes, the relationship was not as clear as with Items A-E above. The QMM Task Force discussed whether or how these additional attributes could be addressed by the PDA tool with the following outcomes:

1. “Optimizing set-up and cleaning procedures are documented as best practice process and rolled out through the whole plant.”

While the PDA model does not specifically describe set-up and cleaning procedures, the concept is covered in the Process Ownership element covering major programs within the site. Level 4 states that “process owners are actively collecting feedback across the organization as input to continuous improvement.” The task force believes that ownership of the set-up and cleaning procedures meeting the PDA element Level 4 would lead to best practices and be rolled out throughout the whole plant.

2. “All potential bottleneck machines are identified and supplied with additional spare parts.”

This St. Gallen attribute is covered in PDA’s Technical Excellence element. Level 2 of the Manufacturing Technologies element talks about how the “firm is trying to maintain equipment beyond its typical useful life and breakdowns impact productions schedules.” This would be the potential bottlenecks described in this St. Gallen attribute. Level 2 in the PDA model is a relatively low level of maturity. The case study that accompanies the training of the tool discusses establishing a “bone yard of spare parts” to prevent such bottlenecks. The higher maturity level of this element talks about “Proactive investments as a part of standard operating plans” as the best way to address them.

3. “For product and process transfers between different units or sites, standardized procedures exist that ensure a fast, stable and compliant knowledge transfer.”

This St. Gallen attribute is also a very specific task that would be part of the PDA element on Process Ownership and Engagement in which the process owner for product and process transfer has established a robust program and drives further continuous improvement to that program (also see item 1).

4. “Manufacturers have joint improvement programs with suppliers to increase performance.”

This St. Gallen attribute points to an apparent gap between the two lists. The PDA tool is very site-specific and does not address relationships with suppliers or between multiple sites. However, the task force believes the internal culture of an organization carries over to its relationships with suppliers and customers, and predicts improving systems, behaviors and performance internally would result in greater maturity in interactions with suppliers. There is an opportunity for the PDA task force to explore a maturity model on supplier management or add to the current PDA model if industry or health authorities believe there is value in addressing it.

5. “The firm ranks its suppliers and conducts supplier qualification and audits.”

There is a gap between this St. Gallen attribute and the PDA model similar to Item 4 above. Addressing this gap could be integrated into any discussion of adding a supplier management attribute or element to the PDA model in the future.

Conclusions and Next Steps