End User Validation Requirements for COTS Software

by Mark Matis
May 1999


Commercial Off-The-Shelf (COTS) software applications are routinely used in the pharmaceutical industry. COTS software applications include any software applications that are configured for use for a specific function without altering the basic program. In other words, the off-the-shelf software can be used by the end user with little or no adaptation. The four main reasons for using these types of software programs are:

A frequently overlooked element for COTS use is the end user validation requirements. This article will discuss the validation requirements for COTS software, provide points to consider during the product selection phase, discuss some of the common pitfalls and misconceptions associated with COTS applications, and define the documentation and quality systems necessary to achieve and maintain an adequate validation status for a COTS system.

Regulatory Requirements

Validation of computerized systems used in producing, managing, and reporting data for pharmaceutical activities is required by the US Code of Federal Regulations, Title 21, parts 210 and 211, as well as in other related pharmaceutical areas which will not be included in this article. A recognized problem area in software validation includes the end user validation requirements for COTS applications. A point repeatedly made during audits is that the practice of "validation through use" is not acceptable, and it is the end user’s responsibility for the validation of a software application prior to its release for production use in the FDA-regulated environment.

Two widely recognized references for computerized system validation, the PDA Technical Report No.18, Validation of Computer-Related Systems and the GAMP (Good Automated Manufacturing Practice) Guide for Validation of Automated Systems in Pharmaceutical Manufacture, present widely accepted and recognized validation concepts and procedures. These reports draw from the essential steps in the life-cycle validation approach, all of which should be evaluated and interpreted in any computerized-system qualification project. Much of the development, design, and test requirements are completed by the vendor; however, it is the end user’s responsibility to verify that the vendor has provided the application in accordance with these defined procedures.

Vendor/Product Audit

Planning for and definition of a computerized system project should include careful and well-thought consideration of the four main areas of concern - software, hardware, quality systems policies and procedures, and controlled function. From the COTS perspective, defining adequate qualification activities is essential. The user requirements, system specifications, and validation requirements for all computerized systems must be defined as if the application were to be custom-developed by the end user, and these requirements and specifications should be referenced during the vendor and product audit process. Validation of a software application by a third party outside of the actual production environment is a practice that is not acceptable by regulatory standards. Also, interpreting a "certificate of validation" from a software vendor as an adequate level of validation is not sufficient.

Qualification Activities

The user should define validation procedures and requirements prior to performing qualification activities. These documents are the cornerstone documents in any validation effort, and they should be developed in clear, concise terms. Establishment of these documents will result in the development of concise qualification tests and verifications to demonstrate the proper functionality of all defined user requirements, business practices, and functional requirements for the computerized system.

Establishing policies and procedures to define responsibilities and requirements for any type of computerized systems implementation and ongoing support are essential. COTS applications range in complexity from the basic "one person" development/test/management systems to integration/implementation teams involving multiple departments, locations, third-party system integrators, and defined cross-functional implementation teams. Implementing adequate quality systems and procedures for all levels of computerized system validation efforts should be completed to maintain the validation status of the computerized system.

Common Pitfalls

Providing general references to vendor-provided documentation (i.e., User Manuals) for COTS should not be performed by the end user. Although reference to provided manuals is an accepted practice, specific rather than general reference to documentation is necessary. Also, a thorough review and approval of any document designated as a system specification by the end user should be required. Vendor-supplied information generally provides instruction and specifications for all functionality of the application, but without proper configuration, no assurance of accuracy can be assumed or implied. Therefore, a general statement of acceptance of vendor-provided User Manuals, and a lack of documented user requirements and system specifications, should be avoided by the end user. Without a formal document stating what the validation requirements are, no clearly defined point of completion is established and no clear state of validation is defined.

Without adequate testing requirements and defined specifications for any software application, creating complete and unambiguous tests and acceptance criteria is not possible. If the approach is that the vendor has provided a "validated" product, it is the end user’s responsibility to verify that documented evidence exists for the operation of the software application as it is installed and configured in the production environment. Part of the vendor/product audit process should include an assessment of software development, testing, documentation, and control procedures established by the vendor, and assurance that these polices and practices are enforced by the vendor. By definition, a system cannot be validated outside of the intended production environment that it is to be used in.


The end user is responsible for the validation status of a software application as it is installed in the production environment. Computer-related system validation, as defined in the PDA Technical Report No. 18, is "establishing documented evidence which provides a high degree of assurance that a specific computer-related system will consistently operate in accordance with pre-determined specifications." To accurately assess the validation status of a system, the validation requirements must be clearly defined and documented. To maintain the state of validation of a system, the end user is responsible for establishing and implementing supporting quality systems and procedures. Two essential points to be verified during a vendor/product audit are verification that the vendor has provided the software product in accordance with defined development policies and procedures and verification that the supplied application will meet the predefined user requirements and system specifications. Validation efforts completed by the end user should be completed to demonstrate that the installed and configured application meets predefined end user requirements.

Copyright © 2018 ISPE