On December 22, 2021, FDA published a request for comments on the newly issued draft guidance for industry entitled “Validation and Verification of Analytical Testing Methods Used for Tobacco Products” in the Federal Register (86 FR 72603). The recommendations provided address analytical testing of tobacco product constituents, ingredients, additives and stability testing of finished product. This guidance is to provide information and recommendations on how tobacco product manufacturers can produce validation and verification data for analytical methods used to support regulatory submissions for premarket approval. When finalized this draft guidance will represent the current thinking of the agency.

We applaud the agency for its decision to provide guidance to laboratories and other stakeholders to inform method validation and verification of analytical methods. This will aid in the standardization of assays with established accuracy, precision, repeatability, and reproducibility, resulting in the generation of data that can be used to make product comparisons and regulatory decision making.

McKinney Specialty Labs submitted the following general comments listed below to the agency with the goal of providing clarity and strengthening the draft guidance document. Read our full comments, including an additional 18 specific comments to Docket No. FDA-2021-D-0756.

Specific areas and/or actions that FDA CTP should contribute to include:

  1. Finalization of the HPHC list for ENDS e-liquid and aerosol. Potential additions were posted for comment in 2019[9] but no further action has been apparent since then.
  2. Creation of HPHC lists for the newer-to-market products such as HTP and modern oral products. These products feature very different matrices relative to traditional tobacco products, likely contain or emit, respectively, far fewer HPHCs, and as such warrant individualized HPHC lists.
  3. Involvement in the creation of standardized methods for the newer-to-market products. FDA CTP should increase its involvement with collaborative bodies to develop standardized testing methods for existing and any newly added HPHCs. It should also encourage independent testing facilities to adopt these new standardized methods when generating data for FDA submissions. Such actions can only serve to increase FDA CTP’s confidence in the quality of the data it uses for regulatory decision making.
  4. Provide guidance and expectations for aerosol sample generation. The utility of the data FDA CTP receives for making cross-product analyses in aerosol-generating products is governed by how comparable the aerosol generation is between those products. It is in FDA CTP’s best interest to have greater involvement in developing standardized methods for aerosol generation under conditions that provide the most valuable information for assessing a product’s potential impact on public health.
  5. Provide guidance for the study of long-term method variation. For stability studies, where HPHCs are measured over time, it is important to understand any temporal variation that may occur in an analytical method so as not to convolve it with test sample variation.[10]

3. FDA should identify suitable certified reference productsThe provided guidance states that “a validation of the ASP should include comparative measurements of sample extracts/smoke from a certified reference product (e.g., 1R6F, CM8, CRP1, etc.) and of the tobacco product category that is the subject of the validation.” This recommendation is wholly valid and good practice, but it does not acknowledge that no such reference products exist for the newer-to market products. Furthermore, what would the reference product be for aerosol-generating products such as ENDS and HTP? The device itself, the nicotine-containing component (e-liquid or tobacco substrate), or a combination of both? To add further complexity, both systems are available in a number of types – tank-based (open, refillable), pod-based (closed, exchangeable), and disposable (closed, sealed) for ENDS, and electrically-heated (eHTP), aerosol-heated (aHTP), and carbon-heated (cHTP) for HTP.[11] Should one device type for each product class be used as a comparator for all types, or should a reference product for each product class variant be developed? CORESTA recently published a collaborative study report on the evaluation of a reference device (Aspire NautilusTM tank with an EvolvTM power unit) for e-cigarette aerosol generation[12] that it intends to use as a monitor device in future aerosol studies. Whether this device becomes a reference product for wider use remains to be seen, but it represents a step in the right direction.

If reference products are to be created for the more newly developed nicotine product types, then their design, development, and adoption will require significant input from FDA CTP in order for them to be appropriate for comparative testing purposes. FDA must participate in this crucial work.

4. Delineation of Analytical Measurement Procedure and Analytical Sampling Procedure

One of the major differences that these draft guidelines contain is the separation of the method validation into two parts, “analytical measurement procedure” and “analytical sampling procedure,” which other guidance documents address as a single concept. For example, the International Conference on Harmonization method validation guidance, ICH Q2(R1), terms this the “Analytical Procedure.” This delineation is also in contrast to guidance issued by other FDA centers, such as FDA CDER and CBER[13] and the FDA Foods Program Regulatory Science Steering Committee (RSSC),[14] which represents multiple centers. Furthermore, the currently adopted standardized methods for tobacco product analysis (both CORESTA and ISO) do not separate these aspects during their own development and validation. It would, therefore, be helpful to know what FDA CTP’s rationale is behind this choice as its implementation would represent a significant departure from current practices, as well as an additional burden with regard to experiments and documentation. Given this, the value of this separation becomes questionable since the guidance provides recommendations and not requirements. In our opinion, the guidance document would be clearer and more streamlined without this distinction.

Regardless of whether the delineation between the analytical measurement and sampling procedures is warranted, its implementation within the document suffers since the format and layout introduce further confusion due to inconsistencies and a lack of clarity throughout. For example, the first clear description of the delineation between analytical and sampling procedures occurs in Section V, paragraph 1 (page 13), and some version of it would have been better placed in Section VI, paragraph 3 (page 5), when the separation of the two elements was first introduced. The second and third paragraphs on page 6 do attempt to delineate the two, but some confusion is introduced by the use of the term “sample stock solution,” which is not defined in Section III (only “stock solution” is defined as referring to “a concentrated solution that may be diluted…for use in testing”). Under the definitions for “Analytical Procedure” and “Analytical Sampling Procedure,” the term “homogeneous sample” is mentioned alongside “stock solution,” with the inference being that this is the product sample that will be tested since step-by-step instructions are provided for its preparation. However, this term’s own definition could contradict its usage in this context since it is defined as a “portion of the material under test that is internally consistent and representative of the bulk of the material.” This could be taken to mean a sample extract that has been prepared through all the required sampling steps or simply a sample that has been prepared to undergo subsequent extraction and testing (i.e., compositing of material from multiple product cans to create a bulk sample, grinding of tobacco, etc.). Whether the final guidance maintains the delineation or not, there does need to be greater consistency in the terms and definitions used throughout so as not to add confusion and ambiguity.

5. Total Error Probability and Measurement Uncertainty

We find the inclusion of “total error probability” to be an interesting concept and certainly feel that it could present a viable alternative to the separate consideration of accuracy and precision for determining method suitability. It is certainly straightforward and simple to apply. However, we feel the term “total error probability” is somewhat of a misnomer as it implies the result is the probability that there is an error present. As defined both in the guidance and the source material,[15] the probability being determined is of an assay value of 100% providing a passing result within defined acceptance criteria. As such, the “total error probability” actually provides an indication of how reliable the method is, and it is inversely proportional to the error present.

In the use of “total error probability,” the guidelines group accuracy and intermediate precision together to determine method suitability. ISO 17025 requires that accuracy and intermediate precision be determined separately along with the use of Measurement Uncertainty to capture determinate and indeterminate errors. No mention of Measurement Uncertainty is made within the guidance, though there is a recommendation that tobacco product manufacturers consider using accredited testing facilities. Consideration of the ISO 17025 requirements by which accredited testing facilities must abide would be welcomed.

Many of our method validations are based on the use of certified reference tobacco products (University of Kentucky, CORESTA) for which known (or accepted) values of many analytes have been determined. How does the use of these products for intermediate precision determination fit with total error probability since accuracy in this instance would be based on the closeness of the result to the established values rather than from spiking studies?

[1] International Conference on Harmonisation (2005), Validation of Analytical Procedures: Text and Methodology Q2(R1). (link)

[2] (a) FDA Center for Drug Evaluation and Research (CDER) and Center for Biologic Evaluation and Research (CBER), Guidance for Industry – Analytical Procedures and Methods Validation for Drugs and Biologics (July, 2015) (link); (b) FDA Office of Regulatory Affairs (ORA) Laboratory Procedure, ORA-LAB.5.4.5, APPENDIX 1-ORA Validation and Guidance for Human Drug Analytical Methods (August, 2014) (link); (c) FDA CDER and Center for Veterinary Medicine (CVM), Guidance for Industry-Bioanalytical Method Validation (May, 2018) (link); (d) FDA Office of Foods, Guidelines for the Validation of Chemical Methods for the FDA Foods Program, 3rd ed. (October, 2019) (link); (e) FDA CVM Guidance for Industry-Validation of Analytical Procedures: Methodology (July, 1999) (link).

[3] CORESTA CRM & ISO Correspondence List (Feb 2022) (link)

[4] National Academies of Sciences, Engineering, and Medicine 2018. Public Health Consequences of E-Cigarettes. Washington, DC: The National Academies Press. (link)

[5] CORESTA Recommended Method No. 81 – Routine Analytical Machine for E-Cigarette Aerosol Generation and Collection – Definitions and Standard Conditions (June, 2015). (link)

[6] ISO 20768:2018 Vapour products — Routine analytical vaping machine — Definitions and standard conditions (link)

[7] FDA Center for Tobacco Products, Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems. Guidance for Industry (June, 2019). (link)

[8] Oldham et al., Insights from analysis for harmful and potentially harmful constituents (HPHCs) in tobacco products, Reg. Tox Pharm 70(1), 138-148, 2014. (link)

[9] 84 Fed. Reg. 38,032 (August 5, 2019). (link)

[10] Morton, M.J.; Gardner, W.P.; Agnew-Heard K.A.; Miller, J.H., “Temporal variability of analytical testing for aerosol products and impact on number of replicates”. Verbal presentation at the 72nd Tobacco Science Research Conference, September 16-19, 2018, Memphis, TN, 2018 (link)

[11] CORESTA HTP Task Force, Heated Tobacco Products (HTPs): Standardized Terminology and Recommendations for the Generation and Collection of Emissions (July, 2020). (link)

[12] CORESTA E-Vapour Subgroup, 2019 Collaborative Study: Reference Device for e-Cigarette Aerosol (January, 2021). (link)

[13] FDA Center for Drug Evaluation and Research (CDER) and Center for Biologic Evaluation and Research (CBER), Guidance for Industry – Analytical Procedures and Methods Validation for Drugs and Biologics (July, 2015). (link)

[14] FDA Office of Foods, Guidelines for the Validation of Chemical Methods for the FDA Foods Program, 3rd ed. (October, 2019). (link)

[15] Ermer, J. and P. W. Nethercote (2014). Method Validation in Pharmaceutical Analysis. Singapore, Wiley-VCH. (link)