"Test results on herbal supplements are complex . . . and can be misleading."

An interview with Mark Blumenthal, Executive Director, American Botanical Council; Editor & Publisher, HerbalGram.
19 December 2000

Mark Blumenthal founded the American Botanical Council (ABC) in 1988. His organization is dedicated to promoting the safe and effective use of medicinal plants and herbs. In particular, ABC educates the public, government agencies, research institutions, industry, and the media about scientific research as a means to guide decisions about producing and consuming herb-based products.

Last October, Wyn Snow of SupplementQuality.com (SQ) asked Mark about a range of issues in botanical quality and testing. His responses highlight the complexity of supplement testing while giving the interested consumer some insight into how responsible supplement producers are approaching an important aspect of supplement quality. Our questions -- and Mark's responses -- follow . . .

SQ: Over the past year we've seen a lot of headlines in the media about supplements "not measuring up" to the potency levels on the labels. Are these stories accurate?

A: Test results on herbal supplements are complex, and depending on how they are reported, can be misleading.

A good example is The Boston Globe story on St. John's wort, which I described in an article that was also published in Whole Foods Magazine. Initially, the headline was intended to read "5 of 7 products fail to meet label claims" -- but they were comparing apples and oranges.

They were looking for the 0.3% hypericin content that many manufacturers state on their labels as the standardization parameter. However, as most herbal experts know, this is done only as a quality assurance (QA) mechanism; there is no correlation between this chemical marker and clinical results for depression. Hypericin is biologically active, but it is linked to potential photosensitization in fair-skinned individuals, not to anti-depressant action.

So first of all, testing needs to be done for the appropriate chemical marker. Next, it's important to have a second lab confirm the results of the analysis. It's also crucial for everyone to be using the same test. Results from the HPLC method developed in Germany in 1991 are not the same as those from the 1986 DAC (German Pharmaceutical Codex) method.

Some companies use the older, 1986 DAC method for testing their products because extract material that is standardized that way is less expensive. They can lower the QA cost of testing their raw material by 20%. There is no rule or policy as to which method to use for testing -- so if a company makes a claim based on the older DAC method and its product is tested by the newer HPLC method, of course the two results don't match.

So the Globe reporter called me and other experts to comment on the results she was seeing. We explained the problems they were facing and recalculated what they thought were "fails" on their one laboratory test.

According to the older 1986 method of standardization, all the products passed, except for one that was off by only about 15 percent (which is not much as lab assays go). As a result, the lead of the story changed to "We thought it would be easy" -- and the thrust of the story changed to why it's not easy to test product quality after all.

Just as there are challenges and variability in the quality of herbal products, there is also wide ranging quality in media reports on herbal products.

SQ: So the Globe story is an example of good, knowledgeable reporting on herbal testing. Can you describe a worst case scenario?

A: In April 2000, D Magazine in Dallas did a story called "Herbal Remedy Ripoffs." They bought 5 brands of 7 kinds of herbal supplements, and sent them to an FDA-approved lab in Fort Worth.

The first problem was that they sent the samples with the labels showing, so the testing was not blinded.

Second, this "FDA-approved lab" was not familiar with botanicals. They tested the products using non-validated methods from the Merck Manual, not the INA's Methods Validation program.

Third, the magazine did not use a second lab to check the accuracy of the test results, and didn't have a chemist or pharmacognosist checking for accuracy of the test design. With some of the herbs tested, D Magazine claimed they did not contain sufficient levels of "active" ingredients, but these were not the real active ingredients at all -- they were marker compounds. So neither the magazine reporter nor, as it appears, the analytical lab had a clear understanding of what they were doing.

Because of these problems, the results of this test were close to useless. Comparing the Globe story with that of D Magazine is totally black and white.

SQ: What has your experience with HerbalGram 's Ginseng Evaluation program (GEP) taught you about how to do testing?

A: When we started the ginseng testing program at the American Botanical Council, we too thought it would be easy -- and it's not. We learned a lot of things about the procedures to follow.

First, use two labs. If their results don't match, you know there is a problem and can investigate.

Second, have knowledgeable chemists, people who know what they are doing.

Third, use methods developed through the Methods Validation Program (MVP) of the Institute for Nutraceutical Advancement (INA) or the Association of Official Analytical Chemists (AOAC). INA has been very successful in determining which methods give consistent results and can be used with a wide range of formulations.

When it came to choosing a test method for ABC's Ginseng Evaluation Program, we did not have access to INA's Methods Validation Program (it did not yet exist), and AOAC had not developed a method. In fact, our GEP method for ginseng testing has been submitted to 15 laboratories around the world for validation under AOAC's strict peer review procedure -- and will become the standard testing method to analyze dosage forms of commercial ginseng products (such as tablets, capsules and soft-gels for both American and Asian ginseng species.).

Fourth, make sure the samples are properly blinded. Don't send a sample to the lab with the brand name still on it.

Fifth, store all samples in fireproof cabinets (or other reasonable way to ensure they are not tampered with) to ensure proper chain of custody.

Sixth, use peer reviews to make sure your methods are responsible and fair.

Seventh, have a good reference standard in the testing devices. Without this, your results won't be accurate. In fact, purity of the reference standard is an issue in and of itself.

SQ: What do you think of the ConsumerLab testing program? (ConsumerLab.com regularly tests a variety of dietary supplements and botanicals, then publishes the names of passing brands on its Web site.)

A: For the most part, I agree with the intention of ConsumerLab -- to test commercial products and publish their results. I think they are doing an excellent job -- especially with respect to the type of analytical protocols they use in their testing, using the most up-to-date testing methods and buying products off the shelf in a randomized fashion. And of course, companies whose products are not included in the first round can request ConsumerLab's Ad Hoc testing program for their products.

However, there is talk in the industry that the U. S. Pharmacopoeia (USP) is considering developing a testing program -- which would be both less expensive and more comprehensive in covering more products and more brands. The implications for this initiative, if it is developed, are not clear at this time, although it would likely mean a lower cost for industry. Having two separate testing programs may not be ultimately viable, but there are many issues at play here.

SQ: What is the biggest quality challenge facing the botanical supplement industry today?

A: The fact that there are no published federal rules for good manufacturing practices (GMPs). This means the industry does not have a level playing field for production quality.

It's been six years since DSHEA was passed, mandating more stringent GMPs for dietary supplements. Since then, three industry trade organizations gave the FDA their recommendations for GMP regulations -- which the FDA published three years ago as an advanced notice of proposed rulemaking (ANPR) on GMPs. And still we don't have a final rule. Supposedly, the FDA will publish a proposed final rule by the end of 2000.

By the time the GMP regulations are published, maybe we'll have new recommendations from the industry. Also, there's the compliance period. And what about the exclusion for small business? How will they define a small business? $100 million in sales? Less than 300 employees?

So it's totally unclear right now what kind of GMP regulations any particular company will soon have to follow. Some companies are going for really high GMPs based on the rules for pharmaceuticals. Others are following the currently required level -- which is food GMPs. The recommended GMPs for supplements are between the two: more stringent than food and less stringent than pharmaceuticals. But we don't know yet what the FDA will publish in its final regulations.

SQ: What do you think of the current trend towards standardization of herbal products?

A: The question is: "Standardization to what and by what method?"

In HerbalGram's Ginseng Evaluation Program, some manufacturers were claiming a particular level of ginsenosides -- which are active, marker compounds in American and Asian ginseng root -- based on one method, while others were claiming the same level using HPLC. So you have two products, both claiming standardization to 3-4%, but using different methods to test that "standardization". The result being that the two don't match.

"Standardization" simply means that the manufacturer says the product contains a guaranteed specified range of a chemical compound or group of compounds. For example, standardized milk thistle products typically claim to contain 70-80% silymarin (a name for three flavonolignans). But this is highly concentrated and not typical of most standardized herbal products, which is usually either at a 3-5% level or at 24%, depending on the particular herb and parameters of standardization.

Research is now investigating which of the chemicals are the correct ones to standardize on. In St. John's wort, as noted earlier, the hypericin marker is related to photosensitivity and is not related to anti-depressant activity. The compound called hyperforin has been shown to be more directly correlated to anti-depressant activity.

Also, many herbs have more than one biologically active compound which work synergistically -- so standardization to only one compound may not be the best way to make a product that delivers consistent health results. It also depends on what clinical outcome you want. Some products might be standardized to one compound (or group of compounds) for one desired effect, and to another chemical or group of chemicals for another. It's not a cut and dried issue.

Also, standardization is not just a matter of chemistry. With some companies, standardization starts with the seed and horticulture. Most plants grow best in specific environmental conditions. There's also the matter of selecting the best seed -- one that yields a preferred chemical profile in an extract or essential oil. Plus, there are cultural issues of using good harvesting and processing methods, and so on.

A great example of this is a German company that makes a line of chamomile products for external use. All their chamomile extract is made from chamomile grown in Spain from a specific strain of seeds. This strain was selected because years of research show that this chemotype of chamomile produces the optimum extract from the perspective of biological activity. So standardization is not as simple as some people make it out to be.

Quality is a complex issue. Standardization is only one of many factors that contribute to making a quality product. In some cases, there is compelling evidence to support the safety and efficacy of nonstandardized products. Francis Brinker, a naturopathic physician and a leading author in botanical medicine, wrote a well-referenced 16-page article in HerbalGram #46 about the "Variations in Effective Botanical Products" -- the case for non-standardized herbs being given their due. He cited much European research on non-standardized herbal preparations that indicates their safety and efficacy.

Just as we want biological diversity and cultural diversity, there is a strong case for herbal diversity. There are many different uses for herbs, and various types of dosage forms are appropriate for different applications. For example, I don't think I must have a chemically standardized chamomile tea-bag for me to enjoy a flavorful cup of tea that also gives me the added benefit of helping me digest a big meal!

About the American Botanical Council's Ginseng Evaluation Program

ABC began the Ginseng Evaluation Program (GEP) in 1993, motivated by the concerns of research scientists and veterans of the ginseng and herbal industries -- concerns that some ginseng products might be mislabeled (meaning that they did not contain the amount of ginseng listed on the label) or adulterated with any of a variety of contaminants. ABC purchased randomized, off the shelf samples of approximately 500 ginseng products throughout the US and Canada, and sent them to laboratories for testing.

The passage of DSHEA in 1994 changed the regulatory environment for herbal supplements, including ginseng, so the GEP evolved into three phases:

Results of the GEP will be reported in two special issues of HerbalGram devoted to ginseng. Standardized phase test results are being published in HerbalGram #51, due out in March 2001; the regular and new lot testing phases will be reported in #52, due in June 2001. HerbalGram is the journal of ABC and the Herb Research Foundation.

ABC's website has a full description of the GEP program at www.herbalgram.org/browse.php/defaulthome.