Deliverable 28: Specification of low risk products REBECA ...

Deliverable 28:
Specification of low risk products
REBECA
Regulation of Biological Control Agents
Specific Support Action
Project no. SSPE-CT-2005-022709
Contract Start Date: 01-06-2006
Duration: 24 months
Project Coordinator: Ralf-Udo Ehlers, Christian-Albrechts-University of Kiel.

Document Classification
Title
Specification of low risk products
Deliverable 28
Reporting Period 2
Contractual Date of Delivery Project Month 12, February 2007
Actual Date of Delivery December 2007
Authors
Work package
Dissemination
Nature
Version
Keywords
Anita Fjelsted, Rüdiger Hauschild, Hermann Strasser
WP7
Public
Report
01.00, Final
Low risk
Document History
The document is based (i) on the recommendation expressed in a questionnaire
sent to participants prior to the Salzau workshop in September 2006, (ii) on
recommendations expressed during discussions at the Salzau workshop in
September 2006 and (iii) on recommendations expressed at the regulators meeting
in September 2006 (iv) the QPS work carried out within an EFSA working group and
the final opinion of the Scientific Committee (v) a risk index model proposed by
Tobias Längle and Hermann Strasser.
Document Abstract
At present no definition or criteria of low risk plant protection products or active
substances exists in the EU regulation. However, it is being introduced in the new
proposal for an EU regulation of plant protection products, whish is still being
negotiated.
During the REBECA project period various stakeholders have given their opinion on
which BCAs should be regarded as low risk active substances/products and why.
Also the work initiated within EFSA on Qualified Presumption of Safety (QPS) which
relates to this subject is discussed within this document. The micro-organisms
suggested to be given a QPS status is included in Annex 1 and in Annex 2 a
publication by Längle and Strasser introduce a newly developed risk index system,
which can be of significance in the definition of “low risk” products. The suitability of
the model was demonstrated by calculating the risk scores for 17 selected wellstudied
biological control agents and chemical products used for similar purposes.
The authors conclude that the score of “low risk” products should not exceed 100,
whereas a threshold of 500 seems justified for the term "reduced risk".
1

Table of contents
Introduction................................................................................................................. 3
Questionnaire on low risk ........................................................................................... 5
Regulator’s experiences in defining low risk products ................................................ 6
QPS – Qualified Presumption of Safety...................................................................... 6
USA: Minimal Risk Pesticides (25b list)...................................................................... 8
Low risk semiochemicals and botanicals.................................................................... 9
Low risk microbials ..................................................................................................... 9
Annex I: List of taxonomic units proposed for QPS status........................................ 10
Annex 2: Developing a risk indicator to comparatively assess environmental risks
posed by microbial and conventional pest control agents ........................................ 12
Abstract .................................................................................................................... 13
Introduction............................................................................................................... 13
Overview of current risk indices................................................................................ 15
Environmental Impact Quotient (EIQ)....................................................................... 15
Norwegian Indicator (NARI)...................................................................................... 16
Québec Pesticide Risk Indicator (IRPeQ)................................................................. 17
Canadian Agri-Environmental Standards (NAESI) ................................................... 17
ERBIC Risk Indicator................................................................................................ 18
Defining a Risk Indicator suitable to compare biological and convential pesticides.. 18
Proposed risk indicator and rationale ....................................................................... 19
Basic components and their integration.................................................................... 19
Scoring and rationales.............................................................................................. 20
Conclusions and envisaged applications.................................................................. 25
Acknowledgements .................................................................................................. 28
References ............................................................................................................... 29
2

Introduction
The main purpose of the REBECA project is to speed up the registration process of
BCAs in order to speed up the market introduction of such products. One way of
doing this would be to differentiate between low risk and high risk active substances
in the EU and national evaluation processes and in this way make a fast track
regulatory system for the low risk substances (which, among others, would be
expected to be a number of BCAs). In order to do that, it is necessary to establish a
definition or criteria of low risk substances that can be used to place the active
substances into one of these two categories even prior to a risk assessment process.
However, this is quite difficult. This document describes activities carried out within
REBECA in order to investigate whether it would be possible to make such a
differentiation at an early stage of the evaluation and registration process, with the
view of obtaining a faster introduction of new low risk products on the market.
Legal framework
Plant protection products
In the present EU regulation on pesticides (Directive 91/414/EEC) there is no
differentiation between low risk and higher risk active substances.
However, the Commission’s proposal for a new pesticide Regulation 2006/0136
(COD) which is still being negotiated among EU member states contains separate
paragraphs relating to “low-risk substances”, “basic substances” and “substances of
concern”. Article 22 extends the period of approval from the normal 10 years to 15
years for low risk active substances.
Based on the still ongoing negotiations the Commission has published a revised
proposal in order to seek agreement between the member states. In the most recent
amended proposal of Regulation 2006/0136, which is dated 11 March 2008, the
following definition of low risk is included:
Low risk: of a nature considered inherently unlikely to cause an adverse effect on
humans, animals or the environment.
Further more a number of criteria are listed in Article 22 for low risk substances. The
active substances can not be regarded as low risk if they are classified as one of the
following:
• carcinogenic
• mutagenic
• toxic to reproduction
• very toxic
• toxic
• sensitising
• explosive
3

Further more the substances which are qualified as the following can not be regarded
as low risk either:
• persistent (half life of less than 60 days)
• endocrine disrupter
• bioaccumulative and non readily-degradable.
Article 46 sets timelines for the authorization of plant protection products based on
low risk substances. The member state shall within 90 days decide whether to
approve an application for authorisation of a low-risk plant protection product. This
period should only be 60 days in case an authorisation has already been granted for
the same low-risk plant protection product by another Member State located in the
same zone. However, in case the Member State will need additional information, it
shall set a time limit not exceeding 6 months for the applicant to supply it.
These timeframes are shorter than those suggested for active substances that are
not regarded as of low risk (article 36). For those the timeframe is 12 months with the
possibility of asking for additional information within a period of further 6 months.
Article 23 provides criteria for basic substances and extends the period of their
approval to an unlimited time. The basic substances will have to be applied included
into a separate list. Article 28 states that plant protection products only containing
basic substances (from this list) do not need to go through a national authorization in
order to be placed on the market.
The criteria for low risk substances are clearly made with chemical active substances
in mind. First of all, there is a general risk of micro-organisms being sensitizers,
which would thus right away disqualify them as low risk substances. However, so far
no proper guidelines are available that can be used to carry out studies in order to
investigate the sensitising properties of micro-organisms. In the data requirements for
micro-organisms (Annex IIB to Directive 91/414/EEC) which are listed in Dir.
2001/36/EC, it is mentioned that it is not necessary to present data on sensitisation,
due to this lack of guidelines, but in this case the micro-organism is considered to be
sensitising.
.
Secondly, the three terms: persistence, bioaccumulative and non readily-degradable
and endocrine disrupters are all terms originating from the classification of chemical
active substances. These criteria do not take into account that e.g. micro-organisms
are naturally occurring substances.
Biocides
In the Biocide directive 98/8/EC the active substances regarded as being of low risk
are included into a specific list: 1A. The criteria for substances to be included into this
list are quite similar to the criteria which are now suggested included in the new
regulation on plant protection product. However, the biocide criteria does not include:
toxic, very toxic, explosive and endocrine disrupters.
4

Questionnaire on low risk
Prior to the REBECA workshop held in Salzau 18-22 September 2006 a
questionnaire was sent to all participants in which they were asked to list active
substances (BCAs consisting of micro-organisms, botanicals, semiochemicals or
macrobials) which they would regard as being of low risk and to give their reasoning
for such proposals for low risk substances. Further more the participants were asked
to give a definition and/or criteria for low risk substances.
46 persons replied to the questionnaire (9 regulators, 12 persons representing the
industry, 22 from the scientific community and 3 from consultancies).
The participants representing the industry and the scientific community all gave lists
of active substances which they regarded as of low risk. In particular the participants
gave a long list of macrobials. However, also the semiochemicals, in particular the
SCLP were mentioned by representatives from both the industry and regulatory
authorities as a category of low risk. It was mentioned by several participants that if
SCLP were applied in concentrations similar to the background concentration
occurring in areas with high densities of the pest insect, they should definitely be
regarded as of low risk. Baculoviruses was another group of active substances that
was mentioned by many participants as being of low risk.
A number of botanicals were listed as well. These were products which are also used
for human consumption.
Arguments for listing these as low risk were:
Long history of safe use
Micro-organisms that frequently cause natural epizootics in presence of the host pest
Micro-organisms which are ubiquitous in soils around the world
Micro-organisms that do not grow at 37 °C
Narrow host range/very specific
Low persistence
Substances used for human consumption (e.g. rapeseed oil, garlic oil, olive oil)
Substances used as household products (e.g. for cleaning)
During the discussion of the questionnaire at the REBECA workshop in Salzau the
general opinion expressed by regulators and the European Commission (DG Sanco)
was, that it would not be possible to establish a list of substances of low risk prior to a
risk assessment, i.e. a list of substances that would not need a risk assessment.
However, all regulators seemed to agree, that there was a need for a
definition/criteria of low risk substances for the new EU regulation of pesticides, but,
such criteria will be applied only after the risk assessment has been carried out and
will rather determine which substances will get an Annex I inclusion of a longer
period (15 years) and an easier/faster process for national registration. As mentioned
already, the text of the Commission proposal for a new EU regulation of pesticides is
still being discussed and negotiated among EU member states.
5

Regulator’s experiences in defining low risk products
At the REBECA stakeholder meeting for regulators which took place in Salzau,
Germany on 18 September 2006 (a meeting attended by 27 regulators from Europe,
Australia and USA) the participants discussed the possibility of the national
authorities to give priority to low risk products during the evaluation and authorisation
process.
This issue had been discussed in Sweden, the Netherlands and in the UK. The
purpose in all three countries was to increase the number of such products at their
market e.g. by reducing the fee requested for low risk products and in the
Netherlands and the UK also to provide further guidance to applicants in order to
speed up the preparation of dossiers and the subsequent evaluation of those dossier.
However, none of the regulatory authorities in the three countries found the term “low
risk” very helpful, simply due to the difficulties in defining such a category. In the UK
the Pesticide Safety Directorate (PSD) has not used the term low risk products in
their BioPesticide Scheme but instead the term alternative products (however, also
without a specific definition). For this product group they have lowered the fees, are
arranging pre-submission meetings, they have increased the web-site information of
the regulatory process, established a specific contact point in PSD for these product
types (a champion) and the applicants can be guided throughout the process of
putting together an application.
A somewhat similar project is taking place in the Netherlands, where the project is
called GENOEG. It is also aiming at getting further low risk products on the market.
In the Netherlands they have used the term natural pesticides rather than low risk
products. Via this project the applicants can get up to 100,000 € co-finance for
registration fees and extra studies needed for the risk assessment, and the
regulatory authority here also help applicants put together good dossiers and invite
applicants for pre-submission meetings.
QPS – Qualified Presumption of Safety
At several REBECA workshops the EFSA initiative on developing a QPS concept
(Qualified Presumption of Safety) was discussed. The reason being that it was
anticipated, that if the microbial plant protection products were included in the
development of this new concept, it would be a way of defining groups of low risk
micro-organisms, and a way of obtaining a faster evaluation and market introduction
of microbial plant protection products.
The development of a QPS concept was initiated in 2003 by a working group
consisting of members of several former (EC) scientific committees. The work was
continued within an EFSA working group. The aim was to develop a scheme that
would harmonize the risk assessment of micro-organisms throughout the various
EFSA panels and a scheme developed as a tool for setting priorities within the risk
assessment of micro-organisms used in food/feed. By using this tool risk assessors
will for some micro-organisms be able to take a generic approach in the risk
assessment instead of a full case-by-case assessment, and in this way make better
use of assessment resources by focussing on those organisms that present greatest
risk or uncertainties, and which would need a case-by-case risk assessment.
6

It is proposed, that a safety assessment of a defined taxonomic group should be
made based on four pillars (establishing identity, body of knowledge, possible
pathogenicity and end use). If the taxonomic group did not raise safety concerns or, if
safety concerns existed, but could be defined and excluded the grouping could be
granted QPS status. Thereafter, any strain of the micro-organisms given QPS status
would be freed for further safety assessments other than satisfying any qualifications
specified.
The final opinion of the Scientific Committee (including 4 appendices) was adopted
on 19 November 2007. Table 1 contains the 4 groups of micro-organisms included in
the concept. The committee explains in this document that the group consisting of
filamentous fungi could not be recommended QPS status. Further more they explain
that all strains belonging to the Bacillus cereus sensu lato group (e.g. Bacillus
thuringiensis) should not be given a QPS status either, since it is known that the vast
majority of strains within this group are toxin producers and thus can not meet the
required qualifications.
Table 1. The four groups of micro-organisms which are so far considered in the QPS
concept and the number of species proposed for QPS status so far
Group of micro-organism
Number of species proposed for QPS
status so far
non-spore forming gram positive bacteria
Bacillus spp.
yeasts
commonly encountered filamentous fungi
48 species
13 species
11 species
None
The Scientific Committee writes as follows in their opinion of 19 November 2007:
“The Scientific Committee is of the opinion that the use of strains from the B. cereus
group should be avoided whenever there is a possibility of human exposure whether
intended or incidental. The B. cereus group is therefore excluded from consideration
for QPS status.
There is an artificial distinction held between B. cereus and B. thuringiensis (used for
plant protection) which has little scientific basis. The plasmid encoding the
insecticidal enterotoxin, which provides the phenotypic distinction for B. thuringiensis,
is readily lost, particularly when grown at 37 ºC, leaving an organism
indistinguishable from B. cereus. Consequently it is likely that B. thuringiensis has
been the causative organism of some instances of food poisoning but identified as B.
cereus because clinical investigations would have failed to recognise the
distinguishing features characteristic of B. thuringiensis.
However, the Scientific Committee recognises that B. thuringiensis has value to the
industry as a means of biological pest control and that its widespread use for this
purpose may not lead to significant human exposure.”
7

Bacteria directly consumed by humans only qualify for QPS status, if they are free of
acquired resistance to antibiotics of importance in clinical and veterinary medicine.
Furthermore, all bacteria capable of toxin production should be demonstrated to be
free of any toxigenic potential.
It is important to stress that QPS does not carry any legal status.
Since neither B. thuringiensis nor any of the filamentous fungi are included on the list
of species proposed for QPS status, the QPS in its present form does not offer a
generic approach to the safety assessment of most micro-organisms used as
biological control agents. Never the less, the EFSA Scientific Committee considers
that it may be possible to devise robust use qualifications which would allow a QPS
approach for further groups of micro-organisms relevant for biological control in the
future. The system is developed in order to provide a generic assessment system for
use within EFSA that can be applied to all requests for the safety assessment of
micro-organisms deliberately introduced into the food chain or used as producer
strains for food/feed additives. This implies, that when industry applies for Annex I
inclusion of micro-organisms belonging to microbial taxonomic units, which are now
included in the list of organisms for which a QPS status is proposed (e.g. Bacillus
subtilis and B. pumilus) with the intention to market these in plant protection
products, the industry can in their dossier argue that the species are given QPS
status, and that the risk for consumer health (due to exposure from residues on
crops) is likely to be low when these strains are applied as plant protection products.
This information can be used as a waiver for residue data for micro-organisms given
QPS status. The list of taxonomic units for which QPS status has been proposed can
be found in Annex 1.
The applicability of the QPS approach for broad use of micro-organisms as plant
protection products needs to be discussed further.
USA: Minimal Risk Pesticides (25b list)
In the USA, there is a list of substances that can be used as pesticides without any
registration, however, they still need a residue limit, or exemption, for food or feed
uses. These substances are called Minimal Risk Pesticides, as described in the US
Code of Federal Regulation, 40CFR 152.25(f). The list contains many essential oils 1 .
All inerts must be on EPA’s 4A inert list, all ingredients must be identified on the
label, and the label may not contain false or misleading claims. This regulation was
developed by an EPA workgroup in 1994 and revised in accordance with public
comments for a final Federal Register publication in 1996. The EPA has experienced
a problem since it has been difficult identifying exactly which chemical substances
are included under the names listed. Currently, CAS numbers are used to describe
the substances on the EPA inert substance classification lists.
1 Currently, the list includes the following substances: castor oil, cedar oil, cinnamon and cinnamon oil,
citric acid, citronella and citronella oil, cloves and clove oil, corn gluten meal, corn oil, cottonseed oil,
dried blood, eugenol, garlic and garlic oil, geraniol, gernanium oil, lauryl sulfate, lemongrass oil,
linseed oil, malic acid, mint and mint oil, peppermint and peppermint oil, 2-phenethyl propionate (2-
phenylethyl propionate), potassium sorbate, putrescent whole egg solids, rosemary and rosemary oil,
sesame (includes ground sesame, plant) and sesame oil, sodium chloride (common salt), sodium
lauryl sulfate, soybean oil, thyme and thyme oil, white pepper and zinc metal strips.
8

Low risk semiochemicals and botanicals
In REBECA Deliverable 18 (Positive list of low risk candidate botanicals and
semiochemicals), gives a discussion on low risk semiochemicals and botanicals and
provide lists of such substances which REBECA propose should be given low risk
status.
Low risk microbials
Baculoviruses
Baculoviruses in general have low risk for all organisms except their specific hosts.
As stated in the “OECD Consensus document No 20 on information used in the
assessment of environmental applications involving baculoviruses” from January
2002, «Baculoviruses are naturally occurring pathogens of arthropods. Their host
range is exclusively restricted to arthropods. No member of this virus family is
infective to plants or vertebrates». Likewise, no sensitisation was observed for
baculoviruses so far. The OECD Consensus Document concludes that «No adverse
effect on human health has been observed in any of these investigations indicating
that the use of baculovirus is safe and does not cause any health hazards.»
The majoritiy of baculoviruses has a very restricted host range, which mainly
comprises one or a few species of the same genus, rarely different genera of the
same family. Baculoviruses with a broader host range are the exception. Therefore,
risks for non-target species can be excluded as well.
Low risk bacterial and fungal products
In REBECA Deliverable 12 (Positive list of low risk candidate microbials), gives a
discussion on low risk microbials and provide lists of such substances which
REBECA propose should be given low risk status.
This recommendation is based (i) on a case by case evaluation of microbial
biocontrol agents, assessed by international experts, recognised by REBECA
consortium, (ii) the safety data fact sheet published by the US Environment
Protection Agency (EPS) and (iii) publication of the European Council regulations,
reporting the opinion of the safe use of Annex I listed micro-organisms.
Annexes
1. List of taxonomic units proposed for QPS status
2. Publication: Developing a risk indicator to comparatively assess environmental
risks posed by microbial and conventional pest control agents.
9

Annex I.
List of taxonomic units proposed for QPS status
Gram-Positive Non-Sporulating Bacteria 2
Species
Bifidobacterium adolescentis Bifidobacterium bifidum
Bifidobacterium animalis Bifidobacterium breve
Corynebacterium glutamicum
Lactobacillus acidophilus
Lactobacillus amylolyticus
Lactobacillus amylovorus
Lactobacillus alimentarius
Lactobacillus aviaries
Lactobacillus brevis
Lactobacillus buchneri
Lactobacillus casei
Lactobacillus crispatus
Lactobacillus curvatus
Lactobacillus delbrueckii
Lactococcus lactis
Lactobacillus farciminis
Lactobacillus fermentum
Lactobacillus gallinarum
Lactobacillus gasseri
Lactobacillus helveticus
Lactobacillus hilgardii
Lactobacillus johnsonii
Lactobacillus
kefiranofaciens
Lactobacillus kefiri
Lactobacillus mucosae
Lactobacillus panis
Bifidobacterium longum
Lactobacillus paracasei
Lactobacillus paraplantarum
Lactobacillus pentosus
Lactobacillus plantarum
Lactobacillus pontis
Lactobacillus reuteri
Lactobacillus rhamnosus
Lactobacillus sakei
Lactobacillus salivarius
Lactobacillus
sanfranciscensis
Lactobacillus zeae
Leuconostoc citreum Leuconostoc lactis Leuconostoc mesenteroides
Pediococcus acidilactici Pediococcus dextrinicus Pediococcus pentosaceus
Propionibacterium.
freudenreichii
Streptococcus thermophilus
Qualifications
QPS status applies only
when the species is
used for production
purposes.
Bacillus 6
Species
Bacillus amyloliquefaciens
Bacillus atrophaeus
Bacillus clausii
Bacillus coagulans
Bacillus fusiformis
Bacillus lentus
Bacillus licheniformis
Bacillus megaterium
Bacillus mojavensis
Bacillus pumilus
Bacillus subtilis
Bacillus vallismortis
Geobacillus
stearothermophillus
Qualifications
Absence of emetic food
poisoning toxins with
surfactant activity.*
Absence of enterotoxic
activity.*
* When strains of these QPS units are to be used as seed coating agents, testing for toxic
activity is not necessary, provided that the risk of transfer to the edible part of the crop at
harvest is very low (section 4.3 of Appendix C).
2
Absence of acquired antibiotic resistance should be systematically demonstrated unless cells are
not present in the final product.
10

Yeasts
Species
Debaryomyces hansenii
Hanseniaspora uvarum
Kluyveromyces lactis
Pichia angusta
Kluyveromyces
marxianus
Pichia anomala
Qualifications
Saccharomyces bayanus
Schizosaccharomyces
pombe
Xanthophyllomyces
dendrorhous
Saccharomyces
cerevisiae
Saccharomyces pastorianus
(synonym of Saccharomyces
carlsbergensis)
S. cerevisiae, subtype
S. boulardii is
contraindicated for
patients of fragile
health, as well as for
patients with a central
venous catheter in
place. A specific
protocol concerning
the use of probiotics
should be formulated
11

Annex 2
Developing a risk indicator to comparatively assess environmental risks posed
by microbial and conventional pest control agents
Tobias Laengle 1,2 , and Hermann Strasser 1*
1 Institute of Microbiology, University Innsbruck, Technikerstrasse 25,
A 6020 Innsbruck, Austria
2 Pest Management Centre, Agriculture and Agri-Food Canada, Central Experimental Farm,
Building #57, 960 Carling Ave, Ottawa, Ontario K1N 8L4, Canada
* Corresponding author: Email: hermann.strasser@uibk.ac.at;
phone: +43-512-507-6008; fax: +43-512-507-2938;
Keywords:
plant protection; microbial biocontrol agents; risk assessment; risk indicator;
registration; comparative risk assessment
12

Abstract
Selected biological control agents and conventional pesticides were used to critically
review the applicability of a newly developed risk indicator (RI) system. Five basic
components are proposed for the calculation of the overall environmental risk score:
persistence of the active ingredient, dispersal potential, range of non-target
organisms that are affected, and direct and indirect effects on the ecosystem.
Several risk measurement systems were reviewed, risk categories in the proposed
system were modified from a widely-accepted model (i.e. ERBIC model).
Additionally, one new category was implemented to assess the risks to vertebrate
non-target species.
Besides a detailed discussion of the new risk indicator model, the suitability of the
model was demonstrated by calculating the risk scores for seventeen selected
products. It became obvious, that the environmental risk score greatly varied within
the assessed chemical products, and also, yet at a much lower level, within the group
of biological products. The use pattern greatly influenced the estimated
environmental risk posed by any given product. The overall environmental risk score
varied between 24 (Coniothyrium minitans, soil application) and 4.275 (DDT, foliar
spray).
The proposed model can be used to communicate environmental risk and to design
lower risk integrated pest management strategies. It is recommended, that the
proposed risk indicator system may serve to define low risk (i.e., RI ≤ 100) and
reduced risk (i.e., 500 ≥RI > 100) pesticides. Yet, it remains debatable whether RI will
be useful in determining acceptability of data waivers. Use pattern, application
method, persistence, growth temperature range and taxonomic relatedness to
known/suspected pathogens should all be considered when justifying data waivers.
Introduction
In recent years, significant progress has been made in the development of fungal
biocontrol agents (BCAs) for the suppression of pests (insects, nematodes), weeds
and diseases of a wide range of forest, horticultural and agricultural crops.
Nevertheless, relatively few of these products have reached the market: For
instance, at the time of writing this manuscript no mycoinsecticide has been
registered in the European Union under the harmonized registration procedure of
Council Directive 91/414/EEC. Likewise, no fungal insecticides have been approved
under the Pest Control Act in Canada. Today, only 17 fungal insecticides comprising
five fungal species are registered in all 30 OECD countries (Kabaluk & Gazdik,
2007).
The biocontrol industry and regulators agree that relatively few microbials have been
registered in recent years in part due to insufficient experience with such products
and the fact that methods and tools used for risk assessment are still not applicable
to biological systems (Scientific Committee on Plants, 2002; International Biocontrol
Manufacturers Association, 2003; Strasser & Kirchmair, 2006). An additional problem
for microbial pesticides, however, lies in the structure of the biopesticide industry:
most biopesticide companies are small and medium sized enterprises with limited
financial resources and experience in the registration of plant protection products,
respectively. This fact is important to point out because these enterprises often
13

cannot afford the high costs for a successful registration of their biological control
agents (BCAs), which are in most cases niche products.
Microbial pesticides are generally regarded as posing lower risks to human health
and the environment than chemical pesticides (OECD, 2007). Many governments
have responded to growing public demands for safer means of plant protection and
have recognized that the obstacles for safer biological pesticides need to be
addressed. In the European Union, the multidisciplinary REBECA (acronym;
Regulation of Biological Control Agents) consortium sought to find more appropriate
protocols to address data requirements for biological pesticides and thereby facilitate
access to lower risk biological controls (REBECA, 2007).
In Canada, the pesticide regulatory authority, Health Canada’s Pest Management
Regulatory Agency (PMRA) has waived all registration review fees for microbial
pesticides. To complement this measure, the federal department of Agriculture and
Agri-food Canada has set up a regulatory support program for biopesticides needed
by the grower community. The new Canadian pesticide legislation explicitly requires
that the registration to lower risk products will be facilitated by the regulator
(Government of Canada, 2002). Both in Canada and in the United States the
respective regulatory agencies have established data requirements that reflect the
current scientific knowledge about the risks posed by microbial pest control agents
(Pest Management Regulatory Agency, 2001; Environmental Protection Agency,
2006).
Unlike chemical pesticides, microbial agents may infect other living organisms
causing diseases. Potential adverse effects of microbial pesticides include the
displacement of non-target micro-organisms and allergenic, toxic, and pathogenic
effects on humans or other non-target organisms (Cook et al., 1996; OECD, 2003).
The key challenge in risk assessment method development is to establish protocols
and guidelines that enable an efficient yet responsible risk assessment.
In the area of exotic natural enemies van Lenteren et al. (2003) proposed the use of
a risk indicator developed in the ERBIC project (Hokkanen et al., 2003) as an
objective approach to a comparative risk assessment of “classical biocontrol agents”
(definition as in Eilenberg et al., 2001). This approach has been implemented in a
guideline for the risk assessment of arthropods as part of the OECD Plant Protection
Programme (OECD, 2004a).
Legislators in Canada (Government of Canada, 2002) and US EPA (US EPA, 2000)
have recognized the value of objectively comparing the risks of different pesticides
for the same pattern of use and allowing regulators to consider the risks of other
pesticides when registering a new product. A similar approach has been suggested
by the European Parliament (European Parliament, 2003).
Several authors have suggested concepts to compare the health or environmental
risks or hazards of conventional pesticides, but, unfortunately, the applicability of
these tools to biological control agents is limited. The availability of a tool to
objectively compare the environmental impact of biological and conventional
pesticides is desirable in light for the promotion of safe biological control options and
the measurement of resulting risk reduction in agricultural use.
In this paper we propose such a tool for microbial BCAs by using data gathered in
the EU funded BIPESCO (FAIR6-CT-98-4105) and RAFBCA (QLK1-CT-2001-01391)
research projects, as well as data from public regulatory documents and scientific
literature. Selected microbial pest control agents were used to critically review the
14

applicability of the proposed environmental risk indicator. Furthermore, advantages of
the suggested risk indicator are compared to existing risk indicators used for
conventional pesticides.
Overview of current risk indices
Numerous attempts have been made to compare the risks associated with different
pesticides to one another. The complexity of these systems varies widely and ranges
from a simple grouping of pesticides into toxicity classes, to more sophisticated
measures, which use numerical data such as toxicity endpoints used for regulatory
purposes. A thorough review of risk comparison systems currently used in different
countries has been conducted elsewhere (Reus et al., 1999; Reus et al., 2002;
OECD, 2004b; Mineau & Whiteside, 2005). Selected models are introduced here
followed by a discussion of their merits and shortcomings in comparing the risks of
microbials to those of conventional pesticides.
Environmental Impact Quotient (EIQ)
Among first risk indices developed for pesticides was the Environmental Impact
Quotient (EIQ), which was published in 1992 (Kovach et al., 1992). The EIQ is used
predominantly in North America, and classifications for new pesticides are regularly
updated on Cornell University’s website (http://www.nysipm.cornell.
edu/publications/eiq/).
Essentially, the base EIQ for an active ingredient is the average risk calculated for
the three sub-indices which describe the risk to (i) producers, (ii) consumers, and (iii)
the environment. These sub-indices are calculated by using regulatory data such as
dermal, oral and chronic toxicity to mammals (for consumer and producer risk), and
toxicity to fish, birds, bees, other arthropods (for environmental risk). Various
weighting coefficients, as well as soil and plant half life of the substance are also
used as factors in the calculation. The "field EIQ" is derived by multiplying the base
EIQ with the application rate and number of applications per year.
While a good estimate to roughly assess the impact of pesticides, there are some
major points of criticism that limit the usefulness of the EIQ.
Firstly, the EIQ solely relies on the data established for the active ingredient. The
type of application (e.g. soil incorporation, foliar spray), and formulation type are not
taken into account. These are significant factors that will affect exposure to the
pesticide and as such have a major influence on the risk, and hence the potential
impact, resulting from a pesticide application. The EIQ should therefore be better
described as a hazard quotient rather than a risk quotient.
Secondly, the EIQ averages producer, consumer and environmental risks, which may
result in a significant risk to one component being averaged out and overseen by a
lower risk to others. For instance, a relatively high toxicity to fish or birds would not
be appropriately accounted for if the risk to humans and mammals is low.
Thirdly, weighting factors used on the EIQ are not explained or justified in the
original publication and seem to be assigned arbitrarily. For instance, Mineau et al.
(2005) noted that the environmental section of the EIQ is biased towards arthropods.
With respect to the EIQ's application to microbials, there is often insufficient data
available to feed into the EIQ equation, because of a higher likelihood of studies
being waived in the registration process. Furthermore, the factors affecting the risks
15

esulting from microbial pest control products are not necessarily determined by the
same variables as those for conventional chemical pesticides.
Norwegian Indicator (NARI)
The Norwegian Agricultural Inspection service has established separate pesticide
risk indicators to assess human health and environmental risk with the purpose of
tracking risk reduction over time. This was aimed to assist a government initiative,
which had a declared goal of reducing pesticide risk by 25 % between 1998 and
2002, and also encompassed the introduction of risk based taxes on pest control
products (Norwegian Agricultural Inspection Service, 2002). Compared to risk indices
implemented in other OECD countries (OECD, 2004b), the Norwegian model is a
step towards a more integrated model that attempts to include both exposure and
toxicity. For the calculation of the Norwegian environmental risk indicator, scores are
assigned for (i) terrestrial adverse effects, (ii) aquatic adverse effects, (iii) leaching
potential, (iv) persistence, and (v) bioaccumulation. Scores in the terrestrial and
aquatic categories are calculated using both toxicity of and exposure to a substance;
values are assigned in two different groups of organisms (bees/earthworms/birds and
fish/aquatic invertebrates/plants, respectively), the highest value in each category is
used for the calculation. Values for persistence, mobility and bioaccumulation are
also assigned according to set intervals. The environmental risk indicator is then
calculated as the squared sum of all components. Modifications of the indicator are
possible, defined for specific scenarios such as seed treatments or greenhouse
applications. Microbial pest control agents are routinely assigned a value of one
(lowest risk).
The inclusion of exposure considerations, as well as the breakdown into separate
environmental and health indices in the Norwegian model marks a significant
advantage over the EIQ. Certainly, this allows for a more differentiated assessment
of pesticide risks to humans and the environment. The advantage of the Norwegian
system is that it is entirely based on data commonly requested by registration
authorities, and as such data to calculate the risk score should be readily available,
at least within governments.
However, the Norwegian system also has some significant weaknesses. Most
importantly, the indicator is simply a sum of its subcomponents, which is not
reflective of interactions between the different components. For instance a high level
of persistence, mobility or toxicity will each increase the risk posed to aquatic and
terrestrial organisms. However, a very short half-life will significantly mitigate a high
level of toxicity, whereas a highly persistent substance of the same toxicity will
expose and possibly kill multiple times more non-target organisms.
Another shortcoming of the Norwegian approach is the treatment of microbial pest
control products. The simple flat-rate assignment of the minimal risk score to all
microbial active ingredients may be a pragmatic approach useful in the context in
which the Norwegian indicator is used but does not allow for a differentiated
comparison between conventional chemicals and biological pesticides. For a
regulatory comparative risk assessment or a farm level decision tool, a more
sophisticated assessment of microbial pest control products is required.
16

Québec Pesticide Risk Indicator (IRPeQ)
The Indicateur de risque des pesticides du Québec (IRPeQ, Onil et al., 2007), is
largely based on the Norwegian approach, but was fine-tuned to be applicable to
agricultural practices and conditions in Québec, Canada. The tool is novel insofar as
it is accompanied by a website that allows farmers to enter individualised information
and assists informed decision-making on a field level.
Modifications of the Norwegian model include a different weighting assigned to
persistence, aquatic and terrestrial effects, and the addition of separate sub-indices
for birds and bees. None of the major shortcomings of the Norwegian model outlined
above were improved in the system adapted for Québec.
Canadian Agri-Environmental Standards (NAESI)
An entirely new approach to assess comparative risks of pesticides was presented
by the Canadian Wildlife Service of Environment Canada (Mineau, 2002; Mineau &
Whiteside, 2005; Mineau et al., 2008). The starting point of this study was the
analysis of comprehensive sets of actual field data, which included field studies and
reported incidents resulting in losses of non-target organisms due to pesticide
applications. The initial study focused on birds, and the author attempted to correlate
field mortality data to known chemical and toxicological properties recorded during
the registration of these compounds through the application of different mathematical
models. These approaches used factors such as oral toxicity, half-life, and other
factors, and the deducted model allowed the correct prediction of bird mortality in 85
% of tested cases (Mineau & Whiteside, 2005).
These techniques were further developed and refined under the Canadian National
Agri-Environmental Standards Initiative (NAESI), where, among others, acceptable
standards were set for pesticide effects on all environmental compartments. The
NAESI approach allows a statistically calibrated prediction of pesticide-caused losses
suffered by birds, mammals, non-target arthropods, edaphic invertebrates
(particularly earthworms) and aquatic organisms (Mineau et al., 2008).
As such, this model is the only available system that allows for the direct prediction of
field mortality on the basis of laboratory parameters. Hence, this approach breaks
new ground by providing a calibrated prediction of adverse effects as opposed to a
theoretical risk score.
Regarding the application of this approach to microbial pesticides, there is, in theory,
no compelling reason why it could not be used for microbial active ingredients.
However, as indicated by the authors, the method relies on the ability to link field
mortality directly to the pesticide application. The longer the time between exposure
and measurement of damage, the more difficult it is to establish such a correlation in
the field. This can certainly cause problems for microbial pest control agents
particularly those that have an infective rather than toxic mode of action. For example
entomopathogenic fungi can often have a considerable latent period of several days
or longer. Further the availablity of the required data is often limited for microbials.
Nevertheless, the NAESI approach has great potential, and its implementation in risk
assessment models is highly recommended.
17

ERBIC Risk Indicator
To our knowledge, the only attempt to numerically describe the environmental risks
of biological organisms was developed by the New Zealand Environmental Risk
Management Authority (ERMA). First developed by Hickson et al. (2000) to assess
the risks related to the introduction of genetically modified organisms into New
Zealand, this approach was further developed in the European Union research
project ERBIC (Evaluating Environmental Risks of Biological Control Introductions
into Europe; Hokkanen et al., 2003; van Lenteren et al., 2003). In part, this approach
has also been adopted by the OECD for the review of inundative releases of
beneficial insects (OECD, 2004a).
The underlying principle for the ERBIC/ERMA system is the definition of risk as the
product of the probability of an effect, and the magnitude of its impact. The risk of
adverse effects of a certain organism is scored and calculated on the basis of this
principle in five separate risk categories, which are then added up to form the overall
risk indicator. The categories used are (i) the risk of establishment in non-target
habitat, (ii) dispersal potential, (iii) host range, (iv) direct and (v) indirect effects.
One advantage of this system is the very simple calculation and ease with which to
interpret the risk score, hence allowing a direct and meaningful comparison between
different agents.
The authors of the ERBIC report have attempted to compare the obtained risk scores
to values obtained for some conventional chemicals and microbial control agents for
illustrative purposes. While this may be useful for a rough comparison, it is necessary
to exercise caution when applying the system beyond its original scope without
modification, because the risk categories were clearly defined with the goal of
assessing insect biocontrol agents.
In the context of this paper, a key weakness of the ERBIC/ERMA approach is that
the applied risk categories are treated as independent of each other. As was
discussed for the Norwegian Indicator, the simple addition of sub-indices does not
reflect the complexity of the interaction between all defined categories.
Despite this weakness, the basic principles in calculating sub-indices are valid and
merit considerations when constructing a risk indicator for a related purpose.
Defining a Risk Indicator suitable to compare biological and convential
pesticides
When building a risk indicator for pesticides, a number of factors need to be taken
into account.
Upfront, it is important to differentiate between ‘risk indicator’ systems, which are
intended to summarize the environmental risks for the purpose of making
environmental policy decisions and communicating risk, and ‘impact assessment
systems’, which can be used to accurately predict impacts on a particular
environmental component (Levitan, 2000; Mineau et al., 2008). With the exception of
the NAESI approach, all the systems presented above fall under the ‘risk indicator’
category.
Upon review of several reports on this topic (Reus et al., 2002; Mineau & Whiteside,
2005) we deemed the following factors to be most important for successfully deriving
a risk indicator model to allow for a adequate comparison between conventional and
microbial control agents:
18

Firstly, it is necessary to have a specific purpose in mind for which the risk
assessment is to be used. Further, the system should be relatively simple to use and
understand to allow for broad acceptance (this should, however, not be achieved at
the expense of accuracy). To be useful in judging environmental risk it is critical for
an indicator system to score risk, not hazard.
Moreover, a risk indicator should discriminate between higher and lower risk
substances, should avoid a bias resulting from different mechanisms of action, and
should take into account both acute and chronic effects where possible. In this
context it should be noted, that the actual numerical differences are less significant
than the trends that they show.
It is critical that the pesticide application method and formulation type are factored
into the environmental risk assessment. This is necessary because the use pattern of
a substance dictates how different groups of organisms are exposed.
Finally, a system should be adaptable to accommodate new scientific information
without major revision and should allow for input of expert judgement where data
gaps are identified.
We consider these factors essential for such a system to yield valid and valuable
results for the comparative assessment of environmental pesticide risks.
Proposed risk indicator and rationale
Based on the above arguments we propose a risk indicator for balanced comparative
assessment of both conventional and biological pesticides. The main emphasis of
our system is to focus on microbial pesticides, but the proposed structure should also
enable the assessment of substances with other modes of action such as
semiochemicals or growth regulators.
Basic components and their integration
Five basic components are proposed for the calculation of the risk indicator (RI).
These are the persistence of the substance (P), the dispersal potential (D), the range
of non-target organisms that are affected (N), direct effects (E D ) and indirect effects
(E I ) on the ecosystem.
Each of the component values consists of a ‘likelihood’ and a ‘magnitude’ factor. Both
values score on a scale between 1 and 5, resulting in a component range from 1 to
25, where 25 marks the highest risk. The direct effects score (E D ) is multiplied by a
weighting factor (W) if vertebrates or other groups of specific importance are
affected.
These components are modified from the ERBIC model to accommodate microbial
and conventional chemical pesticides. The category of vertebrate toxicity was added
to give special consideration to this group.
Because both persistence and dispersal potential increase the potential for negative
effects in the other categories, we propose the following integration of the variables
to calculate the overall environmental risk score as follows:
RI = (P+D)*[N+(E D *W)+E I ]
19

The multiplication of non-target risks with persistence and dispersal factors is in our
opinion the only appropriate way to account for the elevated risk resulting from
increased exposure of non-targets.
There are three factors at play which affect environmental pesticide risk: these are
label rates, formulation type, and application method. We consider it necessary to
calculate the risk indicator on an application basis for the end-use product, as label
rates, application method and formulation will strongly affect environmental exposure.
Scoring and rationales
Persistence. Persistence of an active ingredient in the environment is an important
factor in determining its risk because it strongly influences the likelihood for nontarget
organism exposure. However, it is a difficult task to define a scoring system
where conventional chemicals and microbials can be fairly compared. Clearly, living
organisms can have an entirely different behaviour in the environment than
chemicals in that they can proliferate in the environment. On the other hand microorganisms
often have a narrow host range and may, in the absence of a suitable host
for proliferation, degrade in the environment similarly to chemical substances.
Further, it is important to note that, from a risk assessment perspective, an organism
or substance naturally present in the environment must be regarded differently than a
new species or substance introduced into an ecosystem.
We postulate that a naturally occurring substance or organism will pose no additional
risk to the environment if introduced into a comparable system at similar
concentrations. For instance, the concentrations of entomopathogenic fungi often
heavily fluctuate depending on host densities and micro-climatic conditions, and
concentrations found during naturally occurring epizootics are often as high as those
found after artificial inoculations (Kessler, 2004; Laengle, 2005).
Therefore, a relatively high and persistent concentration of an indigenous organism in
the environment, even if a result of artificial inoculation, does not necessarily add an
environmental risk. Contrarily, persistence of non-indigenous micro-organism in the
environment also means prolonged exposure of potential non target organisms that
may never have been previously exposed to the micro-organism. Likewise, persistent
or increasing concentrations of a micro-organism in the absence of its natural host
could be an indicator of vegetative growth or even multiplication in non-target hosts.
To account for these considerations in a systematic manner, we propose to score the
persistence component in our risk indicator as laid out in Table 1. The persistence
score for both chemicals and microbials is dependent on its half-life in the absence of
the target host, where the half-life is the highest value from all environmental
compartments. The reduced risk for indigenous micro-organisms and naturally
occurring substances is taken into account by assigning a low persistence score if
concentrations return to levels comparable to natural concentrations within one or
two years of application.
Table 1 Scores assigned for persistence of the assessed active ingredient. Values
are assigned based on its half-life in the environmental compartment where the agent
is most stable, or based on the percentage of CFUs (colony forming units) of BCA
found one or two years post application (in target absence).
20

Persistence factor
Score
Persistence in target absence (use column 2 or 3, depending on available
information)
1 2 ( = 1)
No BCA detectable in soil 1 yr after application
(or at levels found naturally for indigenous T 0.5 < 30 d
species)
2 2 ( = 2)
> 0 % - 16 % of original CFUs 1 yr after application
T
(for indigenous species: at natural levels after 2 0.5 = 0.1 a - 0.25 a (36 d - 91
d)
yrs)
3 2 (= 9) 16 % - 40 % of original CFUs 1 yr after application T 0.5 = 0.25 a - 0.75 a (91 d -274
d)
4 2
40 % -62 % of original CFUs 1 yr after application T
(= 16)
0.5 = 0.75 a – 1.5 a
5 2 No significant reduction of CFUs 1 yr after
T
(= 25) application
0.5 > 1.5 a
T 0.5 stands for half life in the absence of the target.
For the purpose of the proposed risk indicator system, the persistence of biological
and conventional pesticides scores as described in Table 1. Where there is variation
of persistence in different environmental compartments, the highest value is applied
to determine the persistence score.
Dispersal. As with persistence, the dispersal potential of a substance or organism
greatly influences the likelihood of non-target exposure. The risk level resulting from
this component is dependent on the distance of dispersal and the quantity of
dispersed material.
Many factors affect the dispersal potential of pest control products. These include
spray drift, bioaccumulation, leaching and run-off. Specific to microbials is the
dispersal by infected organisms. We propose to calculate the dispersal risk factor as
detailed in Table 2 by multiplying the score for the maximum dispersed distance with
the score given for dispersed quantity. The dispersal factor is calculated on an
application basis, and will therefore vary significantly between, for instance, a spray
application and a seed treatment with the same active ingredient.
Table 2 Scoring of dispersal factor on the basis of dispersal distance and quantity.
Dispersal factor
Score Distance Quantity
1 < 10 m < 1 %
2 < 100 m < 5 %
3 < 1 000 m < 10 %
4 < 10 000 m < 25 %
5 > 10 000 m > 25 %
21

For a given chemical pesticide, the dispersal factor would be calculated by
determining the maximum dispersal distance and quantity, taking into account spray
drift, run-off and leaching, dispersal via bioaccumulation, and through evaporation.
Similarly, when scoring for micro-organisms spray drift, leaching and run-off will be
considered. Additionally, the dispersal through infected organisms must be
accounted for, as this can involve the transport of significant quantities of infectious
material over long distances.
In the absence of more detailed knowledge the following generalized worst case
assumptions are applied: for spray applications, drift is assumed to disperse 10 % of
the applied active ingredient up to 100 m (Pest Management Regulatory Agency,
2007); for microbials pathogenic to insects, unless other information is available, a
dispersal distance of up to 1000 m is assumed due to the possibility of infected
insects spreading the inoculum.
Range of non target effects. Most bacteria and fungi have, under certain
conditions, the potential to act as opportunistic pathogens and infect species
normally not susceptible to the organism. These conditions include high inoculum
concentrations, climatic conditions suitable for the micro-organism, and/or
circumstances under which the immune response of the potential host is weakened.
It is, therefore, important to differentiate between the physiological and the ecological
host range (Onstad & McManus, 1996; Jaronski et al., 1998; Jaronski et al., 2003;
Meyling et al., 2005). For an environmental risk assessment only the ecological host
range of a biological control agent is of interest. As a consequence, laboratory
studies on non-target organisms should be designed to reflect natural conditions.
Likewise for chemicals, the target range assessment should be done at
concentrations realistic for a typical application scenario. Where available NOEL
levels, dose-response curves or risk quotients (estimated environmental
concentration over toxicity) should be used for this assessment.
Table 3 Assignment of scores for non-target effects. Modified from Hickson et al.
(2000) and Hokkanen et al. (2003) to better suit the inclusion of chemical pesticides
in the proposed system.
Range of non-target effects
Score Likelihood Magnitude
1 1 species Genus
2 2-3 species Family
3 4-10 species Order
4 11-30 species Class
5 > 30 species >= Phylum
Once the range of possible affected non-targets is determined, the score in this
category can be calculated as described in Table 3. Should significant indirect effects
occur, the number of affected species (irrespective of the taxonomic level) must be
included when scoring for the range of non-target species.
While the scoring guidance for non-targets described above is relatively
straightforward, the antagonistic suppression of microbial growth to control plant
22

diseases is difficult to capture in this section. Effects described above focus on nontarget
mortality - an approach not suited to describe the dynamics of a microbial
community. Therefore, in the absence of more detailed knowledge, the likelihood and
magnitude values in this category are set to three for antagonistic micro-organisms.
As in the previous category, the risk score results from the multiplication of likelihood
and magnitude of the effect.
Direct Effects. Whereas the host/target range describes the spectrum of organisms
affected by the pest control product, the direct and indirect effects characterise the
impact on the most sensitive non-target organism or group. This portion of the risk
indicator is calculated by multiplication of the likelihood of an adverse effect
(mortality) with its magnitude. The likelihood of non target mortality can, for instance,
be deduced from dose/response data or an ‘impact assessment system’ such as the
NAESI approach presented above (compare Mineau et al., 2008).
Somewhat more challenging is the assessment of the non-target effects for
antagonistic organisms used for disease suppression. These organisms do not kill
their targets, but rather suppress the growth of disease-causing micro-organisms by
establishing in their ecological niche. Capturing the suppression of non target microorganisms
by an antagonist in the proposed indicator will only be considered if a
substantial long-term suppression and/or consequential indirect effects are expected.
For instance, suppression of a saprotrophic basidiomycete fungus in the soil after
application of an antagonist to wild blueberry would be considered as a non-target
effect, whereas transient changes in the soil or phylloshpere microbiota would not.
The scoring regime for direct effects, described in Table 4, allows for an assessment
based on likelihood and magnitude of the effect (derived from ERBIC system), or
alternatively on the basis of the highest risk quotient as determined from a
deterministic risk assessment approach as described by PMRA (Pest Management
Regulatory Agency, 2007).
Table 4. Risk of direct effects. The overall score is either the product of scores
assigned for likelihood and magnitude, or the square of the score assigned for the
highest risk quotient as described by the PMRA.
Direct Effects
Score Likelihood Magnitude Score
Highest
Quotient
1
Very unlikely (<
< 5 % mortality
1 %)
1 2 ( = 1) < 0.1
2 Unlikely (1 %-10 %) < 50 % mortality 2 2 ( = 4) < 1
3
Possible (10 %- > 50 % mortality or > 10 %
3 2 ( = 9)
50 %)
short term suppression
< 10
4 Likely (50 %-80 %)
> 50 % mortality ore > 10 %
4 2 (= 16)
permanent suppression
< 100
> 10 % long term
5 Very likely (> 80 %) suppression or local 5 2 (= 25) > 100
extinction
Risk
23

Indirect Effects. Indirect effects in this context are defined as changes in the
ecosystem that are a consequence of a direct effect caused by the use of a pesticide.
For instance, the reduction of pollinators will reduce the reproduction of plants that
rely on these pollinators. Likewise, pesticide effects on earthworms, aquatic plants, or
microbial activity will temporarily or permanently alter the ecosystem the product is
applied to. Because of their indirect nature, the level of uncertainty in assessing
these effects is greater than for the other components of the risk indicator system,
and the assessment must mainly rely on expert judgement. However, the estimation
of how severe the occurrence of a non-target effect is on an ecosystem level is
considered significant for the appropriate assessment of pesticide risk to the
ecosystem.
Table 5. Score for indirect effects on the ecosystem. Assignments are generally
based on expert judgement as indicated by the ecosystematic position of organisms
affected by direct effects, and the severity of anticipated effects.
Indirect Effects
Score Likelihood Magnitude
1 Very unlikely no significant impact on whole ecosystem
2 Unlikely
minor and short-term impact on parts of
ecosystem
3 Possible
significant short-term impact on parts of
4 Likely
ecosystem
significant long-term impact on parts of
ecosystem
5 Very likely significant long-term impact on whole ecosystem
Table 5 describes the scoring mechanism for indirect effects. As a general guidance,
substances that score high in the host range and direct effects sections will in most
cases have a higher likelihood for indirect effects.
Vertebrate toxicity. We propose to apply a weighting factor to direct effects (E D ) to
increase the value for non-target losses if vertebrates are affected. The weighting
scheme is described in Table 6.
Table 6 Factors assigned whether or not direct effects concern vertebrates.
Vertebrate toxicity
Factor Magnitude
1 No vertebrates are affected
1.5 Poikilothermic vertebrates are affected by non-target effects
2 Homeothermic vertebrates are affected
24

Demonstration of Risk Indicator using selected pest control agents
In order to demonstrate the validity of the proposed risk indicator we applied this
system to a number of well-studied biological control agents and selected chemical
products used for similar purposes. Organisms scored were Bacillus thuringiensis,
Beauveria brongniartii, Beauveria bassiana, Coniothyrium minitans, Metarhizium
anisopliae, Pantoea agglomerans, Pseudomonas fluorescens, Trichoderma
harzianum; conventional pesticides assessed were atrazine, chlorpyrifos, benomyl,
DDT, methyl bromide, phorate and streptomycin. Indices were calculated using open
literature and published regulatory documents. The results are displayed in Table 7.
The organisms with the lowest risk indicator were soil applied fungi with very narrow
host ranges when used in environments to which they are native. These organisms
consistently scored low in all categories. Biocontrol agents with broader host ranges
delivered by spray application typically had a higher dispersal potential and also
scored higher under direct and indirect effects, but remained about one magnitude or
more below conventional chemical alternatives.
The highest scoring substances were DDT, methyl bromide, and chlorpyriphos.
These scores were largely a consequence of high persistence and dispersal
potential, combined with wide target ranges and high values assigned for direct and
indirect effects.
On average, biopesticides had an approximately 40 times (range 9-200) lower risk
indicator than conventional products used for the same purpose.
Conclusions and envisaged applications
The proposed risk indicator is, to our knowledge, the first indicator to allow a direct
numerical comparison of relative environmental risks posed by microbials and
conventional chemical pesticides.
While microbials are often reported to pose low risks to the environment (OECD,
2007), it is of critical importance for the credibility of the promoters of microbial pest
control products to be able to underline such generic statements with solid data.
The presented framework permits the unbiased generation of an environmental risk
score for biological and chemical products on the basis of scientific and regulatory
data.
Key advantages of the proposed system compared to previously available risk
indicator systems are (i) the applicability to biological and conventional pesticides to
allow a direct comparison between products, (ii) the ability to score the risk on an
application basis rather than on an active ingredient basis, (iii) the flexibility of the
system that permits the use of regulatory data or published literature, and (iv) a
readily understandable output. The latter allows for a broader discussion beyond the
highly specialised expert community of the environmental advantages certain
products may offer from an environmental risk perspective.
In our analysis of selected products we found that the environmental risk score
greatly varied within the assessed chemical products, and also, yet at a much lower
level, within the group of microbial products. Further to this, it was demonstrated, that
the use pattern of a product has a great influence on the estimated environmental
risk posed by a specific product.
25

Table 7 Risk scores and calculated risk indicator for selected microbial and conventional pest control products.
Active Ingredient
Persistence factor
Dispersal
factor
Distance
Quantity
Host Range Direct
Effect
Species
Taxonomic
level
Likelihood
Magnitude
Indirect
Effects
Likelihood
Magnitude
Vertebrate effects
Risk Score
Sources
Bacillus thuringiensis (foliar spray) 4 2 3 4 4 3 2 2 3 1 280
Beauveria brongniartii (soil) 1 3 1 2 1 1 1 1 1 1 16
Beauveria bassiana (foliar spray) 1 3 3 4 4 3 2 2 2 1 260
Beauveria bassiana (soil) 1 3 1 4 4 3 2 1 2 1 96
Joung & Coté (2000); Pest Management
Regulatory Agency (2006c)
Kessler et al. (2004); Kessler (2004);
Laengle et al. (2005); Laengle (2005);
Traugott et al. (2005)
Vänninen et al. (2000); BIPESCO (2001);
Hokkanen et al. (2003); Butt (2004);
Laengle (2006); US EPA, (2006a)
Vänninen et al. (2000); BIPESCO (2001);
Hokkanen et al. (2003); Butt (2004);
Laengle (2006); US EPA, (2006a)
Coniothyrium minitans (soil) 1 1 3 1 1 1 1 2 2 1 24 US EPA (2002b)
Metarhizium anisopliae (soil) 1 3 1 4 4 3 2 1 2 1 96
Metarhizium anisopliae (foliar
spray)
Pantoea. agglomerans (foliar
spray)
1 3 3 4 4 3 2 1 2 1 240
1 3 2 3 3 2 1 1 1 2 98
Vänninen et al. (2000); BIPESCO (2001);
Hokkanen et al. (2003); Butt (2004);
Meyling et al. (2005)
Vänninen et al. (2000); BIPESCO (2001);
Hokkanen et al. (2003); Butt (2004);
Meyling et al. (2005)
Pest Management Regulatory Agency
(2006a)
26

Pseudomonas fluorescens (foliar
spray)
1 2 3 3 3 2 1 1 1 1.5 91
Trichoderma harzianum (soil) 4 1 1 3 3 3 2 2 2 1 95
Chlorpyrifos (foliar spray) 9 5 4 5 5 5 5 5 3 2 2610
Lindemann et al., (1982); Beattie & Lindow
(1995); Beattie & Lindow (1999); Lindow &
Brandl (2003); Lindow & Suslow (2003);
Moore et al. (2004); US EPA (2006b)
Pest Management Regulatory Agency
(2006b)
US EPA (2002a); Pest Management
Regulatory Agency (2003a)
Benomyl (foliar spray) 16 2 3 5 5 5 4 5 3 2 1760 Extension Toxicology Network (1996a)
Methyl bromide (fumigation) 25 5 5 5 5 5 3 3 3 2 3200 Extension Toxicology Network (1996b)
Streptomycin (foliar spray) 9 3 3 5 5 3 3 5 3 1 882 US EPA (2006)
Atrazine (foliar spray) 25 5 4 5 5 5 5 3 3 1.5 3218
Pest Management Regulatory Agency
(2007); Mineau et al. (2008)
DDT (foliar spray) 25 5 4 5 5 5 5 5 4 2 4275 Extension Toxicology Network (1996c)
Phorate (granular) 9 5 3 5 5 5 5 3 3 2 2016
Pest Management Regulatory Agency
(2003b)
27

These results demonstrate that the proposed system enables a clear differentiation
between products and use patterns.
We acknowledge that no risk indicator scoring system can replace the risk
assessment frameworks used by regulatory bodies to reach a decision on the
acceptability of risks posed by a certain product. However, we can certainly envisage
the proposed risk indicator helping in regulatory decisions:
The system could help in defining low risk products. We consider a risk score of 100
to be a reasonable cut-off value for "low risk" products, whereas a threshold of 500
seems justified for the term "reduced risk". Regulatory implications should reduce
data requirements or acceptance data waivers upon the submission of a well
documented risk score, and a significant reduction of registration/submission fees for
"low risk" and "reduced risk" products. An example for such reduced fees is the
Canadian PMRA, which has waived review fees for certain reduced-risk products
such as microbial and semiochemical pesticides. Further, the risk score enables a
measurement of risk reduction strategies and the impact risk-mitigation restrictions.
A ranking of risks posed by various pest control products would allow for more
targeted efforts to reduce pesticide risks in agriculture, and could thus be highly
beneficial for the environment.
For industry representatives, the calculation of the environmental risk score for both
conventional chemicals and microbial agents could assist in the decision on which
product to pursue at a relatively early stage of development. As indicated above, a
risk score that is well documented with data and/or scientific literature could be used
to support the request to waive certain environmental data requirements in support of
registration.
Finally, the risk indicator data allows the activation of clauses in pesticide legislation
that allow regulators to consider relative risk in comparison with other products
available for the same uses (such as those in Canada’s updated Pest Control
Products Act).
We believe that the proposed system can help stakeholder groups (i.e. risk
assessors, academia, and industry) to facilitate discussions regarding the regulatory
approaches to microbial and other biological pest control organisms. Maximum
benefit from this system could be achieved through establishing a web-based
database that could serve as a reference for experts, growers, and the interested
public and allow informed decisions in areas ranging from product development and
registration to product choice on the farm level.
Acknowledgements
This work was supported by the European Commission, Quality of Life and
Management of Living Resources Programme (QoL), Key Action 1 on Food, Nutrition
and Health, QLK1-2001-01391 and Specific Support Action, SSPE-022709. We also
wish to thank Brian Belliveau (Pest Management Regulatory Agency of Canada),
Anita Fjelsted (Danish Ministry of the Environment), Stefan Jaronski (USDA ARS
Northern Plains Agricultural Research Laboratory), Olaf Strauch (University of Kiel),
and Stefan Hutwimmer (University Innsbruck), for their helpful discussion and kindly
reviewing the manuscript.
28

References
Beattie GA & Lindow SE (1999) Bacterial colonization of leaves: a spectrum of
strategies. Phytopathology 89: 353-359.
Beattie GA & Lindow SE (1995) The secret life of foliar bacterial pathogens on
leaves. Annu Rev Phytopathology 33: 145-172.
BIPESCO (2001) WP6: Ecological studies and field trials. Biocontrol of important soil
dwelling pests by improving the efficacy of insect pathogenic fungi. EC project
FAIR6 CT-98-4105. Final report (Strasser H, ed), pp. 262-384, Innsbruck, Austria.
Butt TM (2004) Risk Assessment of Fungal Biocontol Agents. EC Project QLK1-CT-
2001-01391. Final Report, Swansea, UK.
Cook RJ, Bruckart WL, Coulson JR, Goettel MS, Humber RA, Lumsden RD, Maddox
JV, McManus ML, Moore L & Meyer SF (1996) Safety of Microorganisms
Intended for Pest and Plant Disease Control: A Framework for Scientific
Evaluation. Biol Control 7: 333-351.
Eilenberg J, Hajek A & Lomer C (2001) Suggestions for unifying the terminology in
biological control. BioControl 46: 387-400.
Environmental Protection Agency (2006) 40 CFR parts 158 and 172 pesticides; data
requirements for biochemical and microbial pesticides; proposed rule. [EPA–HQ–
OPP–2004–0415; FRL–7763–4] RIN 2070–AD51. Federal Register 71: 12072-
12117.
European Parliament (2003) European Parliament resolution on the Commission
report entitled: Evaluation of the active substances of plant protection products
(submitted in accordance with Article 8(2) of Council Directive 91/414/EEC on the
placing of plant protection products on the market) (COM(2001) 444 - C5-
0011/2002 - 2002/2015(COS)). Plant protection products. P5_TA(2002)0276.
Official J Eur Union C 187 E: 173-179.
Extension Toxicology Network (1996a) Benomyl. Pesticide Information Profiles.
http://extoxnet.orst.edu/pips/benomyl.htm, accessed Jun 18, 2007.
Extension Toxicology Network (1996b) Methyl Bromide. Pesticide Information
Profiles. http://extoxnet.orst.edu/pips/methylbr.htm, accessed Jun 18, 2007.
Extension Toxicology Network (1996c) DDT (dichlorodiphenyltrichloroethane).
Pesticide Information Profiles. http://extoxnet.orst.edu/pips/ddt.htm, accessed Jun
18, 2007.
Government of Canada (2002) Pest Control Products Act. P-9.01.
http://laws.justice.gc.ca/en/ShowFullDoc/cs/P-9.01/, accessed Jun 20, 2007.
Hickson R, Moeed A & Hannah D (2000) HSNO, ERMA and risk management. New
Zealand Sci Rev 57: 72–77.
Hokkanen HMT, Babendreier D, Bigler F, Burgio G, Kuske S, van Lenteren JC,
Loomans AJM, Menzler-Hokkanen I, van Rijn PCJ, Thomas MB, Tommasini MG
& Zeng QQ (2003) Evaluating Environmental Risks of Biological Control
Introductions in Europe. FAIR-CT97-3489.
http://honeybee.helsinki.fi/MMSBL/MAEL/Hankkeet/ERBIC/ERBIC%20FINAL%20
REPORT.pdf, accessed May 13, 2007.
International Biocontrol Manufacturers Association (2003) IBMA Position Concerning
Revision of Directive 91/414 CEE. Position Papers.
http://www.ibma.ch/pdf/20030905_revision_91414_position_ibma.pdf, accessed
Feb 5, 2008.
29

Jaronski ST, Goettel MS & Lomer CJ (2003) Regulatory requirements for ecotoxicological
assessments of microbial insecticides - How relevant are they?
Environmental Impacts of Microbial Insecticides: Need and Methods for Risk
Assessment (Hokkanen, H. M. T. & Hajek AE, eds), pp. 237-260. Kluwer
Academic Publishers, Dordrecht.
Jaronski ST, Lord J, Rosinska J, Bradley CA, Hoelmer K, Simmons G, Osterlind R,
Brown C & Staten R (1998) Effect of a Beauveria bassiana-based
mycoinsecticide on beneficial insects under field conditions. The 1998 Brighton
Conference. Pests & Diseases 2: 651-657.
Joung K & Coté J (2000) A review of the environmental impacts of the microbial
insecticide Bacillus thuringiensis. Technical Bulletin No. 29. http://dsppsd.pwgsc.gc.ca/Collection/A54-9-29E.pdf,
accessed Feb 15, 2008.
Kabaluk T & Gazdik K (2007) Directory of Microbial Pesticides for Agricultural Crops
in OECD Countries.
http://www4.agr.gc.ca/resources/prod/doc/pmc/pdf/micro_e.pdf, accessed Feb 14,
2008.
Kessler P (2004) Influence of soil factors on virulence, growth and survival of the
fungus Beauveria brongniartii, a specific bioocontrol agent of the European
cockchafer (Melolontha melolontha). Dissertation Thesis, Swiss Federal Instiute
of Technology, Zürich.
Kessler P, Enkerli J, Schweizer C & Keller S (2004) Survival of Beauveria brongniartii
in the soil after application as a biocontrol agent against the European cockchafer
Melolontha melolontha. BioControl 49: 563-581.
Kovach J, Petzoldt C, Degni J & Tette J (1992) A Method to Measure the
Environmental Impact of Pesticides. NY Food and Life Sci Bull 139: 1-8.
Laengle T (2006) Beauveria bassiana (Bals.-Criv.) Vuill. – A biocontrol agent with
more than 100 years of history of safe use. Workshop on Current Risk
Assessment and Regulation Practice Salzau, Germany, September 18-22, 2006.
http://www.rebeca-net.de/downloads/Beauveria%20bassiana.pdf, accessed Jun
18, 2007.
Laengle T (2005) Beauveria brongniartii (Sacc.) Petch as a fungal biocontrol agent:
Environmental risk assessment. Dissertation Thesis, Leopold-Franzens University
Innsbruck, Austria.
Laengle T, Pernfuss B, Strasser H & Seger C (2005) Field efficacy evaluation of
Beauveria brongniartii against Melolontha melolontha in potato cultures. Sydowia
57: 54-66.
Levitan L (2000) “How to” and “why”: assessing the enviro–social impacts of
pesticides. Crop Prot 19: 629-636.
Lindemann J, Constantinidou HA, Barchet WR & Upper CD (1982) Plants as sources
of airborne bacteria, including ice nucleation-active bacteria. Appl Environ
Microbiol 44: 1059-1063.
Lindow SE & Brandl MT (2003) Microbiology of the phyllosphere. Appl Environ
Microbiol 69: 1875-1883.
Lindow SE & Suslow TV (2003) Temporal dynamics of the biocontrol agent
Pseudomonas fluorescens strain A 506 in flowers in inoculated pear trees.
Phytopathology 93: 727-737.
Meyling NV, Eilenberg J & Nielsen C (2005) Non-target effects of insect pathogenic
fungi. IOBC wprs Bull 28: 13-20.
30

Mineau P (2002) Estimating the probability of bird mortality from pesticide sprays on
the basis of the field study record. Env Tox Chem 21: 1497-1506.
Mineau P & Whiteside M (2005) Development of Comparative Environmental Risk
Assessment Tools for Pesticides in Support of Standard Development At
Environment Canada. National Agri-Environmental Standards Initiative (NAESI).
Report No. 1-17.
Mineau P, Dawson T, Whiteside M, Morrison C, Harding K, Singh L, Längle T &
McQueen DAR (2008) Environmental Risk-based Standards for Pesticide Use in
Canada. National Agri-Environmental Standards Initiative Synthesis Report
(Draft).
Moore ERB, Tindall BJ, Martins dos Santos VAP, Pieper DH, Ramos J & Palleroni
NJ (2004) Pseudomonas: Nonmedical. The Prokaryotes (web version).
http://141.150.157.117:8080/prokPUB/chaphtm/392/COMPLETE.htm, accessed
Jun 19, 2006.
Norwegian Agricultural Inspection Service (2002) Pesticide Risk Indicators for Health
and Environment – Norway. Translated from Norwegian.
http://landbrukstilsynet.mattilsynet.no/vedlegg/Norwegian_pesticide_risk_indicator
s.doc, accessed Feb 5, 2008.
OECD (2007) Biological Pesticide Registration. Environment Directorate.
http://www.oecd.org/document/8/0,2340,en_2649_34383_31962760_1_1_1_1,00
.html, accessed Jun 20, 2007.
OECD (2004a) Guidance for Information Requirements for Regulation of
Invertebrates as Biological Control Agents (IBCAs). Environment Directorate.
ENV/JM/MONO(2004)1. http://www.oecd.org/dataoecd/6/20/28725175.pdf,
accessed Feb 5, 2008
OECD (2004b) Summary Report of the OECD Project on Pesticide Terrestrial Risk
Indicators. http://www.oecd.org/dataoecd/53/59/34896939.pdf, accessed April 28,
2007.
OECD (2003) Guidance for Registration Requirements for Microbial Pesticides.
Environment Directorate. ENV/JM/MONO(2003)5.
http://www.oecd.org/dataoecd/4/23/28888446.pdf, accessed Feb 5, 2008
Onil S, Dion S, St-Laurant L & April M (2007) Indicateur de risque des pesticides du
Québec. Governement du Québec, Ministère de l’Agriculture, des Pêcheries et de
l’Alimentation du Québec. http://www.inspq.qc.ca/pdf/publications/602-
IndicateurDeRisqueDesPesticides.pdf, accessed Feb 5, 2008.
Onstad DW & McManus ML (1996) Risks of host range expansion by parasites of
insects. Bioscience 46: 430-435.
Pest Management Regulatory Agency (2007) Re-evaluation of Atrazine
(Environmental Assessment). Proposed Acceptablity for Continuing Registration
PACR2007-05. www.pmra-arla.gc.ca/english/pdf/pacr/pacr2007-05-e.pdf,
accessed Jun 18, 2007.
Pest Management Regulatory Agency (2006a) Pantoea agglomerans Strain C9-1.
Regulatory Note REG2006-13. http://www.pmraarla.gc.ca/english/pdf/reg/reg2006-13-e.pdf,
accessed Jun 18, 2007.
Pest Management Regulatory Agency (2006b) RootShield Biological Fungicide
Trichoderma harzianum Rifai strain KRL-AG2. Regulatory Note REG2002-01.
http://www.pmra-arla.gc.ca/english/pdf/reg/reg2002-01-e.pdf, accessed Jun 18,
2007.
31

Pest Management Regulatory Agency (2006c) Re-evaluation of Bacillus
thuringiensis. Proposed Acceptablity for Continuing Registration PACR2006-09.
http://www.pmra-arla.gc.ca/english/pdf/pacr/pacr2006-09-e.pdf, accessed Jun 29,
2007.
Pest Management Regulatory Agency (2003a) Phase 2 of the Re-evaluation of
Chlorpyrifos. Proposed Acceptablity for Continuing Registration. PACR2003-03.
http://www.pmra-arla.gc.ca/english/pdf/pacr/pacr2003-03-e.pdf, accessed Jun 18,
2007.
Pest Management Regulatory Agency (2003b) Re-evaluation of Phorate. Proposed
Acceptablity for Continuing Registration PACR2003-01. http://www.pmraarla.gc.ca/english/pdf/pacr/pacr2003-01-e.pdf,
accessed Jun 29, 2007.
Pest Management Regulatory Agency (2001) Guidelines for the Registration of
Microbial Pest Control Agents and Products. Regulatory Directive DIR2001-02.
http://www.pmra-arla.gc.ca/english/pdf/dir/dir2001-02-e.pdf, accessed Jun 18,
2007.
REBECA (2007) Regulation of Biological Control Agents. http://rebeca-net.de,
accessed February 2, 2008.
Reus J, Leendertse P, Bockstaller C, Fomsgaard I, Gutsche V, Lewis K, Nilsson C,
Pussemier L, Trevisan, M, van der Werf H, Alfarroba F, Blümel S, Isart J,
McGrath D, Seppälä T (1999) Comparing environmental risk indicators for
pesticides. Results of the European CAPER Project. Centre for Agriculture and
Environment, Utrecht.
Reus J, Leendertse P, Bockstaller C, Fomsgaard I, Gutsche V, Lewis K, Nilsson C,
Pussemier L, Trevisan M, van der Werf H, Alfarroba F, Blümel S, Isart J, McGrath
D & Seppälä T (2002) Comparison and evaluation of eight pesticide
environmental risk indicators developed in Europe and recommendations for
future use. Agric, Ecosystems & Environ 90: 177-187.
Scientific Committee on Plants (2002) Opinion of the Scientific Committee on Plants
on tthe Working Document from the Commission establishing Annex VI to Council
Directive 91/414/EEC (Document SANCO/108/2002). Document SCP/ANNEX VI
B/002-Final http://europa.eu.int/comm/food/fs/sc/scp/out147_ppp_en.pdf,
accessed Aug 8, 2006
Strasser H & Kirchmair M (2006) Potential health problems due to exposure in
handling and using biological control Agents. An ecological and societal approach
to biological control (Eilenberg J & Hokkanen H. M. T., eds), pp. 275-293.
Springer, Dordrecht.
Traugott M, Weissteiner S & Strasser H (2005) Effects of the entomopathogenic
fungus Beauveria brongniartii on the non-target predator Poecilus versicolor
(Coleoptera: Carabidae). Biol Control 33: 107-112.
US EPA (2006) Report of the Food Quality Protection Act (FQPA) Tolerance
Reassessment Progress and Risk Management Decision (TRED) for
Streptomycin. EPA 738-R-06-012.
http://www.epa.gov/oppsrrd1/REDs/streptomycin_tred.pdf, accessed Jun 19,
2007.
US EPA (2002a) Interim Reregistration Eligibility Decision for Chlorpyrifos. EPA 738-
R-01-007. http://www.epa.gov/oppsrrd1/REDs/chlorpyrifos_ired.pdf, accessed
Jun 19, 2007.
32

US EPA (2002b) Coniothyrium minitans strain CON/M/91-08. Biopesticides
Registration Action Document.
http://www.epa.gov/pesticides/biopesticides/ingredients/tech_docs/brad_028836.
pdf, accessed Jun 29, 2007.
US EPA (2006a) Beauveria bassiana Strain GHA (128924) Technical Document.
http://www.epa.gov/pesticides/biopesticides/ingredients/tech_docs/tech_128924.h
tm, accessed Sep 12, 2006.
US EPA (2006b) Frost-Preventing Bacteria: Pseudomonas fluorescens A506
(006438); Pseudomonas fluorescens 1629RS (006439); Pseudomonas syringae
742RS (006411) Products. Regulating Pestidices. Ingredient Fact Sheets.
http://www.epa.gov/pesticides/biopesticides/ingredients/product/prod_006438.htm
, accessed Sept. 14, 2006.
US EPA (2000) The role of use-related information in pesticide risk asssessment and
risk management. Science Policy. Office of Pesticide Programs.
http://www.epa.gov/oppbead1/use-related.pdf, accessed February 2, 2008.
van Lenteren, JC, Babendreier D, Bigler F, Burgio G, Hokkanen HMT, Kuske S,
Loomans AJM, Menzler-Hokkanen I, van Rijn PCJ, Thomas MB, Tommasini MG
& Zeng QQ (2003) Environmental risk assessment of exotic natural enemies used
in inundative biological control. BioControl 48: 3-38.
Vänninen I, Tyni-Juslin J & Hokkanen H (2000) Persistence of augmented
Metarhizium anisopliae and Beauveria bassiana in Finnish agricultural soils.
BioControl 45: 201-222.
33