ISO/TC 34/SC 005 - Izba Mleka

© ISO 2010 – All rights reserved© ISO 2010 – All rights reserved© ISO 2010 – All rights reserved 

ISO/ISO/ISO/TC 343434/SC 005005005 

Date: 2010-12-72010-12-72010-12-7 

ISO/DIS 16297ISO/DIS 16297ISO/DIS 16297 

IDF 161 

ISO/ISO/ISO/TC 343434/SC 005005005/WG 

Secretariat: NENNENNEN 

Milk — Bacterial count — Protocol for the evaluation of alternative 

methodsMilk — Bacterial count — Protocol for the evaluation of 

alternative methodsMilk — Bacterial count — Protocol for the evaluation 

of alternative methods 

Lait — Dénombrement bactériologique — Protocole pour l'évaluation de méthodes alternativesLait — 

Dénombrement bactériologique — Protocole pour l'évaluation de méthodes alternativesLait — 

Dénombrement bactériologique — Protocole pour l'évaluation de méthodes alternatives 

Warning 

This document is not an ISO International Standard. It is distributed for review and comment. It is subject to 

change without notice and may not be referred to as an International Standard. 

Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of 

which they are aware and to provide supporting documentation. 

Document type: International StandardInternational StandardInternational Standard 

Document subtype: 

Document stage: (40) Enquiry(40) Enquiry(40) Enquiry 

Document language: EEE 

T:\PRG Work\IDF Working Bodies\SCSA\2012\S07\S07 ISO corrected DIS 16297 IDF 161 

120511.docH:\DATA\IDF\IDF 161 2011-12 ---\ISO 16297 2012-01-03 AA 20022012 IA 2012-05- 

11.docxC:\Users\ba523\AppData\Local\Microsoft\Windows\Temporary 

Internet 

Files\Content.Outlook\SWD7669A\ISO 16297 2012-01-03 AA 20022012.docx STD Version 2.2

ISO/DIS 16297 

Copyright notice 

This ISO-IDF document is an International Technical Specification and is copyright-protected by ISO and 

IDF. Except as permitted under the applicable laws of the user's country, neither this ISO-IDF document nor 

any extract from it may be reproduced, stored in a retrieval system or transmitted in any form or by any 

means, electronic, photocopying, recording or otherwise, without prior written permission being secured. 

Requests for permission to reproduce this document for the purpose of selling it should be addressed as 

shown below or to ISO's member body in the country of the requester: 

ISO copyright office 

IDF Central Secretariat 

Case postale 56° Silver Building Boulevard Auguste Reyers 70 

CH-1211 Geneva 20 Swiss 

1030 Brussels Belgium 

Tel. + 41 22 749 01 11 Tel. + 32 2 733 98 88 

Fax + 41 22 749 09 47 Fax + 32 2 733 04 13 

E-mail copyright@iso.ch 

E-mail info@fil-idf.org 

Web www.iso.ch 

Web www.fil-idf.org 

Reproduction for sales purposes may be subject to royalty payments or a licensing agreement. 

Violators may be prosecuted. 

ii 

© ISO 2010 – All rights reserved© ISO 2010 – All rights 

reserved© ISO 2010 – All rights reserved


Contents 

Page 

Foreword ...................................................................................................................................................... iviviv 

Foreword ......................................................................................................................................................... vvv 

Introduction .................................................................................................................................................. vivivi 

1 Scope .................................................................................................................................................. 111 

2 Normative reference .......................................................................................................................... 111 

3 Terms and definitions ....................................................................................................................... 111 

4 Transformation of results ................................................................................................................. 111 

5 Attributes of the alternative method ................................................................................................ 111 

5.1 Description of the method to be evaluated......................................................................................... 2 

5.2 Measuring range .................................................................................................................................... 2 

5.3 Carry-over .......................................................................................................................................... 444 

5.4 Calibration and standardizationError! Bookmark not defined.Fel! Bokmärket är inte definierat.Fel! Bokmärket 

5.5 Precision............................................................................................................................................. 666 

5.6 Evaluation of factors affecting the results...................................................................................... 777 

6 The alternative method as an estimate of the reference method ................................................. 777 

6.1 Evaluation of factors affecting the estimation ............................................................................... 777 

6.2 Measurement protocol ...................................................................................................................... 888 

6.3 Calculations ....................................................................................................................................... 888 

6.4 Range of validity ................................................................................................................................ 988 

6.5 Accuracy of the estimate .................................................................................................................. 999 

6.6 Attributes of the alternative method expressed in units of the reference method .................... 999 

7 Rating of the elaborated attributes ............................................................................................ 101010 

Annex A (informative) EXPRESSION OF PRECISION PARAMETERS ................................................. 111111 

Bibliography .............................................................................................................................................. 121212 


reserved© ISO 2010 – All rights reserved 

iii


Foreword 

ISO (the International Organization for Standardization) is a worldwide federation of national standards 

bodies (ISO member bodies). The work of preparing International Standards is normally carried out through 

ISO technical committees. Each member body interested in a subject for which a technical committee has 

been established has the right to be represented on that committee. International organizations, governmental 

and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the 

International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization. 

International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. 

The main task of technical committees is to prepare International Standards. Draft International Standards 

adopted by the technical committees are circulated to the member bodies for voting. Publication as an 

International Standard requires approval by at least 75 % of the member bodies casting a vote. 

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent 

rights. ISO shall not be held responsible for identifying any or all such patent rights. 

ISO 16297|IDF 161 was prepared by Technical Committee ISO/TC 34, Food products, Subcommittee SC 5, 

Milk and milk products and the International Dairy Federation (IDF). It is being published jointly by ISO and 

IDF. 

All work was carried out by the Joint ISO-IDF Project Group of the Standing Committee on Statistics and of 

Automation under the aegis of its project leader, Mrs. I. Andersson (SE) . 

iv 




Foreword 

IDF (the International Dairy Federation) is a non-profit organization representing the dairy sector worldwide. 

IDF membership comprises National Committees in every member country as well as regional dairy 

associations having signed a formal agreement on cooperation with IDF. All members of IDF have the right to 

be represented at the IDF Standing Committees carrying out the technical work. IDF collaborates with ISO in 

the development of standard methods of analysis and sampling for milk and milk products. 

The main task of Standing Committees is to prepare International Standards. Draft International Standards 

adopted by the Standing Committees are circulated to the National Committees for endorsement prior to 

publication as an International Standard. Publication as an International Standard requires approval by at least 

50 % of IDF National Committees casting a vote. 

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent 

rights. IDF shall not be held responsible for identifying any or all such patent rights. 

ISO 16297|IDF 161 was prepared by Technical Committee ISO/TC 34, Food products, Subcommittee SC 5, 

Milk and milk products and the International Dairy Federation (IDF). It is being published jointly by ISO and 

IDF. 

All work was carried out by Joint ISO-IDF Project Group (S07) of the Standing Committee on Statistics and 

Automation under the aegis of its project leader, Mrs. I. Andersson (SE) . 

This first edition of ISO 16297|IDF 161 cancels and replaces IDF 161A:1995, which has been technically 

revised. 



v


Introduction 

Any quantitative measurement in microbiology should consider that the microbiological state in a sample must 

be regarded as one point within the co-ordinates of a multidimensional system, which is to be projected onto 

the one-dimensional scale of the method applied i.e. plate count, flow cytometry. Aspects such as flora (types 

and numbers of micro-organisms and their distribution), growth phase, sub-lethal damage, metabolic activity 

and history, influence to a greater or lesser extent any parameter that is measured. It is evident that any 

projection of an n-dimensional situation onto an one-dimensional scale is bound to provide a picture of the real 

situation that will be rather restricted. In this respect one has to bow to the inevitable, regardless of which 

method of measurement is preferred. 

The term reference (or official or anchor) method in this International Standard means a method internationally 

recognized by experts, used in legislation, or by agreement between the parties. The evaluation of an 

alternative method has to refer to the reference method and has to be based on the examination of suitable 

samples for its intended use. 

vi 



DRAFT INTERNATIONAL STANDARD ISO/DIS 16297 

Milk — Bacterial count — Protocol for the evaluation of 

alternative methods 

1 Scope 

This International Standard gives guidelines for the evaluation of alternative methods for bacterial count in raw 

milk from animals of different species. 

Note: The document is considered complementary to ISO 16140 and ISO 8196|IDF 128. 

2 Normative references 

The following referenced documents are indispensable for the application of this document. For dated 

references, only the edition cited applies. For undated references, the latest edition of the referenced 

document (including any amendments) applies. 

ISO 5725-1, Accuracy (trueness and precision) of measurement methods and results -- Part 1: General 

principles and definitions 

ISO 5725-2, Accuracy (trueness and precision) of measurement methods and results -- Part 2: Basic method 

for the determination of repeatability and reproducibility of a standard measurement method 

ISO 8196-1 | IDF 128-1, Milk — Definition and evaluation of the overall accuracy of alternative methods of 

milk analysis — Part 1: Analytical attributes of alternative methods 

ISO 8196-2 | IDF 128-2, Milk — Definition and evaluation of the overall accuracy of alternative methods of 

milk analysis — Part 2: Calibration and quality control in the dairy laboratory 

ISO 11095, Linear calibration using reference materials 

ISO 16140, Microbiology of Food and animal feeding stuffs – Protocol for the validation of alternative methods 

ISO 21187 | IDF 196:2004, Milk – Quantitative determination of bacteriological quality – Guidance for 

establishing a conversion relationship between routine method results and anchor method results and its 

verification 

3 Terms and definitions 

For the purposes of this document, the terms and definitions given in ISO 8196-1|IDF 128-1 and ISO 8196- 

2|IDF 128-2 apply. 

4 Transformation of results 

A prerequisite for statistics most common in the evaluation of measuring methods is the approximation of a 

normal distribution of the data. The exponential multiplication of microorganisms usually leads to a right-tailed 

distribution of quantitative microbiological parameters. Thus, in general transformation of the raw data is 

necessary for approximation of normality. This is usually a common logarithmic transformation or a square 

root transformation for low bacteria levels. The most appropriate transformation can be checked by comparing 

histograms. In the examples outlined in the following, not corresponding to low bacteria levels, common 

logarithmic transformation was used. All statistics are then computed from the transformed data and only the 

final results are re-transformed to give a more expressive idea of the situation to the user (see also Annex A). 

5 Attributes of the alternative method 



1


NOTE The parameters outlined in the following text do not need to be evaluated completely for every alternative 

method. For example the measuring range (see 5.2) of the plate loop method is determined by the loop(s) used. 

NOTE 

Results are expressed in the units of the routine method unless specified otherwise. 

5.1 Description of the method to be evaluated 

5.1.1 Description 

The description of the method under study should be in line with the checklist in 5.1.2. 

Most of the information is found in the specification of the method given by the responsible supplier or any 

other source (author) of the described technique. 

5.1.2 Checklist 

a) Principle of the method. 

b) Parameter and/or unit. 

c) Technical design of the measurement procedure. 

d) Field of application: 

1) Purpose: for example, research, screening, milk grading, etc.; 

2) Substrate: for example, raw milk, fermented milk products. from cows 

e) Supplier(s) of instrument, reagents, standards. 

f) Specification of the method given by producer or author: 

1) Prerequisites for sampling (often compared to the situation of fat analysis); 

2) Possibilities for sample preservation (reagent(s), storage condition(s)); 

3) Quantitative (units: method under study and/or reference method) and qualitative (the kind of microorganisms 

covered) spectrum; 

4) Precision (in units of the method under study and/or in reference method units); 

5) Accuracy of the estimate (in reference method units); 

6) Samples per hour; 

7) List of references. 

5.2 Measuring range 

5.2.1 Lower limit of quantification 

The lower limit of quantification is defined as the average of blank values of the reagents plus the n-fold of its 

standard deviation; generally, n = 10. See also ISO 16140. 

5.2.2 Upper limit of quantification 

2 © ISO 2010 – All rights reserved© ISO 2010 – All rights 



The upper limit of quantification is determined by the highest possible reading of the method or 

methodological limitations, for example, coincidence effects, inaccuracy in the upper measuring range, 

clogging of filters. Coincidence is when two or more elements of the measurand are detected simultaneously 

and identified as only one unit. For example with flow cytometry, if two cells pass the detector simultaneously 

they will be detected as one. The coincidence effect will be higher with higher concentrations of a measurand. 

For the assessment of the upper limit of quantification the following procedure is recommended: 

A total of 10 milk samples with very high bacterial counts are diluted with low count milk in steps of 0,25 times 

the common logarithm, which is equivalent to a dilution factor of 1,78. Each dilution step is analysed. 

For each of the 10 samples a dilution that is not influenced by coincidence or methodological background 

noise is chosen as basis for the evaluation. Starting from the measuring value of this dilution step the 

expected values for each of the other dilution steps are computed. 

For a given level of measurand coincidence effect is characterized by the coincidence factor, that is the 

number or fraction of elements interpreted as only one and expressed by the ratio of expected value to the 

measured (observed) value 

f c = C exp /C obs , where 

C exp is the mean concentration expected from the dilution ratio; 

C obs is the mean concentration measured 

The coincidence factor is plotted against the measuring level. By visual inspection of the plot the values which 

are not yet influenced by coincidence can be derived. An example is given in Figure 1. 

In many cases it might be possible to correct for the coincidence effect, because in general it is a systematic 

effect and has a functional relation to the level of the parameter. 

10 

f c 

8 

6 

LOQ u 

4 

2 

0 

4 5 6 7 

x 

Key 

x 

f c 

Log bacterial count of milk samples 

Coincidence factor 

f c for individual milk samples 

Trendline 

Square markers 

Dashed line 

Solid line Coincidence factor: 1,0 

LOQ u 

Upper limit of quantification 

Figure 1 — Coincidence factor with regard to total bacterial count in raw milk 



3


5.2.3 Linearity of the instrument signal 

The relationship between the instrument readings and the expected values should be linear within the 

concerned range of microbial bacterial counts. Deviations from linearity may stem from non-specific signals 

and coincidence effects. 

A linearity check is at first performed visually using appropriate figures graphsso as to obtain an clear picture 

impression of the shape of the relationship. Whenever deviation from linearity appears evident, a quantitative 

parameter is used as a testcalculated to indicate whether the observed trend is acceptable or not. 

To achieve this, use can be made of a high microbial bacterial count milk diluted serially with low microbial 

bacterial count milk, resulting in a set of at least five eight samples covering the concentration range of 

interest. 

Measure all samples at least four times and calculate the average result for each sample. This gives the 

measured value per sample. Use the measured values for Measure the high microbial count milk and the low 

microbial count milk in at least quadruplicate and calculate the average result for each sample. to Ccalculate 

values for the intermediate samples from the applied mixing ratios. per sample, resultingThis results in an 

expected value for each sample. Then measure all samples in at least quadruplicate and calculate the 

average result for each sample, which is equivalent to the measured value per sample. Aapply linear 

regression with the expected values per sample on the x-axis and the measured values per sample on the y- 

axis. Calculate the residuals e i = y i – (b*x i + a) from the regression. Plot the residuals e i (y axis) versus the 

expected values (x axis) on a graph. A visual inspection of the data points will usually yield sufficient 

information about the linearity of the signal. Any outlying residual should lead to delete deletion of the related 

result and to renewal of the calculation process before applying the further test. 

When observed, tThe curving can be expressed by the ratio, r, by using the following equation: 

r 

where 

 

e 

e 

 

max min 

( M 

max 

M 

min 

100 

) 

e max 

e min 

is the numerical value of the maximum residual from the regression; 

is the numerical value of the minimum residual from the regression; 

M max is the numerical value of the upper measured value for the concerned set of sampleshigh count 

milk; 

M min 

is the numerical value of the lower measured value for the concerned set of sampleslow count milk 

The ratio, r, should be less than 4 5 % (?). 

5.3 Carry-over 

Carry-over effects might occur in analytical systems that operate continuously. It derives from the transfer of a 

certain portion of sample material from one test sample to the next or further sample(s). 

Due to the design of the mechanized process of analysis not only the next sample but also samples in a later 

position can be influenced as e.g. due to incubation wellsvials with a periodic circulation. 

This effect can be tested by analysing consecutively milk with high bacterial count and blank samples. Carryover 

will cause an increase of blank sample values compared to normal blank sample value (value of blank 

sample analysed after another blank sample). 

The carry-over can be expressed as percentage of the corresponding preceding milk sample, 




For evaluation of carry-over, the number of samples and the bacterial count of the milk samples should be 

high enough to estimate the carry- over with sufficient certainty. The samples should be representative of the 

routine samples, especially regarding the storage time (longer storage time leading to higher milk viscosity 

and potentially higher carry over). Relative uncertainty of the estimation should be less than 20%. One way of 

setting up the test is described in the example below. 

For detailed and theoretical aspects and alternative setups of carry-over estimation it is referred to ISO 8196-3 

| IDF 128-3 [1]. 

Example: 

One way to estimate the carry overcarry-over effect is to analyse at least 10 sets of samples, each set 

containing one milk sample with very high bacterial count followed by two blank samples. Blank samples could 

be water or milk with negligible bacterial count. 

(milk, blank 1 , blank 2 ) 1 , (milk, blank 1 , blank 2 ) 2 , … (milk, blank 1 , blank 2 ) n , 

The relative carry-over (%) can be calculated for each sample set and then averaged: 

x 100, 

, where 

COR i = relative carry-over in the i th sample set 

c b1i = result of the first blank sample in the i th sample set 

c b2i = result of the second blank sample in the i th sample set 

c mi = result of the milk sample in the i th sample set 

COR = relative carry-over 

n = number of sample sets 

Even a very low carry-over effect might be relevant if the corresponding preceding sample has a very high 

level in comparison to the next one. It might even cause the result of the next sample to exceed a given limit. 

In many cases it is possible to correct for the carry-over effect, because, in general, it is a systematic effect 

and has a functional relation to the level of the corresponding preceding sample. Carry-over should preferably 

be below 1 %. 

An example of carry-over effect is given in Figure 2. The results of blank solutions analysed immediately after 

high count samples (blank 1 ) are plotted against the results of the corresponding preceding milk samples. From 

the graph the measuring level of preceding milk samples which can lead to an increase of the blank values 

above the accepted level can be derived. The relation between sample and blank values can be approximated 

by a function, e.g. a polynomial. 



5


Key 

x 

Bacterial count of milk samples in units/ml 

Bacterial count of blank solution analysed after milk sample in 

units/ml 

Results with individual sample sets 

Trend line 

y 

Square markers 

Dashed line 

Solid line Carry-over: 0% 

Carry over in this example is 1% 

5.4 Stability 

Figure 2 — Carry-over effect with regard to total bacterial count in raw milk 

It is essential to check the stability of the instrument with suitable samples. 

For many microbiological methods, reference materials of the measured component are not available or its 

widespread application under field conditions is not possible, due to short shelf life and thus restricted 

transportability. 

This deficiency has to be compensated for by a reference material substitute or a ring test procedure. The 

relevant characteristics of a reference material substitute should be as similar as possible to the nature of the 

components and the substrate in which the measurement takes place. 

In those cases where reference material substitutes with longer shelf life are available, the stability of 

instrumental methods should be checked throughout the working day and also during the period between 

instrument standardization operations (quality control in the laboratory). 

Protocols for standardization and stability checks are described in ISO 8196-2 | IDF 128-2. 

5.5 Precision 

5.5.1 Terms and definitions 

For the definitions of precision, repeatability and reproducibility, and guidance on their determination, see ISO 

5725-1, ISO 5725-2, ISO 8196-1|IDF 128-1 and ISO 16140. 

5.5.2 Repeatability 




The repeatability can be estimated from a large number (n = 50 - 100) of measurements in duplicate made 

from samples covering the whole measuring range. If the repeatability is dependent on the level, it should be 

specified as a function of the level, otherwise an average value will suffice. 

For total bacterial count Tthe acceptability limits for the intra-laboratory repeatability standard deviation (s r ) 

are: 

0,09 log 10 units for contamination levels ≥ 2 x 10 4 cfu/ml 

0,12 log 10 units for contamination levels < 2 x 10 4 cfu/ml. 

5.5.3 Reproducibility 

The reproducibility should be estimated through an interlaboratory study according to ISO-5725-2 from 

duplicate measurements in representative samples at lower, medium and upper level in the measuring range, 

preferably obtained from at least 8 collaborators. 

The organizer, using all recorded data shall determine which results are suitable for use in data analysis. The 

organizer shall examine the raw data and other collected information to ascertain that all collaborators have 

performed the analyses as prescribed. When there is evidence that results might be obtained under 

inappropriate conditions and/or the method has not been followed strictly, these or all results from the 

collaborator are excluded for further analysis. No outlier tests are performed on the selected data. 

If no relationship exists between repeatabilty and the level, this can also be assumed to be true for the 

reproducibilty. If there is a relationship between the reproducibilty and the level, it should be specified. 

For total bacterial count tThe acceptability limit for the reproducibility standard deviation (s R ) is 0,16 log 10 units. 

5.6 Evaluation of factors affecting the results 

All non-bacteriological factors associated with the properties of the milk sample which could disturb the 

measurements by the alternative method should be evaluated. Examples of factors are somatic cell count, 

composition of milk, history of milk, sampling of milk, preservation of milk, species and breed of animals. 

It should be carefully considered which effects different factors may cause, and experiments should be 

designed taking these into account. 

Examples 

If it can be expected that the linearity may be affected by a certain factor (e.g. fat content), the linearity test 

should be repeated using samples with a low and high content of this affecting factor. If it is expected that the 

repeatability may be affected, the repeatability test should be repeated using samples with high and low 

content. Certain preservatives may affect the level of counts, to check for this analyse a series of samples with 

and without addition of preservative. 

6 The alternative method as an estimate of the reference method 

This clause addresses the analysis of the interrelations between the results of the alternative method and the 

reference method. For the establishment and verification of a conversion relationship, see ISO 21187 | IDF 

196. 

The analysis of the relation between two methods is based on the examination of test materials with both 

methods, covering the field of application and its spectrum of samples to be analysed with the method under 

study. 

6.1 Evaluation of factors affecting the estimation 

All factors (bacteriological and non-bacteriological) associated with the properties of the milk sample that 

could affect the estimation - that is, the relation between reference and routine method - are to be considered 



7


in order to make sure that . Ssamples chosen to evaluate the relationship should beare representative to the 

normal routine samples regarding these properties. 

Factors influencing the relation can be bacteriological or non-bacteriological, for example. 

These influences can be type of bacteria, growth phase, storage condition, sample preservation, geographic 

differences, seasonal variations, species and breed of the animals from which the milk originates, method of 

milking, disinfection, feeding methods or individual supplier. 

6.2 Measurement protocol 

6.2.1 Design 

The evaluation of the alternative method as an estimate of the reference method requires a great amount of 

different samples (typically 100 per log step). A minimum number of samples can be calculated according to 

ISO 21187 | IDF 196:2004, Annex A. 

Samples should: 

a) be natural milk samples; 

b) uniformly cover the whole range of interest; 

c) be representative of the routine samples to be analysed especially taken into account the above 

mentioned factors. 

Samples are analysed in duplicate with the reference method as well as with the alternative method at the 

same time or close to it. 

6.3 Calculations 

6.3.1 Visual check of a scatter diagram 

Before any calculation is made a scatter diagram should be checked visually to obtain a first impression of the 

character of the relationship and determine whether the expected relationship between the methods is 

approximated. 

6.3.2 Regression 

To find a useful function to describe the relationship, several statistic approaches can be used which all have 

their particularities and limits, but by combining the results a sufficiently exact solution can be approximated. 

6.3.2.1 Regression models 

After suitable (usually logarithmic) data transformation, the relationship between the alternative method and 

the reference method is assumed to be linear within the range of interest. In that case the relationship can be 

calculated by means of a simple linear regression model (y = abx + ba). 

If the relationship is not linear, it is recommended to seek advice from a statistician. Methods to check for 

linearity are specified in ISO 8196-3 | IDF 128-3, or use a regression program which gives the probability of 

lack of fit or non-linearity. 

If the repeatability error on for one or both methods after data transformation is still dependent on the level, 

apply a weighted least-squares method in accordance with ISO 11095. In such cases, it is recommended to 

seek advice from a statistician. 

In the general case where regression is applied to calibration, the vertical y-axis (dependent variable) is used 

for the reference method and the horizontal x-axis (independent variable) is used for the alternative method 




(Ordinary Least Square regression) since it is assumed that the repeatability error of the reference method is 

generally much larger than the repeatability error of the alternative method (ratio >2). When necessary, the 

random error of the alternative method can be reduced with more replicates (see ISO 8196-2). 

6.3.2.2 Outliers 

Outliers should be carefully scrutinized. No data should be discarded unless there is a sound microbiological 

reason to do so. For outlier evaluation see ISO 21187 | IDF 196. 

6.4 Range of validity 

The range that can be adequately described by a chosen function shall be expressed in units of the reference 

method. 

6.5 Accuracy of the estimate 

The accuracy of the estimate is a measure of the reliability of the estimation of the value with one method from 

the measured value of the other. 

The residual standard deviation s y,x from the linear regression (y = abx + ba) gives an average measure for the 

accuracy of the estimate throughout the whole range where this linear regression is valid. It should be 

estimated for the circumstances that also apply in the routine application of the alternative method. If, for 

example, the alternative method is routinely applied in singular, single results of the alternative method 

against the average of duplicate results of the reference method should be used for the estimation of s y,x . 

In general the value of the reference method has to be estimated from the routine result. Thus, in accordance 

with ISO 8196-3 | IDF 128-3, the routine results are regarded as the independent variable x and the reference 

results as dependent variable y (see Figure 3). The accuracy of the estimate is expressed in units of the 

reference method and is independent on whether or not the method is properly calibrated and standardized. 

This has the advantage that different routine methods can be compared with respect to this attribute. 

For total bacterial count the acceptability limit for the accuracy standard deviation (s y,x ) is 0,40 log 10 units. 

6.6 Attributes of the alternative method expressed in units of the reference method 

Again, the results hereafter depend on the function by which the relation is described. Thus, the function used 

has to be clearly stated. 

After the appropriate regression function is identified, all attributes of the alternative method, which are 

expressed in the units of the alternative method, can be transformed onto the reference scale by using the 

regression formula. 

For all parameters elaborated in clause 5, which are expressed in the units of the alternative method, 

equivalence in units of the reference method can be computed using the regression function. If in the example 

given in Figure 3Fel! Hittar inte referenskälla., 5 000 units of the routine method (lg value = 3,7) corresponds 

to a certain attribute, e.g. the limit of quantification, that value has to be included in the regression function as 

x. The value calculated for y is the corresponding average reference value lg value =3,12 or 1 318 units). 

The corresponding confidence level can be derived by including sy,x. 



9


Key 

x 

y 

Round markers 

Solid line 

Routine method results (logarithmic values) 

Reference/anchor method results (logarithmc values) 

Results of individual milk samples 

Trendline 

Figure 3 — Relation between the results of a routine method and a reference method 

In this way the measuring range (5.2), the carry-over (5.2.3) and the precision (5.5) of the alternative method 

can be translated onto the scale of the reference method. Thus, once evaluated for different alternative 

methods, these parameters can be compared. 

7 Rating of the elaborated attributes 

In general a final report should summarize the elaborated attributes of the alternative method under study. A 

rating of the results should be given by expert opinion, especially in the light of the requirements set for the 

alternative method by its intended use. 

Among others the following aspects might be of special importance. 

a) Range of the parameter of the routine method to be expected under the conditions of its intended use. For 

example with milk grading, the routine method should have adequate discriminative power in the range of 

interest, thereby also having still a margin for situations where a future shift to stricter grading limits is 

anticipated. 

b) If a grading scheme is not based on single results but on the averages of the results of several 

measurements (for example, throughout a month), an appropriate measuring range of the routine method 

with regard to the parameter’s distribution in the population is of special importance: low measurements 

must not be disturbed by method background noise, otherwise the "buffer" to compensate for accidentally 

high measurements in a series is lost. This may lead to an unfair grading. 

c) The representativeness of sampling and preservation facilities plays an important role in the rating of a 

routine method for the quantitative determination of the bacteriological quality of milk. 




Annex A 

(informative) 

Expression of precision parameters 

This tThe text in this Annex is in line toline with the one described bysuggestions prepared by Piton & Grappin 

[11]. 

To take into account the fact that the logarithmic transformation of data does not provide parameters which 

are easy to use, it is proposed to characterize the repeatability and reproducibility of microbiological methods 

by the following three different parameters: 

a) the logarithmic standard deviation; 

b) the geometric relative standard deviation; 

c) the critical difference between duplicates. 

The repeatability and reproducibility standard deviations s r * and s R *, expressed on a logarithmic scale, are the 

most straightforward statistical parameters and should be reported. 

Unfortunately, these values cannot be used directly to draw inferences about the range of dispersion of results 

one may expect when performing replicate measurements. If the geometric mean = gives a 

satisfactory measure of the location of the median of the original log-normal population, the geometric 

standard deviation (s g = ) should be used as a multiplication factor to measure the dispersion of the data 

about the geometric mean. 

When considering the distribution curve of microbiological analytical data before and after logarithmic 

transformation, where z = lg x, the range for z of * ± s* is equivalent to that for x being obtained by multiplying 

and dividing by . In other words, * + s* is equivalent to × and * - s* is equivalent to / . 

A geometric coefficient of variation (GCV), expressed as a percentage, can be substituted, using the formula 

C V,g = ( 1) × 100. This GCV will be the second parameter to be used to characterize the repeatability 

and reproducibility of the method. For example, if s r * = 0,07, then C V,g = ( - 1) × 100 = 17,5 %. The limits 

of the range of variation of the original results about the geometric mean corresponding to ± s* will be 

× [(100+17,5)/100] = × 1,175 and / [(100 +17, 5)/100] = / 1,175. 

The third parameter, which is very useful for the analyst, is the critical relative difference between two 

measurements (or RD 95 ) which indicates that, when two measurements are performed according to the same 

technique under conditions of repeatability or reproducibility, the highest result should not exceed the lowest 

by more than ( - 1) x100 in 95 % of the cases. For example, with s r * = 0,07, the critical relative 

difference between two results is: RD 95 = ( - 1) x 100 = 57 %. 

In other words, when performing two independent measurements, the highest result should not exceed the 

lowest by more than 57 % in 95 % of cases. If the lowest result is 100 000 cfu/ml, the highest should be lower 

than 157 000 cfu/ml in 95 % of cases. 



11


Bibliography 

[1] ISO 8196-3 | IDF 128-3, Milk — Definition and evaluation of the overall accuracy of alternative methods of 

milk analysis — Part 3: Protocol for the evaluation of alternative quantitative methods of milk analysis 

[2] Deming, W.E., Statistical Adjustment of Data. John Wiley & Sons, New York 1943. 

[3] GRAPPIN, R., DASEN, A., FAVENNEC, P., Numération automatique et rapide des bactéries du lait cru à l'aide 

du BactoScan [Rapid automatic enumeration of bacteria in untreated milk using BactoScan]. Lait 1985, 

65, pp. 123-147. 

[4] JARVIS, B., EASTER, M.C., Rapid methods in the assessment of microbiological quality; experiences and 

needs. Journal of Applied Microbiology 1987, 63, pp. 115s-126s. 

[5] KREYSZIG, E. 2 ed. Statistische Methoden und Anwendungen. Vandenhoeck und Ruprecht, Göttingen 

1967. 

[6] MANDEL, J., Fitting Straight Lines When Both Variables are Subject to Error. Journal of Quality Technology 

1984, 16, pp. 1-14. 

[7] MANNERS, J.G., Trends in microbiological testing of milk and dairy products. Australian Journal of Dairy 

Technology 1985, 40, pp. 73-75. 

[8] NIEUWENHOF, F.F.J., HOOLWERF, J.D., Suitability of Bactoscan for the estimation of the bacteriological 

quality of raw milk. Milchwissenschaft 1988, 43, pp. 577-580, 585-586. 

[9] NIEUWENHOF, F.F.J., HOOLWERF, J.D., LANGEVELD, L.P.M., WAALS, C.B.V.D., Modern methods of assessing 

the bacteriological quality of milk. Zuivelzicht 1985, 77, pp. 668-671, 699-701. 

[10] O'CONNOR, F., Rapid test methods for assessing microbiological quality of milk. Australian Journal of Dairy 

Technology 1984, 39, pp. 61-65. 

[11] PITON, C., GRAPPIN, R., A model for statistical evaluation of precision parameters of microbiological 

methods: application to dry rehydratable film methods and IDF reference methods for enumeration of total 

aerobic mesophilic flora and coliforms in raw milk. Journal - Association of Official Analytical Chemists 

1991, 74, pp. 92-103. 

[12] REICHMUTH, J., SUHREN, G., UBBEN, E.H., HEESCHEN, W., Assessment of a routine method: suitability as 

measuring method with regard to the reference method by the example of fluorescence microscopic 

counting of individual microorganisms and macrocolony counting in raw milk. Kieler Milchwirtschaftliche 

Forschungsberichte 1989, 41, pp. 15-33.

ISO/TC 34/SC 005 - Izba Mleka

Create successful ePaper yourself

Delete template?

Save as template?