SIST EN ISO 17994:2014

SLOVENSKI STANDARD
SIST EN ISO 17994:2014
01-september-2014
1DGRPHãþD
SIST EN ISO 17994:2004
Kakovost vode - Zahteve za primerjavo relativne ponovitve mikroorganizmov z
dvema kvantitativnima metodama (ISO 17994:2014)
Water quality - Requirements for the comparison of the relative recovery of
microorganisms by two quantitative methods (ISO 17994:2014)
Wasserbeschaffenheit - Anforderungen für den Vergleich der relativen Wiederfindung
von Mikroorganismen durch zwei quantitative Verfahren (ISO 17994:2014)
Qualité de l'eau - Exigences pour la comparaison du rendement relatif des
microorganismes par deux méthodes quantitatives (ISO 17994:2014)
Ta slovenski standard je istoveten z: EN ISO 17994:2014
ICS:
07.100.20 Mikrobiologija vode Microbiology of water
13.060.70 Preiskava bioloških lastnosti Examination of biological
vode properties of water
SIST EN ISO 17994:2014 en
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

---------------------- Page: 1 ----------------------

SIST EN ISO 17994:2014

---------------------- Page: 2 ----------------------

SIST EN ISO 17994:2014

EUROPEAN STANDARD
EN ISO 17994

NORME EUROPÉENNE

EUROPÄISCHE NORM
February 2014
ICS 07.100.20; 13.060.70 Supersedes EN ISO 17994:2004
English Version
Water quality - Requirements for the comparison of the relative
recovery of microorganisms by two quantitative methods (ISO
17994:2014)
Qualité de l'eau - Exigences pour la comparaison du Wasserbeschaffenheit - Anforderungen für den Vergleich
rendement relatif des microorganismes par deux méthodes der relativen Wiederfindung von Mikroorganismen durch
quantitatives (ISO 17994:2014) zwei quantitative Verfahren (ISO 17994:2014)
This European Standard was approved by CEN on 8 February 2014.

CEN members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for giving this European
Standard the status of a national standard without any alteration. Up-to-date lists and bibliographical references concerning such national
standards may be obtained on application to the CEN-CENELEC Management Centre or to any CEN member.

This European Standard exists in three official versions (English, French, German). A version in any other language made by translation
under the responsibility of a CEN member into its own language and notified to the CEN-CENELEC Management Centre has the same
status as the official versions.

CEN members are the national standards bodies of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia,
Finland, Former Yugoslav Republic of Macedonia, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania,
Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and United
Kingdom.





EUROPEAN COMMITTEE FOR STANDARDIZATION
COMITÉ EUROPÉEN DE NORMALISATION

EUROPÄISCHES KOMITEE FÜR NORMUNG

CEN-CENELEC Management Centre: Avenue Marnix 17, B-1000 Brussels
© 2014 CEN All rights of exploitation in any form and by any means reserved Ref. No. EN ISO 17994:2014 E
worldwide for CEN national Members.

---------------------- Page: 3 ----------------------

SIST EN ISO 17994:2014
EN ISO 17994:2014 (E)
Contents Page
Foreword .3
2

---------------------- Page: 4 ----------------------

SIST EN ISO 17994:2014
EN ISO 17994:2014 (E)
Foreword
This document (EN ISO 17994:2014) has been prepared by Technical Committee ISO/TC 147 "Water quality"
in collaboration with Technical Committee CEN/TC 230 “Water analysis” the secretariat of which is held by
DIN.
This European Standard shall be given the status of a national standard, either by publication of an identical
text or by endorsement, at the latest by August 2014, and conflicting national standards shall be withdrawn at
the latest by August 2014.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. CEN [and/or CENELEC] shall not be held responsible for identifying any or all such patent rights.
This document supersedes EN ISO 17994:2004.
According to the CEN-CENELEC Internal Regulations, the national standards organizations of the following
countries are bound to implement this European Standard: Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech
Republic, Denmark, Estonia, Finland, Former Yugoslav Republic of Macedonia, France, Germany, Greece,
Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal,
Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the United Kingdom.
Endorsement notice
The text of ISO 17994:2014 has been approved by CEN as EN ISO 17994:2014 without any modification.


3

---------------------- Page: 5 ----------------------

SIST EN ISO 17994:2014

---------------------- Page: 6 ----------------------

SIST EN ISO 17994:2014
INTERNATIONAL ISO
STANDARD 17994
Second edition
2014-02-15
Water quality — Requirements for the
comparison of the relative recovery of
microorganisms by two quantitative
methods
Qualité de l’eau — Exigences pour la comparaison du rendement
relatif des microorganismes par deux méthodes quantitatives
Reference number
ISO 17994:2014(E)
©
ISO 2014

---------------------- Page: 7 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO 2014
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2014 – All rights reserved

---------------------- Page: 8 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms, definitions and symbols . 1
3.1 Terms and definitions . 1
3.2 Symbols and abbreviated terms. 3
4 Principle . 3
5 Basic requirements for a comparison study . 4
5.1 General . 4
5.2 Description of methods . 4
5.3 Types of samples . 4
5.4 Number of samples and participating laboratories . 5
5.5 Counting and confirming. 7
6 Calculations. 8
6.1 Preliminary editing of the raw data . 8
6.2 Basic relative differences . 8
6.3 Half-width of the confidence interval . 9
7 Evaluation . 9
7.1 Preliminary evaluations . 9
7.2 Two-sided evaluation .10
7.3 One-sided evaluation .12
8 Test report .13
Annex A (informative) Flowchart .14
Annex B (informative) Comparison studies .15
Annex C (informative) Derivation of equation for calculation of the number of samples .17
Annex D (informative) Example of a two-sided evaluation
.19
Bibliography .22
© ISO 2014 – All rights reserved iii

---------------------- Page: 9 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2, www.iso.org/directives.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of any
patent rights identified during the development of the document will be in the Introduction and/or on
the ISO list of patent declarations received, www.iso.org/patents.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical Barriers
to Trade (TBT) see the following URL: Foreword - Supplementary information
The committee responsible for this document is ISO/TC 147, Water quality, Subcommittee SC 4,
Microbiological methods.
This second edition cancels and replaces the first edition (ISO 17994:2004), which has been technically
revised.
iv © ISO 2014 – All rights reserved

---------------------- Page: 10 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

Introduction
This International Standard specifies criteria and procedures for comparing the average quantitative
results obtained by two microbiological analytical methods, one of which may, but need not, be a
standard or reference method.
The methods considered are based on counts of colonies or of positive and negative liquid enrichment
tubes (MPN).
© ISO 2014 – All rights reserved v

---------------------- Page: 11 ----------------------

SIST EN ISO 17994:2014

---------------------- Page: 12 ----------------------

SIST EN ISO 17994:2014
INTERNATIONAL STANDARD ISO 17994:2014(E)
Water quality — Requirements for the comparison of the
relative recovery of microorganisms by two quantitative
methods
WARNING — Persons using this document should be familiar with normal laboratory practice.
This document does not purport to address all of the safety problems, if any, associated with its
use. It is the responsibility of the user to establish appropriate safety and health practices and to
ensure compliance with any national regulatory conditions.
IMPORTANT — It is absolutely essential that tests conducted in accordance with this document
be carried out by suitably qualified staff.
1 Scope
This International Standard specifies an evaluation procedure for comparing two methods with
established performance characteristics according to ISO/TR 13843 and intended for the quantification
of the same target group or species of microorganisms.
This International Standard provides the mathematical basis for the evaluation of the average relative
performance of two quantitative methods against chosen criteria for the comparison. It does not provide
data for assessment of the precision of the methods being compared. It is appropriate that the precision
of methods is assessed as part of their performance characterization.
This International Standard does not provide methods for the verification of method performance
characterization in a single laboratory.
2 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated
references, the latest edition of the referenced document (including any amendments) applies.
ISO 8199, Water quality — General guidance on the enumeration of micro-organisms by culture
ISO/TR 13843, Water quality — Guidance on validation of microbiological methods
3 Terms, definitions and symbols
3.1 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
3.1.1 General terms
3.1.1.1
comparison trial
interlaboratory method comparison that involves laboratories which perform paired measurements on
several of their own samples with two different methods
© ISO 2014 – All rights reserved 1

---------------------- Page: 13 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

3.1.1.2
not-different method
method considered quantitatively not different to another method when the mean difference between
their confirmed counts and stipulated difference lie between predetermined stipulated limits, taking
into account all sources of variation
Note 1 to entry: This difference can be assessed by the average relative difference of their confirmed counts.
3.1.1.3
predetermined stipulated limit
permitted average difference (based on a “confidence interval” designated −2L to +2L) between results
obtained by each method, based on professional practices or regulatory requirements
Note 1 to entry: Reference [1] suggests that, in international and interlaboratory method performance tests, a
limit of 2L = 10 % for setting the “confidence interval” be the predetermined stipulated limit for drinking water,
and this has been widely used. However, for environmental waters, such as bathing waters, Reference [2] proposes
a predetermined stipulated limit of 2L = 20 %.
3.1.1.4
reference method
method of analysis internationally recognized by experts or by agreement between the parties
Note 1 to entry: As a rule, the reference method is a standard or a commonly used method.
3.1.1.5
standard measurement uncertainty
measurement uncertainty expressed as a standard deviation
(3)
[SOURCE: ISO/IEC Guide 99:2007 , 2.30]
3.1.1.6
trial method
any method which is to be tested for comparison with a reference method
3.1.2 Specific terms
3.1.2.1
count
observed number of objects
EXAMPLE Colonies or cells of microorganisms, plaques of bacteriophages
Note 1 to entry: In this International Standard, the result of an MPN estimation is also considered to be a count.
3.1.2.2
presumptive count
colony count or most probable number (MPN) estimate based on the number of colonies or fermentation
tubes that have an outward appearance that is interpreted as typical of a target organism
3.1.2.3
confirmed count
presumptive count multiplied by the confirmation coefficient
3.1.2.4
relative difference
RD
difference between two results, a and b, measured on a relative (natural logarithmic) scale
2 © ISO 2014 – All rights reserved

---------------------- Page: 14 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

3.2 Symbols and abbreviated terms
A (symbol for the idea of) trial method
a test result by method A
a test result (confirmed count) of method A in sample i
i
B (symbol for the idea of) reference method
b test result by method B
b test result (confirmed count) of method B in sample i
i
i subscript indicating a series
k coverage factor used for calculating the confidence interval
L smallest microbiologically significant mean relative difference between the results by
methods A and B
MPN most probable number quantitative method
n number of samples
s experimental standard deviation of the relative difference (standard uncertainty)
2
s experimental variance
s standard deviation of the relative difference (standard uncertainty) of the mean
x
W half-width of the confidence interval
x relative difference
x value of the relative difference between a and b in sample i
i i i
x
arithmetic mean of x (i = 1,2 … n)
i
x value of the relative difference at the lower confidence limit, derived by subtracting the
L
value of the half-width of the confidence interval from the mean
x value of the relative difference at the upper confidence limit, derived by adding the value of
U
the half-width of the confidence interval to the mean
2
X experimental Poisson index of dispersion
y conditional variable used in computing the number of samples for comparison testing
4 Principle
This International Standard is based on the principle of the paired t-test (see Annex C).
The basic data are pairs of confirmed counts (a , b ) obtained from the examination of two equal portions
i i
taken from the same vessel of a carefully mixed test sample, one determination (count) per method. The
complete design consists of a large number of similar determinations.
In this International Standard, two methods are considered quantitatively “not different” if the mean
relative difference of the paired confirmed counts does not differ significantly from zero and the
© ISO 2014 – All rights reserved 3

---------------------- Page: 15 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

confidence interval does not extend beyond the level of the predetermined stipulated limit. The decision
rules based on the above principle are given in 7.2 and 7.3 and a flow chart is given in Annex A.
5 Basic requirements for a comparison study
5.1 General
Both methods shall have data on detailed performance characteristics, derived in accordance with the
guidance outlined in ISO/TR 13843.
The most important basic requirement of comparison studies is a wide range of samples. Participation
by a number of laboratories is preferable, allowing the expansion of the sample range over large
geographical areas. Also the credibility of a general conclusion is commonly believed to require the
participation of several laboratories. However, the inclusion of a wide range of sample types by a single
laboratory will also be valid. The result of the comparison is generally valid only within the range of
sample types studied. Advice on the conduct of comparison studies is given in Annex B.
It is essential that all laboratories taking part in a collaborative study have recognized quality assurance
systems in use and apply approved basic techniques of cultivation.
5.2 Description of methods
It is important to note that the principles of operation of the two methods being compared should be
well understood and that the significance of any differences in the methods on the outcome of the
comparative assessment should be recognized. This is particularly important if the confirmed results
from each method are based on different principles. Any differences should be detailed in the test report
(see Clause 8).
Performance characteristics data shall be derived in accordance with ISO/TR 13843. Such data for the
methods shall be compared in order to assess potential differences in performance.
EXAMPLE Methods for the enumeration of coliform bacteria based on possession of the enzyme β-galactosidase
have been shown to produce higher counts than those based on the fermentation of lactose due to the detection
of a greater range of coliform bacteria.
5.3 Types of samples
The requirements for method comparisons differ somewhat from the daily routine situation. It is useful
and often necessary to preselect or prepare special samples. Samples for method comparisons shall
contain enough target organisms that the likelihood of scoring a zero count is small.
Samples for method comparisons shall represent types that are included in the scope of both methods.
Natural samples are ideal. Samples to be tested shall represent those water source types relevant to the
geographical and environmental area where the method is applied. The water types to be tested shall
be included to the scope of the methods under evaluation. Appropriate samples may also be prepared
by dilution, spiking, or mixing of different kinds of water to achieve the desired population in a suitable
density. Spiking with pure cultures shall be considered the last resort.
To avoid the inhibition of the target organisms by other organisms, ensure that the concentration of
total bacteria in a sample is not too high. Consult ISO 8199 to ascertain the ranges of the colony counts
for different cultivation methods.
It can be appropriate to influence the microbial population of existing samples to simulate situations
encountered in routine laboratory practice. Such modifications could be the applications of disinfectants
(e.g. chlorine, ozone or UV, Reference [1]), different ranges of temperature or the influence of daylight,
in order to simulate different environmental situations from where the samples for laboratories can
originate.
4 © ISO 2014 – All rights reserved

---------------------- Page: 16 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

5.4 Number of samples and participating laboratories
5.4.1 Number of laboratories
The number of laboratories participating in comparison trials shall be sufficient to obtain a representative
result for the relative recovery of the two methods being tested. Factors that shall be considered when
deciding on the number of participating laboratories in a comparison trial include:
a) whether the alternative method is being assessed as a replacement for the reference method;
b) whether the comparison trials are for statutory or verification purposes;
c) the need to cover the range of geographical areas and water types for which the alternative method
may be used;
d) the need to consider seasonal variability in occurrence of the target organisms;
e) the number of test results needed for the assessment of relative recovery;
f) the number of laboratories with sufficient capacity and technical expertise available to participate.
It can be acceptable to have a limited number of participating laboratories analysing a wide range of
water types rather than a higher number of laboratories analysing a narrower range of water types
appropriate for the methods being compared.
NOTE Several successful comparisons have been achieved with three to six laboratories. In theory, it is
possible that one laboratory is able to conduct a suitable comparison study provided they have access to a wide
enough range of sample types for which the methods have been characterized.
5.4.2 Number of samples
It is not possible to determine beforehand the exact number of samples required for a valid comparison.
The number depends on the actual difference observed, on the experimental standard deviation and on
the difference considered significant. This International Standard includes an adequacy clause based on
a “predetermined stipulated limit” and the half-width of the confidence interval. If the data are found
inadequate for deciding that the methods are either “different” or “not different”, more samples are to
be collected and examined.
If the methods happen to differ markedly, a small number of samples might suffice to determine this
fact. It is therefore advisable to proceed in stages. The first stage should be planned to detect large
differences between the methods. If large differences are not found (result inconclusive), more samples
are taken until the system is able to detect the average relative difference that corresponds to the
predetermined stipulated limit chosen at the beginning of the trial.
© ISO 2014 – All rights reserved 5

---------------------- Page: 17 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

The total number of samples, n, for a two-sided evaluation (7.2) that would be sufficient for the detection
of a given average relative difference at about 95 % confidence depends on the experimental variance
according to Formula (1):
2
4s
n= (1)
2
L
where
n is the number of samples required for the detection of a difference L;
L is the smallest microbiologically significant mean relative difference;
s is the experimental standard deviation.
EXAMPLE A rather frequently observed value for the experimental standard deviation of the relative
difference is approximately s = 80. In order to detect an average relative difference of 10 % (L = 10 %), n =
25 600/100 = 256 samples is expected to be sufficient for a two-sided evaluation.
For a one-sided evaluation (7.3) the corresponding number of samples can be calculated according to
Formula (2):
2
3s
n= (2)
2
L
The rationale for the derivation of Formulae (1) and (2) is presented in Annex C.
High variability of counts can be experienced due to irregular behaviour of either laboratories or the
range of sample types analysed. Whether it is warranted to continue a comparison of methods study
can be ascertained by an examination of the standard deviation of the mean relative difference. Valid
comparisons are indicated by standard deviations of less than 100.
NOTE With some recent methods based on chromogenic substrates, it is possible to estimate two bacterial
groups simultaneously. One of the groups can be 10 or more times as numerous as the other. It is possible that
the number of samples sufficient for making a final decision of equivalence with the more numerous type is not
sufficient for an organism present in low numbers.
6 © ISO 2014 – All rights reserved

---------------------- Page: 18 ----------------------

SIST EN ISO 17994:2014
ISO 17994:2014(E)

5.4.3 Number of additional samples
Formula (3) can be applied to estimate the number of additi
...