NF VALIDATION AFNOR CERTIFICATION VALIDATION OF THE METHOD

Transcription

1 NF VALIDATION AFNOR CERTIFICATION VALIDATION OF THE METHOD VIDAS Listeria monocytogenes II VIDAS LMO2 (ref ) Certificate number: BIO 12/11 03/04 For the detection of Listeria monocytogenes with an enrichment step at 37 C in Fraser broth Protocol a broad range of foods and environmental samples or with an enrichment step at 30 C in LPT broth Protocol for meat products, dairy products, seafood products SUMMARY REPORT JANUARY 2017 V1 Expert laboratory: Manufacturer : ISHA biomérieux 25 avenue de la République Chemin de l Orme MASSY MARCY L ETOILE FRANCE FRANCE This report of analysis concerns only objects subjected to analysis. Its reproduction is authorized only under the shape of complete photographic facsimile. It contains 77 pages. Only some assays reported in this document are covered by the accreditation of the Section Laboratory of COFRAC. They are identified by the symbol (*). Assays performed at ISHA: 25, avenue de la République Massy

2 Table of contents 1. Introduction Validation history Validation repositories and expert laboratories Review of the alternative method Principle of the method Protocol for use Reference method Application scope Method comparison study general protocol Relative accuracy, specificity and sensitivity Number and nature of samples Artificial contamination of samples Results Calculation and interpretation for sensitivity Analysis of discordant results Initial validation study Renewal study Calculation and interpretation of data Relative level of detection study Experimental design Results and calculation of the RLODs Interpretation and conclusion Inclusivity and exclusivity study Practicability study Interlaboratory study Organization of the study Results Interpretation of the results Summary of the results Calculation of sensitivities, relative accuracy and false positive ratio Determination of the acceptability limit and conclusion Determination of the relative level of detection Extension study (January 2016) specific protocol Sensitivity study Number and nature of samples Institut Scientifique d'hygiène et d'analyse 2/77

3 Artificial contamination of samples Results Calculation and interpretation for sensitivity Analysis of discordant results Interpretation according to the standard ISO/FDIS : Study of storage of the enriched LPT broths Extended confirmation procedure Relative level of detection study Experimental design Results and calculation of the RLODs Interpretation and conclusion Inclusivity and exclusivity study Inclusivity test protocol Results Conclusion Practicability Conclusions Initial validation for a secondary enrichment in Fraser broth at 37 C Second extension for a secondary enrichment in LPT broth at 30 C Institut Scientifique d'hygiène et d'analyse 3/77

4 1. Introduction The present document introduces the results of the study for the validation AFNOR Certification of the method VIDAS Listeria monocytogenes II with an enrichment step at 37 C in Fraser broth or with an enrichment step at 30 C in LPT broth Validation history The VIDAS LMO2 method has been initially validated by AFNOR Certification in March 2004 under the certificate number BIO 12/11 03/04 for the detection of Listeria monocytogenes in food products and environmental samples with a secondary enrichment step at 37 C in Fraser broth. The method received three extensions and three renewals: December 2004: extension in accordance with the amended EN ISO standard, December 2006: Performance of inter laboratory study in accordance with EN ISO standard, March 2008: first renewal, March 2012: second renewal, January 2016: extension for the addition of a new protocol including a secondary enrichment step in LPT broth at 30 C, June 2016: third renewal and extension for the addition of a new category: composite foods. This document introduces all the results obtained during the different validation steps Validation repositories and expert laboratories The initial validation study of the method, its first extension and the first and second renewal were performed according to the standard ISO (2003) and its amendment A1:2011. All data from these studies were reinterpreted according to the FDIS version of the new standard ISO/FDIS (2015) during the third renewal of the method in June The second and third extensions were also interpreted according to the standard ISO/FDIS (2015) with the agreement of the Technical Board. Concerning the initial validation study, the first extension, the first and the second renewals, results reported in the present report were obtained during the assays conducted by the SERMHA, Institut Pasteur de Lille within the framework of the NF Validation trademark, according to the current requirements. Concerning the following extension studies and the third renewal, results reported in the present report were obtained during the assays conducted by the Institut Scientifique d Hygiène et d Analyse (ISHA) within the framework of the NF Validation trademark, according to the current requirements Review of the alternative method Principle of the method The VIDAS LMO2 method is based on an immuno enzymatic assay, enabling the detection of Listeria monocytogenes antigens using the ELFA (Enzyme Linked Fluorescent Assay) method using the VIDAS automated system. Each test is broken down into two components: The single use SPR used both for the solid phase and as a pipetting system for the test. The interior of the SPR is coated with anti Listeria monocytogenes antibodies absorbed on its surface. The strip contains all of the ready to use reagents required for the test: washing solution, anti Listeria monocytogenes antibodies conjugated with alkaline phosphatase and substrate. An aliquot of the enrichment broth is dispensed in the strip and then all the steps are performed automatically Institut Scientifique d'hygiène et d'analyse 4/77

5 by the VIDAS analytical module. The reaction medium is cycled in and out of the SPR several times, whose duration is specifically calculated to activate the reaction. The fluorescence intensity is measured by the VIDAS optical system at 450 nm and expressed as a relative fluorescence value (RFV), interpreted by the VIDAS system as follows: Test value (TV) = TV < 0.05 and TV > 0.05 sample RFV standard RFV test negative test positive Protocol for use The validated protocol is as follows: pre enrichment in half Fraser broth, incubated for 24 to 26 hours at 30 C ± 1 C, subculture in complete Fraser broth (0.1 ml in 10 ml), incubated for 24 to 26 hours at 37 C ± 1 C NB: In the comparative study of methods, the minimum incubation time (i.e. 24 hours) was tested. The extension protocol is as follows: pre enrichment in LPT broth, incubated for 24 to 32 hours at 30 C ± 1 C, subculture in LPT (0.1 ml or 1 ml in 10 ml), incubated for 18 to 24 hours at 30 C ± 1 C The VIDAS LMO2 test is then performed using an aliquot of the unheated Fraser broth, or the unheated LPT broth. Samples found to be positive using the VIDAS LMO2 test are confirmed through isolation on agar: PALCAM, Oxford or a chromogenic agar: if PALCAM or Oxford agar is used, the characteristic colonies are confirmed using the tests set out in the methods standardized by CEN or ISO, if a chromogenic agar is used, the presence of typical Listeria monocytogenes colonies is sufficient to confirm the VIDAS LMO2 result. The protocol of the method is set out in appendix Reference method The reference methods used were as follows: initial validation: EN ISO (1996), "Horizontal method for the detection and enumeration of Listeria monocytogenes Part 1: Detection method" validation extension: EN ISO /A1 (2004), "Horizontal method for the detection and enumeration of Listeria monocytogenes Part 1: Detection method" The outline of the method is set out in appendix Application scope The scope of validation is as follows: a broad range of foods and environmental samples for the general protocol, meat products, dairy products and fishery products for the specific protocol. No restrictions on the use of the method are mentioned. Institut Scientifique d'hygiène et d'analyse 5/77

6 2. Method comparison study general protocol 2.1. Relative accuracy, specificity and sensitivity The relative sensitivity is the ability of the alternative method to detect the analyte when it is detected by the reference method. The sensitivity study aims to determine the difference in sensitivity between the reference and the alternative method Number and nature of samples In total, 390 samples contaminated and non contaminated with Listeria monocytogenes were tested using both the EN ISO /A1:2004 reference method and the VIDAS LMO2 method: 327 samples in 2004 for the initial validation distributed in five categories, 63 samples in 2016 for the third renewal in one category. The different kinds of samples analyzed are presented in table 1. Table 1 : Number and nature of samples analyzed for the category Composite foods (* : positive by any method) Category Types Number of Number of positive results * negative results Total Raw meats Meat products Minced products to be cooked Ready to eat, ready to reheat Total Raw milk cheese Dairy products Pasteurized milk cheese Milks Total Ready to eat vegetables Vegetal products Cooked products Frozen products Total Smoked products Seafood products Raw products Cooked products Total Surface samplings Environmental Process waters samples Residues Total Ready to eat Composite foods Ready to reheat Pastry and derived, ovoproducts Total TOTAL Artificial contamination of samples Artificial contamination was carried out using stressed strains in accordance with the requirements of the EN ISO (2003) standard in 2004, the ISO/FDIS (2015) standard in 2016 and to the requirements of the AFNOR Validation Technical Board linked to these standards. Institut Scientifique d'hygiène et d'analyse 6/77

7 In 2004, fourteen different strains were used to carry out 50 artificial contaminations, and at least three different types of stress were trialed (cold, heat and saline stress). For eight products, contaminations were conducted by mixing with a naturally contaminated product. In 2016, thirteen different strains were used to carry out 42 artificial contaminations, using a seeding protocol. In total, 88 positive results out of 190 were obtained following artificial contaminations, i.e. 46.3%. The samples and the strains used for the artificial contaminations are presented in appendix Results Raw data are shown in appendix 3. Table 2 shows the results for the two methods. Table 2 : results of the sensitivity study for both methods (A+ = confirmed positives, A = immediate negatives and negative after confirmation of a presumptive positive, * = no unconfirmed VIDAS LMO2 positive results, PA: positive agreement, NA: negative agreement, ND: negative deviation, PD: positive deviation, PP: presumptive positive before confirmation, A+/R+ : confirmed positive, A /R negative immediately and negative after confirmation of the presumptive positive, (a): if ALOA and Oxford are taken as the media used with the EN ISO reference method, (b): if ALOA and PALCAM are taken as the media used with the EN ISO reference method) Category Meat products Dairy products Vegetal products Seafood products Environmental samples Composite foods All categories Response Positive reference method (R+) Negative reference method (R ) Total Positive alternative method (A+) PA = 32 PD = 2 34 Negative alternative method (A ) ND = 0 NA = Total Positive alternative method (A+) PA = 29 PD = 0 29 Negative alternative method (A ) ND = 1 NA = Total Positive alternative method (A+) PA = 30 PD = 0 30 Negative alternative method (A ) ND = 0 NA = Total Positive alternative method (A+) PA = 33 PD = 0 33 Negative alternative method (A ) ND = 1 (a) or 0 (b) NA = 29 (a) or30 (b) 30 Total 34 (a) / 33 (b) 29 (a) or30 (b) 63 Positive alternative method (A+) PA = 31 PD = 0 31 Negative alternative method (A ) ND = 0 NA = Total Positive alternative method (A+) PA = 30 PD = 0 30 Negative alternative method (A ) ND = 1 NA = Total Positive alternative method (A+) PA = 185 PD = Negative alternative method (A ) ND = 3 (a) NA = 200 (a) or ND = 2 (b) * or ND = 201 (b) * 203 Total 188 (a) or 187 (b) 202 (a) or 203 (b) 390 For seafood products, sample 49 is found as a negative deviation if Oxford agar media is used for the second agar media of the reference method and as a negative agreement if PALCAM agar media is used for the second agar media of the reference method. Institut Scientifique d'hygiène et d'analyse 7/77

8 Calculation and interpretation for sensitivity All results were used to calculate the sensitivity for the alternative method and the reference method, the relative trueness and the false positive ratio (cf. table 3). Table 3 : values in % of sensitivity for the two methods, relative trueness and false positive ratio for the alternative method (PA: positive agreement, NA: negative agreement, ND: negative deviation, PD: positive deviation, PP: presumptive positive before confirmation, SE alt : sensitivity for the alternative method, SE ref : sensitivity for the reference method, RT: relative trueness, FPR: false positive ratio for the alternative method, (a): if ALOA and Oxford are taken as the media used with the EN ISO reference method, (b): if ALOA and PALCAM are taken as the media used with the EN ISO reference method) Category PA NA ND PD N PPND PPNA SE alt SE ref RT FPR Meat products ,0% 94,1% 97,0% 0 Dairy products ,7% 100,0% 98,6% 0 Vegetal products ,0% 100,0% 100,0% 0 Seafood products (a) ,1% 100,0% 98,4% 0 Seafood products (b) ,0% 100,0% 100,0% 0 Environmental samples ,0% 100,0% 100,0% 0 Composite foods ,8% 100,0% 98,4% 0 All categories (a) ,4% 98,9% 98,7% 0 All categories (b) ,9% 98,9% 99,0% Analysis of discordant results Initial validation study During the initial validation study, four (a) or three (b) results are discordant between the two methods regarding the agar media used for the reference method Reminder : (a) :, ALOA and Oxford, (b) : ALOA and PALCAM. Positive deviations : Two samples gave positive supplementary results (a and b) regarding the reference method in the category Meat products : a chicken sample and a sausage sample, naturally contaminated. These two samples were contaminated by Listeria monocytogenes and by Listeria other than L. monocytogenes. With the reference method, only L. innocua and L. welshimeri were identified on selective agar media. Ottaviani and Agosti agar media were full of colonies and it was not possible to distinguish any halo typical of the colonies of L. monocytogenes. All colonies isolated from this agar media led to an identification different from L. monocytogenes. The confirmations realized from the alternative method allowed identifying Listeria monocytogenes regardless the agar media used. The proportion of colonies typical of Listeria other than Listeria monocytogenes was inferior to the one observed in the reference method, probably because of an incubation time of the Fraser Institut Scientifique d'hygiène et d'analyse 8/77

9 brother shorten of 24 hours in the alternative method. That s why it was possible to identify and confirm easier colonies typical of Listeria monocytogenes. Negative deviations: Two samples gave false negative results regarding the reference method: a sample of Saint Nectaire cheese (a and b)and a sample of salmon with basil (a), naturally contaminated. The sample of cheese was highly contaminated in L. innocua and poorly contaminated in L. monocytogenes. The test VIDAS LMO2 was negative. The isolations realized from the Fraser broth allowed finding L. monocytogenes on only three chromogenic agar media. However Listeria innocua was found on the seven agar media used for the confirmation of the alternative method. The Fraser broth was incubated 24 more hours and a test VIDAS LMO2 was realized again and found negative. It s important to note that, for the reference method, Listeria monocytogenes was not able to be found on the Ottaviani and Agosti agar media isolated from the Fraser broth. For the sample of salmon with basil, with the reference method, Listeria monocytogenes was only able to be found on the Oxford agar media streaked from the Fraser broth, suggesting that the samples was poorly contaminated. This sample became negative when considering the agar media PALCAM and Ottaviani and Agosti as the media for the reference method, and so negative concordant with the alternative method. The confirmations realized for the alternative method didn t allow isolating Listeria monocytogenes Renewal study For the category Composite foods, only one sample is discordant and found as a negative deviation. It s a sample of Piemontese salad, artificially contaminated. The VIDAS LMO2 test gave a negative result, although the confirmations reveal the presence of L. monocytogenes in the samples. The analysis was renewed three times from the broth stored at 5±3 C and didn t allow obtaining a positive result. The concentration in L. monocytogenes in the enriched broth was probably very low, as typical colonies were only found from the Ottaviani and Agosti agar media isolated from the half Fraser broth with the reference method Calculation and interpretation of data For each category and for all categories, the difference between ND and PD and the sum of ND and PD is calculated. The value obtained is compared to the acceptability limits defined by the ISO/FDIS (2015) standard. Table 4 shows the difference and the sum of ND and PD for all categories and the acceptability limits. Table 4: acceptability limits Category (ND PD) (ND+PD) Acceptability limit (AL) (ND PD) (ND+PD) Meat products Dairy products Vegetal products Seafood products (a) Seafood products (a) Environmental samples Composite foods All categories (a) All categories (b) Observation (ND PD) < AL (ND+PD) < AL Institut Scientifique d'hygiène et d'analyse 9/77

10 The observed values (ND PD) and (ND+PD) are below the acceptability limits for each category and for all categories. The alternative method produces results comparable to the reference method Relative level of detection study The level of detection is the measured analyte concentration, obtained by a given measurement procedure, for which the probability of detection is x, e.g. LOD50 is the level of detection for which 50 % of tests give a positive result. The relative level of detection (RLOD) is defined as the level of detection at P = 0.50 (LOD50) of the alternative (proprietary) method divided by the level of detection at P = 0.50 (LOD50) of the reference method Experimental design In 2004, five "food product strain matrix" pairs were studied in parallel using the reference method and the VIDAS LMO2 method with six replicates per contamination level tested. Artificial contamination was carried out in accordance with the requirements of the EN ISO standard and the Microbiology Technical Board. In 2016, for the category Composite foods, three levels of contamination were prepared consisting of a negative control level, a low level, and a higher level. Only one strain of the target analyte is used to contaminate the low and the high level. The negative control level shall not produce positive results. Five replicates are tested for this level. The low level shall be the theoretical detection level, it has been contaminated at CFU per test portion to obtain fractional recovery results. Twenty replicates are tested for this level. The higher level shall be just above the theoretical detection level, it has been contaminated at 2 3 CFU per test portion. Five replicates are tested for this level. Food samples were contaminated using the seeding protocol. Bulk contaminations were performed on the matrix for the different levels of contamination, then the matrix was stored at 5±3 C for two days before analysis. In 2004 and in 2016, an enumeration of the mesophilic aerobic flora was performed on the matrices. Table 5 details the couples matrix strain tested. Table 5 : couples matrix strain used for the determination of the RLOD of the method Matrix Strain Origine Raw milk L. monocytogenes 1/2b Raw milk Maroilles Rillettes L. monocytogenes 1/2c Minced meat Red cabbage L. monocytogenes 4b Salad Smoked salmon L. monocytogenes 1/2a Slices of smoked salmon Process water L. monocytogenes 1/2c Surface sampling from a milk factory Piemontese salad L. monocytogenes 1/2a Salad ham and raw vegetables Results and calculation of the RLODs Raw results are shown in appendix 4. The RLOD is defined as the ratio of the LODs of the alternative method and the reference method: RLOD= LOD alt / LOD ref. The RLODs calculations were performed according to the standard ISO/FDIS : 2015 using the Excel spreadsheet available for download at Values of the RLODs are presented in table 6. Institut Scientifique d'hygiène et d'analyse 10/77

11 Table 6 : RLODs values for the two categories (RLOD: the estimated relative level of detection value, RLODU: the upper limit of the 95% confidence interval for RLOD, RLODL: the lower limit of the 95% confidence interval for RLOD, b=ln(rlod): logarithm of the RLOD value, sd(b): standard deviation of b, z Test statistic: absolute value of the test statistic of the z Test with the null hypothesis H0: b=0, p value: p value of the z Test) Category RLOD RLODL RLODU b=ln(rlod) sd(b) z Test p Acceptability statistic value limit Meat products Dairy products Vegetal products Seafood products Environmental samples Composite foods Interpretation and conclusion The RLODs values are below the acceptability limit set at 1.5, meaning that, as stated in ISO/FDIS : 2015, the maximum increase in LOD of the alternative versus the reference method is not considered as relevant in consideration of the fitness for purpose of the method. In conclusion, alternative and reference methods show similar LODs values for the detection of Listeria monocytogenes in the categories tested Inclusivity and exclusivity study The inclusivity and exclusivity of the method are defined by analyzing, respectively, 50 positive strains and 30 negative strains. This study, carried out for the validation of the VIDAS LMO2 method (with phase of enrichment at 30 C) in 2002 was deemed valid with regard to the EN ISO standard. Reminder (2002 validation study): The various Listeria monocytogenes strains were cultured in Fraser broth. The various Listeria nonmonocytogenes strains were cultured in Fraser broth and the non Listeria strains were cultured in nutrient broth in such a way as to obtain sufficient concentrations to conduct the test. The VIDAS LMO2 test was then performed. Fifty Listeria monocytogenes strains, 15 Listeria other than Listeria monocytogenes and 28 other genera were tested using the VIDAS LMO2 test from the first stage of the protocol. All the Listeria monocytogenes strains provided a positive result and no cross reactions were observed. The results are set out in Appendix Practicability study The practicability of the alternative method will be documented according to the criteria defined by AFNOR Certification. 1. Component storage conditions (see package insert) Shelf life of unopened products (see package insert) 2. Conditions for use after first use (see package insert) The test storage temperature is 2 8 C. The shelf life of tests is indicated on the kits. Each reagent should be stored between +2 C and +8 C. Institut Scientifique d'hygiène et d'analyse 11/77

12 3. Lead time to obtain results Negative samples Step Lead time obtained VIDAS LMO2 method Lead time obtained reference method ISO Performance of primary enrichment D0 D0 Inoculations of various secondary enrichment broths (half Fraser) D1 D1 Performance of VIDAS LMO2 test D2 / Isolation of selective broths on selective agar / D1 & D3 Negative results obtained - if there are no characteristic colonies - if the VIDAS LMO2 test is negative - if the VIDAS LMO2 test is positive and the confirmation is negative D2 D3 to D5 D5 Positive samples: Step Lead time obtained VIDAS LMO2 method Lead time obtained reference method ISO Performance of primary enrichment D0 D0 Inoculations of various secondary enrichment broths D1 D1 Performance of VIDAS LMO2 test and isolation on selective agars D2 / Isolation of selective broths on selective agar / D1 & D3 Confirmation tests: Genus - Isolation on TSAYE - Gram stain, catalase Species - CAMP test, hemolysis, TSBYE broth - Use of carbohydrates - Isolation on chromogenic agar Positive results obtained - after confirmation with reference method tests - if API strips used - if isolated on chromogenic agar D3 D4 D4 D5 D2 D10 D5 D3 to D4 D2 to D5 D3 to D6 D3 to D6 D4 to D7 D9 to D12 4. Common step with the reference method The enrichment broth, temperature and time is the same for the two methods. Institut Scientifique d'hygiène et d'analyse 12/77

13 3. Interlaboratory study A validation extension was obtained in December 2006 following the completion of the interlaboratory study in accordance with the EN ISO standard Organization of the study Sixteen laboratories received samples of pasteurized milk. The strain used for contamination of the pasteurized milk was a strain of Listeria monocytogenes (L32), isolated from a raw milk cheese. The contamination rates obtained and the estimated precisions are set out in the table below: sixteen laboratories received samples. The contamination rates obtained and the estimated precisions are set out in the table below: Level Samples Targeted theoretical rate (b/25 ml) Actual level (b/25 ml of sample) Level Low level High level As a result of transport conditions, only 14 laboratories carried out the tests, as two laboratories did not receive the samples within the lead time Results Results obtained by participating laboratories Positive results after confirmation obtained with the reference method Contamination levels L0 L1 L2 Laboratories Obtaine d N. of samples Obtaine d N. of samples Obtaine d N. of samples Laboratory A Laboratory D Laboratory E Laboratory F Laboratory G Laboratory H Laboratory I Laboratory J Laboratory K Laboratory L Laboratory M Laboratory N Laboratory O Laboratory P Total Institut Scientifique d'hygiène et d'analyse 13/77

14 Positive results after confirmation obtained with the alternative method Contamination levels L0 L1 L2 Laboratories Obtaine d N. of samples Obtaine d N. of samples Obtaine d N. of samples Laboratory A Laboratory D Laboratory E Laboratory F Laboratory G Laboratory H Laboratory I Laboratory J Laboratory K Laboratory L Laboratory M Laboratory N Laboratory O Laboratory P Total Analysis of the results Laboratory F was excluded from the final statistical analysis of the results, as four samples had not been analyzed due to leaks. Results of thirteen laboratories are thus kept for the statistical analysis. At level 0, no positive result is observed. At levels 1 and 2, all samples are found positive by the two methods for all the laboratories Interpretation of the results Summary of the results Table 7 details per method and per level the results obtained during the study. Table 7 : tests results for the two methods (PA: positive agreement, NA: negative agreement, ND: negative deviation, PD: positive deviation, PP: presumed positive before confirmation) Level Alternative method Reference method MR+ MR Total MA+ PA = 0 PD = 0 0 L0 MA ND = 0 NA = Total MA+ PA = 104 PD = L1 MA ND = 0 NA = 0 0 Total MA+ PA = 104 PD = L2 MA ND = 0 NA = 0 0 Total MA+ PA = 208 PD = L0+L1+L2 MA ND = 0 NA = Total Institut Scientifique d'hygiène et d'analyse 14/77

15 Calculation of sensitivities, relative accuracy and false positive ratio Based on the data of table 7, the following parameters are calculated: Sensitivity for the alternative method: Sensitivity for the reference method: 100% = 100% 100% = 100.0% Relative accuracy: 100% = 100% False positive ratio for the alternative method: 100% = 0% where N is the total number of samples (NA + PA + PD + ND) and FP is false positive results Determination of the acceptability limit and conclusion The difference between (ND PD) and the sum of (ND + PD) is calculated. The observed values shall not be higher than the acceptability limits (AL) defined by the ISO/FDIS (2015). The AL is not met when the observed value is higher than the AL. When the AL is not met, investigations should be made (e.g. root cause analysis) in order to provide an explanation of the observed results. Based on the AL and the additional information, it is decided whether the alternative method is regarded as not fit for purpose. The reasons for acceptance of the alternative method in case the AL is not met shall be stated in the study report. As no fractional recovery result is obtained either at level L1 or at level L2, calculations are performed from the data of these two levels. The different values observed are detailed in the table below: Table 8 : values obtained for the determination of the acceptability limit Acceptability limits (AL) Number of collaborators (ND PD) (ND+PD) (ND PD) (ND+PD) The values (ND PD) and (ND+PD) are inferior to the AL, so the requirements of the standard ISO/FDIS : 2015 are fulfilled. The performance of the alternative method and the reference method can be considered as equivalent Determination of the relative level of detection This evaluation is performed according to Annex F of ISO/FDIS :2015 and using the excel spreadsheet as described in this standard. As there is limited experience with the interpretation of this approach, the results are used only for information. Results are shown in the table below : Table 9 : values obtained for the determination of the relative level of detection RLOD RLODL RLODU b=ln(rlod) sd(b) z Test statistic p value 1,000 0,468 2,135 0,000 0,379 0,000 1,000 Institut Scientifique d'hygiène et d'analyse 15/77

16 4. Extension study (January 2016) specific protocol The extension study performed in 2015 for a presentation of the results in January 2016 concerned the addition of a specific protocol for meat products, dairy products and seafood products, using the LPT broth for the enrichment. Assays were performed according to the ISO/FDIS :2015 standard and to the specific rules of the Technical Committee linked to this standard Sensitivity study The relative sensitivity is the ability of the alternative method to detect the analyte when it is detected by the reference method. The sensitivity study aims to determine the difference in sensitivity between the reference and the alternative method. During the validation study, only the minimal incubation times of the two broths of the alternative method were tested. All samples of the alternative method were confirmed by direct streaking of the enriched secondary LPT broth on an ALOA Petri dish. Typical colonies were confirmed: by the confirmation tests of the ISO /A1 method, including the purification step, by the observation of the presence of typical colonies of Listeria monocytogenes. Moreover, as part of the application of the standard ISO/FDIS : 2015, for negative samples of the alternative method, the subcultures in LPT were re incubated for a total of 48 hours and then streaked onto an ALOA Petri dish to check the absence of Listeria monocytogenes in the sample. The storage of the 10 ml LPT broth tubes was also studied: after a storage of 72 hours at 5±3 C, positive and discordant samples were re analyzed by the alternative method. The confirmation was performed by streaking onto ALOA and the observation of the presence of typical colonies of Listeria monocytogenes Number and nature of samples The different kinds of samples analyzed are presented in table 10. A total of 191 samples was analyzed. Table 20 : Number and nature of samples analyzed (* : positive by any method) Category Type Number of negative Number of positive Total results * results Raw products (including deep frozen, fresh, seasoned) Meat Ready to eat and processed meat products products Fermented or dried meat products (raw and cooked) Total Dairy products Fishery products Raw milk cheese Other raw milk products Heat processed milk and dairy products Total Raw products (fresh, deep frozen) Smoked, cured, marinated products Ready to eat products Total Total Institut Scientifique d'hygiène et d'analyse 16/77

17 Artificial contamination of samples Artificial contaminations were performed using the seeding protocol mentioned in the standard ISO/FDIS : 2015: 47 samples were artificially contaminated at a level lower or equal to 3 CFU/25g using 22 different strains of Listeria monocytogenes. No more than 3 positive results were obtained using the same strain. Overall 91 samples gave a positive result by at least one of the method and 50.5 % of them were naturally contaminated. The detail of the artificial contaminations is in appendix Results Raw data are shown in appendix 7. Table 11 shows the results for the two methods for a subculture of 1 ml and 0.1 ml of the enriched LPT broth in 10 ml of LPT broth. Table 11 : results of sensitivity for both methods with a subculture volume of 1 ml (PA: positive agreement, NA: negative agreement, ND: negative deviation, PD: positive deviation, PP: presumptive positive before confirmation, A+/R+ : confirmed positive, A /R negative immediately and negative after confirmation of the presumptive positive) Subculture volume : 1 ml Subculture volume : 0.1 ml Category Meat products Dairy products Fishery products All categories Response Alternative method positive (A+) Alternative method négative (A ) Alternative method positive (A+) Alternative method négative (A ) Alternative method positive (A+) Alternative method négative (A ) Alternative method positive (A+) Alternative method négative (A ) Reference method (*) positive (R+) Reference method (*) négative (R ) Reference method (*) positive (R+) Reference method (*) négative (R ) PA= 21 PD= 5 PA= 21 PD= 5 ND= 4 incl. 0 PPND NA= 40 incl. 1 PPNA ND= 4 incl. 0 PPND NA= 40 incl. 1 PPNA PA= 22 PD= 5 PA=22 PD= 4 ND= 3 incl. 0 PPND NA= 30 incl. 0 PPNA ND= 3 incl. 0 PPND NA= 31 incl. 0 PPNA PA= 26 PD= 3 PA= 26 PD= 3 ND= 2 incl. 0 PPND NA= 30 incl. 0 PPNA ND= 2 incl. 0 PPND NA= 30 incl. 0 PPNA PA= 69 PD= 13 PA= 69 PD= 12 ND= 9 incl. 0 PPND NA= 100 incl. 1 PPNA ND= 9 incl. 0 PPND NA= 101 incl. 1 PPNA Institut Scientifique d'hygiène et d'analyse 17/77

18 Calculation and interpretation for sensitivity All results were used to calculate the sensitivity for the alternative method and the reference method, the relative trueness and the false positive ratio according to the standard ISO/FDIS : Results are indicated in tables 12 and 13. Notes: PA: positive agreement, NA: negative agreement, ND: negative deviation, PD: positive deviation, PP: presumptive positive before confirmation. Table 12 : sensitivity for the two methods, relative accuracy and false positive ratio for a subculture volume of 1 ml according to the standard ISO/FDIS : 2015 (SE alt : sensitivity for the alternative method, SE ref : sensitivity for the reference method, RT: relative trueness, FPR: false positive ratio for the alternative method) Category PA NA ND PD N PPND PPNA SE alt SE ref RT FPR Meat products % 83.3% 87.1% 2.5% Dairy products % 83.3% 86.7% 0.0% Fishery products % 90.3% 91.8% 0.0% All categories % 85.7% 88.5% 1.0% Table 13 : sensitivity for the two methods, relative accuracy and false positive ratio for a subculture volume of 0.1 ml according to the standard ISO/FDIS : 2015 (SE alt : sensitivity for the alternative method, SE ref : sensitivity for the reference method, RT: relative trueness, FPR: false positive ratio for the alternative method) Category PA NA ND PD N PPND PPNA SE alt SE ref RT FPR Meat products % 83.3% 87.1% 2.5% Dairy products % 86.2% 88.3% 0.0% Fishery products % 90.3% 91.8% 0.0% All categories % 86.7% 89.0% 1.0% Analysis of discordant results Positive deviations : Sample 1, 14, 34, 36, 39, 63, 87, 105, 127, 141, 143 and 152 A positive result is obtained by the alternative method (with 1 ml or 0.1 ml of subculture) whereas a negative result is obtained by the reference method. Due to the difference of sampling between both methods, no cell of L. monocytogenes may have been taken in the sampling for the reference method. Sample 2 A positive result is obtained by the alternative method with 1 ml of subculture whereas a negative result is obtained by the reference method. Due to the difference of sampling between both methods, no cell of Listeria may have been taken in the sampling for the reference method. Negative deviations : Sample 3, 16, 19, 30, 32, 131, 136 and 144 A positive result is obtained by the reference method whereas a negative result is obtained by the alternative reference. A confirmation according to the ISO/FDIS : 2015 was applied on subcultures of LPT incubated 48h at 30±1 C. The results of this protocols did not allow finding typical colonies. Institut Scientifique d'hygiène et d'analyse 18/77

19 Due to the difference of sampling between both methods, and use of naturally contaminated samples or seeded samples with low level of contamination, no cell of L. monocytogenes may have been present in the sampling of the alternative method. Sample 41 A positive result is obtained by the reference method whereas a negative result is obtained by VIDAS system from naturally contaminated samples. However the protocols of confirmation of the alternative method allowed finding typical colonies which were confirmed as L. monocytogenes. For this sample, the enrichment did not allow to reach the threshold of the VIDAS assay Interpretation according to the standard ISO/FDIS : 2015 Table 14 show the difference between negative deviations and positive deviations and the acceptability limits. Table 14: acceptability limits Volume of Volume of Acceptability Category transfer : 1 ml transfer : 0.1 ml limit (AL) (ND PD) (ND PD) Meat products Dairy products Fishery products All categories Observation (ND PD) < AL : The observed values (ND PD) are below the acceptability limit for each category and for all categories. The alternative method with subculture volume of 1 ml or of 0.1 ml produces comparable results to the reference method Study of storage of the enriched LPT broths A stability study of the secondary enriched broths stored at 5±3 C for 72 hours was performed on all positive and discordant samples. After storage, the broths were re analyzed and confirmed with an isolation on ALOA (results in appendix 8). One sample found positive after enrichment (transfer of 1 ml only) was tested negative after storage of the LPT broth for 72 hours at 2 8 C. This result did not modify the conclusion for the conservation of the broths Extended confirmation procedure None of the negative samples were confirmed positive after enrichment of the LPT broth for 48 hours at 37 C Relative level of detection study The level of detection is the measured analyte concentration, obtained by a given measurement procedure, for which the probability of detection is x, e.g. LOD50 is the level of detection for which 50 % of tests give a positive result. The relative level of detection (RLOD) is defined as the level of detection at P = 0.50 (LOD50) of the alternative (proprietary) method divided by the level of detection at P = 0.50 (LOD50) of the reference method Experimental design Three couples matrix strain were studied in parallel by both methods. For each category of the scope, one relevant type of food product is selected. Three levels of contamination per type were prepared consisting of Institut Scientifique d'hygiène et d'analyse 19/77

20 a negative control level, a low level, and a higher level. Only one strain of the target analyte is used to contaminate the low and the high level. The negative control level shall not produce positive results. Five replicates are tested for this level. The low level shall be the theoretical detection level, it has been contaminated at CFU per test portion to obtain fractional recovery results. Twenty replicates are tested for this level. The higher level shall be just above the theoretical detection level, it has been contaminated at 2 3 CFU per test portion. Five replicates are tested for this level. Food products were contaminated using the seeding protocol. Bulk contaminations were performed on the matrices for the different levels of contamination, then the matrices were stored at 5±3 C for two days before analysis. Simultaneously, a total viable count was performed on a portion of non contaminated matrix to estimate the concentration of mesophilic aerobic flora. A detection of Listeria monocytogenes using the reference method was also performed to check the absence of the target analyte in the matrix. Table 15 details the three couples matrix strain tested. Table 15 : couples matrix strain used for the determination of the RLOD of the method Category Matrix type Strain Code Strain origin Meat products Pork rillettes Listeria monocytogenes 1/2c LIS.4.33 Minced meat Dairy products Raw milk Listeria monocytogenes 1/2b LIS.4.32 Raw milk Fishery products Salmon rillettes Listeria monocytogenes 1/2a LIS.4.12 Smoked salmon Results and calculation of the RLODs Raw results are shown in appendix 9 for the two volumes of subculture in LPT secondary enrichment. The RLOD is defined as the ratio of the LODs of the alternative method and the reference method: RLOD = LOD alt / LOD ref The RLODs calculations were performed according to the standard ISO/FDIS : 2015 using the Excel spreadsheet available for download at Values of the RLODs are presented in table 16. They are the same for a subculture volume of 1 ml or 0.1 ml. Table 16 : RLODs values for the three categories for the two volumes of subculture in LPT tubes (RLOD: the estimated relative level of detection value, RLODU: the upper limit of the 95% confidence interval for RLOD, RLODL: the lower limit of the 95% confidence interval for RLOD, b=ln(rlod): logarithm of the RLOD value, sd(b): standard deviation of b, z Test statistic: absolute value of the test statistic of the z Test with the null hypothesis H0: b=0, p value: p value of the z Test) Category RLOD RLODL RLODU b=ln(rlod) sd(b) z Test statistic p value Meat products Dairy products Fishery products The alternative and the reference method show similar detection levels. Acceptability limit 2.5 Institut Scientifique d'hygiène et d'analyse 20/77

21 Interpretation and conclusion The RLODs values for the two volumes of subculture in LPT secondary enrichment broth are below the acceptability limit set at 2.5, meaning that, as stated in ISO/FDIS : 2015, the maximum increase in LOD of the alternative versus the reference method is not considered as relevant in consideration of the fitness for purpose of the method. In conclusion, alternative and reference methods show similar LODs values for the detection of Listeria monocytogenes in the categories tested Inclusivity and exclusivity study Inclusivity and exclusivity of the method were already determined for the initial validation of the method with different enrichment broths: half Fraser and Fraser, known to be more selective than the LPT broth. During the presentation of the extension project of the method, the Technical Committee accepted that the exclusivity was not to be determined again. The inclusivity of the extended protocol was tested according to the protocol detailed below. The list of the strains and the results are shown in appendix Inclusivity test protocol Each L. monocytogenes strain was cultivated twice before inoculation in LPT broth (about 1 to 100 CFU/225 ml). The complete protocol of the alternative method was applied with the minimum time of incubation Results The 50 Listeria monocytogenes strains tested were detected by the alternative method (cf. appendix 10) Conclusion The selectivity of the method is satisfactory Practicability The practicability of this new protocol of the VIDAS LMO2 method is similar to the one of the general protocol: negative results can be obtained in three days and positive results in four to eleven days depending on the confirmation step chosen. Institut Scientifique d'hygiène et d'analyse 21/77

22 5. Conclusions 5.1. Initial validation for a secondary enrichment in Fraser broth at 37 C This study concerned 390 samples of a broad range of foods and environmental samples. The performance of the VIDAS LMO2 method is equivalent to that of the EN ISO /A1:2004 reference method. The relative trueness obtained is 99.0%, the relative sensitivity is 98.9% and the false positive ratio is 0%, based on the calculations stipulated by the ISO/FDIS (2015) standard. There were three discordant results: two positive deviations and two negative deviations. The observed values (ND PD) are below the acceptability limit for each category and for all categories. The relative level of detection of the VIDAS LMO2 method was evaluated by artificially contaminating six different products representative of the six categories tested. It is between and Listeria monocytogenes cells per 25 g or ml of sample for both methods. The method offers good specificity, as all the strains of Listeria monocytogenes were detected (inclusivity) and no cross reactions were observed among the Listeria other than Listeria monocytogenes or the non Listeria strains tested (exclusivity). The interlaboratory study results obtained for all 13 laboratories selected demonstrate that the alternative method and the reference method have equivalent relative accuracy and sensitivity values and in the same level as those obtained in the comparison study Second extension for a secondary enrichment in LPT broth at 30 C This study concerned 191 samples of three categories of products: meat products, dairy products and seafood products. With regard to the volume of subculture, the relative sensitivity obtained is from 90.0% to 90.1%, the relative trueness from 88.5% to 89.0% and the false positive ratio is 1.0%, based on the calculations stipulated by the ISO/FDIS (2015) standard. The observed values (ND PD) are below the acceptability limit for each category and for all categories. The RLODs values for the two volumes of subculture in LPT secondary enrichment broth are below the acceptability limit set at 2.5, meaning that, as stated in ISO/FDIS : 2015, the maximum increase in LOD of the alternative versus the reference method is not considered as relevant in consideration of the fitness for purpose of the method. The VIDAS LMO2 method with an enrichment step at 37 C and the reference method show similar LODs values for the detection of Listeria monocytogenes in the categories tested. The selectivity of the alternative method is correct. Massy, January 5 th, 2016 François LE NESTOUR Innovation Biology Unit Manager Institut Scientifique d'hygiène et d'analyse 22/77

23 APPENDICES Institut Scientifique d'hygiène et d'analyse 23/77

24 Appendix 1 Analytical protocols Institut Scientifique d'hygiène et d'analyse 24/77

25 ISO /A1:2004 STANDARD Prepare test sample (x g or x ml) Dilute to 1:10 in ½Fraser broth Incubation for 24 h at 30 C ± 1 C Isolate onto Listeria agar OAA and Palcam Transfer 0,1 ml of ½ Fraser enrichment in 10 ml Fraser 1 Incubate for 24 h Incubation for 48 h at 37 C ± 1 C Plating on selective Listeria agar : OAA and Palcam Incubation for 24 h at 37 C ± 1 C Presence of typical colonies NO Re incubation 24 h à 37 C ±1 C YES Response : Presence of Listeria monocytogenes YES Confirmation YES Presence of typical colonies NO NO Response : Absence of Listeria monocytogenes Institut Scientifique d'hygiène et d'analyse 25/77

26 PROTOCOL FOR THE VIDAS LMO2 ALTERNATIVE METHOD Prepare the test sample by weighing 25 g of product. Dilute in 225 ml of half-fraser broth Incubate for hrs at 30 C ± 1 C Transfer 0.1 ml of primary enrichment broth into 10 ml of complete Fraser medium. Incubate for hrs at 37 C ± 1 C Store the secondary enrichment broth at 2-8 C for subsequent confirmation of tests Place 0.5 ml in a strip and pass it through the VIDAS system. Kit results: positive YES NO Absence of Listeria monocytogenes in 25 g Isolate on Palcam or Oxford agar or chromogenic medium Incubate for 24 hrs at 37 C ± 1 C Presence of characteristic colonies YES NO Reincubate, if necessary, for 24 hrs at 37 C ± 1 C. Response: Absence of Listeria monocytogenes in 25 g NO Presence of characteristic colonies YES Confirmations, as required Institut Scientifique d'hygiène et d'analyse 26/77

27 SPECIFIC PROTOCOL Primary enrichment : X g of sample + 9 X ml of LPT broth Secondary enrichment : 0.1 to 1 ml enriched primary broth + 10 ml of LPT broth Incubation 24 32h at 30±1 C Incubation 18 24h at 30±1 C VIDAS LMO2 Confirmation of positive results : Streaking on ALOA Petri dish with the enriched secondary LPT broth incubation 24±3 h at 37±1 C (+ 24±3 h) Typical colonies confirmed: by the confirmation tests of the ISO /A1 method, including the purification step, by the observation of the presence of typical colonies of Listeria monocytogenes. Institut Scientifique d'hygiène et d'analyse 27/77