A: There are a number of international standards and guidelines for the operation of
proficiency tests.
ISO/IEC Guide 43: Proficiency testing by interlaboratory
comparisons
This is, at present, the major international guideline for the operation
of proficiency tests.
Part 1 defines the quality requirements for the development
and operation of proficiency testing schemes.
Part 2 elaborates on the selection
and use of proficiency testing schemes by laboratory accreditation
bodies.
ILAC Guide 13: Guidelines for the requirements for the competence of
providers of proficiency testing schemes
These guidelines are based on the technical
elements of ISO/IEC Guide 43-1 and on the relevant elements of the laboratory competence
standard ISO/IEC 17025, including the management system requirements and the technical
requirements for the characterisation, homogeneity and stability testing of PT test items.
Relevant clauses of ISO 9000 are included to eliminate the need for separate recognition of
a provider's quality management system. Although they do not have the status of an
international standard, the ISO/IEC and the ILAC guides are used for the accreditation
of PT providers. To set common criteria for the evaluation of the competence of PT
providers, the ISO/IEC guide is currently being revised and upgraded to become ISO/IEC 17043 –
General requirements for proficiency testing.
ISO 13528: Statistical methods for use
in proficiency testing by interlaboratory comparisons
This sets out how the assigned
value and performance assessment criteria may be established and describes the options for
the various scoring systems.
A: The day-to-day operation of each scheme is the responsibility of LGC Standards
Proficiency Testing. Individual schemes are managed by a team of Scheme Coordinators, to cover
reporting, customer service and technical functions. For some schemes, external advisors may also
be used to provide the full range of relevant knowledge and expertise needed to operate the
scheme effectively.
A: Yes depending upon the scheme in question. Advisors are selected on the basis
of their technical knowledge and experience of the industry to which the scheme is related.
Advisors may be used on an ad-hoc basis and contacted when specific issues need to be
addressed. Alternatively, formal advisory groups may be used. Advisory Groups consist of
members who may or may not be participants on the scheme but who are experienced in the field
of testing covered by the PT scheme.The composition and terms of reference of each Advisory
Group will be agreed on a scheme-by-scheme basis. Membership of the Advisory Groups is subject
to change but members’ names are available on request.
A: Yes one or two schemes are operated jointly with a partner organisation.
Where schemes are operated jointly, a Management Committee may be set up to address
operational issues for the scheme.
A: Participants are advised to participate in the scheme(s) that are most
appropriate to their own area of testing. Where necessary, staff at LGC Standards
Proficiency Testing can advise on which scheme is most suitable for participants.
For each scheme, a scheme description and application form will be available, containing
information about the test materials included in the scheme, and the intended distribution
dates. Participants are invited to complete the application form, indicating which test
materials they wish to receive during the scheme year. If the availability of test materials
changes during the scheme year, participants are kept fully informed.Once a participant joins a
scheme and has selected the test materials required, a participant schedule is provided as
confirmation.
A: In order to ensure confidentiality, participants in all schemes are allocated a
unique laboratory reference number. This number enables results to be reported without divulging
the identities of participant laboratories. Only LGC Standards Proficiency Testing and the
laboratory itself know this number.
A: Certain schemes have a minimum level of participation, whilst others have
completely flexible participation. Third parties, such as retail groups, regulatory bodies
and accreditation bodies may recommend minimum levels of participation. To gain the benefit
from trend analysis, participation in a minimum of four rounds over a scheme year is
recommended.
A: Fees for participation are reviewed annually and the current fees for each scheme
are available on application. Payment terms are detailed on the application form and invoice.
Participants are advised that delays with payments may result in test materials and reports
being withheld until payment is made.
A: The vast majority of samples are manufactured by LGC Standards Proficiency
Testing. Where this is not possible, test materials are carefully sourced to meet the needs
of participants. Wherever practical, test materials will be as similar as possible to those
routinely tested by participating laboratories. However, in some cases, in order to achieve
the required degree of homogeneity and stability, test materials may be in the form of simulated
samples or concentrated spiking solutions. The range of test materials will usually be varied
from round to round in order to be realistic and challenging. Details of individual test material
types are available in the relevant scheme description.
A: Test materials are sent in appropriate packaging and under conditions intended
to protect the contents during transit. All samples are sent using priority courier.
Overseas customers must provide relevant documents to prevent delay in customs such as import
permits and may be required to pay import duties locally. Once packages have been delivered,
LGC Standards Proficiency Testing cannot be held responsible if they subsequently fail to
reach the correct personnel or are not stored under the recommended conditions.Participants are
asked to check the contents of packages immediately on receipt and to contact LGC Standards
Proficiency Testing if there are any problems with the condition of the test materials or
accompanying documentation.
A: The samples are all stable at the stated storage temperatures for at least the
period of the Proficiency Test round. Studies have shown there is no significant difference
between results of samples tested the day after despatch and those tested on the deadline date.
There is also no evidence that results are influenced by different climatic conditions of
participating countries.Distance travelled does not affect sample results. We have undertaken
studies on a number of our PT test samples comparing the average result according to distance
travelled, and no correlations have been found. Stability consideration is an important part of
the design and feasibility process for a PT scheme, where transport conditions such as
temperature, humidity, pressure, exposure to x-rays etc are taken into account.
A: It is important for laboratories to understand how to get the optimum benefit
from PT participation. To do this, a laboratory must participate in an open and honest fashion,
being prepared to, on occasion, be evaluated as unsatisfactory. If PT is to achieve its aims,
laboratories need to treat the PT samples the same as routine samples, and staff must be
encouraged to treat them appropriately and learn from their results in a constructive manner.
Laboratories will learn very little about the quality of their routine work if the PT samples
are given special treatment, such as carrying out a much higher number of analyses, in order to
be evaluated as satisfactory. This may in fact compromise the quality of routine measurements as
a disproportionate level of effort is being expended for the PT.
A: Unless otherwise instructed, participants should analyse the test materials
using any method that they feel is appropriate. Participants are asked to treat the PT material
in the same way as a routine sample. Participants may be asked to state their method when
reporting results. It is important that this information is accurate as results are analysed
and reported according to the method stated.
A: Deadlines are specified for the return of results, to ensure the timely
issue of assigned values and reports to participants. For each scheme a closure date will
therefore be specified. For certain tests there may also be a date specified by which
examination of the sample is recommended to have been commenced. This is to ensure that
sufficient time is available to complete the test and report results in time for the
deadline date.
A: It is recommended that results and calculations are checked thoroughly before
reporting. Once submitted and received, results may only be amended at the discretion of the
scheme coordinator. No changes can be made after the deadline has passed. Results should be
reported clearly in the form and units requested. If calculations are used, unless instructed
otherwise, the laboratory is to report only the final calculated result. Part of the challenge
of proficiency testing is the ability to perform calculations and transcribe results correctly.
LGC Standards Proficiency Testing administrators cannot interpret or calculate results on
participants’ behalf.
A: The aim when evaluating measurement uncertainty is to combine the
effects of all the errors, that will influence the measurement result,
into a single value. There are many different guides available which
provide advice on evaluating measurement uncertainty. Two specific
guides that are internationally recognised, are
- ISO (BIPM, IEC, IFCC, IUPAC, IUPAP & OIML) "Guide to the Expression of Uncertainty in Measurement"
- EURACHEM/CITAC Guide "Quantifying Uncertainty in Analytical Measurement"
Further information on approaches to evaluating measurement uncertainty
may also be available from your national accreditation body. For
example, UKAS M3003 "The expression of uncertainty and confidence in
measurement".
The EURACHEM/CITAC Guide can be downloaded, along with further
information on evaluating measurement uncertainty, from the website
www.nmschembio.org.uk
A: It is possible, but must be regarded as a very rough estimate, and
is not an approach addressed in many guides to evaluating measurement
uncertainty. However, two documents that do address the use of PT data
are:
- NORDTEST Report TR 537 "Handbook for Calculation of Measurement
Uncertainty in Environmental Laboratories"
- ISO/TS 19036 "Microbiology of food and animal feeding stuffs -
Guidelines for the estimation of measurement uncertainty for
quantitative determinations"
A: The robust standard deviation provided in the PT report for a
specific method can be taken as an estimate of the reproducibility
standard deviation for the PT round for that specific method
A: Participants are asked to return results by the given deadline in order to
ensure that their results are included in the statistical analysis and the scheme report.
Results received after the closure date will not be included in the report. For schemes where
a generic report is issued, this is available to all participants subscribing to the round
regardless of whether their results were submitted or not.
A: It defeats the objective of taking part in proficiency testing if participants
are not returning genuine results. Certain measures are built into the schemes to try and
prevent collusion but, ultimately the responsibility rests with each participating laboratory
to behave in a professional manner
A: ISO 13528 states:‘Statistical methods for use in proficiency testing by
interlaboratory comparisons’ sets out how the assigned value and performance assessment
criteria can be established and describes the options for the various scoring systems.
The assigned value is the value selected as being the best estimate of the ‘true value’
for the parameter under test. The method used to determine the assigned value may vary
depending upon the particular scheme and test material and is detailed in the relevant
scheme description. When the assigned value is determined from the consensus value of participant
results, robust statistical methods are used for the consensus value, the estimated standard
uncertainty and the robust standard deviation. For qualitative tests, participant results are
compared against the intended result based on formulation.For schemes where the result is
subjective rather than quantifiable, a model answer produced by appropriate experts will be
published in the report.For microbiology samples, all participant results are transformed by
converting them to log10 before the statistical analysis is undertaken.
A: Once the assigned value for the parameters under test has been established,
participant laboratories are assessed on the difference between their result and the assigned
value, with this difference being represented by a performance score called a z-score. This
provides a simple and consistent measure of performance which is the key to monitoring
competence and implementing an improvement programme as required.
A:Results can be expressed in a form that is easy to interpret and understand;
Results can be summarised in graphical or tabular form to depict overall performance;
A performance score allows participants to directly compare their own result with others;
If consistent statistical values are applied, a performance score enables participants
to monitor and trend their own performance over time. It is important to interpret any
performance score in the full context of the overall results and in the context of a
laboratory’s own quality control measures.
A: The participant’s result, x, is converted into a z-score using the following
formula;
For small data sets, there will be increased uncertainty around the assigned value if using
consensus values from participants’ results. In such cases, z-scores may not be provided,
or may be given for information only.The z-score expresses performance in relation to the
assigned value and standard deviation. A z-score of 2 represents a result that is a
distance of 2 x SDPA from the assigned value. A fixed value for SDPA is preferable as
this enables z-scores to be compared from round to round to demonstrate general trends.
For each scheme, the value of SDPA and the method used to derive it is reported in the
scheme description and/or report.
A: The SDPA expresses the acceptable difference between the laboratory result and
the assigned value. An acceptable z-score represents a result that does not deviate from the
assigned value by more than twice the SDPA. The method used to determine the SDPA may vary
depending upon the particular scheme and test material and is detailed in the relevant scheme
description.
A: There are many sources of variation in microbiological testing and the SDPA
used to assess performance therefore needs to be fit-for-purpose and take all possible
sources of variation into account. From experience and historical data, LGC Standards PT uses a
fixed SDPA value of 0.35 log10 for the majority of microbiological tests.
A: For quantitative examinations, participant performance is assessed using the
z-score, and the following interpretation is given to results.|z| ≤ 2.00 Satisfactory result2.00
< |z| < 3.00 Questionable result |z| ≥ 3.00 Unsatisfactory result For qualitative
examinations, laboratories reporting the assigned result will be considered correct.
A: You can calculate your z score by visiting our website www.lgcpt.com and you will
find instructions in the Reports and Assigned Values page.
A: You can do this by Trend Analysis. A single result simply reflects the
performance of the laboratory on the particular day that the test was carried out and
therefore gives limited information. Frequent participation in PT schemes over time can
give greater insight into long-term performance and can help identify where internal bias
may be occurring. One of the best methods of summarising z-scores over time is graphically
as this gives a clear overview and is less prone to misinterpretation than numerical methods.
Participants are therefore advised to monitor their PT results over time. Further information
regarding interpretation and trend analysis of proficiency results is given in the IUPAC
‘International Harmonised Protocol for the Proficiency Testing of Analytical Chemistry
Laboratories’, 2006 and ISO 13528.Trend Analysis is an integral part of our PORTAL software.
A: Summary reports are sent electronically and the full report may be downloaded
from the LGC Standards PT website. Paper copies are also available for an additional charge.
The contents of reports vary from scheme to scheme but include details of the composition of
test materials, the assigned values, and tabular and/or graphical representations of participants’
results.
A: Yes we can produce reports tailored to a customer's specific requirement.
There will be an additional charge for administration and computer programming costs.
Copyright to all reports remains with LGC Standards Proficiency Testing but permission is
granted to participants to make copies for their own internal use, for example for quality
control and regulatory purposes. No other copies may be made without obtaining permission.
A: Communication with participants will be carried out through scheme-related
documentation, e-mails, letters, newsletters, memos, fax, or through LGC Standards regional
offices. Open meetings may also be organised and all interested parties invited to attend.
Feedback from participants is encouraged. Comments on any aspect of the scheme are welcome either
by phone, fax, e-mail, letter, or website www.lgcpt.com
A: A single poor result is not indicative of overall laboratory performance but neither is a single good result.
Ideally, PT results should be monitored over time to detect unusual bias or repeated unsatisfactory results indicate
poor performance. There are many possible reasons for a single unsatisfactory result including statistical chance.
It is therefore important to interpret the results from PT schemes within the context of an all-round quality assurance
programme, including internal quality control, use of validated methods and reference materials.
There are numerous potential causes of unsatisfactory performance in a PT scheme which may
include analytical and non-analytical errors:Analytical errorsCalibration / instrument problems;
Extraction / clean-up;Interferences / matrix effects;Diagnostic kits / reagents;Analyst /
method performance.Non-analytical errorsCalculation / transcription;Reporting format /
units;Storage;Sample defects.Samples are subjected to rigorous quality control testing
before being distributed to participants, and are unlikely to be the cause of a poor z-score.
All possible reasons for a poor z-score should be investigated fully in order to identify the
most likely cause and to enable action to be taken to prevent recurrence. Repeat samples are
available after every distribution, but it is most important to investigate and understand the
reason(s) for the failure, document this fully, and carry out corrective actions before
repeating a test.