AS 2542.1.3:2014 pdf – Sensory analysis Part 1.3: Methodology-Selection,training and monitoring of selected assessors and expert sensory assessors.
Significant variation between assessors indicates the presence of bias, I.e. one or more assessors give(s)
scores consistently higher or lower than the others. Significant variation between samples indicates that the assessors as a panel successfully differentiate between the samples. A significant assessors—samples interaction indicates that two or more of the assessors have a different perception of the dissimilarities between two or more samples. In some cases, an assessors—samples interaction may even reflect a disagreement about the ranking of the samples.
Although ANOVA is appropriate for scoring, it is not appropriate for some forms of rating. If, for instance, a ranking procedure Is used, then non-parametric methods like the Friedman test might be more appropriate (see ISO 8587(71).
6.5 Qualitative descriptive analysis
No additional specific selection procedure Is advocated among those already outlined.
Assessors are chosen on the basis of their performance in the various exercises, in particular those
specified in 5.6 and 5.9,2.
6.6 Quantitative descriptive analysis
6.6.1 Assessors are chosen for their skills and their aptitude to follow an Intensive training in order to become expert sensory assessors.
Panel leaders should evaluate the performance of the selected assessors, over a period of time and on the products concerned. Any selected assessors who show good repeatability, good acuity or particular aptitude regarding specific attributes (e.g. a taint) of classes of materials. should be considered for use on panels of expert assessors.
Moreover desirable characteristics of candidates Include:
a) a memory for sensory attributes;
b) an ability to communicate with other assessors;
c) an ability to verbalize descriptions of products.
The extent to which selected assessors possess these characteristics varies. So a complementary selection could be performed or the training programme adjusted accordingly.
6.6.2 II controls or reference samples (see ISO 5492) have been provided, candidates shall be tested for their ability to recognize and describe them.
6.6.3 Assessors shall assess approximately six samples using the vocabulary and score sheet developed as specified in 5.9.2. The samples shall he presented in triplicate In a property balanced order. Each descriptor for each assessor shall then be subjected to analyses similar to those specified in 6.4 and described in Annex H or to other multidimensional methods of analysis (ISO 132991H1).
6.7 Assessors for particular assessments
Despite being selected as the most suitable candidates, selected assessors may fluctuate in their performance during training. With descriptive analysis, It can often prove an advantage to select the better performers or to divide the assessors Into subgroups following a programme of assessments and before additional training or any complex statistical examination of the data. For this purpose, the procedures used are the same as those specified In 6.4.
8.2 Principle
The principles of monitoring performance are based on:
— participation in various sensory tests (depending on their specificitles) for expert sensorlal assessors;
— production of product or material profiles with one or more inter- or intrasesslon repetitions for expert sensory assessors;
— participation in interlaboratory tests according to ISO 5725161 within the same sector of activity (suppliers or subcontractors working on the profiles of the same products).
8.3 Analysis of the results
8.3.1 General
The analysis of the results obtained allows the assessment of the performance of the group as a whole as well as the individual performances of the assessors (see Annex A).
8.3.2 Assessment of the performance of the group as a whole Different methods can be used. e.g. ANOVA:
— single factor ANOVA (products) In order to assess the discriminatory ability;
— three factor ANOVA (products, assessors, sessions) as well as superposing of the profiles of the two or three sessions in order to check reproducibility by studying thi session factor and the session- product interaction;
— three factor ANOVA (products, assessors session)) In order to ensure homogeneity by studying the product—assessor interaction.
Other statistical techniques, e.g. principal component analysis (PCA). discriminant factor analysis (DFA) generalized Procrustes analysis, calculation of the RV coefficients (coefficient enabling the assessment of the degree of similarity of two matrices, allow the agreement of the assessors with one another and with respect to the group conclusion to be examined.