Brereton Chemometrics

If, as is usual in chemistry, there are many more than two measurements, it is simply necessary to extend the concept of distance to one in multidimensional space. Although we cannot visualise more than three dimensions, computers can handle geometry in an indefinite number of dimensions, and the idea of distance is easy to generalise. In the case of Figure 4.36 it is not really necessary to perform an elaborate computation to classify the unknown, but when a large number of measurements have been made, e.g. in spectroscopy, it is often hard to determine the class of an unknown by simple graphical approaches.

4.5.5.2 Limitations

This conceptually simple approach works well in many situations, but it is important to understand the limitations.

The first is that the numbers in each class of the training set should be approximately equal, otherwise the ‘votes’ will be biased towards the class with most representatives. The second is that for the simplest implementations, each variable assumes equal significance. In spectroscopy, we may record hundreds of wavelengths, and some will either not be diagnostic or else be correlated. A way of getting round this is either to select the variables or else to use another distance measure, just as in cluster analysis. Mahalanobis distance is a common alternative measure. The third problem is that ambiguous or outlying samples in the training set can result in major problems in the resultant classification. Fourth, the methods take no account of the spread or variance in a class. For example, if we were trying to determine whether a forensic sample is a forgery, it is likely that the class of forgeries has a much higher variance to the class of nonforged samples. The methods in the Sections 4.5.2 to 4.5.4 would normally take this into account.

However, KNN is a very simple approach that can be easily understood and programmed. Many chemists like these approaches, whereas statisticians often prefer the more elaborate methods involving modelling the data. KNN makes very few assumptions, whereas methods based on modelling often inherently make assumptions such as normality of noise distributions that are not always experimentally justified, especially when statistical tests are then employed to provide probabilities of class membership. In practice, a good strategy is to use several different methods for classification and see if similar results are obtained. Often the differences in performance of various approaches are not entirely due to the algorithm itself but in data scaling, distance measures, variable selection, validation method and so on. Some advocates of certain approaches do not always make this entirely clear.

4.6 Multiway Pattern Recognition

Most traditional chemometrics is concerned with two-way data, often represented by matrices. However, over the past decade there has been increasing interest in threeway chemical data. Instead of organising the information as a two-dimensional array [Figure 4.38(a)], it falls into a three-dimensional ‘tensor’ or box [Figure 4.38(b)]. Such datasets are surprisingly common. In Chapter 5 we discussed multiway PLS (Section 5.5.3), the discussion in this section being restricted to pattern recognition.

J

J

I

Figure 4.38

(a) Two-way and (b) three-way data

24

6

20

Figure 4.39

Possible method of arranging environmental sampling data

Consider, for example, an environmental chemical experiment in which the concentrations of six elements are measured at 20 sampling sites on 24 days in a year. There will be 20 × 24 × 6 or 2880 measurements; however, these can be organised as a ‘box’ with 20 planes each corresponding to a sampling site, and of dimensions 24 × 6 (Figure 4.39). Such datasets have been available for many years to psychologists and in sensory research. A typical example might involve a taste panel assessing 20 food products. Each food could involve the use of 10 judges who score eight attributes, resulting in a 20 × 10 × 8 box. In psychology, we might be following the reactions of 15 individuals to five different tests on 10 different days, possibly each day under slightly different conditions, and so have a 15 × 5 × 10 box. These problems involve finding the main factors that influence the taste of a food or the source of pollutant or the reactions of an individual, and are a form of pattern recognition.

Three-dimensional analogies to principal components are required. There are no direct analogies to scores and loadings as in PCA, so the components in each of the three dimensions are often called ‘weights’. There are a number of methods available to tackle this problem.

4.6.1 Tucker3 Models

These models involve calculating weight matrices corresponding to each of the three dimensions (e.g. sampling site, date and metal), together with a ‘core’ box or array,

Figure 4.40

Tucker3 decomposition

which provides a measure of magnitude. The three weight matrices do not necessarily have the same dimensions, so the number of significant components for the sampling sites may be different to those for the dates, unlike normal PCA where one of the dimensions of both the scores and loadings matrices must be identical. This model (or decomposition) is represented in Figure 4.40. The easiest mathematical approach is by expressing the model as a summation:

l=1 m=1 n=1

where z represents what is often called a core array and a, b and c are functions relating to each of the three types of variable. Some authors use the concept of ‘tensor multiplication’, being a 3D analogy to ‘matrix multiplication’ in two dimensions; however, the details are confusing and conceptually it is probably best to stick to summations, which is what computer programs do.

4.6.2 PARAFAC

PARAFAC (parallel factor analysis) differs from the Tucker3 models in that each of the three dimensions contains the same number of components. Hence the model can be represented as the sum of contributions due to g components, just as in normal PCA, as illustrated in Figure 4.41 and represented algebraically by

G

xijk ≈ aig bjg ckg g=1

Each component can be characterised by one vector that is analogous to a scores vector and two vectors that are analogous to loadings, but some keep to the notation of ‘weights’ in three dimensions. Components can, in favourable circumstances, be assigned a physical meaning. A simple example might involve following a reaction by recording a chromatogram from HPLC–DAD at different reaction times. A box

J

Figure 4.41

PARAFAC

whose dimensions are reaction time × elution time × wavelength is obtained. If there are three factors in the data, this would imply three significant compounds in a cluster in the chromatogram (or three significant reactants), and the weights should correspond to the reaction profile, the chromatogram and the spectrum of each compound.

PARAFAC is difficult to use, however, and, although the results are easy to interpret physically, it is conceptually more complex than PCA. Nevertheless, it can lead to results that are directly interpretable physically, whereas the factors in PCA have a purely abstract meaning.

4.6.3 Unfolding

Another approach is simply to ‘unfold’ the ‘box’ to give a long matrix. In the environmental chemistry example, instead of each sample being represented by a 24 × 6 matrix, it could be represented by a vector of length 144, each measurement consisting of the measurement of one element on one date, e.g. the measurement of Cd concentration on July 15. Then a matrix of dimensions 20 (sampling sites) × 144 (variables) is produced (Figure 4.42) and subjected to normal PCA. Note that a box can be subdivided into planes in three different ways (compare Figure 4.39 with Figure 4.42), according to which dimension is regarded as the ‘major’ dimension. When unfolding it is also important to consider details of scaling and centring which become far more complex in three dimensions as opposed to two. After unfolding, normal PCA can be performed. Components can be averaged over related variables, for example we could take an average loading for Cd over all dates to give an overall picture of its influence on the observed data.

This comparatively simple approach is sometimes sufficient but the PCA calculation neglects to take into account the relationships between the variables. For example, the

Figure 4.42

Unfolding

relationship between concentration of Cd on July 15 and that on August 1 is considered to be no stronger than the relationship between Cd concentration on July 15 and Hg on November 1 during the calculation of the components. However, after the calculations have been performed it is still possible to regroup the loadings and sometimes an easily understood method such as unfolded PCA can be of value.

Problems

Problem 4.1 Grouping of Elements from Fundamental Properties Using PCA

Section 4.3.2 Section 4.3.3.1 Section 4.3.5 Section 4.3.6.4

The table below lists 27 elements, divided into six groups according to their position in the periodic table together with five physical properties.

1.Standardise the five variables, using the population (rather than sample) standard deviation. Why is this preprocessing necessary to obtain sensible results in this case?

2.Calculate the scores, loadings and eigenvalues of the first two PCs of the standardised data. What is the sum of the first two eigenvalues, and what proportion of the overall variability do they represent?

3.Plot a graph of the scores of PC2 versus PC1, labelling the points. Comment on the grouping in the scores plot.

4.Plot a graph of the loadings of PC2 versus PC1, labelling the points. Which variables cluster together and which appears to behave differently? Hence which physical property mainly accounts for PC2?

5.Calculate the correlation matrix between each of the five fundamental parameters. How does this relate to clustering in the loadings plot?

6.Remove the parameter that exhibits a high loading in PC2 and recalculate the scores using only four parameters. Plot the scores. What do you observe, and why?

Problem 4.2 Introductory PCA

Section 4.3.2 Section 4.3.3.1

The following is a data matrix, consisting of seven samples and six variables:

The scores of the first two principal components on the centred data matrix are given

1.Since X ≈ T .P, calculate the loadings for the first two PCs using the pseudoinverse, remembering to centre the original data matrix first.

2.Demonstrate that the two scores vectors are orthogonal and the two loadings vectors are orthonormal. Remember that the answer will only to be within a certain degree of numerical accuracy.

3.Determine the eigenvalues and percentage variance of the first two principal components.

Problem 4.3 Introduction to Cluster Analysis

Section 4.4

The following dataset consists of seven measurements (rows) on six objects A–F (columns)

1.Calculate the correlation matrix between the six objects.

2.Using the correlation matrix, perform cluster analysis using the furthest neighbour method. Illustrate each stage of linkage.

3.From the results in 2, draw a dendrogram, and deduce which objects cluster closely into groups.

Problem 4.4 Classification Using Euclidean Distance and KNN

Section 4.5.5 Section 4.4.1

The following data represent three measurements, x, y and z, made on two classes of compound:

1.Calculate the centroids of each class (this is done simply by averaging the values of the three measurements over each class).

2.Calculate the Euclidean distance of all nine objects from the centroids of both classes A and B (you should obtain a table of 18 numbers). Verify that all objects do, indeed, belong to their respective classes.

3.An object of unknown origins has measurements (0.5, 0.3, 0.3). What is the distance from the centroids of each class and so to which class is it more likely to belong?

4.The K nearest neighbour criterion can also be used for classification. Find the distance of the object in question 3 from the nine objects in the table above. Which are the three closest objects, and does this confirm the conclusions in question 3?

5.Is there one object in the original dataset that you might be slightly suspicious about?

Problem 4.5 Certification of NIR Filters Using PC Scores Plots

Section 4.3.2 Section 4.3.5.1 Section 4.3.3.1 Section 4.3.6.4

These data were obtained by the National Institute of Standards and Technology (USA) while developing a transfer standard for verification and calibration of the x-axis of NIR spectrometers. Optical filters were prepared from two separate melts, 2035 and 2035a, of a rare earth glass. Filters from both melts provide seven well-suited adsorption bands of very similar but not exactly identical location. One filter, Y, from one of the two melts was discovered to be unlabelled. Four 2035 filters and one 2035a filter were available at the time of this discovery. Six replicate spectra were taken from each filter. Band location data from these spectra are provided below, in cm−1. The expected location uncertainties range from 0.03 to 0.3 cm−1.

1.Standardise the peak positions for the 30 known samples (exclude samples Y).

2.Perform PCA on these data, retaining the first two PCs. Calculate the scores and eigenvalues. What will the sum of squares of the standardised data equal, and so what proportion of the variance is accounted for by the first two PCs?

3.Produce a scores plot of the first two PCs of this data, indicating the two groups using different symbols. Verify that there is a good discrimination using PCA.

4.Determine the origin of Y as follows. (a) For each variable subtract the mean and

divide by the standard deviation of the 30 known samples to give a 6 × 7 matrix stand X . (b) Then multiply this standardised data by the overall loadings, for the first PC to give T = stand X .P and predict the scores for these samples. (c) Superimpose

the scores of Y on to the scores plot obtained in 3, and so determine the origin of Y.

Problem 4.6 Simple KNN Classification

Section 4.5.5 Section 4.4.1

The following represents five measurements on 16 samples in two classes, a and b:

1.Calculate the 16 × 16 sample distance matrix, by computing the Euclidean distance between each sample.

2.For each sample, list the classes of the three and five nearest neighbours, using the distance matrix as a guide.

3.Verify that most samples belong to their proposed class. Is there a sample that is most probably misclassified?

Problem 4.7 Classification of Swedes into Fresh and Stored using SIMCA

Section 4.5.3 Section 4.3.6 Section 4.3.5 Section 4.3.2 Section 4.3.3.1

The following consist of a training set of 14 swedes (vegetable) divided into two groups, fresh and stored (indicated by F and S in the names), with the areas of eight GC peaks (A–H) from the extracts indicated. The aim is to set up a model to classify a swede into one of these two groups.

In addition, two test set samples, X and Y, each belonging to one of the groups F and S have also been analysed by GC:

1.Transform the data first by taking logarithms and then standardising over the 14 training set samples (use the population standard deviation). Why are these transformations used?

2.Perform PCA on the transformed PCs of the 14 objects in the training set, and retain the first two PCs. What are the eigenvalues of these PCs and to what percentage