Stabilizing predicted values from NIR using REP files

REP files ("repp files") make calibrations less sensitive to environmental and sample variations. The abbreviation REP has usually been referred to as repeatability although it would be more correct to let it stand for reproducibility. 

To illustrate the effect of using REP files to stabilize NIR calibrations, data from the sugarcane analysis system in the Okinawa and Kagoshima prefectures was used. The calibration file contained 482 samples collected with an NIR analyser. Later this instrument would act as the network master. The sugarcane sucrose content ranged from 10 to 20% using polarimetry as the reference method (hence the name Pol in Cane), and the calibration data were collected during a period of four years. MPLS was used for the regression based standard normal variate and detrending pretreatments using the first derivative (1, 4, 4, 1). 

Figure 1. Accuracy of sucrose determination in sugar cane (pol in cane) without any REP file.

 

Figure 2. Accuracy of sucrose determination in sugar cane (pol in cane) when using a REP file. As can be seen when comparing to Figure 1, the increase in accuracy error is negligible.

The REP file must contain NIR spectra. Reference data are not necessary and this is a very convenient point. The REP file is built up from samples recorded at different conditions. Spectra from the same sample scanned at different temperatures are given the same sample number, for example "1". Spectra from another sample, also scanned at different temperatures, can be given the sample name "2". If a sample has been scanned across several instruments, all of those spectra can be given sample name "3". If a sample has been scanned at different packing pressures, its spectra at different repacks can be given the sample name "4". All spectra with the same sample name will come from the same physical sample, but recorded at different environmental conditions. The variability in the spectra for the same physical sample is then automatically used in the calibration step to systematically down-weight the wavelengths with large variability, as found in the REP file. 

Figure 3. No REP file: Five samples analyzed at low, medium and high temperatures (ca. 10, 25 and 35 °C), repacked replicates.

 

Figure 4. With REP file: Five samples analyzed at low, medium and high temperatures (ca. 10, 25 and 35 °C), repacked replicates.

 

Figure 5. No REP file: 10 samples were scanned without any repack in 11 instruments located in different factories. The standard deviation was 0.33%.

 

Figure 6. With REP file: 10 samples were scanned without any repack in 11 instruments located in different factories. The standard deviation was 0.11%.

From the negligible increase in accuracy error it, was concluded that the accuracy in practice would be the same with and without the REP file (0.30 to 0.31%), see Figures 1 and 2. However, accuracy for one (master) instrument is not the only important thing to consider. In Figures 3 and 4 it can be seen that there is less variability around the averages due to temperature when a REP file is used. Without REP file, the variation at different temperatures was 0.68% (measured as standard deviation) and when using a REP file, it came down to 0.31%. Although this is still quite large compared to the SEP(C), this means that the predicted results are only about half as much influenced by the temperatures. This is beneficial as the NIR instrument can be used in environments with different temperatures to a larger extent. The reproducibility errors were reduced from 0.33 to 0.11%. This is a very clear improvement and for this particular case, the reproducibility error came down to 1/3 of the SEP(C). This is a tremendous improvement making the contribution from variations between instruments to the total network accuracy measured as the total network-SEP(C) becomes negligible. Reproducibility is the most important thing in any science, including analytical chemistry and quantitative (payment) analysis based on NIR spectrometry. 

The use of REP files is justified by a negligible increase in accuracy error combined with a clear improvement in insensitivity to temperature as well as tremendous improvement in reproducibility. REP files should therefore always be used. 

Asst. Prof. Eizo Taira is acknowledged for sharing these data.