Meta-Analysis of Commercial-Scale Trials as a Means to Improve Decision-Making Processes in the Poultry Industry: A Phytogenic Feed Additive Case Study

The feed conversion ratio (FCR) is, in addition to the cost of the feed, the most influential variable in the cost structure of poultry production^1,2 and consequently, drives the economic efficiency of poultry operations. As a result, the FCR represents an important response variable in nutrition experiments, irrespective of whether or not they are complex, as metabolism studies^3,4, or as simple as the experiments that are usually conducted to evaluate nutritional and feeding interventions. Standard nutritional experiments are frequently used to examine changes in nutrient requirements⁵, use of supra-nutritional nutrient levels to modulate physiological responses, the inclusion of feed additives to optimize performance and the application of feeding strategies in broilers or layer hens⁶.

The trend to produce antibiotic-free broilers is pressing the allied industry to develop technologies that help to overcome the multimodal action mechanisms of antimicrobial growth promoters⁷. An important area of research is related to the use of plant-derived products (phytogenics) to exert positive effects. Indeed, oregano (Origanum vulgare) represents a widely studied plant-derivative, as its essential oil and its main secondary metabolites (carvacrol and thymol)8 have shown several biologically important activities, including antimicrobial^9,10, antioxidant^11,12, endogenous enzyme activity promoting^13,14 and prebiotic¹⁵ properties, as well as its ability to promote intestinal mucosa structure and health¹⁶ and prevent coccidia^17,18. However, the overall effect of oregano essential oil on broiler performance could be challengedependent¹⁹ and may vary if the chemical composition is inconstant²⁰.

In this regard, it becomes a complex task to perform an experiment to test these technologies, while also satisfying statistical power and meeting growing conditions similar to the industry, where natural pathogenic challenges limit the expression of the genetic potential²¹. The main reason for this is that the larger the experimental unit, the lower the statistical power, as less experimental units will be available²². In contrast, statistical power can be increased if more replications are made available using smaller floor pens or cages; however, the growing conditions would become less similar to the commercial ones, which would lower the challenging conditions.

One of the main limiting aspects faced by the industry and researchers is to design experiments that are sensitive enough to detect numerically small effects^3,4, such as those expected in FCR when phytogenic feed additives are tested. Usually, most of these can be economically justified with an improvement in FCR lower than 1.5%; however, the design of experiments offering such statistical sensitivity is not only a complex task²³ but is also rare. As a result, detecting these small effects becomes extremely unlikely if the study is performed under commercial conditions to test a particular technology in a real usage scenario.

In this context, meta-analysis of independent studies has been proposed as a strategy to increase statistical power^24,25. Consequently, this is expected to support decision-making processes based on commercial-scale experiments where statistical sensitivity is insufficient, or when the expected effect is relatively low but still economically relevant. Therefore, the objective of this case study was to determine the overall effect of a phytogenic feed additive on the performance variables of broilers. In addition, we sought to compare these results with those from independent experiments included in the analysis.

Experiments: Nine independent experiments (EX1 to EX9) were performed and included a total of 622,496 broilers (Table 1). In all experiments, 1-day-old male Cobb 500 chicks were used, from 1-42 days of age. Within each experiment, birds were randomly allocated to the experimental units (EU): Whole broiler houses (7 experiments), floor pens (1 experiment), or cages (1 experiment). In experiments, reused litter based on rice husk was used as a bedding material and when cages were used, a screen was placed over the floor wiring to successfully retain the litter. In the nine experiments, corn-soybean meal-based pelleted diets that were formulated following the nutritional guidelines of the genetic line²⁶, were fed ad libitum to the birds under a four-phase feeding program (pre-starter, 0-8 day; starter, 9-18 day; grower, 19-28 day; finisher, 29-42 day) as shown in Table 2. Two dietary treatments were tested: T1, control diet and T2, control diet+the additive at a 0.05% inclusion rate, fed continuously from 1-42 day. In all cases, the treatments were randomly assigned to the EUs. The tested phytogenic oregano-derived commercial product (blind-coded as PHE780 by LIAN Development and Service Co., Lima, Peru) provided no less than 45 g of carvacrol per kg of product.

The response variables were final body weight (BW, g bird⁻¹), feed intake (FI, g bird⁻¹), FCR (g g⁻¹) mortality (%) and European Production Efficiency Factor (EPEF) following the calculation reported by Marcu et al.²⁷

 

In broiler houses, the BW was obtained by weighing 10 sub-samples of 50 birds, each one in different locations within the house and the FI was calculated assuming that all of the feed provided was eaten. In the floor pens and cages, the BW was obtained by weighing all the birds and the FI was calculated as the actual net amount of feed eaten.

Analyses of variance: Data were first analyzed independently by experiment under completely randomized designs and thereafter, data were combined and analyzed under a completely randomized block design, considering the experiment itself as the blocking factor²⁸. Normality of the data was determined using the Shapiro-Wilk test²⁹ and the existence of outliers was determined by Grubbs test³⁰. The response variables with non-normal distributions were analyzed with Kruskal-Wallis test³¹. In all cases, results were considered statistically significant when p≤0.05.

The additive linear model for the analysis of each independent experiment was Y_ij = μ + Ti + ε_ij, where Y_ij is the observed value in the i-th treatment (i: 1,...t) and j-th replication (j: 1,...r); μ is the effect of the general mean; T_i is the effect of the i-th treatment; ε_ij is the effect of the experimental error in the i-th treatment and j-th replication; t is the number of treatments; r is the number of replications in the i-th treatment; being that ε_ij~ N(µ,σ²) and independently, where N denotes the normal distribution among replications and σ² is the variance among the experimental error of the different EU.

In contrast, the additive linear model for the analysis of variance of the whole data was Y_ijk = μ + Ti + β_j + ε_ijk, where Y_ijk is the observed value in the i-th treatment (i: 1,...t), j-th block (j: 1,...p) and k-th replication (k: 1,...r); μ is the effect of the general mean; Ti is the effect of the i-th treatment; β_j is the effect of the j-th block; ε_ijk is the effect of the experimental error in the i-th treatment, j-th block and k-th replication; t is the number of treatments; p is the number of blocks; r is the number of replications in the i-th treatment; being that εijk~N(μ,σ²) and independently, where N denotes the normal distribution among replications and σ² is the variance among the experimental error of the different EU.

Meta-analyses: Independent meta-analyses were performed for each single response variable to determine the overall effect size, its 95% confidence interval (CI_95%) and its probability with Wald test³² and the existence of heterogeneity using a random-effects model with Cochran test³³ (Q statistic) and its corresponding probability with chi-square test³⁴. In all cases, results were considered statistically significant when p≤0.05.

The heterogeneity was determined considering the following linear additive model: y_i = µ+u_i +e_i, where y_i is the observed effect size in the i-th experiment (i: 1,...k) (and also, y_i = θi +e_i, where θi is the unknown true effect in the i-th experiment; e_i is the intra-experimental sampling error in the i-th experiment); u_i is the inter-experimental deviation regarding the overall effect size in the i-th experiment; e_i is the intra-experimental sampling error in the i-th study; k is the number of experiments; N denotes the normal distribution of the random inter-experimental deviation (u) and the intraexperimental sampling error (e); being that u_i~N(0,τ²) y e_i~N(0,ν_i) and both independently, where τ2 indicates the heterogeneity (variability among the true effects in the different experiments) and ν_i is the approximately known sampling variance of the estimated effect size in the i-th experiment.

To adjust the model, a weighted least square method was applied, implying that the adjusted model provides an estimate of ¯θ_w = Σw_iθ_i /Σw_i, where is the true weighted average effect size; w_i is the weighing factor considered, θ_i is the true effect size in the i-th experiment; that is, is the weighted average of the true effects (θ_i) in the set of k studies, with weights equal to the inverse of the corresponding variances (w_i = 1/ν_i).

In addition, the goodness of fit of model residues were evaluated with the Shapiro-Wilk test (normal if p>0.05). In cases where the residues were non-normally distributed, the data were analyzed again to determine the probability associated to the global effect size but this time, with applying a permutation test with 10,000 iterations.

Finally, the presence of bias within the data of each response variable was evaluated through the Egger regression test to determine the asymmetry of the distribution of the data, based on both the effect sizes and the precision of each experiment. Trim and Fill analysis was then performed to estimate the effect size values that would compensate distribution imbalances, if they existed, and if so, their magnitude and influence on the overall effect size were determined. As a result, each variable eventually had two sets of effect sizes: dO, being the set of effect sizes calculated from the experiments and dA, being the set of effect sizes that also included the values estimated through the Trim and Fill analysis. Thereafter, the bias was considered relevant if the Egger test was significant (p≤0.05) and if the CI_95% of the overall effect sizes, calculated with both the adjusted data (dA) and with the original data (dO), were not overlapped.

Software and informatics resources: Grubbs test for the detection of outliers was performed with GraphPad Prism 7 software₃₅. Kruskal-Wallis tests and variance analyses were performed in SAS 9.4 using NPAR1WAY with Wilcoxon restriction and GLM procedures, respectively³⁶. The goodness of fit to the normal distribution and meta-analyses routines were performed with stats and Metafor 2.0-0³⁷ packages in R 3.5.2 version programming language³⁸ using RStudio 1.1.456 as an interface³⁹.

Analyses of variance: No outliers were detected in the data from each independent experiment; however, the Shapiro-Wilk goodness of fit test showed non-normally distributed mortality values; therefore, the data of this variable were analyzed with the Kruskal-Wallis test. The results found in each of the nine experiments and in the combined analysis are shown in Table 3. The highest percentage differences between treatments in BW, FCR and EPEF were +5.28, -4.50 and +6.65% in experiments EX6, EX5 and EX1, respectively; however, even these differences were not statistically significant (p>0.12). Similarly, the combined analysis of the nine experiments under a completely randomised block design showed no statistically significant effects on any of the tested variables (p>0.08).

Meta-analyses results: Table 4 shows the meta-analyses results. Test for the goodness of fit of model residuals found mortality values being non-normally distributed; therefore, the overall effect size p-value for this variable was recalculated by applying a permutation test. BW (Fig. 1), FI (Fig. 2), mortality (Fig. 3) and EPEF (Fig. 4) showed no significant (p>0.05) overall effect sizes and had CI_95% with limit values with opposite mathematical signs (positive, negative). A statistically significant (p<0.05) overall effect size was found in FCR (Fig. 5), with a CI_95% with negative limit values. The Trim and Fill tests determined and estimated possibly missing BW, FI and mortality values; however, the CI_95% of the adjusted overall effect sizes for all these variables, were overlapped with the CI_95% calculated with the original data; therefore, if biases existed, they were not considered to be relevant. In addition, Egger tests did not detect statistically significant bias (p>0.50) and no statistically significant heterogeneity was found among experiments in any of the tested variables (p>0.23).

This study investigated the effect of a phytogenic feed additive on the performance of broilers. We sought to explore three different approaches to analyze the data from nine 

Fig. 1: Forest plot of the effects of a phytogenic feed additive on the body weight of 42-day-old broilers (nine experiments)

Fig. 2: Forest plot of the effects of a phytogenic feed additive on the feed intake of 42-day-old broilers (nine experiments)

Fig. 3: Forest plot of the effects of a phytogenic feed additive on the mortality of 42-day-old broilers (nine experiments)

experiments to increase the likelihood of finding statistically significant effects, if they existed. The aim of this study was to determine a suitable method to improve decision-making processes related to nutrition and feeding strategies in the poultry industry.

The results showed that neither analyzing the data from the different experiments independently under completely randomised designs, nor combining all the data under a block design, led to statistically significant effects in any of the tested variables. The lack of sensitivity to detect differences as big as +5.28, -4.50 and +6.65% in BW, FCR and EPEF, respectively, was influenced by the low number of replications used in the experiments²¹. However, this is the usual scenario faced by the industry when evaluating nutrition or feeding 

Fig. 4: Forest plot of the effects of a phytogenic feed additive on the European production efficiency factor of 42-day-old broilers (nine experiments)

Fig. 5: Forest plot of the effects of a phytogenic feed additive on the feed conversion ratio of 42-day-old broilers (nine experiments)

strategies under actual commercial-scale conditions⁴⁰. Under such situations, there are three main consequences: (1) Companies take positive decisions but without the desired confidence and, consequently, become short-lived, (2) decision-making processes become complex and longer; or (3) no decision is taken, status quo is maintained and the opportunity to improve results may be lost. In addition, it has been reported that when it is more difficult for a person to make decisions based on rigorous reasoning, it ultimately leads to a more intuitive and heuristic thinking process due to decision fatigue; consequently, less judicious decisions are taken⁴¹.

Although none of the independent experiments showed significant effects on the studied variables (p>0.05), this should not be interpreted as that the evaluated product does not produce an effect on these response variables. Instead, this may be explained by the fact that in hypothesis testing, the null hypothesis (that both means are equal) can only be rejected and not proved^42,43. In this regard, under the Neyman-Pearson dichotomous approach, a p-value greater than the pre-established α level of significance in a hypothesis test of the difference of two means determines that the null hypothesis must be exhaustively accepted as true. However, the Fischer approach considers the p-value as a continuous measure of the strength of evidence⁴⁴ and states that the absence of a significant effect could only indicate that, if such an effect exists, it is not sufficiently large to be detected by an experiment of the size used⁴⁵.

Although the meta-analyses did not detect effects on BW, FI, mortality, or EPEF, we demonstrated an improvement in FCR that was due to the feed additive tested, in that the supplemented birds converted feed to body weight more efficiently (35 g less feed per kg body weight obtained). No significant heterogeneity was detected among experiments (p>0.87), indicating that the effect of the feed additive on the FCR was not inconsistent across the nine experiments. In addition, the CI of the effect size in FCR (0.0006 to 0.0686 less FCR points) indicates that, regardless of the accuracy of the estimation of the effect, the real effect of the phytogenic on feed efficiency is positive⁴³.

The effect of the tested phytogenic feed additive found in FCR agrees with previous reports about the effect of oregano essential oil on the FCR of broilers^17,46-49. This finding is consistent with the antimicrobial^9,10, antioxidant^11-13, endogenous enzyme activity promoting^10,14, prebiotic¹⁵, anticoccidial¹⁸ and gut mucosa promoting effects¹⁶ of oregano essential oil that have been previously shown. Besides, previous studies have reported positive effects of oregano essential oil on intestinal mucosa structure, nutrient absorption capacity, bone mineralization and overall performance⁵⁰. It has been reported that the effect of oregano essential oil on broiler performance could be challenge-dependent¹⁹; however, in the current study, the experiments were conducted under commercial conditions, unavoidably implying certain intestinal challenges, since reused litter material was used in all experiments^51-53.

In the present study, when combining the data from all the experiments under a completely randomized block design, the statistical power for FCR increased and, therefore, the p-value (p = 0.085) was lower in comparison to that observed in individual experiments (p-values: 0.151-0.867). However, the p-value was not only not considered significant but also was 85% higher than that obtained for the overall effect on FCR through meta-analysis (p = 0.046).

Thus, in the present analysis, we demonstrate how meta-analyses of the results obtained in different experiments favour the probability of detecting an effect, when it exists, that may not be evident in independent experiments. In this regard, although the meta-analyses carried out using random effects models do not guarantee that the inclusion of additional studies increases the statistical power of the analysis, in general, it does increase the statistical power in comparison to the independent studies^24,25. This is particularly useful when the critical response variable in an experiment is FCR, as usually a small percentage effect, even less than 2%⁵⁴, is sufficient for the poultry producer to justify making a favourable decision regarding the nutritional benefit of the feeding strategy tested. In addition, significant effect sizes obtained by meta-analysis also allows the nutritionist to make a cost-sensitivity analysis^55,56. Previous meta-analyses have detected small percentage effects on FCR in broilers⁵⁴, layer hens⁵⁷ and pigs⁵⁸; however, to the best of our knowledge, a meta-analysis approach has not yet been reported for analysing commercial size trials with a low number of replications to help improve statistical sensitivity.

Experiment standardization is a common strategy to increase the sensitivity of the test; however, this also reduces the reproducibility of the results⁵⁹. In this regard, meta-analysis of commercial-scale experiments not only allows the sensitivity of the analysis to be increased²⁴ but also preserves the reproducibility of the results, as they are performed in conditions less homogeneous than those of a highly controlled research facility. Therefore, the higher the systematic variation, the greater the reproducibility of the experiment⁵⁹. Finally, in poultry nutrition research, statistical sensitivity and growing conditions similar to the industry are commonly opposite objectives, as the more sensitive a design is, the more replications it takes and the smaller they become²¹; however, a meta-analysis can go some way to help solve this dichotomy.

In the present study, we tested a phytogenic feed additive, based on oregano essential oil, providing no less than 45 g carvacrol per kg of product and fed at an inclusion rate of 0.05%, continuously from 1-42 day. Based on the observed results, it can be concluded that the tested product improved the FCR of broilers under commercial-scale conditions, in that it increased the efficiency of converting feed into BW (35 g less feed per 1 kg of BW obtained). In addition, the analysis of the nine conducted experiments using a meta-analysis approach improved the statistical power to a greater magnitude than that observed by applying a block design. Moreover, the meta-analysis was sensitive enough to detect a statistical significance that, otherwise, would have remained undetected.

This study demonstrated that meta-analysis is a useful technique to improve statistical power and to help find statistically significant differences, if they exist, when testing nutritional interventions under commercial conditions. We postulate that the use of meta-analysis in the poultry industry would help industry nutritionists and researchers to establish a more efficient but still simple, system to evaluate nutrition interventions, including feed additives and consequently, provide a means to facilitate and objectivize decision-making processes.