A new method to compare treatments in unreplicated on-farm experimentation

The design and analysis of on-farm experimentation (OFE) have received growing attention because of the availability of precision machinery that promotes data collection. Even though replicated trials are the most recommended designs, on-farm trials with no replication are used in scenarios where variable rate technology is not available. Despite the abundance of georeferenced data within each plot harvested with yield monitor, treatments are not replicated. This paper presents an approach to statistically analyze unreplicated OFE promoting field-specific inference of treatment effects. Statistical tools for spatial data are coupled with permutation tests to determine the statistical significance between treatment means. The new methodology (OFE-mean test) involves: (1) calculation of effective sample size (ESS) given the underlying spatial structure, (2) ANOVA permutation test on a random sample of ESS, and (3) generation of the empirical distribution of p-values from repetition of step two. The median of this empirical distribution is regarded as the p-value associated with the no treatment effect hypothesis. The OFE-mean test is illustrated using several OFE trials comparing two treatments under different scenarios: with and without treatment differences. Additional assessment is carried out under simulated scenarios with different levels of spatial correlation, variability, and mean differences between treatments. The OFE-mean test had high power to detect mean differences higher than 15% for all spatial structures when total variability was lower than 30%. After treatment effects were removed, no type I error occurred in real data. The test can be easily extended to cover scenarios with more than two treatments. R scripts and sample files to run the OFE-mean test are provided.