Block designs for sequential block factors

Introduction

Comparative experiments in agriculture and biology often involve the estimation of treatment effects against a background of high non-treatment variability. Effective control of background variability is essential for good treatment estimation and the most common method of control is the block design. The aim of good block design is to group experimental units into homogeneous blocks where the precision of comparison within blocks is expected to be greater than the precision of comparison between blocks. Good block design should maximize the treatment information estimated within blocks and should minimize the treatment information confounded between blocks.

The most basic type of block design is the complete randomized blocks design where every treatment occurs in every block in proportion to its replication. The treatment replication need not be equal for all treatments but all treatments in the same block need to occur in some fixed proportion relative to their replication. Complete randomized blocks provide full efficiency on all treatment comparisons and are the designs of choice for experiments with a small number of treatments. However, because every block must contain every treatment at least once, complete randomized blocks may be too large to give good within-block homogeneity of variance. In that situation, sub-division into smaller, more homogeneous incomplete blocks may be necessary to provide improved precision of estimation, especially for large field crop trials with many treatments.

Sometimes it can be assumed that two or more mutually orthogonal sets of crossed blocks will give improved control of nuisance effects. For example, in field trials it is sometimes assumed that a row-and column blocks design will provide for the control of mutually orthogonal trend effects running along rows and running along columns respectively. However, this is a very strong assumption and may well be unrealistic especially in large field trials where patterns of variability may not fit a simple additive row-and-column model.

Prior to the development of modern statistical software, the statistical analysis of experiments was a major constraint on the complexity of a design. Nowadays, modern software such as the R packages ‘lme4’ (Bates et al 2015) and lmerTest’ (Kuznetsova et al 2017) will allow efficient analysis of virtually any experimental design no matter how complex. Block designs have been extensively researched but often the emphasis has been to fit a relatively simple design for a pre-conceived blocks model. The modern paradigm for empirical model building is to fit model terms in order of importance until no further improvement in fit is possible judged against a suitable measure of model fit such as the Akaike information criterion (AIC) or the Bayesian information criterion (BIC). Empirical model building is commonly applied to treatment effects but there is no reason why the same approach should not be used for block effects. The aim of the ‘blocksdesign’ package is to provide a design of sufficient generality to ensure that the unknown ‘true’ block structure of an experiment to be effectively modelled at the analysis stage by empirical model fitting methods.

Block sizes

A single set of complete replicate blocks will almost always improve the efficiency of an experiment but, for medium or large size experiments, further sub-division into smaller nested sub-blocks is often beneficial and designs with a single level of nesting are widely used in agricultural research. An important study on the effects of block size on precision of estimation of treatments effects in large field trials was undertaken by Patterson and Hunter (1983). They reviewed 244 cereal variety trials arranged in generalized incomplete block designs and found that the mean improvement in yield efficiency due to the incomplete blocks was 1.43 relative to complete randomized blocks. Results varied greatly from one trial to another, however, and the distribution of efficiencies was positively skewed across the set of trials. Precision was more than doubled in one tenth of the trials and in another tenth the lattice arrangement had little effect on accuracy. Nevertheless, in half the trials the efficiency of blocking was 1.23 or more. They also modelled the background field variability of 166 of the trials assuming an empirical exponential variance function.

Letting 2*phi(x) represent the estimated variance of the difference between two plots separated by a distance x, they calculated the semi-variance \(phi(x)\) that best fitted the observed variances of the 166 trials to be:

\[phi(x) = 0.209 (1 - 0.7250*942^{x})\]

They then considered example designs for 48 varieties in three replicate blocks and estimated the theoretical variances of incomplete blocks of sizes 4, 6 or 8 based on the assumed semi-variance phi(x). They used these estimates to estimate the overall efficiency of the different block sizes including inter-block information and found the estimated efficiencies of block sizes 4, 6 and 8 were 1.460, 1.523 and 1.500 respectively. They concluded that for trials with up to about 64 treatments, the nested block size should be approximately equal to the square root of the number of treatments.

Multi-level nesting

The Patterson & Hunter (1983) study compared different block sizes assuming block designs with a single level of nesting and a single nested block size. However, with multi-level nesting it is feasible to have a range of different block sizes in the same design so there is no reason why a design should be restricted to a single size of nested block. In particular, with a nested multi-stratum block design it is possible to include blocks of both size 8 and size 4 in the same design thus spanning the range of block sizes considered by Patterson & Hunter (1983). Modern methods of analysis allow for routine analysis of designs with multi-level nesting and we will examine a single design with three levels of nesting and three block sizes in the same design.

Assume a design for 3 replicates of 48 treatments with three complete replicate blocks of size 48, 18 nested sub-blocks of size eight, 36 nested sub-sub-blocks of size four and 72 nested sub-sub-sub-blocks of size two. The first example in the ‘blocksdesign::design’ function shows how the ‘blocksdesign’ package (Edmondson 2017) can be used to fit a block design of this size and the following table shows the efficiency factors of the design (extracted by using the \(\$blocks\_model\) extractor):

Table 1. Efficiency factors for nested blocks model

Additive model	DF	Efficiency
reps	2	1
reps+sub1	17	0.8769
reps+sub1+sub2	35	0.7275
reps+sub1+sub2+sub3	71	0.4113

The D-efficiencies of the four block sizes from the top stratum downwards were estimated to be 1, 0.877, 0.7275 and 0.4113. By comparison, the D-efficiencies of the same four block sizes from separate designs with a single nested level (not shown) were estimated to be: 1, 0.877, 0.7281 and 0.4118. Thus the efficiencies of the multi-level nested design were very slightly reduced for the third and fourth levels relative to a single-level nested block design of comparable block size. However, these reductions were very small and of no practical importance for real experimental designs. In our experience, the effect of repeated nesting on the efficiency of nested block designs is generally negligible although we have not made any formal study of this effect.

NB the \(\$blocks\_model\) extractor provides efficiency factors for both an ‘Additive_model’ and a ‘Multiplicative_model’. The ‘Multiplicative_model’ efficiency factors are intended to provide efficiency factors for individual blocks in designs with crossed block factors: for fully nested block designs, there should be no difference between the two models.

Multi-level nesting: simulations

The efficiencies of the multi-stratum example design discussed above were examined numerically by using 1000 simulated data sets based on the Patterson and Hunter (1983) \(phi(x)\) function. The following four mixed models were fitted to each simulated data set using the ‘lmer’ function of the ‘lme4’ package (Bates et al. 2014):

1. A single-level nested blocks design with 3 replicate blocks of size 48 and 18 sub-blocks of size 8 nested within replicate blocks.
2. A single-level nested blocks design with 3 replicate blocks of size 48 and 36 blocks of size 4 nested within replicate blocks.
3. A multi-level nested blocks design with 3 replicate blocks of size 48, 18 sub-blocks of size eight and 36 sub-sub-blocks of size four.
4. A multi-level nested blocks design with 3 replicate blocks of size 48, 18 sub-blocks of size eight, 36 sub-sub-blocks of size four and 72 sub-sub-sub-blocks of size 2.

All four models were fitted by maximum likelihood and for each simulation the Chi-square difference in fit between models 2 and 3, between models 1 and 3 and between models 3 and 4 was calculated. Maximum likelihood models with nested random effects and the same fixed effects can be compared by a Chi-square deviance test and Fig 1 shows the Chi-square deviances for model 2 versus model 3, Fig 2 shows the Chi-square deviances for model 1 versus model 3 and Fig 3 shows the Chi-square deviances for model 4 versus model 3. In ‘lme4’, variance components are constrained to be positive so the model is non-standard and the probability levels of the deviances do not follow the usual Chi-square distribution with 1 d.f (Stram & Lee 1994). Instead, Dr. Hans-Peter Piepho, (University of Hohenheim; personal communication) recommends that significance tests should be implemented by halving the p-value of the standard Chi-square test on 1 d.f. Hence, the critical 5% test statistic for the deviance will be the 10% Chi-square critical value which is 2.706 on 1 d.f with a similar derivation for other critical p-values.

Fig 1 shows the effects of adding large blocks of size 8 after fitting the medium blocks of size 4 and shows that for almost 40% of the simulations the large blocks gave a large improvement in model fit (Chi-square>10) over-and-above the effects of the medium blocks.

Fig 2 shows the effects of adding medium blocks of size 4 after fitting the large blocks of size 8 and shows that for about 25% of the simulations the medium blocks gave a large improvement in model fit (Chi-square>10) over-and-above the effects of the large blocks.

Fig 3 shows the effects of adding small blocks of size 2 after fitting both large blocks of size 8 and medium blocks of size 4. In most of the simulations, the extra improvement in model fit due to the small blocks was small (Chi-square<5) but for a few simulations there was a substantial improvement in model fit (Chi-square>10).

In summary, these results show that for large field experiments with spatial semi-variance function given by \(phi(x)\), multi-level nesting can provide a substantial improvement in model fit compared with a single level of nesting. Multi-level nesting can control variability over a range of scales of measurement by providing a range of block sizes that are nested within the same design. Multi-level nesting automatically spans a range of block sizes therefore the issue of finding a single ‘optimum’ block size is redundant.

Crossed blocks

When an experiment has two or more independent sources of background variability, such as the rows and columns of a field crop trial, it can be advantageous to accommodate both sources of variability simultaneously using a crossed blocks design. The simplest crossed blocks design is a Latin square which has v replicates of v treatments arranged in v rows and v columns with a complete treatment replicate in each row and a complete treatment replicate in each column. There are v*v treatment plots and v*v row-by-column intersections therefore each row-by-column intersection contains a single plot. Latin squares have proven useful in a wide range of disciplines but, because Latin squares require as many replications as there are treatments, they quickly become impracticable as the number of treatments increases.

Latin squares can be generalized either by relaxing the constraint that each row and each column contain a complete set of treatments or by relaxing the constraint that each row and each column intersect in a single plot. The first generalization allows designs where each row or each column is an incomplete block while the second generalization allows designs where each row-by-column intersection is an incomplete block. Examples of the first type are incomplete Latin squares (discussed in Cochran and Cox 1957) while examples of the second type are semi-Latin squares (discussed by Preece & Freeman 1983). An example of a row-and-column design that includes both types of generalization simultaneously is the incomplete Trojan square (Edmondson 1998).

A full analysis of a generalized crossed blocks design requires a full mixed model type analysis with a full factorial analysis of the crossed block effects. However, for designs with a single plot in each crossed blocks intersection, the crossed blocks interaction effect is confounded with the treatments effects and for these designs only the crossed blocks main effects are estimable. For designs with multiple plots in each row-by-column intersection, it may be possible to separate the crossed block interaction effects from the treatment effects and for these designs the crossed block factor interactions can be estimated and fitted in the usual way.

Crossed blocks: a literature example

Durban et. al. (2003) examined data from an experiment with two replicates of 272 spring barley varieties arranged in an array of 34 beds (east-west) and 16 rows (north-south). The experiment was designed as a row-and-column design with 16 rows and 34 columns subject to the constraint that rows 1-8 contained one complete set of treatment replicates whilst rows 9-16 contained the other set. Thus the original analysis of the block design had two fixed replicate block effects with 16 random row and 34 random column effects.

The assumptions underlying a standard analysis of a row-and-column design are that row and column effects are additive and that, after allowing for treatment and additive row and column effects, the residual plot variances will be homogeneous with a flat correlation structure. Durban et. al. (2003) showed that even after fitting a full mixed model for rows, columns and replicate block effects, the genotype adjusted plot residuals for this data set were far from homogeneous. This shows that a simple additive row-and-column analysis was far from adequate for this experiment.

Durban et. al. (2003) fitted a semi-parametric loess smoothing model (Hastie 2017) to the genotype adjusted plot residuals of the original row-and-column block analysis. In this vignette, we will use ‘post-blocking’ to show that a block model based on a generalized row-and-column design with a range of block sizes and block structures can explain virtually all the positional variation in this data set.

Crossed blocks: post-blocking

The original row-and-column design had 16 beds and 34 columns which gave a design with very long rows (beds). Simple additivity over a large number of plots in a field trial is unrealistic therefore we examine a post-blocked design with a hierarchy of nested column block sizes that allow for the estimation of row-by-column block interaction effects. A maximal partition of the column blocks gives a hierarchy of two, four, eight, 16 and 34 nested column blocks. However, a partition with only two nested column blocks is not useful because it is not realistic to estimate a blocks variance for only two blocks. Furthermore, a partition with 16 column blocks was found to be not useful because the mixed model failed to converge with 16 random column blocks. Therefore, for the purposes of this study, we will consider a hierarchy of 4, 8 and 34 column blocks only. Four blocks is rather few for estimating a random block effects model but as the overall model gave satisfactory convergence a partition including only four column blocks seems justified. As 4 and 8 are not exact divisors of 34, the column block sizes cannot be exactly equal and we use a set of four large column blocks, Col1, of widths 9,8,8,9, a set of eight medium column blocks, Col2, of widths 5,4,4,4,4,4,4,5 and a set of 34 column blocks, Col3, of width one. The two complete replicate blocks were fitted as fixed effects.

Crossed blocks: choice of model

Model selection proceeded by fitting a sequence of block models in some assumed order and by retaining only those effects that explained a significant amount of variability. There is no obvious assumed order for fitting for the three sets of column blocks Col1, Col2, Col3 but it can be assumed that if a factorial interaction effect is included in any particular model then the main effects marginal to that interaction effect must also be included (Nelder 1977). Using this constraint, Table 2 tests the fit of the various nested column block effects by comparing model ‘goodness of fit’ statistics for various models obtained by dropping column block terms from a fully saturated model (line 1 of the table). These models were fitted by the lme4 package using maximum likelihood and the change in goodness of fit was assessed by the change in the AIC (or BIC) statistic relative to the full model where the smaller the AIC (or BIC) the better the fit of the model.

Table 2. Goodness of fit of column block model terms

Blocks Model	AIC	BIC	logL	dev	df
Reps + Rows + Col1 + Rows:Col1 + Col2 + Rows:Col2 + Col3	280.9	1484	139.6	-279.1	264
Reps + Rows + Col1 + Col2 + Rows:Col2 + Col3	285.8	1485	136.1	-272.2	265
Reps + Rows + Col2 + Rows:Col2 + Col3	284.5	1479	135.7	-271.4	266
Reps + Rows + Col1 + Rows:Col1 + Col2 + Col3	302.6	1502	127.7	-255.4	265
Reps + Rows + Col1 + Rows:Col1 + Col3	305.9	1501	125	-250	266
Reps + Rows + Col1 + Rows:Col1 + Col2 + Rows:Col2	361.2	1560	98.4	-196.8	265

Line 2 of the table shows the effect of omitting the Rows:Col1 interaction effect from the fully saturated model while line 3 shows the effect of omitting both the Rows:Col1 interaction effect and the Col1 main effect simultaneously. There was little change in the AIC or the BIC statistic in either case which shows that the Col1 block factor had little effect on the overall fit of the design and can be omitted completely without any significant loss of precision of estimation.

Line 4 shows the effect of omitting the Rows:Col2 interaction effect from the fully saturated model while Line 5 shows the effect of omitting both the Rows:Col2 interaction effect and the Col2 main effect simultaneously. There was a substantial increase in the AIC and the BIC statistic in both cases which shows that both the Cols2 main effects and the Rows:Col2 block interaction effects are important for a good model fit.

Line 6 shows the effects of omitting Col3 from the fully saturated model and the large increase in both the AIC and the BIC statistics shows that the Col3 main effects are important for model goodness of fit even after including all other column block terms. There is no estimate of the Rows:Cols3 interaction effects because these effects are fully confounded with the treatment effects.

In summary, the best fitting model is shown on line 3 of Table 2 and includes the main effects of Col2 and Col3 and the Rows:Col2 interaction but omits all effects due to Col1.

Crossed blocks: graphical analysis

Durban et al. (2003) examined trends within rows by using coplots (Cleveland 1993) of genotype yield residuals plotted against bed position for each row individually. Fig 4 shows coplots of the REML residuals from the original row-and-column blocks model and Fig 5 shows coplots of the REML residuals from the best fitting model from Table 2 (line 3).

Comparison of the residual plots of the original row-and-column analysis (Fig 4) with the residual plots of the best multi-level crossed-blocks analysis (Fig 5) shows that the post-blocked design gave a much improved model fit. Fig 4 shows strong evidence of residual within-row trend effects whereas Fig 5 shows very little evidence of any residual within-row trend effects.

The residual error variance of the original row-and-column analysis (Fig 4) was 0.0724 whereas the residual error variance of the best multi-level crossed-blocks analysis (Fig 5) was 0.0510. This shows a very substantial improvement of 1.42 in the precision of estimation of treatment effects due to the additional block constraints of the multi-level crossed-blocks analysis.

Generalized crossed blocks designs

The post-blocked analysis of the spring barley variety trial shows that, for large field trials, a multi-level block analysis can substantially improve the precision of estimation of treatment effects. As the assumption that row and columns blocks are additive may not be fully valid, it is usually desirable, where possible, to construct designs that are efficient both for crossed block additive effects and for crossed-block interaction effects. However, except for certain special designs such as Trojan designs, it is usually not possible to optimize both the additive effects and the interaction effects of crossed block factors simultaneously. The ‘blocksdesign’ package adopts a compromise approach by optimizing a weighted combination of the additive block effects information matrix and the multiplicative block effects information matrix. The weighting factor w can vary between 0 and 1 inclusively where a weight of zero give a fully additive crossed blocks model, a weight of 1 gives a fully multiplicative crossed blocks model and an intermediate weighting gives a weighted combination of the two models: \[ Information(combined) = (1-w)*Information(additive) + w*Information(multiplicative) \] The default weighting is 0.5.

The following tables show additive and multiplicative blocks efficiency factors for hierarchically nested column block designs for the spring barley variety trial assuming weights of 0, 0.5 or 1.

Efficiency factors for additive effects of crossed blocks model

Additive model	DF	weight=0	weight=.5	weight=1
Reps	1	1	1	1
Reps+Rows	15	0.9648	0.9648	0.9648
Reps+Rows+Col1	18	0.9605	0.9603	0.9573
Reps+Rows+Col1+Col2	22	0.9507	0.9506	0.9476
Reps+Rows+Col1+Col2+Col3	48	0.8875	0.8872	0.887

Efficiency factors for multiplicative effects of crossed blocks model

Multiplicative model	DF	weight=0	weight=.5	weight=1
Reps	1	1	1	1
Reps.Rows	15	0.9648	0.9648	0.9648
Reps.Rows.Col1	63	0.841	0.8441	0.8442
Reps.Rows.Col1.Col2	127	0.6749	0.6784	0.6785
Reps.Rows.Col1.Col2.Col3	543	NA	NA	NA

The efficiency factors for the compromise design should show additive efficiency factors close to those found for the fully additive design and multiplicative efficiency factors close to those found for the fully multiplicative design. Comparison of the sets of efficiency factors for the three designs does, indeed, show that the compromise design is almost as efficient as the fully additive design for additive block effects and almost as efficient as the fully multiplicative design for multiplicative block effects. We can therefore conclude that a weighted design with a suitable weighting factor will provide a design that is robust for different assumptions about the fitted crossed blocks model at the analysis stage.

Code for the above design is included in the examples code for the \(blocksdesign::design\) function. Code for the efficiency factors of the post-blocked design based on the actual design used in the experiment is also included (not shown here).

Discussion

The advantages of simplicity, flexibility and robustness mean that block designs are the designs of choice for most agricultural experiments. Wherever possible, replicated experiments should be divided into complete replicate blocks and the complete replicate blocks should be used to account for sources of non-treatment differences between the experimental units. For example, with large experiments, cultural operations such as planting or harvesting might occur over several days and it is often useful to use the main blocks as management units so that all cultural operations on the same replicate block are carried out on the same day (Bailey 2008 Chapter 4). Sometimes the physical constraints of an experiment mean that the natural block size does not coincide with the size of a replicate block in which case the main blocks cannot be complete treatment replicates. In that situation, the best option is to allocate treatments to main blocks in such a way that the efficiency of the main block design is maximized (see Bailey 2008 Chapter 11).

The Patterson & Hunter (1983) paper examined the effects of different block sizes on the efficiency of resolvable nested block designs assuming an empirical exponential variance function for the correlations between plots within replicate blocks. They assumed a design with 3 replicates of 48 treatments and examined the efficiency of nested blocks of size 8, 6 or 4, using different resolvable nested block design for each block size. In this vignette, the Patterson & Hunter (1983) example has been generalized by simulating the efficiencies of recursively nested blocks of size 8 and 4 in the same hierarchically nested block design and it has been shown that including two or more nested block sizes in the same design can give substantial improvement in precision relative to a design including either block size separately. The range of block sizes could be further extended by nesting an additional set of blocks to give blocks of size 2 in the bottom stratum of the design but that option will not be further explored here.

The trials assessed in the Patterson & Hunter (1983) paper were cereal trials mostly with long thin plots arranged side by side. This arrangement is convenient for one-dimensional blocking and most of the trials in the Patterson and Hunter study had one-dimensional blocks. Plots that are more nearly square are less easily blocked in one-dimension and may require two-dimensional blocks. Often, simple nested blocks are quite adequate for two-dimensional blocking provided that the blocks are as compact as possible in both dimensions.

The Durban et al. (2003) paper re-examined the analysis of an experiment for two replicates of 272 spring barley genotypes with 16 rows crossed with 34 beds and found that the original row-and-column type analysis was inadequate. Even after removing the additive effects of the rows and the beds, significant trends between beds within rows remained. As an alternative to the original analysis, they showed that a semi-parametric spatial model gave a significant reduction in the residual variability and improved the estimation of treatment effects. In our study, we assumed a post-blocked design and substitute the original single set of 34 column blocks by a hierarchy of nested column blocks with up to three levels of nesting.

Fig 4 shows coplots of the residuals from the original rows-and-beds design while Fig 5 show coplots of the residuals from the best hierarchically nested blocks design with two levels of nesting. Fig 4 shows large trends in the individual rows whereas Figs 5 show that rows are effectively trend free. The main reason that the additive row-and-column analysis was so unsuccessful is because, as shown in the maximum likelihood analysis of the various possible fitted models, there was a large and significant interaction between column blocks and row blocks that was not properly accommodated in the original row-and-column analysis. After fitting a suitable columns-by-rows interaction term, the resulting multi-level analysis gave an excellent fit to the observed data.

Although not discussed in this vignette, we also examined the fit of the spring barley data using generalized additive models (Wood 2017) from the gamm4 package (Wood & Scheipl 2017). In general, we found that the fit of the GAM models was less good than the fit of the block models although further development of the GAM analysis might improve the fit of the GAM model. We note that the post-blocked model fitted to the barley trial data was not optimized for the assumed block design therefore we would expect that the model fit of the hierarchical block design would be improved still further if the design was properly optimized as discussed in the examples for the \(blocksdesign::design\) function. We would also expect that a design optimized for a multi-level nested blocks analysis would also be efficient for a GAM model although this conjecture needs further confirmation.

The outstanding problem in the design and analysis of hierarchical multi-level block models is the issue of model selection. Muller et al. (2013) give an excellent review of model selection for linear mixed models at the analysis stage but the problem of model selection at the design stage involves different issues. Selection of a suitable blocks model at the design stage is critical for enabling the selection of a suitable blocks model at the analysis stage. The risks of selection bias and model over-fitting are well known and further work on the right balance between efficiency of estimation and the risk of over-fitting of nested block designs would be valuable.

Acknowledgement: I would like to thank Prof. Dr. Hans-Peter Piepho of the University of Hohenheim for much useful advice and encouragement to publish this paper.

REFERENCES

ATKINSON, A.C, DONEV, A.N. & TOBIAS, R. D. (2007). Optimum Experimental Designs, with SAS. Oxford, Oxford University Press

BAILEY, R.A. (2008). Design of Comparative Experiments (Cambridge Series in Statistical and Probabilistic Mathematics), CUP

BATES D., MAECHLER M., BOLKER B., WALKER S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1-48. :10.18637/jss.v067.i01.

BOLKER, B (and others) (2018), GLMM FAQ http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#should-i-treat-factor-xxx-as-fixed-or-random 19:51 08 June 2018

COCHRAN, W. G. & COX, G. M. (1957). Experimental designs 2nd edition New York: Wiley.

COOK, R. D. & NACHTSHEIM, C. J. (1980). A comparison of algorithms for constructing exact D-optimal designs, Technometrics, 22, 315-324.

DURBAN, M., HACKETT, C., MCNICOL, J., NEWTON, A., THOMAS, W., & CURRIE, I. (2003). The practical use of semi-parametric models in field trials, Journal of Agric. Biological and Envir. Stats., 8, 48-66.

EDMONDSON, R. N. (1998). Trojan square and incomplete Trojan square designs for crop research Journal of Agricultural Science, Cambridge, 131, 135–142.

EDMONDSON. R. N. (2018). blocksdesign: Nested and Crossed Block Designs for Factorial, Fractional Factorial and Unstructured Treatment Sets. R package version 2.8.

KUZNETSOVA A, BROCHOFF PB & CHRISTENSEN RHB (2017). “lmerTest Package: Tests in Linear Mixed Effects Models.” Journal of Statistical Software, 82(13), pp. 1-26. :10.18637/jss.v082.i13 (URL: http://doi.org/10.18637/jss.v082.i13).

MULLER, S., SCEALY, J. L. & WELSH, A. H. (2013). Model Selection in Linear Mixed Models, Statistical Science, 28, 135–167

NELDER, J. A. (1977). “A Reformulation of Linear Models”. Journal of the Royal Statistical Society. 140 (1): 48–77. :10.2307/2344517. (Section 2.1: The Neglect of Marginality)

PATTERSON H.D. & HUNTER, E.A. (1983). The efficiency of incomplete block designs in National List and Recommended List cereal variety trials. J. Agric. Sci. Camb.,101, 427-433.

PREECE, D. A and FREEMAN, G. H (1983). Semi-Latin Squares and Related Designs. Journal of the Royal Statistical Society. Series B (Methodological). Vol. 45, 267-277

PIEPHO, H. P., BUCHSE, A. & EMRICH, K. (2003). A Hitchhiker’s Guide to Mixed Models for Randomized Experiments. Journal of Agronomy and Crop Science, 189, 310-322.

R Core Team (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

WOOD, S. N. (2017). Generalized Additive Models: An Introduction with R, 2nd Edition, Chapman & Hall/CRC Texts in Statistical Science.

WOOD, S. N. & SCHEIPL, F. (2017). gamm4: Generalized Additive Mixed Models using ‘mgcv’ and ‘lme4’. R package version 0.2-5.https://CRAN.R-project.org/package=gamm4