In an experimental design, we distinguish between random and fixed factors. The “levels” of the random factors are quasi-random samples from a population of persons (subjects) or material (items). To avoid confusion with levels of fixed factors we will refer to levels of random factors as instances. For fixed factors, usual (quasi-)experimental ones, we must specify whether they are between- or within-subjects and between- or within-items. For a given fixed factor all four combinations are possible in principle. We also need to decide on a counterbalancing scheme; a common example is a Latin square applied to all or a subset of the factors.
In this vignette, we illustrate how to set up an experiment using subject (Subj) and item (Item) as random factors. In this fictive experiment, the words of a text are presented serially one at a time at a slow, medium, or high rate (i.e., fixed factor Speed with three levels) at the center of the screen. A second factor is cognitive load varying whether subjects have keep six digits in memory while reading or not (i.e., fixed factor Load with two levels yes and no).
Typically, in such an experiment, (1) each subject reads different texts (Item) in the 2 x 3 experimental conditions; (2) each subject reads the same number of texts in each condition; (3) across subjects each text (item) is presented equally often in the six experimental conditions. The final example in this vignette implements this design. However, for didactic reasons, we first show how within/between-subject and within/between-item features of the factors are specified without counterbalancing. These designs are preferred if repeated exposure to the same stimuli in an experimental condition does not have any confounding effects on the measure.
In the first version of the experimental design, each subject sees each text in each of 2 x 3 experimental conditions. Thus, the two fixed factors Speed and Load are both within-subject and within-item. A minimum of six subjects and six texts is required for a complete within-subject/within-item design. We summarize the design with the design formula:
Load(2) x Speed(3) x 6 Item x 6 SubjThis is a completely crossed design. Note the difference between specification of levels for fixed and instances for random factors. The product of numbers in the formula informs about the number of observations generated by the design. In this case: 216 observations.
design1 <-
fixed.factor("Speed", levels=c("slow", "medium", "fast")) +
fixed.factor("Load", levels=c("yes", "no")) +
random.factor("Subj", instances=6) +
random.factor("Item", instances=6)
codes1 <- arrange(design.codes(design1), Subj, Item)[c(3, 4, 2, 1)]
codes1## # A tibble: 216 x 4
## Subj Item Load Speed
## <fct> <fct> <fct> <fct>
## 1 Subj1 Item1 yes slow
## 2 Subj1 Item1 yes medium
## 3 Subj1 Item1 yes fast
## 4 Subj1 Item1 no slow
## 5 Subj1 Item1 no medium
## 6 Subj1 Item1 no fast
## 7 Subj1 Item2 yes slow
## 8 Subj1 Item2 yes medium
## 9 Subj1 Item2 yes fast
## 10 Subj1 Item2 no slow
## # … with 206 more rowstail(codes1, 10)## # A tibble: 10 x 4
## Subj Item Load Speed
## <fct> <fct> <fct> <fct>
## 1 Subj6 Item5 yes fast
## 2 Subj6 Item5 no slow
## 3 Subj6 Item5 no medium
## 4 Subj6 Item5 no fast
## 5 Subj6 Item6 yes slow
## 6 Subj6 Item6 yes medium
## 7 Subj6 Item6 yes fast
## 8 Subj6 Item6 no slow
## 9 Subj6 Item6 no medium
## 10 Subj6 Item6 no fast#xtabs( ~ Subj + Item + Load + Speed, codes1)
xtabs(~ Load + Speed, codes1)## Speed
## Load slow medium fast
## yes 36 36 36
## no 36 36 36xtabs(~ Subj + Load + Speed, codes1)## , , Speed = slow
##
## Load
## Subj yes no
## Subj1 6 6
## Subj2 6 6
## Subj3 6 6
## Subj4 6 6
## Subj5 6 6
## Subj6 6 6
##
## , , Speed = medium
##
## Load
## Subj yes no
## Subj1 6 6
## Subj2 6 6
## Subj3 6 6
## Subj4 6 6
## Subj5 6 6
## Subj6 6 6
##
## , , Speed = fast
##
## Load
## Subj yes no
## Subj1 6 6
## Subj2 6 6
## Subj3 6 6
## Subj4 6 6
## Subj5 6 6
## Subj6 6 6xtabs(~ Item + Load + Speed, codes1)## , , Speed = slow
##
## Load
## Item yes no
## Item1 6 6
## Item2 6 6
## Item3 6 6
## Item4 6 6
## Item5 6 6
## Item6 6 6
##
## , , Speed = medium
##
## Load
## Item yes no
## Item1 6 6
## Item2 6 6
## Item3 6 6
## Item4 6 6
## Item5 6 6
## Item6 6 6
##
## , , Speed = fast
##
## Load
## Item yes no
## Item1 6 6
## Item2 6 6
## Item3 6 6
## Item4 6 6
## Item5 6 6
## Item6 6 6The first command generates the list design1. The function design.codes() extracts the generated variable coding as a dataframe in the tibble format. After resorting and rearranging the variables, the code is converted to the long format (i.e, N=216). Obviously, having subjects read each text six times may lead to practice effects that would need to be taken into account by counterbalancing the order in which texts are presented across subjects.
In the second example, we replace the factor Load with a factor Type of text. We assume that Items 1 to 3 are simple texts and items 4 to 6 are complex texts. Subjects read both simple and complex texts; Type of text is a within-subject factor. Each text (item), however is either simple or complex. Thus, Type is a between-item factor in this design.
Such a design is realized by specifying Type with the groups argument in the corresponding random.factor() command. We generate 3 items (instances) within each of the two levels of the factor Type, that is, as in the first example, we will have again six different items.
Design formula:
Type(2) x Speed(3) x 3 Item[Type] x 6 SubjWe read the item-part of this formula: “3 Items nested under levels of Type.” The total number of different instances for the random factor Item is 3 items x 2 levels of Type, that is 6 items. The design generates 108 observations; it is no longer completely crossed.
design2 <- fixed.factor("Speed", levels=c("slow", "medium", "fast")) +
fixed.factor("Type", levels=c("simple", "complex")) +
random.factor("Subj", instances=6) +
random.factor("Item", groups="Type", instances=3)
codes2 <- arrange(design.codes(design2), Subj, Item)[c(3, 4, 1, 2)]
codes2## # A tibble: 108 x 4
## Subj Item Type Speed
## <fct> <fct> <fct> <fct>
## 1 Subj1 Item1 complex slow
## 2 Subj1 Item1 complex medium
## 3 Subj1 Item1 complex fast
## 4 Subj1 Item2 complex slow
## 5 Subj1 Item2 complex medium
## 6 Subj1 Item2 complex fast
## 7 Subj1 Item3 complex slow
## 8 Subj1 Item3 complex medium
## 9 Subj1 Item3 complex fast
## 10 Subj1 Item4 simple slow
## # … with 98 more rowsxtabs(~ Item + Type, codes2)## Type
## Item simple complex
## Item1 0 18
## Item2 0 18
## Item3 0 18
## Item4 18 0
## Item5 18 0
## Item6 18 0xtabs(~ Subj + Type, codes2)## Type
## Subj simple complex
## Subj1 9 9
## Subj2 9 9
## Subj3 9 9
## Subj4 9 9
## Subj5 9 9
## Subj6 9 9#xtabs( ~ Subj + Item + Type + Speed, codes2)
#xtabs(~ Type + Speed, codes2)
#xtabs(~ Subj + Type + Speed, codes2)
#xtabs(~ Item + Type + Speed, codes2)The tables shows that for Items 1 to 3 all available codes for the factor Type are complex and for Items 4 to 6 all codes are simple. Thus, Type is varied between items. Each item is read three times (three levels of Speed) by six subjects. yielding 18 codes in each of the 6 non-zero cells of the Item x Type table.
Conversely, for all six subjects codes are available for simple and complex items. Thus, Type is varied within subjects. Each text is read three times (i.e., the three speed rates). Therefore, there are 3 texts x 3 levels of speed = 9 codes in each cell of the Subj x Type table.
The command to specify Speed as between_item factor would be:
random.factor("Item", groups="Speed", instances=2)
```
We need 2 instances within each of the 3 levels of _Speed_ to obtain 6 items in total.
**Design formula:**
```
Type(2) x Speed(3) x 2 Item[Speed] x 6 SubjThe total number of items is 2 x 3 = 6. This design generates 72 observations.
In this example, we replace the factor Load (or Type) with a between-subject factor Age, assuming that half the subjects are young and the other half old.
design3 <-
fixed.factor("Speed", levels=c("slow", "medium", "fast")) +
fixed.factor("Age", levels=c("young", "old")) +
random.factor("Item", instances=6) +
random.factor("Subj", groups="Age", instances=3)
codes3 <- arrange(design.codes(design3), Subj, Item)[c(4, 3, 2, 1)]
codes3## # A tibble: 108 x 4
## Subj Item Speed Age
## <fct> <fct> <fct> <fct>
## 1 Subj1 Item1 slow old
## 2 Subj1 Item1 medium old
## 3 Subj1 Item1 fast old
## 4 Subj1 Item2 slow old
## 5 Subj1 Item2 medium old
## 6 Subj1 Item2 fast old
## 7 Subj1 Item3 slow old
## 8 Subj1 Item3 medium old
## 9 Subj1 Item3 fast old
## 10 Subj1 Item4 fast old
## # … with 98 more rowsxtabs(~ Subj + Age, codes3)## Age
## Subj young old
## Subj1 0 18
## Subj2 0 18
## Subj3 0 18
## Subj4 18 0
## Subj5 18 0
## Subj6 18 0xtabs(~ Item + Age, codes3)## Age
## Item young old
## Item1 9 9
## Item2 9 9
## Item3 9 9
## Item4 9 9
## Item5 9 9
## Item6 9 9#xtabs( ~ Subj + Item + Age + Speed, codes3)
#xtabs( ~ Subj + Age + Speed, codes3)The tables show that subjects 1 to 3 are old and subjects 4 to 6 are young (i.e., Age is a between-subject factor) and that all items are read by young and old subjects (i.e., Age is a within-item factor). The formula for this design can be written as: Age(2) x Speed(3) x 6 Item x 3 Subj[Age], yielding 108 observations.
Note that instances specifies the number of instances within groups. To generate code for 25 young and 25 old subjects (i.e., total N=50), we set instances=25.
Design formula:
Age(2) x Speed(3) x 6 Item x 25 Subj[Age] The total number of subjects is 25 x 2 = 50. This design generates 900 observations.
Continuing with the last example, it may also make sense to vary not only Age, but als Speed between subjects. Thus, every subject is either old or young (i.e., a quasi-experimental factor) and is randomly assigned to one of the three Speed conditions (i.e., an experimental factor). For this specification the two factors are included as a vector for the groups argument. For the minimal design we need only 1 instance because 2 x 3 = 6. This means we generate codes for 1 subject in each of the six design cells, but each subjects reads each text in this condition (i.e., there are six measures for each subject.) To get code for 10 subjects in each of the 2 x 3 = 6 design cells (i.e., a total of 60 subjects), we set instances=10.
Design formula:
Age(2) x Speed(3) x 6 Item x 10 Subj[Age x Speed]The total number of subjects is 10 x 2 x 3 = 60. This design generates 360 observations.
design4 <-
fixed.factor("Speed", levels=c("slow", "medium", "fast")) +
fixed.factor("Age", levels=c("simple", "complex")) +
random.factor("Subj", groups=c("Age", "Speed"), instances=10) +
random.factor("Item", instances=6)
codes4 <- arrange(design.codes(design4), Subj, Item)[c(3, 4, 2, 1)]
codes4## # A tibble: 360 x 4
## Subj Item Age Speed
## <fct> <fct> <fct> <fct>
## 1 Subj01 Item1 complex fast
## 2 Subj01 Item2 complex fast
## 3 Subj01 Item3 complex fast
## 4 Subj01 Item4 complex fast
## 5 Subj01 Item5 complex fast
## 6 Subj01 Item6 complex fast
## 7 Subj02 Item1 complex fast
## 8 Subj02 Item2 complex fast
## 9 Subj02 Item3 complex fast
## 10 Subj02 Item4 complex fast
## # … with 350 more rowsxtabs( ~ Subj + Age, codes4)## Age
## Subj simple complex
## Subj01 0 6
## Subj02 0 6
## Subj03 0 6
## Subj04 0 6
## Subj05 0 6
## Subj06 0 6
## Subj07 0 6
## Subj08 0 6
## Subj09 0 6
## Subj10 0 6
## Subj11 0 6
## Subj12 0 6
## Subj13 0 6
## Subj14 0 6
## Subj15 0 6
## Subj16 0 6
## Subj17 0 6
## Subj18 0 6
## Subj19 0 6
## Subj20 0 6
## Subj21 0 6
## Subj22 0 6
## Subj23 0 6
## Subj24 0 6
## Subj25 0 6
## Subj26 0 6
## Subj27 0 6
## Subj28 0 6
## Subj29 0 6
## Subj30 0 6
## Subj31 6 0
## Subj32 6 0
## Subj33 6 0
## Subj34 6 0
## Subj35 6 0
## Subj36 6 0
## Subj37 6 0
## Subj38 6 0
## Subj39 6 0
## Subj40 6 0
## Subj41 6 0
## Subj42 6 0
## Subj43 6 0
## Subj44 6 0
## Subj45 6 0
## Subj46 6 0
## Subj47 6 0
## Subj48 6 0
## Subj49 6 0
## Subj50 6 0
## Subj51 6 0
## Subj52 6 0
## Subj53 6 0
## Subj54 6 0
## Subj55 6 0
## Subj56 6 0
## Subj57 6 0
## Subj58 6 0
## Subj59 6 0
## Subj60 6 0xtabs( ~ Subj + Speed, codes4)## Speed
## Subj slow medium fast
## Subj01 0 0 6
## Subj02 0 0 6
## Subj03 0 0 6
## Subj04 0 0 6
## Subj05 0 0 6
## Subj06 0 0 6
## Subj07 0 0 6
## Subj08 0 6 0
## Subj09 0 6 0
## Subj10 0 6 0
## Subj11 0 6 0
## Subj12 0 6 0
## Subj13 0 6 0
## Subj14 0 0 6
## Subj15 0 0 6
## Subj16 0 0 6
## Subj17 0 6 0
## Subj18 0 6 0
## Subj19 0 6 0
## Subj20 0 6 0
## Subj21 6 0 0
## Subj22 6 0 0
## Subj23 6 0 0
## Subj24 6 0 0
## Subj25 6 0 0
## Subj26 6 0 0
## Subj27 6 0 0
## Subj28 6 0 0
## Subj29 6 0 0
## Subj30 6 0 0
## Subj31 0 0 6
## Subj32 0 0 6
## Subj33 0 0 6
## Subj34 0 0 6
## Subj35 0 0 6
## Subj36 0 0 6
## Subj37 0 0 6
## Subj38 0 0 6
## Subj39 0 0 6
## Subj40 0 0 6
## Subj41 0 6 0
## Subj42 0 6 0
## Subj43 0 6 0
## Subj44 0 6 0
## Subj45 6 0 0
## Subj46 6 0 0
## Subj47 6 0 0
## Subj48 0 6 0
## Subj49 0 6 0
## Subj50 0 6 0
## Subj51 0 6 0
## Subj52 0 6 0
## Subj53 0 6 0
## Subj54 6 0 0
## Subj55 6 0 0
## Subj56 6 0 0
## Subj57 6 0 0
## Subj58 6 0 0
## Subj59 6 0 0
## Subj60 6 0 0xtabs( ~ Item + Age, codes4)## Age
## Item simple complex
## Item1 30 30
## Item2 30 30
## Item3 30 30
## Item4 30 30
## Item5 30 30
## Item6 30 30xtabs( ~ Item + Speed, codes4)## Speed
## Item slow medium fast
## Item1 20 20 20
## Item2 20 20 20
## Item3 20 20 20
## Item4 20 20 20
## Item5 20 20 20
## Item6 20 20 20#xtabs( ~ Subj + Item + Age + Speed, codes4)The tables show that Age and Speed vary indeed between subjects and within items.
In this final example, we modify the very first example such that each subject reads one different texts in each of the six conditions, respecting the constraint that design cells are counterbalanced (i.e., each text is read equally often in each condition, each subject reads the same number of texts in each condition).
For this implementation we (1) add a third random factor defined as Subj-by-Item and (2) specify factors Speed and Load as varying between Subj-by-Item.
We start with the minimal design of 6 subjects reading 6 texts.
Design formula:
Speed(3) x Load(2) x 1 Item[Speed x Load] x 1 Subj[Speed x Load] x
(3 x 2) Item-by-Subj[Speed x Load x Item[Speed x Load] + Subj[Speed x Load]] We have 1 item and 1 subject nested under the levels of the Speed x Load design. There are 36 instances of the random factor resulting from the multiplication of the random factors Item and Subj. The design generates 3 x 2 x 1 x 1 x (3 x 2) 36 observations.
design5 <-
fixed.factor("Speed", levels=c("slow", "medium", "fast")) +
fixed.factor("Load", levels=c("simple", "complex")) +
random.factor("Subj", instances=1) +
random.factor("Item", instances=1) +
random.factor(c("Subj", "Item"), groups=c("Speed", "Load"))
codes5 <- arrange(design.codes(design5), Subj, Item)[c(3, 4, 1, 2)]
codes5## # A tibble: 36 x 4
## Subj Item Speed Load
## <fct> <fct> <fct> <fct>
## 1 Subj1 Item1 slow simple
## 2 Subj1 Item2 slow complex
## 3 Subj1 Item3 medium simple
## 4 Subj1 Item4 medium complex
## 5 Subj1 Item5 fast simple
## 6 Subj1 Item6 fast complex
## 7 Subj2 Item1 slow complex
## 8 Subj2 Item2 medium simple
## 9 Subj2 Item3 medium complex
## 10 Subj2 Item4 fast simple
## # … with 26 more rowsxtabs(~ Subj + Speed + Load, codes5)## , , Load = simple
##
## Speed
## Subj slow medium fast
## Subj1 1 1 1
## Subj2 1 1 1
## Subj3 1 1 1
## Subj4 1 1 1
## Subj5 1 1 1
## Subj6 1 1 1
##
## , , Load = complex
##
## Speed
## Subj slow medium fast
## Subj1 1 1 1
## Subj2 1 1 1
## Subj3 1 1 1
## Subj4 1 1 1
## Subj5 1 1 1
## Subj6 1 1 1xtabs(~ Item + Speed + Load, codes5)## , , Load = simple
##
## Speed
## Item slow medium fast
## Item1 1 1 1
## Item2 1 1 1
## Item3 1 1 1
## Item4 1 1 1
## Item5 1 1 1
## Item6 1 1 1
##
## , , Load = complex
##
## Speed
## Item slow medium fast
## Item1 1 1 1
## Item2 1 1 1
## Item3 1 1 1
## Item4 1 1 1
## Item5 1 1 1
## Item6 1 1 1xtabs( ~ Subj + Item + Load + Speed, codes1)## , , Load = yes, Speed = slow
##
## Item
## Subj Item1 Item2 Item3 Item4 Item5 Item6
## Subj1 1 1 1 1 1 1
## Subj2 1 1 1 1 1 1
## Subj3 1 1 1 1 1 1
## Subj4 1 1 1 1 1 1
## Subj5 1 1 1 1 1 1
## Subj6 1 1 1 1 1 1
##
## , , Load = no, Speed = slow
##
## Item
## Subj Item1 Item2 Item3 Item4 Item5 Item6
## Subj1 1 1 1 1 1 1
## Subj2 1 1 1 1 1 1
## Subj3 1 1 1 1 1 1
## Subj4 1 1 1 1 1 1
## Subj5 1 1 1 1 1 1
## Subj6 1 1 1 1 1 1
##
## , , Load = yes, Speed = medium
##
## Item
## Subj Item1 Item2 Item3 Item4 Item5 Item6
## Subj1 1 1 1 1 1 1
## Subj2 1 1 1 1 1 1
## Subj3 1 1 1 1 1 1
## Subj4 1 1 1 1 1 1
## Subj5 1 1 1 1 1 1
## Subj6 1 1 1 1 1 1
##
## , , Load = no, Speed = medium
##
## Item
## Subj Item1 Item2 Item3 Item4 Item5 Item6
## Subj1 1 1 1 1 1 1
## Subj2 1 1 1 1 1 1
## Subj3 1 1 1 1 1 1
## Subj4 1 1 1 1 1 1
## Subj5 1 1 1 1 1 1
## Subj6 1 1 1 1 1 1
##
## , , Load = yes, Speed = fast
##
## Item
## Subj Item1 Item2 Item3 Item4 Item5 Item6
## Subj1 1 1 1 1 1 1
## Subj2 1 1 1 1 1 1
## Subj3 1 1 1 1 1 1
## Subj4 1 1 1 1 1 1
## Subj5 1 1 1 1 1 1
## Subj6 1 1 1 1 1 1
##
## , , Load = no, Speed = fast
##
## Item
## Subj Item1 Item2 Item3 Item4 Item5 Item6
## Subj1 1 1 1 1 1 1
## Subj2 1 1 1 1 1 1
## Subj3 1 1 1 1 1 1
## Subj4 1 1 1 1 1 1
## Subj5 1 1 1 1 1 1
## Subj6 1 1 1 1 1 1Number of subjects and items increase by six with each increment of the value of the instances argument. For example,
...
random.factor("Subj", instances=10) +
random.factor("Item", instances= 4) +
...will generate codes for 60 subjects and 24 texts.
design6 <-
fixed.factor("Speed", levels=c("slow", "medium", "fast")) +
fixed.factor("Load", levels=c("simple", "complex")) +
random.factor("Subj", instances=10) +
random.factor("Item", instances=4) +
random.factor(c("Subj", "Item"), groups=c("Speed", "Load"))
codes6 <- arrange(design.codes(design6), Subj, Item)[c(3, 4, 1, 2)]
codes6## # A tibble: 1,440 x 4
## Subj Item Speed Load
## <fct> <fct> <fct> <fct>
## 1 Subj01 Item01 slow simple
## 2 Subj01 Item02 slow complex
## 3 Subj01 Item03 medium simple
## 4 Subj01 Item04 medium complex
## 5 Subj01 Item05 fast simple
## 6 Subj01 Item06 fast complex
## 7 Subj01 Item07 slow simple
## 8 Subj01 Item08 slow complex
## 9 Subj01 Item09 medium simple
## 10 Subj01 Item10 medium complex
## # … with 1,430 more rowslength(unique(codes6$Subj))## [1] 60length(unique(codes6$Item))## [1] 24length(unique(paste(codes6$Subj, codes6$Item)))## [1] 1440Design formula:
Speed(3) x Load(2) x 4 Item[Speed x Load] x 10 Subj[Speed x Load] x
(3 x 2) Item-by-Subj[Speed x Load x Item[Speed x Load] x Subj[Speed x Load]] The total number of items is 4 x 3 x 2 = 24; the total number of subjects is 10 x 3 x 2 = 60. The total number of instances of Item-by-Subj is 3 x 2 x (3 x 2) x 4 x 10 = 1440. The design yields 3 x 2 x 4 x 10 x (3 x 2) = 1440 observations.
The examples illustrate some of the basic functionalities. The generalization to a larger number of fixed or random factors and number of levels associated with them should be clear.
The codes generated with the above specifications can be extended with different assignment of presentation orders according to latin.square (default), random.order, or williams. These options will be described in the second vignette.
The function also allows the specifations of fixed effects, variance and correlation parameters to generate input suitable for linear (mixed) models and the determination of statistical power via simulations from the model. The third vignette is a tutorial about these functionalities.
The development of this package was supported by German Research Foundation (DFG)/SFB 1287 Limits of variability in language and Center for Interdisciplinary Research, Bielefeld (ZiF)/Cooperation Group Statistical models for psychological and linguistic data.
sessionInfo()## R version 3.5.1 (2018-07-02)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: macOS 10.14.6
##
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
##
## locale:
## [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] designr_0.1.8 forcats_0.4.0 stringr_1.4.0 dplyr_0.8.3
## [5] purrr_0.3.2 readr_1.3.1 tidyr_1.0.0 tibble_2.1.3
## [9] ggplot2_3.2.1 tidyverse_1.2.1
##
## loaded via a namespace (and not attached):
## [1] Rcpp_1.0.2 cellranger_1.1.0 pillar_1.4.2 compiler_3.5.1
## [5] tools_3.5.1 zeallot_0.1.0 digest_0.6.21 lubridate_1.7.4
## [9] jsonlite_1.6 evaluate_0.14 lifecycle_0.1.0 nlme_3.1-141
## [13] gtable_0.3.0 lattice_0.20-38 pkgconfig_2.0.3 rlang_0.4.0
## [17] cli_1.1.0 rstudioapi_0.10 yaml_2.2.0 haven_2.1.1
## [21] xfun_0.10 withr_2.1.2 xml2_1.2.2 httr_1.4.1
## [25] knitr_1.25 hms_0.5.1 generics_0.0.2 vctrs_0.2.0
## [29] grid_3.5.1 tidyselect_0.2.5 glue_1.3.1 R6_2.4.0
## [33] fansi_0.4.0 readxl_1.3.1 rmarkdown_1.16 modelr_0.1.5
## [37] magrittr_1.5 backports_1.1.5 scales_1.0.0 htmltools_0.3.6
## [41] rvest_0.3.4 assertthat_0.2.1 colorspace_1.4-1 utf8_1.1.4
## [45] stringi_1.4.3 lazyeval_0.2.2 munsell_0.5.0 broom_0.5.2
## [49] crayon_1.3.4