1. Experimental Designs

   Readings: Chambliss & Dunlap; Winer; Collier, Baker, Mandeville & Hayes; Huynh & Feldt; Gaito (1961).
   Computer Programs: psylib ranper; psylib oneway

   1. Review of Logic Behind ANOVA
      
      1. Relation of the F-test to the Central Limit Theorem
      2. Randomization in Experimental Design
         1. Random Selection from the Population
         2. Random Assignment to Groups
      3. Types of Variables in ANOVA
         1. Experimental variables (causality)
         2. Organismic (quasi-experimental) variables
         3. Causality

   2. Assumptions of ANOVA
      1. Empirical Studies of Robustness
         1. What is an Empirical Study
         2. Non-normality
         3. Heteroscedasticity
      2. Assumptions of Repeated Measures ANOVA
         1. Carryover Effects
         2. Randomization vs. Counter-balancing
         3. Equal Measure to Measure Correlations
            (Uniform Variance-Covariance Matrix)
         4. Consequenses of Violation
         5. Epsilon (Box)
         6. Huynh-Feldt Correction for Epsilon
         7. Geiser-Greenhouse Conservative Test

   3. Models of ANOVA
      1. Decomposition of Factorial ANOVA
      2. Representing Interaction Graphically
      3. Expected Values & Expected Mean Squares
      4. Random vs. Fixed Variables in ANOVA
      5. Error terms for SsxAxB; SsxAxBxC; etc.
         (individual vs. pooled error terms)
      6. Ss/(AxB)xC; Ss/Ax(BxC); Ss/(AxB)x(CxD)
      7. Logic Behind the Clifford Analysis

2. Power

   Readings: Cohen; Lane & Dunlap; Dunlap (1981); Dunlap (1982); Levine & Dunlap (1982).
   Computer Programs: psylib power; psylib powr

   1. Interpretation of Power Curves

   2. Power Tables and Power Programs
      1. Power of F and t
      2. Power for Proportions & Chi-Square
      3. Power with Correlation

   3. Estimating Power for Your Own Designs
      1. Estimating Error
      2. Estimating a Meaningful Effect Size
      3. Deciding Upon Acceptable Power
      4. Power Checklist -- Additional Items
         1. Within vs Between Ss Design
         2. Maximum manipulation of IV
         3. Control of population variance
         4. Concomitant variables
         5. Skew in DV
         6. Unequal n
         7. Outliers

   4. Measures of Effect Size (Omega Squared & Others)
      1. History of Effect Size Measures
      2. Problems with Such Measures

3. Tests Subsequent to ANOVA for Main Effects

   Readings: Petrinovich & Hardyck; Games; Dunlap, Powell & Konnerth; Dunlap, Marx, & Agamy, Dunlap (1975).
   Computer Programs: psylib range, psylib dunn, psylib trend

   1. Review of Psychology 212
      1. Orthogonal Comparisons
      2. Bonferroni Tests
      3. Scheffe' Tests

   2. Trend Analysis
      1. Trend Coefficients
      2. Must have scaled variable
      3. Shapes & Patterns rather than Differences

   3. Range Tests
      1. Tukey's Test
      2. Neuman-Keuls
      3. Duncan's Test
      4. Dunnett's Test
      5. Presenting Range Test Results

4. Tests Subsequent to Significant Interaction
   1. Tests of Simple Effects
      1. Benefits and Drawbacks
      2. Satterthwaite's Correction for df

   2. Range Tests on Cell Means

   3. Orthogonal Decomposition of Interactions

   4. Trend
      1. Trend Coefficients for Interaction
      2. Trend Analysis of Interactions
      3. Tukey's Test for Nonadditivity

5. Unequal Sample Sizes - Problems and Solutions
   1. Independent Groups Designs
      1. Method of Unweighted Means
      2. Least Squares Solution

   2. Repeated Measures Designs
      1. Missing data not permitted
      2. Blocking, discarding subjects, or measures
      3. Replacement of data

6. Controlling Concomittant Variables in Research

   Readings: Evans & Anastasiou; Feldt

   1. Control variables vs. Experimental variables

   2. Stratification
      1. Stratifying with two levels only
      2. Curvilinear relations - at least 3 levels needed
      3. Matched sets of subjects
      4. Tests of interaction

   3. Analysis of Covariance (ANCOVA)
      1. Additional assumptions
      2. Tests of regression slopes
      3. Tests of adjusted means
      4. Control of error vs. correction of pre-experimental differences.

7. Simple Correlation

   Readings: Steiger
   Computer Program: psylib steig

   1. Importance of Scatterplot

   2. Simple Regression

   3. Testing for Significance

   4. Tests for Differences Between Independent r's
      1. Fisher's z'-transformation
      2. The Variance Sum Law

   5. Tests for Non-independent r's
      1. With one variable in common
      2. With no variables in common

8. Multiple Regression

   Readings: Darlington

   1. General Model and Normal Equations
      1. General Model
         1. Y_(pred) = B₁x₁ + B₂x₂ + ... + B_px_p + A
         2. Using z-scores: z_y(pred) = b₁x₁ + ... + b_px_p
      2. Normal Equations
         1. b₁ + b₂r₁₂ + ... + b_pr_1p = r_y1
            b₁r₂₁ + b₂ + ... + b_pr_2p = r_y2
            r_p1 + b₂r_p2 + ... + b_p = r_yp
         2. Solving normal equations to get optimal weights
         3. Solving to get raw score equation
            B_i = b_i * S_y/S_i
            A = M_y - B₁*M_x1 - ... - B_p*M_xp

   2. Multiple Correlation Coefficient
      1. R_{y.x1, x2 ... xp} = SQRT(b₁r_y1 + b₂r_y2 + ... + b_pr_yp )
      2. R_{y.x1, x2 ... xp} = r_{y, y(pred)}

   3. Tests of Significance
      1. Test of Significance of R²
      2. Test of Complete vs Reduced Models

   4. Usefulness of Individual Predictors
      1. Significance of b weight
      2. Complete vs. reduced model test
      3. Loadings (how well does each predictor correlate with y_(pred))

   5. Shrinkage - Bias in R
      1. Adding predictors increases R, if only because of chance relationship
      2. If Rho = 0, R equals approximately p/(N-1)
      3. Correction for bias ("shrinkage")

   6. Suppressor Variables
      1. Variable may have low loading but be significant
      2. Identification of suppression
         1. Make all variables correlate positively with criterion; suppressor variable will have negative beta weight.

   7. Polynomial Regression
      1. Example: fit a parabola
      2. Is x² significant?

   8. Dummy predictors
      1. Coding dichotomous variables
      2. Coding multicategory variables with (k-1) dummy variables

   9. Moderated Multiple Regression (MMR)
      1. Test for interaction with continuous variables
      2. Generation of cross product term
         1. Centering is best for interpretation of interaction and main effects
         2. Not centering is best for plotting
      3. Power is generally a problem
      4. Plotting

   10. Mediator variables in Regression
       1. Does w mediate relation between y and x?
          1. r_yx, r_yw, r_xw all significant
          2. r_yx.w < r_yx
             1. r_yx.w = 0 or ns for complete mediation
             2. r_yx.w significantly less than r_yx for partial mediation

Bresnahan & Shapiro. (1966). A general equation and technique for the exact partitioning of chi square contingency tables. Psychol.Bull., 66, 252-262

Camilli & Hopkins. (1978). Applicability of Chi-square to 2 X 2 contingency tables with small expected cell frequencies. Psychol. Bull., 85, 163-167.

Chambliss & Dunlap, Experimental Design and Analysis

Chen, R., & Dunlap, W. P. (1994). A Monte Carlo study on the performance of a corrected formula for epsilon suggested by Lacoutre. Journal of Educational Statistics, 19, 119-126.

Cochran & Cox. (1957). Experimental Designs (2nd Ed.). NY: Wiley.

Cohen, J. (1977). Statistical Power Analysis for the Behavioral Sciences. NY: Academic Press, (Revised Edition)

Cohen, Jacob, & Cohen, P. (1975). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Hillsdale, NJ: Erlbaum (I believe there is a revised edition).

Collier, Baker, Mandeville & Hayes. (1967). Estimates of test size for several procedures based on conventional variance ratios in the repeated measues design. Psychometrika, 32, 339-353

Conover, W.J. (1968). Uses and abuses of the continuity correction. Biometrics, 24, 1028

Conover, W.J. (1974) Some reasons for not using Yates' correction on the 2x2 contingency table. Journal Amer. Stat. Assoc., 69, 374-382

Cortina, J. M., & Dunlap, W. P. (1997). On the logic and purpose of significance testing. Psychological Methods, 2, 161-172.

Darlington, R.B. (1968). Multiple regression in psychological research and practice. Psychol. Bull., 69, 161-182

Davis, D.J. (1969). Flexibility and power in comparisons among means. Psychol. Bull., 71, 441-454

Dunlap, W. P. (1981). An interactive FORTRAN IV program for calculating power, sample size, or detectable differences in means. Behavior Research Methods & Instrumentation, 13, 757-759.

Dunlap, W. P. (1994). Generalizing the common language effect size indicator to bivariate normal correlations. Psychological Bulletin, 116, 509-511.

Dunlap, W. P. (1999). A program to compute McGraw and Wong's common language effect size indicator. Behavior Research Methods, Instruments, & Computers., 31, 706-709.

Dunlap, W. P., Burke, M. J., & Greer, T. (1995). The effect of skew on the magnitude of product-moment correlations. Journal of General Psychology, 122, 365-377.

Dunlap, W. P., Chen, R., & Greer, T. (1994). Skew reduces test-retest reliability. Journal of Applied Psychology, 79, 310-313.

Dunlap, W. P., Dietz, J., & Cortina, J. M. (1997). The spurious correlation of ratios that have common variables: A Monte Carlo examination of Pearson's formula. Journal of General Psychology, 124, 182-193.

Dunlap, W. P., & Landis, R. S. (1998). Interpretations of multiple regression borrowed from factor analysis and canonical correlation. Journal of General Psychology, 125, 397-407.

Dunlap, W. P., Marx, M. S., & Agamy, G. J. (1981). FORTRAN IV functions for calculating probabilities associated with Dunnett's test. Behavior Research Methods & Instrumentation, 13, 363-366.

Dunlap, W. P., & Myers, L. (1997). Approximating power for significance tests with one degree of freedom. Psychological Methods, 2, 186-191.

Dunlap, W. P., Myers, L., & Silver, N. C. (1984). Exact multinomial probabilities for one-way contingency tables. Behavior Research Methods, Instuments, & Computers, 16, 54-56.

Dunlap, W. P., Powell, R. S., & Konnerth, T. K. (1977). A FORTRAN IV function for calculating probabilities associated with the studentized range statistic. Behavior Research Methods & Instrumentation, 9, 373-375.

Evans & Anastasio. (1968). Misuse of analysis of covariance when effect and covariate are confounded. Psychol. Bull., 69, 225-234

Federer, W.T. (1955). Experimental Design. NY: Macmillan.

Feldt, L. S. (1958). A comparison of the precision of three experimental designs employing a concomitant variable. Psychometrika, 23, 335-353.

Gaito, J. (1961). Repeated measures designs and counter-balancing. Psychol. Bull., 58, 46-54

Gaito, J. (1965). Unequal intervals and unequal n in trend analysis. Psychol. Bull., 63, 125-127

Games, P.A. (1971). Multiple comparisons of means. Amer. Educ. Research Journal, 8, 531-565

Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972). Consequences of failure to meet the assumptions underlying the fixed effects analysis of variance and covariance. Review of Educational Research, 42, 237-288.

Greer, T., & Dunlap, W. P. (1997). Analysis of variance with ipsative measures. Psychological Methods, 2, 200-207.

Huynh, H., & Feldt, L. S. (1976). Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics, 1, 69-82.

Lana & Lubin. (1963). The effect of correlation on the repeated measures designs. Educ. & Psychol. Meas., 23, 729-739

Landis, R. S., & Dunlap, W. P. (2000). Moderated multiple regression tests are criterion specific. Organizational Research Methods, 3, 254-266.

Lane, D. M., & Dunlap, W. P. (1978). Estimating effect size: Bias resulting from the significance criterion in editorial decisions. British Journal of Mathematical and Statistical Psychology, 31, 107-112.

Levine, D. W. & Dunlap, W. P. (1982). Power of the F-test with skewed data: Should one transform or not? Psychological Bulletin, 92, 272-280.

McCarroll, D., Crays, N., & Dunlap, W. P. (1992). Sequential ANOVAs and Type I error rates. Educational and Psychological Measurement, 52, 387-393.

Myers, J. L., & Well, A. D. (1995). Research design and statistical analysis. Hillsdale, NJ: Erlbaum.

Petrinovich & Hardyck. (1969). Error rates for multiple comparison methods: some evidence concerning the frequency of erroneous conclusions. Psychol. Bull., 71, 43-54

Steiger, J. W. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245-251.

Tukey, J.W. (1949). One degree of freedom for non-additivity. Journal Amer. Stat. Assoc., 5, 232-242

Wilkinson, L. (1979). Tests of significance in stepwise regression. Psychological Bulletin, 86, 168-174.

Winer, B.J. (1971). Statistical Principles in Experimental Design. NY: McGraw-Hill, (2nd Edition)

Back to Psychology Home Page