Applied Multivariate Analysis

1 Introduction and Preview.- Overview.- Multivariate Analysis: A Broad Definition.- Multivariate Analysis: A Narrow Definition.- Some Important Themes.- Obtaining Meaningful Relations.- Selecting Cutoffs.- Questions of Statistical Inference.- Outliers.- The Importance of Theory.- Problems Peculiar to the Analysis of Scales.- The Role of Computers in Multivariate Analysis.- Multivariate Analysis and the Personal Computer.- Choosing a Computer Package.- Problems in the Use of Computer Packages.- The Importance of Matrix Procedures.- 2 Some Basic Statistical Concepts.- Overview.- Univariate Data Analysis.- Frequency Distributions.- Normal Distributions.- Standard Normal Distributions.- Parameters and Statistics.- Locational Parameters and Statistics.- Measures of Variability.- A Note on Estimation.- Binary Data and the Binomial Distribution.- Data Transformation.- Bivariate Data Analysis.- Characteristics of Bivariate Relationships.- Bivariate Normality.- Measures of Bivariate Relation.- Range Restriction.- Pearson Correlation Formulas in Special Cases.- Non-Pearson Estimates of Pearson Correlations.- The Eta-Square Measure.- Phi Coefficients with Unequal Probabilities.- Sampling Error of a Correlation.- The Z’ Transformation.- Linear Regression.- The Geometry of Regression.- Raw-Score Formulas for the Slope.- Raw-Score Formulas for the Intercept.- Residuals.- The Standard Error of Estimate.- Why the Term “Regression”?.- A Summary of Some Basic Relations.- Statistical Control: A First Look at Multivariate Relations.- Partial and Part Correlation.- Statistical versus Experimental Control.- Multiple Partialling.- Within-Group, Between-Group, and Total Correlations.- 3 Some Matrix Concepts.- Overview.- Basic Definitions.- Square Matrices.- Transposition.- Matrix Equality.- Basic Matrix Operations.- Matrix Addition and Subtraction.- Matrix Multiplication.- Correlation Matrices and Matrix Multiplication.- Partitioned Matrices and Their Multiplication.- Some Rules and Theorems Involved in Matrix Algebra.- Products of Symmetric Matrices.- More about Vector Products.- Exponentiation.- Determinants.- Matrix Singularity and Linear Dependency.- Matrix Rank.- Matrix “Division”.- The Inverse of a 2 × 2 Matrix.- Inverses of Higher-Order Matrices.- Recalculation of an Inverse Following Deletion of Variable(s).- An Application of Matrix Algebra.- More about Linear Combinations.- The Mean of a Linear Combination.- The Variance of a Linear Combination.- Covariances between Linear Combination.- The Correlation between Two Different Linear Combinations.- Correlations between Linear Combinations and Matrix Notation.- The “It Don’t Make No Nevermind” Principle.- Eigenvalues and Eigenvectors.- A Simple Eigenanalysis.- Eigenanalysis of Gramian Matrices.- 4 Multiple Regression and Correlation—Part 1. Basic concepts.- Overview.- Assumptions Underlying Multiple Regression.- The Multivariate Normal Distribution.- A Bit of the Geometry of Multiple Regression.- Basic Goals of Regression Analysis.- The Case of Two Predictors.- A Visual Example.- A Note on Suppressor Variables.- Computational Formulas.- Raw-Score Formulas.- Other Equations for R2.- Determining the Relative Importance of the Two Predictors.- Bias in Multiple Correlation.- The Case of More Than Two Predictors.- Checking for Multicollinearity.- Another Way to Obtain R2.- Residuals.- Inferential Tests.- Testing R.- Testing Beta Weights.- Testing the Uniqueness of Predictors.- Evaluating Alternative Equations.- Cross Validation.- Computing a Correlation from a priori Weights.- Testing the Difference between R2 and r2 Derived from a priori Weights.- Hierarchical Inclusion of Predictors.- Stepwise Inclusion of Predictors.- Other Ways to Handle Multicollinearity.- Comparing Alternative Equations.- Example 1—Perfect Prediction.- Example 2—Imperfect Prediction plus a Look at Residuals.- Example 3—Real Personality Assessment Data.- Alternative Approaches to Data Aggregation.- 5 Multiple Regression and Correlation—Part 2. Advanced Applications.- Overview.- Nonquantitative Variables.- Dummy Coding.- Effect Coding.- Orthogonal Coding.- The Simple Analysis of Variance (ANOVA).- Fixed Effects versus Random Effects.- The Simple ANOVA as a Formal Model.- Results of Regression ANOVAs.- Multiple Comparisons.- Orthogonal versus Nonorthogonal Contrasts.- Planned versus Unplanned Comparisons.- Individual Alpha Levels versus Groupwise Alpha Levels.- Evaluation of Quantitative Relations.- Method I.- Method II.- The Two-Way ANOVA.- Equal-N Analysis.- Unequal-N Analysis.- Fitting Parallel Lines.- Simple Effect Models.- The Analysis of Covariance (ANCOVA).- Effects of the ANCOVA on the Treatment Sum of Squares.- Using Dummy Codes to Plot Group Means.- Repeated Measures, Blocked and Matched Designs.- Higher-Order Designs.- 6 Exploratory Factor Analysis.- Overview.- The Basic Factor Analytic Model.- The Factor Equation.- The Raw-Score Matrix.- Factor Scores and the Factor Score Matrix.- Pattern Elements and the Pattern Matrix.- Error Scores and Error Loadings.- The Covariance Equation.- The First Form of Factor Indeterminacy.- An Important Special Case.- Common Uses of Factor Analysis.- Orthogonalization.- Reduction in the Number of Variables.- Dimensional Analysis.- Determination of Factor Scores.- An Overview of the Exploratory Factoring Process.- Principal Components.- The Eigenvectors and Eigenvalues of a Gramian Matrix.- A Note on the Orthogonality of PCs.- How an Eigenanalysis Is Performed.- Determining How Many Components to Retain.- Other Initial Component Solutions.- Factor Definition and Rotation.- Factor Definition.- Simple Structure.- PC versus Simple Structure.- Graphic Representation.- Analytic Orthogonal Rotation.- Oblique Rotations.- Reference Vectors.- Analytic Oblique Rotation.- The Common Factor Model.- An Example of the Common Factor Model.- A Second Form of Factor Indeterminacy.- Factor Scores.- “Exact” Procedures.- Estimation Procedures.- Approximation Procedures.- Addendum: Constructing Correlation Matrices with a Desired Factor Structure.- 7 Confirmatory Factor Analysis.- Overview.- Comparing Factor Structures.- Similarity of Individual Factors versus Similarity of the Overall Solution.- Case I—Comparing Alternate Solutions Derived from the Same Data.- Case II—Comparing Solutions Obtained from the Same Subjects but on Different Variables.- Case III—Comparing Solutions with the Same Variables but on Different Individuals; Matrix Information Available.- Case IV—Comparing Solutions with the Same Variables but Different Individuals; Matrix Information Unavailable.- Case V—Factor Matching.- Oblique Multiple Groups Tests of Weak Structure.- Basic Approach.- Evaluating the Substantive Model.- Alternative Models.- Performing the Actual OMG Analysis.- Computational Steps.- OMG Common Factor Solutions.- A Numerical Example.- LISREL Tests of Weak Substantive Models.- Specification of Weak Models.- Estimation.- Identification.- Assessment of Fit.- Numerical Examples.- LISREL Tests of Strong Substantive Models.- Causal Models and Path Analysis.- Fundamental Path Analytic Concepts.- Example I: One Exogenous and Two Endogenous Variables.- Example II: Two Exogenous and One Endogenous Variables.- Example III: One Exogenous and Two Endogenous Variables.- Decomposing Observed Correlations.- Causal Models and LISREL.- A Formal Separation of Exogenous and Endogenous Variables.- The Confirmatory Factor Model for Exogenous Variables.- The Confirmatory Factor Model for Endogenous Variables.- The Structural Model.- Addendum: A Program to Obtain Oblique Multiple Groups Solutions.- 8 Classification Methods—Part 1. Forming Discriminant Axes.- Overview.- Discriminant Analysis with Two Groups and Two Predictors.- Graphic Representation.- One Way to Obtain Discriminant Weights.- Another Way to Obtain Discriminant Weights.- Computation.- A Third Way to Obtain Discriminant Weights.- Eigenanalysis of W-1B versus Eigenanalysis of SPw-1SPb.- Strength of Relation.- Discriminant Scores, Discriminant Means, and the Mahalanobis Distance Measure.- The Discriminant Structure.- Interpreting a Discriminant Axis.- Discriminant Analysis with Two Predictors and Three Groups.- Extracting Multiple Discriminant Axes.- Inferring Concentration from the Eigenanalysis.- Numerical Examples.- Depiction of Group Differences in the Diffuse Example.- Varimax Rotation.- Use of Prior Weights.- Heteroscedasticity.- Discriminant Analysis—The General Case.- Components as Discriminant Variables.- Hypothesis Testing.- Confirmatory Factor Analysis.- Stepwise Discriminant Analysis.- 9 Classification Methods—Part 2. Methods of Assignment.- Overview.- The Equal Variance Gaussian Model.- Decision Rules and Cutoff Rules.- Other Properties.- Why a Cutoff Rule?.- Bayesian Considerations.- Why Bayesian Considerations Are Important.- Varying Base Rates.- Bayes’ Theorem in Binary Classification.- Receiver (Relative) Operating Characteristic (ROC) Curves.- Describing Accuracy of Classification (Sensitivity).- The Unequal Variance Gaussian Model.- Other Signal Detection Models.- Strategies for Individual Classification.- Describing Performance.- Bayesian Considerations with Multiple Groups.- Alternative Strategies—An Overview.- A Numerical Example.- Calibration Study.- The Context of Classification.- Classification Based on Salient Variables.- Homoscedasticity Assumed.- Normality Assumed.- Dealing with Heteroscedasticity.- Discriminant Functions and Classification.- Homoscedasticity Assumed.- Normality Assumed.- Dealing with Heteroscedasticity.- Using Multiple Discriminant Scores.- Normality and Multiple Discriminant Functions.- Classification Based on Distance Measures.- Simple Distance.- Fisher’s Classification Functions.- Mahalanobis Distances.- A Summary of Strategic Considerations in Classification.- 10 Classification Methods—Part 3. Inferential Considerations in the Manova.- Overview.- The Two-Group MANOVA and Hotelling’s T2.- The Formal MANOVA Model.- Hotelling’s T2.- Post Hoc Comparisons.- Application of Hotelling’s T2 to a Single Group.- Testing for Equality of Means.- Single Group MANOVA versus Repeated Measures ANOVA.- Tests of Vector Means with Multiple Groups.- The Fundamental Problem.- Testing a Concentrated Structure.- Testing a Diffuse Structure.- Testing for Homogeneity of Covariance.- The Simple MANOVA with Multiple Groups.- SP and Variance-Covariance Matrices.- Applying Box’s M Test.- Specifying the Nature of the Group Differences.- The Multivariate MANOVA.- Terminology and Basic Logic.- A Numerical Example.- The A Effect.- The B Effect.- The AB Interaction.- The MANCOVA.- Numerical Example.- 11 Profile and Canonical Analysis.- Overview.- Profile Similarity.- Similarity of Scalars.- Similarity of Vectors.- Measuring Profile Elevation.- Profile Similarity Based on Elevation Alone.- Numerical Example.- Profile Similarity Based on Shape Information.- A Numerical Example.- Correcting for Elevation.- Correlations, Covariances, and Cross Products as Alternative Indices of Similarity.- Simple and Hierarchical Clustering.- Numerical Example.- Canonical Analysis.- Goals of Canonical Analysis.- Basic Logic.- Redundancy Analysis.- Basic Matrix Operations.- Statistical Inference.- Numerical Example.- Alternatives to Canonical Analysis.- 12 Analysis of Scales.- Overview.- Properties of Individual Items.- Item Formats and Scoring.- Correction for Guessing.- Item Distributions.- Relation of Items to the Scale as a Whole.- Numerical Example.- Test Reliability.- The Logic of Internal Consistency Measures.- The Domain Sampling Model.- Computing rxx and Cronbach’s Alpha.- Numerical Example.- The Standard Error of Measurement.- Estimating What True Scores Would Be in the Absence of Unreliability.- Effects of Changes in Test Length.- Reliability of a Linear Combination of Tests (Scales).- Dimensionality and Item-Level Factoring.- Looking for Difficulty Factors.- Limitations on the Utility of Test Reliability.- Numerical Example I: A Unifactor Scale.- Numerical Example II: A Two-Factor Scale.- Test Validity.- Attenuation.- Appendix A—Tables of the Normal Curve.- Appendix D—Tables of Orthogonal Polynomial Coefficients.- Problems.- References.- Author Index.