Factorial validity

Is relevant to determining whether the scores of a measure correspond to the number and nature of dimensions of theorised dimensions that underlie the construct of interest
Researchers use a data technique known as factor analysis to help determine the number and nature of dimensions associated with the scores derived from a test

Factorial validity:

Pertains to the internal structures of test scores
A tests internal structure is the way that parts of the test are related to each other
The actual (empirical) structure of the test should match the theoretical or intended structure of the test
In a very simplistic sense, factorial validity helps specify what the test measures
That is, with respect to the number of dimensions, and the definition of those dimensions
The definitions of the dimensions are determined by which items ‘load’ onto which dimension
To determine the number of dimensions that underlie an inventory, as well as their definitions, we (can) use a data analytic technique known as factor analysis o Helps us clarify the number of factors within a set of terms

o Helps us determine the which items are linked to which factor, which facilitates the interpretation of those factors

Measurement:

Typically, when we attempt to measure an attribute, we try to do so in such a way that the score represents a single attribute
g. based on six item questions (3 on social interaction, 3 on intellect), if the total was calculated, what would that composite score mean?
- It would be a compromised score, because the items do not all measure the same dimension
- We try to avoid these situations, want composite scores that are ‘factor pure’

Dimensionality:

Generally, three types of tests:
Unidimensional test o Consists of items which all measure one, single factor
Multidimensional test (uncorrelated)

o Consists of items which measure two or more dimensions which are unrelated to each other

Multidimensional test (correlated) o Consists of items which measure two or more dimensions which are correlated with each other (positively or negatively)

Unidimensional tests:

Include items that reflect only a single attribute
The items are not affected by other attributes
Consider a multi choice exam, students get one score on the final exam (presumes the test is also Unidimensional/justifiable to yield one score)

Multidimensional test:

This is a model of a correlated multidimensional test
Each item is linked to one attribute only, but there are there are two distinct attributes which are correlated with each other
This is a model of an uncorrelated multidimensional test, each items is linked to one attribute only.

o There is no link between the two dimensions

Implications of factorial validity:

Test dimensionality has implications for the scoring:

o Scoring o Evaluation and o Use of the test scores

Multidimensional tests with correlated dimensions can produce a variety of scores

o Subtest score – based on the items of a single dimension o Area score- based on the items of a single dimension o Total score – used in higher order composite scores

The psychological meaning of dimensions:

After the number of dimensions have been determined
And the associations between the dimensions have been determined
It is now time to understand the psychological meaning of each test dimension
This is accomplished by evaluating which items load onto each respective dimension

Inter item correlations (hypothetical):

Furthermore, it appears that the two dimensions are not correlated with each other
This is an eye-ball approach to conducting a factor analysis
In practice, it is not especially useful because o There are usually a lot more than six variables included in a factor analysis o The pattern of correlation is not usually as clear cut as those in the above table

Conducting a principle component analysis (PCA):

Note a PCA is almost equivalent to a factor analysis
Conduct a PCA twice:

o rnce to determine the number of components (factors) to extract o Then again based on the number of components that you want extracted

Find the break in the scree plot, this separation is obvious to how many dimensions are in the data o Scree plot consists of the eigenvalues ordered from smallest to largest in a scatter plot

o Eigenvalues are essentially numerical representations of components with respect to their size.

Now that I know how many components to extract from analysis, rerun with the specification of two components.

Communalities:

Represent the percentage of variance associated with a particular variable that was included in the analysis
Generally, want to see communalities that or at least .04 or .09 (depending on the type of data being analysed:
- .04 or greater for items
- .09 or greater for subscales

Component landings:

Are useful when greater than either .20 or .30
- .20 for items
- .30 for subscales
- Hence the minimum expectation of a communality of either .04 or .09 o Communality = sum of the squared component landings

Simple structure:

The degree to which an item (or scale or any variable included in the analysis) is associated with only one substantial loading on a single dimension (i.e. component) and negligible loadings on the remaining dimensions
rne way to achieve simple structure is to rotate the solution, should always be done
You can only rotate a solution if you extract two or more components

Component correlations:

The correlation between the two components is equal to zero
They are two totally separate dimensions with no overlap
You cannot predict how high someone might score on openness to experience if you know how high they were on extraversion

Sample size requirements:

There are no simple guidelines to follow, because the size of the sample required will be dependent upon two main factors o The amount of communality associated with the variables (higher communality means less sample size required)

o The number of variables per factor (higher number of variables per factor means less sample size required)