{"id":4280,"date":"2018-09-18T18:48:18","date_gmt":"2018-09-18T22:48:18","guid":{"rendered":"https:\/\/www.amyork.ca\/academic\/zz\/?p=4280"},"modified":"2019-05-25T23:53:25","modified_gmt":"2019-05-26T03:53:25","slug":"importance-of-reliability","status":"publish","type":"post","link":"https:\/\/www.amyork.ca\/academic\/zz\/psychological-measurement\/importance-of-reliability\/","title":{"rendered":"Importance of reliability"},"content":{"rendered":"<p><strong>Cronbach\u2019s Alpha assumptions:<\/strong><\/p>\n<p>1- That the indicators (items) are essentially tau-equivalent o This implies that each item is an equally strong indicator of the true score scores, but the ma differ by a constant<\/p>\n<p>o Basically this implies that the items can have different means<\/p>\n<p>2- That each item\u2019s error term is uncorrelated with every other item\u2019s error term<\/p>\n<p>3- That the error scores are uncorrelated with the true scores o An assumption associated with all forms of reliability<\/p>\n<p>4- The items used to generate the questionnaire must only measure one attribute or construct o If this is violated, Cronbach\u2019s alpha cannot be used<\/p>\n<p><strong>Standardised coefficient alpha:<\/strong><\/p>\n<ul>\n<li>You apply it to scores that have been converted from a raw score to a standardised score<\/li>\n<li>g. if you had z-scores and you wanted to calculate the level of internal consistency associated with a composite which consisted of a sum of two or more z-scores, you would use the standardized version of coefficient alpha<\/li>\n<\/ul>\n<p><strong>Types of reliability assumption models:<\/strong><\/p>\n<ul>\n<li>Parallel- equal true score variance, equal means, equal error variance<\/li>\n<li>Tau-equivalent- equal true score variance, equal means, unequal error variances<\/li>\n<li>Essentially tau-equivalent- equal true score variance, unequal means, unequal error variances<\/li>\n<li>Congeneric- unequal true score variance (but all greater than zero), unequal means, unequal error variances<\/li>\n<\/ul>\n<p><strong>What Cronbach\u2019s alpha is not?<\/strong><\/p>\n<ul>\n<li>Cronbach\u2019s alpha represents the ratio of true score variance to total variance<\/li>\n<li>It does not imply that a collection of items measure one psychological attribute (or dimension) and one psychological attribute only.<\/li>\n<li>It is fully possible for a collection of items to measure more than one psychological attribute, but still be associated with a reasonably high Cronbach\u2019s alpha<\/li>\n<li>Argued that CA assumes that the collection of items measure only one attribute o This can be tested using factor analysis<\/li>\n<\/ul>\n<p><strong>Factors affecting reliability:<\/strong><\/p>\n<ul>\n<li>Will adding this item reduce the mean inter-item correlation?<\/li>\n<\/ul>\n<p><strong>Sample homogeneity:<\/strong><\/p>\n<ul>\n<li>A more homogenous sample will yield lower reliability estimates than a heterogeneous sample o This is true of all correlations, not just statistics like Cronbach\u2019s alpha which is fundamentally based on correlations<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><strong>Importance:<\/strong><\/p>\n<p>Mental retardation is defined as a person with an IQ score of 70 or less<\/p>\n<ul>\n<li>Some US states, death penalty cannot be administered to mental retards<\/li>\n<\/ul>\n<p><strong>Point estimates and confidence intervals:<\/strong><\/p>\n<ul>\n<li>The standard error of measurement represents the amount of error \u2018around\u2019 a point estimate in standard deviation form<\/li>\n<li>A point estimate is one\u2019s best guess of what a person\u2019s score is on the test o In the context of psychometrics, it\u2019s the score you get from the test o However a confidence interval can be estimated around a point-estimate<\/li>\n<li>A confidence interval reflects a range of values that is often interpreted as a range in which the true score is likely to fall<\/li>\n<li>Se<sub>m<\/sub> = s<sub>o <\/sub>\u221a(1-R<sub>xx<\/sub>) o S<sub>o <\/sub>= standard deviation of the test o R<sub>xx<\/sub> = reliability estimate<\/li>\n<li>Typically, people calculate the 5% confidence interval around a point estimate<\/li>\n<li>rnce the standard error of estimate has been calculated, you can use this info to calculate a confidence interval around the point estimate<\/li>\n<\/ul>\n<p><strong>What confidence level should be used?<\/strong><\/p>\n<ul>\n<li>You probably want to use a 99% CI for a lift or death penalty<\/li>\n<li>You should keep in mind that although high levels of reliability are desired, they are not the only consideration<\/li>\n<li>rften have to sacrifice some reliability for greater validity<\/li>\n<\/ul>\n<p><strong>Reliability standards:<\/strong><\/p>\n<ul>\n<li>Using the standardised coefficient alpha formula, you can work out how large the mean inter-item correlation must be to achieve a reliability of a desired confidence interval<\/li>\n<\/ul>\n<p><em>k<span style=\"text-decoration: line-through;\">r<\/span><\/em><em>ii &#8216;<\/em><\/p>\n<ul>\n<li><em>R<\/em><em>xx<\/em>=1+(<em>k<\/em>\u22121)<em><span style=\"text-decoration: line-through;\">r<\/span><\/em><em>ii&#8217;<\/em><\/li>\n<\/ul>\n<p><strong>Two types of correlations:<\/strong><\/p>\n<ul>\n<li>rbserved score correlation:\n<ul>\n<li>The correlation you get based on the data you have<\/li>\n<li>Will be compromised to the degree to which there is measurement error in your data, when scores are associated with less than perfect reliability<\/li>\n<li>In practice this means the maximum correlation between two sets is not 1.0 but often less than that<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li>The maximum correlation between two variables- <em>r<\/em>max=\u221a<em>R<\/em><em>xx<\/em><em>R<\/em><em>yy <\/em><strong>o <\/strong>The correlation for attenuation formula is the ration of the observed correlation to<\/li>\n<\/ul>\n<p><em>r<\/em><em>x<\/em><em>O <\/em><em>y<\/em><em>O<\/em><\/p>\n<p>the square root of the product of the reliabilities- <em>r<\/em><em>x<\/em><em>t <\/em><em>y<\/em><em>t<\/em>=\u221a<em>R<\/em><em>xx<\/em><em>R<\/em><em>yy<\/em><\/p>\n<ul>\n<li>True score correlation:\n<ul>\n<li>A hypothetical correlation you can estimate, if you know the reliability associate with the scores<\/li>\n<li>Not compromised by measurement error<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Cronbach\u2019s Alpha assumptions: 1- That the indicators (items) are essentially tau-equivalent o This implies that each item is an equally strong indicator of the true score scores, but the&hellip; <a class=\"continue\" href=\"https:\/\/www.amyork.ca\/academic\/zz\/psychological-measurement\/importance-of-reliability\/\">Continue Reading<span> Importance of reliability<\/span><\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[107],"tags":[],"_links":{"self":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts\/4280"}],"collection":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/comments?post=4280"}],"version-history":[{"count":2,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts\/4280\/revisions"}],"predecessor-version":[{"id":4871,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts\/4280\/revisions\/4871"}],"wp:attachment":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/media?parent=4280"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/categories?post=4280"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/tags?post=4280"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}