 |
 |
 |
 |
 |
 |
 |
 |
 |
Reflective Judgment as Goal of Higher Education and Cognitive Ability.
Some of the most interesting real-world problems are those about which even qualified experts can reasonably disagree. Reflective Judgment research seeks to document progressive sophistication in the way that individuals reason about a particular type of problem termed a ``live controversy" or ``ill-structured problem." The ability to reason complexly about such real-world problems is often claimed to be a central goal of college education.
Additionally, the Reflective Judgment model is also a model of cognitive development during adulthood. Many life span developmental psychologists claim that reasoning about such issues represents a major cognitive development during young and middle adulthood.
Specifically, the Reflective Judgment model was developed jointly by Patricia King and Karen Kitchener as one way to document the gains in complex reasoning that students demonstrate during the undergraduate and graduate years.
Background information on the Reflective Judgment Model.
Because the Reflective Judgment model attempts to document reasoning ability about ``live controversies" or ``open-ended questions" it is not possible to judge an answer in terms of its absolute truth or falsity. Rather, the model documents the sophistication with which individuals reason about such issues.
- In our interviews with students, we have found that some individuals evidently are not even aware that such issues don't allow of single correct answers or that relevant evidence plays a role in constructing a perspective on the issue. We have generally referred to these individuals as demonstrating a ``pre-reflective" style of reasoning.
- Still others appreciate the open-ended nature of the problem and the fact that information plays a role in constructing an opinion. Many of these individuals have difficulties in organizing discrepant evidence and opinion. We've termed such reasoning ``quasi-reflective."
- Still others do demonstrate, to varying degrees, the ability to construct an opinion about such issues in a cognitively complex and self-correcting way. Such individuals show truly ``reflective judgment."
Assessment Measures
The Reflective Judgment Interview
One instrument which has been used to document reasoning according to this model is the Reflective Judgment Interview (RJI). This measure involves presenting individuals with 2-4 dilemma topics and soliciting their opinions on the issues by means of a semi-structured interview.
Several studies, both cross-sectional and prospective, have been done using the Reflective Judgment Interview. (For specific citations, I'd refer you to the RJ bibliography).
Educational Level Differences I have taken the time to accumulate all available RJI raw data. If you take a look at all of the available data to date by educational level as displayed in a box plot, three things stand out:
- Over the course of the college undergraduate years, there is a large difference (roughly one standard deviation) (in cross-sectional studies) and a large gain (in prospective studies) as a function of level of education
- Patterns of gains do not appear to be sudden - the RJI appears to document changes which are not the result of a single course or semester-long experience
- The patterns of performance on the interview are generally lower than one would hope.
- About half of the college freshmen assessed score below level 3.5, indicating that their performance is a mixture of pre-reflective and quasi-reflective reasoning.
- Undergraduate seniors, although a standard deviation higher than freshmen, score, on average at level 4, with half of the seniors falling between approximately 3.5 and 4.5. The major accomplishment of undergraduate education appears to be the reduction of pre-reflective reasoning.
- Even graduate students do not uniformly demonstrate reflective reasoning. About half of the beginning graduate students (those with less than three years of graduate experience) score between levels 4 and 5. Scores of about half of the advanced graduate students (those with three or more years of experience) fall between slightly below level 5 and slightly below level 6. As such, graduate student reasoning appears to represent a mix of quasi-reflective and reflective reasoning.
- (It is also possible to look at the smaller subject pool of available high school data to date, which shows that high school students respond at about level 3, indicating a generally pre-reflective approach to the live controversies presented.
Psychometric Properties Generally, the Reflective Judgment Interview has shown excellent internal consistency and has correlated quite well with level of educational attainment. Educational level differences documented by the Reflective Judgment model are not attributable to general verbal ability, traditional measures of critical thinking or age.
The Reflective Judgment Interview requires the use of trained and certified interviewers and raters. For further information on how to become certified to administer and score the Reflective Judgment Interview, contact Karen Kitchener.
Other resources
- In addition to the RJ bibliography I have also made available a short article on our research which was published recently in Assessment Update.
- Sometimes assessment officers and researchers contact me asking me what a reasonable sample size would be for their planned assessment study. I've put together a few initial thoughts on the issue of statistical power in Reflective Judgment studies.