Introduction to Many-Facet Rasch Measurement: Analyzing and Evaluating Rater-Mediated Assessments

Human ratings are prone to error and bias, which can threaten the validity and fairness of performance assessments. This book introduces many-facet Rasch measurement (MFRM) as a psychometric approach to drawing reliable, valid, and fair inferences from rater-mediated assessments. It provides an introduction to the MFRM framework using sample data from a writing performance assessment, encouraging readers to adopt it in their professional context.

Format: Hardback
Length: 242 pages
Publication date: 28 April 2023
Publisher: Peter Lang AG

Human ratings are prone to various forms of error and bias, a longstanding issue in performance assessment. Since the early days, expert raters have often provided different ratings for the same performance, undermining the validity and fairness of the assessment. In such cases, the outcome heavily relies on the specific raters who provide the ratings, posing a significant threat to the reliability and equity of the evaluation. To address this challenge, this book introduces a psychometric approach known as many-facet Rasch measurement (MFRM). Throughout the book, real-world data from a writing performance assessment are utilized to illustrate key concepts, theoretical foundations, and analytical procedures, encouraging readers to adopt the MFRM approach in their current or future professional endeavors.

MFRM is a powerful tool that establishes a coherent framework for drawing reliable, valid, and fair inferences from rater-mediated assessments. It addresses the inherent fallibility of human ratings by employing a hierarchical model that captures the complexity and variability of human judgment. The approach involves rating items on multiple dimensions, such as difficulty, discrimination, and reliability, using a uniform rating scale.

One of the key advantages of MFRM is its ability to provide accurate and reliable measures of performance. By analyzing the data using advanced statistical techniques, MFRM can identify patterns and trends that may not be apparent to human raters. This information can be used to improve the accuracy and fairness of assessments, as well as to develop more effective training and development programs.

Another advantage of MFRM is its ability to handle large amounts of data efficiently. With its hierarchical model, MFRM can analyze large datasets and identify meaningful relationships between variables, even when the number of items and raters is substantial. This makes it particularly useful in situations where assessments are conducted on a large scale, such as in educational institutions or professional organizations.

In addition to its practical applications, MFRM has important theoretical implications. It provides a framework for understanding the nature of human judgment and the factors that influence it, which can have implications for fields such as psychology, education, and business. MFRM also contributes to the development of new measurement theories and methods, which can improve the accuracy and reliability of assessments across different domains.

Overall, this book provides a comprehensive introduction to MFRM, a powerful psychometric approach that establishes a coherent framework for drawing reliable, valid, and fair inferences from rater-mediated assessments. By utilizing real-world data and advanced statistical techniques, MFRM offers a solution to the problem of inevitably fallible human ratings, promoting the development of more effective assessment practices in a wide range of professional contexts.

Weight: 387g
Dimension: 210 x 148 (mm)
ISBN-13: 9783631903049
Edition number: New ed