You are here

Standard Setting
Share
Share

Standard Setting
A Guide to Establishing and Evaluating Performance Standards on Tests

First Edition


December 2006 | 368 pages | SAGE Publications, Inc
"Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests is a well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals. This book would be useful to individuals retained to produce or coordinate standard-setting activites."PSYCCRITIQUES


In Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests, authors Gregory J. Cizek and Michael B. Bunch provide the only "go-to" reference on how to set standards on tests in education, licensure, and certification. This book is comprehensive in scope, practical in nature, and definitive in terms of cataloguing the essential conceptual and procedural fundamentals of setting performance standards.

Key Features:  
  • Offers a detailed description of procedures: In a practical, straightforward, and virtually jargon-free style, this book begins with important foundational and conceptual information about standard setting followed by step-by-step procedures for all currently used methods, including Angoff, Item Mapping, Analystic Judgment, etc. In addition, a full chapter is devoted to future issues and research in standard setting.
  • Enables readers to design, implement, and evaluate a standard setting study: Using this book, readers will be able to: select an appropriate standard setting method; schedule and plan implementation; identify and train qualified standard-setting participants; conduct the standard setting meeting; calculate one or more cut scores; and evaluate the success of the procedure they have implemented.
  • Allows readers to adapt written materials to their situation: Examples are provided in widely available formats such as Excel, SPSS, .pdf and Word files so readers do not have to purchase specialized software or learn to use unfamiliar packages. The book includes downloadable forms, sample correspondence, checklists, data sets, and analytical software for performing calculations required for various standard setting procedures.

Intended Audience:  
This is an excellent supplementary textbook for graduate courses such as Applied Measurement in Education; Psychometric Theory; Advanced Educational Measurement; Special Topics in Measurement and Evaluation; Seminar in Educational and Psychological Measurement in the departments of Educational Psychology, Measurement and Evaluation, Psychology, and others.  

 
SECTION I: FUNDAMENTALS OF STANDARD SETTING
 
1. Contemporary Standard Setting: An Enduring Need
The Need to Make Decisions

 
The Benefits of Standard Setting

 
General Approaches to Standard Setting

 
Current Contexts for Standard Setting

 
 
2. What is Standard Setting?
Kinds of Standards

 
Definitions of Standard Setting

 
Policy Issues and Standard Setting

 
Item Scoring Criteria and Total-Test Performance Standards

 
Conclusions

 
 
3. Common Elements in Setting Preformance Standards
Purpose

 
Choosing a Standard Setting Method

 
Performance Level Labels

 
Performance Level Descriptions

 
Key Conceptualizations

 
Selecting and Training Participants

 
Evaluating Standard Setting

 
Providing Feedback to Participants

 
Professional Guidelines for Standard Setting

 
Evaluating Standard Setting

 
Conclusions

 
 
SECTION II: STANDARD SETTING METHODS
 
4. The Nedelsky Method
Procedures for the Nedelsky Method

 
Alternative Procedures and Limitations

 
 
5. The Ebel Method
Procedures for the Ebel Method

 
Alternative Procedures and Limitations

 
 
6. The Angoff Method and Angoff Variations
Procedures for the Angoff Method

 
Procedures for Angoff Variations

 
Alternative Procedures and Limitations

 
 
7. The Direct Consensus Method
Procedures for the Direct Consensus Method

 
Alternative Procedures and Limitations

 
 
8. The Contrasting Groups and Borderline Group Methods
Procedures for the Contrasting Groups Method

 
An Example Using the Contrasting Groups Method

 
The Borderline Group Method

 
Alternative Procedures and Limitations

 
 
9. The Body of Work and Other Holistic Methods
The Judgmental Policy Capturing Method

 
The Dominant Profile Method

 
The Analytic Judgment Method

 
Summary Analysis of Three Holistic Methods

 
Overview of the Body of Work Method

 
Procedures for the Body of Work Method

 
An Application the Body of Work Method

 
Alternative Procedures and Limitations

 
 
10. The Bookmark Method
Overview of the Bookmark Method

 
An Implementation of the Bookmark Method

 
Alternative Procedures and Limitations

 
 
11. The Item-Descriptor Matching Method
Procedures for the IDM Method

 
Alternative Procedures and Limitations

 
 
12. The Hofstee and Beuk Methods
The Hofstee Method

 
Procedures for Implementing the Hofstee Method

 
The Beuk Method

 
Procedures for Implementing the Beuk Method

 
Alternative Procedures and Limitations

 
 
SECTION III: CHALLENGES AND FUTURE DIRECTIONS IN STANDARD SETTING
 
13. Scheduling Standard Setting Activities
Scheduling Standard Setting for Educational Assessments

 
Scheduling Standard Setting for Credentialing Programs

 
Conclusions and Recommendations

 
 
14. Vertically-Moderated Standard Setting
The Interrelated Challenges

 
A Brief History of Vertically-Moderated Standard Setting

 
What Is Vertically-Moderated Standard Setting?

 
Approaches to VMSS

 
Applications of VMSS

 
An Illustration of VMSS Procedures

 
Alternative Procedures and Limitations

 
 
15. Standard Setting on Alternate Assessments
The Unique Challenges of Alternate Assessment

 
Necessary Conditions for Alternate Assessment Systems

 
A Generalized Holistic (GH) Method

 
Limitations and Alternative Procedures

 
 
16. Special Topics and Next Steps
Rounding

 
Methods of Adjusting Cut Scores

 
Deciding How to Incorporate Uncertainty

 
Generalizability of Standards

 
Using Multiple Methods of Standard Setting

 
Improving Participant Training

 

A well-timed text, given the legislated requirement of performance standards in our K-12 educational system, as well as the continued need in the process of credentialing professionals.  This book would be useful to individuals retained to produce or coordinate standard-setting activites.

PsycCritiques

It is abundantly apparent that the authors know standard setting and themselves have implemented numerous standard-setting procedures using different methods for different types of assessment programs.... Although the book is very practical, someone who has a fair amount of knowledge in standard setting will not feel shortchanged. There is a richness in the discussions of the methods that engages the reader, such as the different modifications of a method and the situational challenge that warranted each modification. Also, when a specific implementation of a method is discussed in detail, the authors are very conscientious in discussing the impetus for the adaptations. The historical perspective presented for some of the methods is quite intriguing.

Journal of Educational Measurement

Recommended for student projects. Excellent for students preparing for greater depth in the topic and specific research projects.

Dr Nancy Sanders
College Of Education, Washington State University - Vancouver
March 15, 2011
Key features
*An overview of the importance of standard setting, current demands, legislation, formats, and licensure and certification contents. *A detailed look at the distinctions between oft-confused concepts such as "content standards" and "performance standards." *A straightforward review of the common elements in setting performance standards. *Detailed inclusion of the various methods (Angoff, Item Mapping, Analytical Judgment, etc.)for setting performance standards. *A full chapter devoted to th future issues and research in standard setting. *Glossary of key terms and a full set of references to aid the reader's literature review.

Sample Materials & Chapters

Chapter 2 - What Is Standard Setting?

Chapter 10 - The Bookmark Method

Chapter 16 - Special Topics and Next Steps

Chapter 3 - Evaluation Form (pdf)

Chapter 3 - Normative Information (xls)

Chapter 3 - Percentages (doc)

Chapter 3 - Reality Information (xls)

Chapter 4 - Nedelsky (xls)

Chapter 5 - Ebel (xls)

Chapter 6 - Angoff (xls)

Chapter 6 - Extended Angoff (xls)

Chapter 6 - yesno (xls)

Chapter 7 - Direct Consensus (xls)

Chapter 8 - Contrasting data (xls)

Chapter 8 - Contrasting data (sav)

Chapter 9 Bow agenda (pdf)

Chapter 9 - Bow Data 1 (xls)

Chapter 9 - Bow Data 2 (xls)

Chapter 9 - Bow Form (xls)

Chapter 9 - Bow samples (pdf)

Chapter 10 - Bookmarktraining, bookmark (PPS)

Chapter 10 - Bookmarktraining, bookmarkNTG (pdf)

Chapter 10 - Bookmarktraining, practice (pdf)

Chapter 10 - Bookmark (xls)

Chapter 11 - IDM Form (xls)

Chapter 13 - Materials 1, agenda (pdf)

Chapter 13 - Materials 1, angoff overview (PPS)

Chapter 13 - Materials 1, bookmark (pdf)

Chapter 13 - Materials 1, bookmark overview (pps)

Chapter 13 - Materials 1, evaluation form (pdf)

Chapter 13 - Materials 1, Holistic overview (pps)

Chapter 13 - Materials 1, letters (pdf)

Chapter 13 - Materials 1, Readiness form (pdf)

Chapter 13 - Materials 1, Security agreement (pdf)

Chapter 13 - Materials 2, Impact (xls)

Chapter 13 - Materials 2, Ordered Item Booklets (xls)

Chapter 13 - Materials 2, PLDs (pdf)

Chapter 13 - Materials 2, Round summaries (xls)

Chapter 13 - Cert scheduling (xls)

Chapter 13 - Ed schedule (xls)

Chapter 13 - Letters (pdf)

Chapter 14 - vmss, Agenda (pdf)

Chapter 14 - vmss, Articulation (pps)

Chapter 14 - vmss, Overall distribution scores_Round 2 (xls)

Chapter 15 - ghm, ghm (xls)

Chapter 15 - ghm, VastDSet (pps)

Chapter 3 - Feedback Form (pdf)

Chapter 3 - Raw Score Distribution (xls)


Sage College Publishing

You can purchase or sample this product on our Sage College Publishing site:

Go To College Site

This title is also available on SAGE Research Methods, the ultimate digital methods library. If your library doesn’t have access, ask your librarian to start a trial.