Nurse Educator's Guide to Assessing Learning Outcomes

Acknowledgments     iiiPreface     xiiiThe Role of Assessment in Instruction     1The Process of Assessment     1Ethical Responsibility     3Assessment and Self-Efficacy     3Assessment Inadequacy     4Assessment Competency Standards     5The Need for a Systematic Approach to Assessment     7Assessment Instruments     7Summary     8The Language of Assessment     9Assessment     9Measurement     10Evaluation     11Formative Evaluation     12Summative Evaluation     12Instructional Objectives     13Learning Outcomes     13Blueprint     13Item Bank     14Test     14Interpreting Test Scores     15Criterion-Referenced Tests     15Norm-Referenced Tests     16High-Stakes Tests     17Grade     18Test Bias     18Reliability     19Validity     20Evidence Based on Test Content     21Evidence Based on ResponseProcesses     22Evidence Based on Internal Structure     23Evidence Based on Relation to Other Variables     24Evidence Based on the Consequences of Testing     25Face Validity     25Basic Test Statistics     26Summary     26Developing Instructional Objectives     27Role of Objectives     28Focus of Instructional Objectives     29Stating Instructional Objectives     30Specific Objectives     30General Objectives     31Learning Outcomes     32Mastery (Performance) Objectives     34Developmental Objectives     35Framework for Writing Objectives     37Number of Objectives     38Number of Learning Outcomes     39Taxonomies     41Using Instructional Objectives     43Criteria for Effective Objectives     46Complete     46Appropriate     46Sound     46Feasible     47Relevant     47Open-Ended     47Delineate Student Behavior     47Shared with Students      48Summary     48Assessing Critical Thinking     49Accreditation Outcomes     49Establishing Evidence of Validity     50Defining Critical Thinking in Nursing     50Sample Definition of Critical Thinking in Nursing     51Instructional Objectives and Learner Outcomes     52Behaviors Associated with the Characteristics of Critical Thinking     53Possesses an Analytical Attitude     53Demonstrates Sensitivity to Context     54Questions Assumptions     54Bases Inquiry on Credible Sources     55Considers a Variety of Solutions     55Pursues Ongoing Evaluation     56Tri-Level Critical Thinking Objective     56Critical Thinking Assessment Plan     57Multiple-Choice Exams and Critical Thinking     59Summary     61Implementing Systematic Test Development     63Initiating Test Development     63Scheduling a Semester's Exams     64Identifying the Purpose of the Test     64Determining the Length of the Test     66Selecting What to Test     66Selecting the Appropriate Assessment Format      67Weighting the Content and Course Objectives     69Developing a Test Blueprint     71Reviewing the Blueprint     73Preparing Students for a Test     74Sharing the Blueprint with Students     74Pop Quizzes     75Determining How Difficult a Test Should Be     75Summary     79Selected-Response Format: Developing Multiple-Choice Items     81Advantages of Multiple-Choice Items     82Limitations of Multiple-Choice Items     82Relevance of Multiple-Choice Items     83Item Writing Logistics     83Style Guide     84Electronic Item Development     84Multiple-Choice Format     85Stem Formats     86Question     86Completion     86Item Writing Guidelines     87General Guidelines     88Characteristics of Effective Stems     91Qualities of Effective Options     95Developing the Correct Answer     103Designing Effective Distractors     105Characteristics to Avoid     109Alternate National Council Licensure Examination (NCLEX) Items      115Fill in the Blank     116Point and Click     117Multiple Response     117Chart/Exhibit     118Drag and Drop     118Item Rationale     119Question Difficulty     119Framing Questions in Terms of the Nursing Process     120Item Shells     121Peer Review     123Summary     123Writing Critical Thinking Multiple-Choice Items     125Characteristics of Critical Thinking Items     125Sequential Reasoning     126Language     127Best Answer Format     129Novel Problems     129Applying Learning Outcomes     131Critical Thinking and the Nursing Process     133Summary     134Selected-Response Format: Developing True-False and Matching Items     135The True-False Item Format     135Advantages of the True-False Item Format     136Disadvantages of the True-False Item Format     136Guidelines for Developing True-False Items     137Characteristics of Effective True-False Items     138Flaws to Avoid With True-False Items     141Forms for True-False Item Writing     144Types of True-False Items     144Matching Exercise     148Advantages of the Matching Exercise     149Disadvantages of the Matching Exercise     149Guidelines for Developing Matching Exercises     149Summary     151Constructed-Response Format: Developing Short-Answer and Essay Items     153Short-Answer Item Format     154Advantages of the Short-Answer Item Format     154Disadvantages of the Short-Answer Item Format     154Guidelines for Developing Short-Answer Items     154Characteristics of Effective Short-Answer Items     155Essay Question Format     159Forms of Essay Questions     159Advantages of the Essay Question     162Disadvantages of the Essay Question     162Guidelines for Developing Essay Items     163Suggestions for Scoring Essay Items     164Developing Essay Items to Assess Critical Thinking     164Scoring Rubrics     166Holistic Scoring Rubrics     166Analytic Scoring Rubric     166Developing Scoring Rubrics     167Summary     169Assembling, Administering, and Scoring a Test     171Assembling a Test     171Arranging Items     172Editing and Proofreading     174Formatting     175Providing Directions     177Reproducing     180Maintaining Security     181Administering a Test     182Computer Test Administration     182Physical Environment     184Psychological Environment     184Academic Dishonesty     185Scoring a Test     190Scoring a Classroom Exam     190Statistical Analysis     191Student Review     192Summary     193Preparing Students for the Licensure Exam: The Importance of NCLEX     197The NCLEX Application Process     198Development of the NCLEX     198The NCLEX Practice Analyses     198The NCLEX Test Plan     199NCLEX Item Development     201Characteristics of NCLEX Items     202Cognitive Levels     202Critical Thinking Ability     202Alternate Formats     202Characteristics of the NCLEX     203CAT Format     203Passing Standard     204Logits     205Examination Length     205Preparing Nursing Students for the NCLEX     207Curriculum Focus     207NCLEX Test Plan Consideration     207Classroom Test Development     208Cross-Referencing the NCLEX Test Plans     209Cross-Referencing for Integrated Processes     210Standardized Examinations     211Progression Policies     213NCLEX Predictor Exit Exams     213Facilitating Student Success     217NCLEX Prep Courses     218Summary     219Establishing Evidence of Reliability and Validity     221Reliability     222Measures of Reliability     222Factors Affecting the Reliability Coefficients of Classroom Exams     226Homogeneity of the Test Content     228Measurement Error     231Validity     234Content Validity Index     234Interpretation of the Validity Coefficient     235Summary     236Interpreting Test Results     237Overall Test Data Analysis      238Measures of Central Tendency     239Measures of Variability     241Reliability Coefficient     242Standard Error of Measurement     243Mean p Value     244Mean Biserial     245Score Distribution     245Histogram     246Individual Item Analysis     248Assigning Test Scores     251Flawed Items     251Returning Scores to Students     253Summary     253Laboratory and Clinical Evaluation     253Relevance of Objectives     253Elements of the Clinical/Laboratory Experience     254Identification of Clinical Action Elements     255Identifying Objectives and Outcomes     256Assessing Learning Needs     256Planning Clinical Learning Experiences     257Instructing and Evaluating     257The Professional Role     259Designing and Using a Performance Tool     259Using a Taxonomy     259Assessment of Mastery Learning     261Assessment of Developmental Learning     263Portfolio Assessment     265Implementing a Clinical Evaluation Tool      266Summary     267Assigning Grades     269Grading Principles     269Philosophy of Grading     271What Meaning Should Each Grade Symbol Carry?     271What Should Failure Mean?     273What Elements of Performance Should Be Incorporated in a Grade?     273How Should the Grades in a Class Be Distributed?     273What Components Should Go Into a Final Grade?     274How Should the Components of the Grade Be Combined?     274What Method Should Be Used to Assign Grades?     274Should Borderline Cases Be Reviewed?     274What Other Factors Should Influence the Grading Policy?     275Developing a Grading Plan     275Weighting Components of a Grade     275Pass/Fail Grading     281Adjusting Grades     281Scaling Grades     282Giving Extra Credit     282Dropping the Lowest Grade     282Summary     283Instituting Item Banking and Test Development Software     285Establishing an Item Bank     286Editing Items     287Selecting a Test Development Software Program      291Hardware Requirements     292Item Banking Facility     295Item Importing and Exporting     295Word Processing     296Item Classification     296Supplemental Fields     297Test Assembly     297Scoring and Reporting     298Storage of Item Date History     299Grade Book Facility     299Cost     300Implementing Test Development Software     302Appointing Test Bank Administrators     302Establishing Procedures     303Incorporating Textbook Item Banks     304Summary     306References     309Appendices     319Basic Test Statistics     319Basic Style Guide     321References     327Targeting Cognitive Levels for Multiple-Choice Item Writing     329Sample Item Stems for Phases of the Nursing Process     333Assessment     333Analysis     334Planning     335Implementation     335Evaluation     336Steps for Implementing a Systematic Assessment Plan     339Relationship of Critical Thinking Characteristics to the Nursing Process     341Index     345