Not all assessments are created equal, even those that claim to be “standards-aligned.” High-quality assessment tools can provide teachers and administrators with actionable information about students’ understanding of key concepts and skills, allowing educators to build a roadmap for helping all students master college- and career-ready academic content. Lower-quality assessment tools can provide some information back to educators, but may not provide an accurate picture of what students know and can do or may not provide the information back in a way that supports educators’ ability to take action using the data.
Here are some key questions the CenterPoint Mathematics Team considers when determining whether an assessment is worth the time and focus of students and educators – i.e. is the assessment high quality?:
- Is the student’s performance on this assessment going to provide actionable information to educators? The purpose of many assessments is to provide educators with information that can help them diagnose specific gaps in student knowledge. The information they get back via reporting tools should be easily interpretable, accessible in real-time, and clear enough to support next steps in instruction. This requires that the assessments are constructed to allow for those kinds of insights into student mastery of the content – in particular by aiming to measure student understanding of specific content with real depth. For example, if teachers want to know what students know and can do with fractions, they need a tool that homes in on student understanding in that area and allows them to use the data to make decisions about where to target instruction so students build that foundational knowledge.
- Are the test questions aligned to the standards being assessed? The first thing to look for is alignment to the content standards being assessed. Close alignment between academic content standards, curriculum, and the assessment means that all three are in sync from a technical standpoint and in terms of overall coherence and intent. The curriculum should closely reflect the breadth and depth of standards, and the assessment tools should provide all students with the opportunity to demonstrate the degree to which they have mastered those content, knowledge, and skills – as appropriate for the type of assessment being administered. (A diagnostic assessment, for example, wouldn’t cover the full set of standards at a grade level and would be focused on pinpointing specific strengths and weaknesses in students’ understanding.)Take eighth grade geometry, for example. The Common Core State Standards (CCSS) include standards related to congruence at eighth grade (see CCSS.MATH.CONTENT.8.G.A.1 through CCSS.MATH.CONTENT.8.G.A.4). There are some traditional test items that ask about congruence in a straightforward way—such as “which of these shapes are congruent”? But, the standards are really focused on understanding that congruence is achieved if a figure can be mapped onto the other using a sequence of rotations, reflections, and translations.Items that assess the concept of congruence from the latter perspective are more closely aligned to the spirit of the standards and will give educators better insight into students' conceptual understanding of the content.
- What knowledge and skills are required to answer each question on the assessment? Does the item require the student to produce evidence of the knowledge and skills that you’re looking for in order to measure whether she understands the concept or mathematical procedure? Also, does the item include all necessary background information to answer the question?
- Are the items accessible? We pay particular attention to whether items adhere to universal design principles and are free of bias and potential sensitivity issues -- in general, ensuring items are accessible to all students. In this way, educators can understand whether a student has mastered the concepts and skills in the standards without any concerns about extraneous issues distracting the student. For example, a question that includes references to a culturally sensitive issue may distract a student from the task at hand and interfere with his ability to show fully what he knows and can do.
- Do the items include application to real-world contexts? To be prepared for college and careers, students need to be able to apply mathematical knowledge and skills to new contexts. High-quality assessments should include items that ask students to use their math knowledge in real-world situations, so that teachers can understand students’ ability to apply that knowledge in different scenarios. Additionally, the context used in items should be authentic and involve mathematics that would be used in real situations. Another caveat: make sure the real-world situations are correct and accurate! We have seen some great items marred by factual errors, which undermines the quality of the question and the assessment itself.
- Are the items engaging to students? High-quality assessments attempt to engage students in multiple ways. For example, questions can be worded so that it’s not immediately obvious how to get to the correct answer, thereby encouraging the student to persevere and think deeply. They may also include real-world contexts that require students to use their knowledge in different situations – and those real-world examples should be interesting and relevant for the student’s grade level. Assessments that make use of technology can also help engage students, although we always want to ensure that we’re using technology to improve the quality of the assessment items and not just for the sake of using technology.
- Are the multiple-choice items of high quality? Multiple-choice items have a mixed reputation, but, if done well, they can be an effective way to quickly assess students’ knowledge of core concepts. One thing to look for is whether the distractors—incorrect answer options—point to common conceptual or procedural misunderstandings that students may have. Good distractors can help educators get more specific information about where students’ misunderstandings lie. (Everything on the assessment should have a purpose!) Another criterion to look for is whether the assessment includes a mix of item types, rather than relying solely on multiple choice items. An array of item types can help measure student understanding in different ways, providing a fuller picture of their mastery of the content.
- Are the items correct? This may seem obvious, but items must be mathematically correct and free from any grammatical or spelling errors that might interfere with students’ ability to understand the question and answer it to the best of her ability.
To illustrate some of these principles, we’ve selected a computer-based sample item that uses drag-and-drop technology enhancements and noted some of the attributes that make this item high quality.
Figure 1: Example Grade 4 Mathematics Item Assessing Student Understanding of Fractions
This fourth-grade item is closely aligned to Number and Operations—Fractions, Extend understanding of fraction equivalence, and ordering (CCSS.MATH.CONTENT.4.NF.A.1 and CCSS.MATH.CONTENT.4.NF.A.2).
Figure 2: Grade 4 Mathematics Standards on Number and Operations-Fractions, Extend understanding of fraction equivalence, and ordering
In this item, students must compare and order the fractions. Efficiency is gained if students notice 6/5 is the only fraction greater than 1 and 5/12 is less than 1/2. They must also compare 3/4 and 7/8, possibly by generating fractions with equivalent denominators or by creating a fraction model. This item requires an understanding of fraction comparison and perseverance in ordering multiple fractions.
It also meets several of the other criteria we’ve outlined above for what constitutes a high-quality assessment question. The item is mathematically correct and free of errors and other unnecessary distractions; it uses accessible language and avoids sensitive issues that may introduce bias. The item type engages students by having them select and place the fractions in the correct order, and it uses a real-world situation to give context to the task at hand.
All of the CenterPoint Mathematics Team members are former mathematics teachers who have created and used many different assessments in the classroom to assess students’ understanding of key mathematical concepts and skills and have built high-quality summative, interim, diagnostic, and formative assessments. The team also brings forth several years of experience as department chairs, math content experts at state departments of education, and now as senior instructional designers for CenterPoint. Altogether, the team has reviewed thousands of assessment items aligned to college- and career-ready standards.