Creating Multiple-Form Summative Assessments

Creating Multiple Forms

Highly Randomized Summative Assignments

_______________________

Pools vs. Forms

The Student Experience of Multiple-Form Assessments

With multiple forms come multiple ways of designing the student experience. Multiple forms create the following types of student experiences: 

• Different students receive different forms on a single attempt.
• All students receive the first form on the first attempt, the second form on the second attempt, and so on. 
• Different students receive different forms on the first attempt, different forms on the second attempt, and so on.

You create each of these experiences by authoring multiple forms and then manipulating the settings surrounding them. 

Note: Forms Should Be Parallel

When you are working with multiple forms, we strongly suggest that you make those forms as parallel as possible. Because different students can receive different forms in some cases, they should receive equal attempts. Also, the same students could receive different forms on different attempts, and those attempts should be as equal as possible. Here are some tips for creating parallel forms:

• Write roughly the same number of questions in each form.
• Cover the same learning objectives and skills in each form.
• Keep questions about equal in difficulty level.
• Think about how to ask the same question in different ways.

________________________________________________

Creating Multiple Forms

Follow these instructions to create multiple forms:

Step 1. Create an assessment page.

Step 2. Create a Pool file.

Step 3. Create assessment sections.

Step 4. Write questions. 

Step 5. Add pool to assessment page and set selection criteria.

     Detailed instructions follow.

 

Step 1: Create an Assessment Page

There are two easy ways to create a basic assessment page. The first is from the Assessment tab in Smart Author: 

1. On the left-hand menu, navigate to the Assessments > Activities tab.

2. Click Add Assessment.

 

3. Select a type (Quiz or Unit Test), add a title, and click Add. 

Or you can use Smart Author to create an assessment page from the content page on which you want to place an assessment: 

1. From the content page in Smart Author, select + Add Section. 

2. Select Assessments & Assignments. 

3. Click Add to Page.

 4. Click Create a new Activity.

 

5. Specify the type (Quiz or Unit Test) and title, then click Add.

 

Step 2: Create a Pool File

To create a pool file:

1. On the left-hand menu, navigate to Assessments > Question Pools. 

2. Click Add Question Pool.

3. Give the new question pool a title that will be easy to identify later, then click Save.

 

Step 3: Create Assessment Sections

Within the pool file, you need to create assessment sections. To create an assessment section, click + Add Assessment Section.

Assessment sections are organizations of content and questions within the pool file. Notice the grey box surrounding the + Add Assessment Section. All questions for that section should be authored within the grey box. 

Remember that a form is the set of questions that a student receives. Create a form by authoring all the questions included in that form in an assessment section. If there are two forms, there should be two assessment sections. If there are three forms, there should be three assessment sections. And so on. It's also helpful to title the section with the form. 

 

Step 4: Write Questions

Add questions and content directly to the assessment sections. You can use any question type available for these assessments.

1. Click + Add Question.

2. Select from any of the question types and click Add to Activity at the bottom of the page.

 

Step 5: Add Pool to Assessment Page and Set Selection Criteria

1. Return to the page on which you placed the assessment.
2. Click Edit Activity
3. Select Add Question from Pool
4. Select the pool that you created and select your selection strategy (how to draw).

The selection strategy allows you to choose how many items to select from the pool and in what order. It's important to understand each of the options, as different configurations of pool files will change how you choose those options. 

Choosing random (with or without replacement) means that different students can receive different forms. The system might choose Form 1 for the first student, Form 1 for the second student, and Form 2 for the third student. Choosing ordered means that the system will select each form in order from the pool. All students will receive Form 1, then Form 2, and so on.

Choosing the right count (Number of questions to draw each time) is critical. Because the assessment section now acts like a single item, you want to set the count to 1, because you want only a single form at a time. If you set it any higher, it will pull more than one assessment section.

______________________________

 

Highly Randomized Summative Assignments

Highly randomized forms are the most complex type of form to author. They provide the most randomization to students, no matter how many attempts you allow them. Questions are grouped into many separate pools and drawn at random so that all students receive an equal number of assessments, but with different combinations of questions. 

 

Ways to Create Highly Randomized Forms

There are several ways to randomize the assessments in terms of how you group questions. 

The most common way to group questions is by learning objective or skill. To use this method, you find the smallest learning component that you are assessing, create questions to assess it, and group them together in a pool. For example, if you wanted three highly randomized forms, you would write three parallel questions against each of the smallest learning components, and then place those three questions in their own pool file. Each form selects one of those three parallel questions for the assessment form. This method generates an assessment in which each question is randomly chosen from a group of equivalent questions. 

There are other ways to group questions. For example, you could write six parallel questions against a learning objective, put them in a pool, and select two randomly from the pool. Doing so will change the combination options for the selection. However, you must be sure that the questions you place in a pool file are equivalent. If you put questions for two different learning objectives in a single pool and select two of those questions randomly from the pool, you could receive two questions from the first learning objective and zero questions from the second. You would not be assessing students equally on the content or against one another. 

Another option is to create parallel question sets and place them in a pool. You might choose this method if you need to use a table or graphic and ask three scaffolding questions against that image. You would want those questions to stay together. If you create three sets of scaffolded questions that are parallel but different, you can place each set within an assessment section in a pool. Now your pool will select one of the three sets of questions. 

 

How to Create Highly Randomized Forms

The general steps are the same as Creating Multiple-Form Assessments: Pools & Forms, except there will be many more pool files attached to the assessment page.

These are the steps:

Step 1. Create an assessment page.

Step 2. Create a pool file.

Step 3. Create assessment sections. For assessments with highly randomized pools, you will repeat this process for every group of questions you have.

Step 4. Write questions. 

Step 5. Add pool to assessment page and set selection criteria. When choosing the Number of questions to draw each time, input the number of pools you want to choose from.