Setting Up Rubrics and Rating Scales

Performance-based evaluation instruments use rubrics and rating scales to provide standardized scoring across observers. This guide covers setting up rubric markers, rating scales, and linking them to evaluation questions.

Understanding Rubrics vs Traditional Forms

Traditional forms present questions in a linear list format. Rubric forms organize questions in a matrix where:

• Rows represent individual indicators or questions
• Columns represent performance levels (e.g., Ineffective, Developing, Effective, Highly Effective)
• Cells contain specific descriptors for each performance level

Rubrics provide clearer performance expectations and more consistent scoring between observers.

Setting Up Rubric Sections

Sections support three display formats: None (standard list), Rubric, and Matrix. To use a rubric layout, set the section’s display format to Rubric.

Converting Sections to Rubric Format

1. Navigate to System Admin → Observe → Sections
2. Select the section you want to configure as a rubric
3. Click Display Format
4. Select Rubric from the display format options
5. Click Update Display Format

The system will convert your section layout while preserving existing questions and answers.

Creating New Rubric Sections

1. Navigate to System Admin → Observe → Sections
2. Click Add Section
3. Enter section details:
   • Name: Section heading (e.g., “Classroom Environment”)
   • Description: Optional explanatory text
   • Display Format: Select Rubric
   • Scorable: Enable if this section contributes to overall scores
4. Click Save

Creating Rubric Markers

Rubric markers define the performance levels that appear as columns (or rows) in your rubric matrix. They are defined at the section level — each rubric section has its own set of markers.

Adding Performance Levels

Required role: system_administrator

1. Navigate to your rubric section
2. Click Add Rubric Marker
3. Configure the performance level:
   • Name: Performance level name (e.g., “Highly Effective”)
   • Description: Optional detailed description
   • Value: Numeric score for this level
   • Display Order: Position in the rubric (lower numbers appear first)
4. Click Save

Repeat this process for each performance level. Common rubric structures include:

• 4-Point Scale: Ineffective (1), Developing (2), Effective (3), Highly Effective (4)
• 3-Point Scale: Beginning (1), Proficient (2), Advanced (3)
• 5-Point Scale: Poor (1), Fair (2), Good (3), Very Good (4), Excellent (5)

Managing Rubric Marker Order

Use the Display Order field to control the sequence of performance levels:

• Lower numbers appear first (leftmost for horizontal rubrics)
• Higher numbers appear last (rightmost for horizontal rubrics)
• Leave gaps (e.g., 10, 20, 30) to allow easy insertion of new levels

Linking Questions to Rubric Markers

Once you’ve created rubric markers, link specific answer choices to performance levels. A section is considered fully configured when all question choices have a corresponding rubric marker assigned.

Connecting Answer Choices to Performance Levels

1. Navigate to your rubric section
2. Click Rubric to open the rubric configuration view
3. For each question’s answer choices, select the corresponding Rubric Marker
4. The system will automatically position the choice under the correct performance level

Scoring Eligibility

A form question is scorable in a rubric context when:

• The question is not a countable type, and
• Either the answer choices have numeric values, or the question’s choices are linked to rubric markers that have values assigned

The system checks these conditions automatically — no manual scoring flag is required.

Setting Up Rating Scales

Rating scales convert numeric scores into meaningful descriptive ratings (e.g., “Meets Expectations”). They can be attached to form sets or to individual sections.

Creating Rating Scales

Required role: system_administrator

Rating scales can be applied at two levels:

Form Set Level Rating Scales

1. Navigate to System Admin → Observe → Form Sets
2. Select your form set
3. Click Add Rating Scale
4. Configure the rating scale:
   • Name: Scale name (e.g., “Overall Performance Scale”)
   • Display Order: Controls the order among multiple scales
   • Active: Enable for use
5. Save the rating scale, then add point ranges (see below)

Section Level Rating Scales

1. Navigate to your section
2. Click Add Rating Scale
3. Configure scale properties
4. Define point ranges

Configuring Point Ranges

For each rating level in your scale:

1. Click Add Point Range
2. Set the scoring parameters:
   • Name: Rating label (e.g., “Meets Expectations”)
   • Minimum Score (minimum_points): Lowest score for this rating (must be ≥ 0)
   • Maximum Score (maximum_points): Highest score for this rating (must be ≥ 0)
   • Display Order: Order among point ranges
3. Save the point range

Important: The system validates that point range minimums and maximums stay within the actual points possible for the scalable object (form set or section). The system will reject ranges that fall outside these bounds.

Example 4-Point Rating Scale:

• Does Not Meet (1.0 - 1.9)
• Approaching (2.0 - 2.9)
• Meets Expectations (3.0 - 3.4)
• Exceeds Expectations (3.5 - 4.0)

Primary Rating Scale Selection

Form sets can have multiple rating scales, but one can be designated as the primary scale for overall evaluation scoring:

1. Navigate to form set rating scales
2. Select the rating scale you want as primary
3. Set it as the Primary Rating Scale
4. Save changes

The primary rating scale determines how overall form set scores are converted to performance ratings in reports.

Weighted Scoring

The system supports weighted scoring at multiple levels: form set → form → section → question.

Weighting Rules

• If any scorable item at a level has a weighted value, all scorable items at that level must have a weighted value
• All weighted values at the same level must add up to 1.0
• Non-scorable items cannot have weighted values (the system clears them automatically)
• Weighted values must be greater than 0 and no greater than 1

Adjustment Factors

When an evaluator does not answer all scored questions, the system automatically applies an adjustment factor so that unanswered questions do not unfairly lower the score. This applies at both the section and form level.

Best Practices

Performance Level Design

• Use clear, actionable language in rubric markers
• Ensure distinct performance levels — avoid overlapping descriptions
• Align to your evaluation standards and district expectations
• Test with observers before full implementation

Scoring Configuration

• Set appropriate point values that reflect performance differences
• Use consistent scales across related form sets
• Consider weighting different sections based on importance
• Validate rating scale ranges — the system enforces that ranges stay within actual points possible

Question Organization

• Group related indicators within sections
• Use parallel structure in question phrasing
• Ensure observable behaviors rather than subjective judgments
• Provide clear performance descriptors for each rubric level

Rubrics provide structured, consistent evaluation while rating scales translate scores into meaningful performance feedback for professional growth.