The details of this chart are less important than the process of creating it. After playing/exploring with online and/or computer based assessments, reading about or watching while teachers and students use online assessments, and taking high stakes assessments ourselves as if we were students, get together with a few other educators and fill out your own chart. Here's a blank chart we give out as a part of a Think-Pair-Share. You might want to divide it into sections and consider the affordances and constraints by user (teacher/student/special needs student/administrator), use (reading/word processing/movie making/note taking/etc.), subject, taxonomy (Bloom/SAMR/etc.), etc. Hopefully you'll revise the chart as you use _ _ _ in a wider variety of ways. This can definitely be combined with ideas of balancing technology, content and pedagogy. (Check out this podcast on TPaCK and SAMR.)

Affordances
Constraints
  • Some environments on some devices have accessibility features which may level the playing field for some learners (i.e. ability to have test read to the student)
  • Assessments can be graded quickly by the computer, saving time for the instructor and providing more immediate feedback to students.
  • It can be easy to share online assessments, questions between classes or teachers
  • Questions can include multimedia
  • Data should be available more readily to inform instruction, but...
  • Multimedia (sound, animation, enahanced user interction, etc.) can be included.
  • Can be adaptive and individualized - branching capabilities, accessibility features, in test feedback, ability to retake (with or without penalty)
  • Familiarity with device
  • Familiarity with assessment environment
  • Ability to review questions/answers can be turned on or off
  • Do students have the necessary typing skills? (Did they have the necessary handwriting skills?)
  • Each tool is quite different
  • Tools crash/freeze
  • Creating assessments can be more confusing or tedious with some tools
  • Observationally, students don't take online test as seriously (especially math, where they would normally work out a problem)
  • it may only be in theory that data is available to inform instruction
  • Some "innovative items are likely to contribute more "artifactual" than valid measurement (Scheuermnn, 2009)
  • Too many possible tools for a teacher to individually evaluate & "free" ones keep changing
  • How is student privacy dealt with?
  • If each teacher uses a different tool or tools it places a high cognitive demand on the students

Questions

  • How can teachers and students benefit from the affordances of online or computer based assessment while mitigating the constraints?
  • Do these assessments benefit all learners equally? Are some students disenfranchised? Are they the same as were disenfranchised by more traditional methods?
  • How should a teacher, student or district handle it when an assessment tool fails in mid assessment? What if it only fails for a portion of the students? Are the answers different if this is for formative assessment or competency assessment when students can retake it?
  • How does the students' prior familiarity with the tool (computer, tablet, phone, etc.) as well as their familiarity (or lack thereof) with the assessment environment affect their scores? Most students are already familiar with multiple choice tests, bubble sheets, etc.
  • Do students have the typing and computer skills required to complete the assessments?
  • Do some teachers falsely believe that using electronic assessments makes their assessments "21st century" assessments? A multiple choice question is still a multiple choice question. But is a handwritten essay the same as a word processed essay?
    • What does the research say on students writing verses typing (in terms of retention, metacognition, reflection)?
  • Do the tools encourage/discourage cheating through the internet? If they are locked down, do they inhibit accessibility tools?
  • Do we have enough working devices? Do we have enough bandwidth? (A few days ago I had little internet access and many students lost their work due to internal updates.)
  • Which devices/environments should a school/district recommend? Which should be avoided?
  • How is student privacy dealt with?


Recommendations

  • Use tools first for ungraded practice tests. Increases student comfort and familiarity. Teacher gains perspective on each tools affordances and constraints compared to traditional assessment tools as well as other online/computer tools.
  • School/district should pilot each tool before teachers use them for high stakes assessments (HS classes)

Resources


Tools To Examine

  • Blackboard or Moodle
  • Schoology
  • Common Core environment