Qualifying good work: the rubric

The first time I ever NOTICED a teacher using a rubric for grading was my sophomore year of high school. I was working on a late assignment for my art class during a study hall (typical of me–I had no urgency with art; perfection took time as far as I was concerned) and my teacher was co-creating a rubric with her freshmen on how they would be assessed for their upcoming project. I had no idea what a rubric was then, and I don’t really have any recollection of being grading that way any time before that. (This was probably around 1995, if you were wondering.) I liked that it seemed more fair to have a break-down of what the teacher was looking for instead of just receiving a B+ at the end, especially if your project was complex… but apparently, I never noticed any of my other teachers thinking this way before that day.

Rubrics are great for communicating to learners what exactly is expected of them AND for keeping instructors consistent and focused in their assessment of student work. (Believe me, it is easy to get cranky and mark things harshly because you’ve had a bad day!) However, it sometimes takes a try or two to get a rubric that works and focuses on the most important parts of an assignment. Sometimes you realize too late that you focused too on one component and missed something else. I have redone rubrics mid-grading because I didn’t think they were fair or right.

Personally, I prefer to use rubrics in a holistic sense, where you “eye up” student performance instead of adding up points. The whole idea of a rubric is to qualify what good work is for a task that isn’t really quantifiable. After all, if it were quantifiable, you would be counting up the right and wrong answers and assigning a percent–so adding up points on a rubric doesn’t make sense to me. Besides, points get skewed easily (simply because you think, “Well, this wasn’t perfect” and circle the “good” column instead of “excellent”) and suddenly you realize that everyone is earning Cs–yet the majority of students actually did meet the objectives and achieved what was asked. Yet, the break-down of individual parts nitpicked their grades away.

As for other handy assessment tools for grading projects, I am fond of using self-assessment, where the learner fills out a rubric or survey on their own learning. I agree with the University of Minnesota’s analysis of grading systems that said learners may initially score themselves too generously if they are inexperienced with self-assessment or do not really know what you are looking for, but much of the data out there indicates that about 80% of self-assessment matches what the instructor thought. I have even noticed that sometimes kids are harder on themselves than I would be, especially if they had an especially challenging task.

Related confession: I have used self-assessment in the past to speed along my grading process so that I wouldn’t have to read/grade as closely… I know that’s bad, but sometimes we all go into survival mode. Eek!

Annotation: Badges for Higher-Ed Assessment

Buell, C. (2013, August 30). Using Badges to Quantify Learning Outcomes at UC Davis. Edcetera. Retrieved October 20, 2013, from http://edcetera.rafter.com/using-badges-to-quantify-learning-outcomes-at-uc-davis/

This article examines the use of badges to measure learning outcomes in higher education, especially as developed by UC Davis and Joanna Normoyle, who won an award for the innovation at the Digital Media and Learning Competition. The idea is to quantify and standardize higher-level thinking skills gained throughout the course of a university degree and award a digital badge for the achievement, potentially making them useful to future employers trying to determine the skill-set of a candidate or even simplifying the process of credit transfer between universities looking for equivalent coursework. Badges in such a system also can be useful for helping learners to track their progress and customize an academic program. UC Davis is officially launching their program with students this fall. By extension, I could see this as an easy, practical way for academic libraries to jump in, partner with departments and get involved in assessing and communicating the information literacy of students.

Setting assessment policies in the syllabus

Part two of the syllabus analysis I began earlier this week…

My sample syllabus (the culinary arts one) addressed assessment as follows:

Assessment Strategies Used: cooking labs in the kitchen, quizzes, tests, homework, and projects.

The instructor neglects to explain if they are using a total points system or are weighting grades based on category. There appear to be 16 course standards, divided into 2-3/unit, sometimes repeating in later units, over 12 units. I suspect that course grades are simply based on competency for each course standard, with each course standard being worth the same value in the final grade.

For example, for this course, Standard 1 is, “Demonstration of proper cooking techniques that result in a quality end product while employing safe and sanitary methods.” Standard 8 is, “Knowledge of food service equipment, including identification and use, mise en place, knife skills, and seasoning.” I am speculating here, but students might be asked to do assignments, tests, quizzes, etc. that give the instructor a piece of evidence/an article that reflects their learning as a competency for each standard–but that it would be at the instructor’s discretion, not pre-announced. The idea is to get kids to focus on learning, not just the acquisition of points.

The syllabus doesn’t actually confirm this thought though. It’s hard to say if it would be a fair assessment of student learning or if it’s qualitative/quantitative assessment without more information on how the gradebook is set up.

To me, removing the points-game is really important. The focus should be on learning, not jumping through hoops. And grades should reflect said learning, not ability to play the game. On the other hand, when students know how they are being held accountable, they have more ownership, which encourages the intrinsic motivation that we want learners to have and leads to real learning!

Duchess of the Kohl Center

My first experience with badges was through iPhone apps like Foursquare and Yelp. In fact, my first check in was February 20, 2011, which was probably about a week after I got my iPhone. I was at the Kohl Center for a Badger men’s basketball game. I can’t say I use the apps a lot, but I would have to say that I’ve checked in more than a few times at the Kohl Center… I like watching basketball, what can I say?

At one point last season (2011-2012), I was named “Duchess of the Kohl Center.” (I think I’ve checked in like 10 or 11 times.) The best badge ever, as far as I am concerned. I wish I had a t-shirt. Too bad being Duchess doesn’t give me the power to keep the old people who sit behind us from talking about colonics and stuff…

Anyway, I don’t think I still possess the title anymore because I think other people can steal them from you if you don’t keep checking in and I don’t go to the Kohl Center in the summer. Technically, it’s probably not even a badge either, but a mayorship or something. Whatever, though, I figure it’s like being President–they have to keep calling you “President” even after your term is up, right?

Needless to say, my experience with badges so far has been pretty silly. I knew a guy once whose mom made his Cub Scout Den change his baby brother’s diapers to earn their “Diaper Duty” badge (High-five to his mom!) At least that badge actually acknowledged a real accomplishment. My Foursquare badge “I’m on a boat” wasn’t exactly hard–I drove my car onto a ferry and then waited.

Deep down, we all like to show off our accomplishments. Badges on a sash, medals on a Varsity Letterman’s jacket, patches and pins on a military uniform. With the exception of the military honors though, these types of recognition mean nothing to job world. So when I read about badges as an alternative assessment tool, I was blown away. I’m sold. As it turns out, the Boy Scouts and Girl Scouts have known what they were doing all along! Maybe that cooking badge you earned as a scout doesn’t seem like much, but I bet for an aspiring chef, it might be quite a big deal to earn a badge for making a perfect crème brulée with a torch. In fact, a crème brulée badge might be currency someday for jobs in a high-scale restaurant.

Standards-based assessment has been all the rage in K-12 public schools lately, especially with the Common Core Standards coming down the road. It’s a major paradigm shift for a lot of teachers to change their grading scales and philosophies from “points” to “competencies.” It’s not because teachers really believe that 84% actually reflects the amount of mastery a student has made–for a lot of teachers, it’s more a question of how to enter a competency into the computer gradebook and have it still spit out a B+. Unfortunately, it is still expected that we use GPAs as a factor in college admissions or scholarships. Sometimes, I’ve felt like it’s futile to even try to change to a standards-based assessment system when higher-ed is going to insist that we boil a kid’s learning back down to a percentage anyway.

UC-Davis sees things differently, however. This year, they won an award for their development of digital “open badges” in the Digital Media and Learning Competition supported by the John D. and Catherine T. MacArthur Foundation and Mozilla.

Students will be able to customize learning goals within the larger curricular framework, integrate continuing peer and faculty feedback about their progress toward achieving those goals, and tailor the way badges and the metadata within them are displayed to the outside world. Students won’t just earn badges—they’ll build them, in an act of continuous learning. (Carey)

As Kevin Carey puts it in his article,  “A Future Full of Badges,” from the Chronicle of Higher Education, “Compared with the new open badge systems, the standard college transcript looks like a sad and archaic thing.” He’s right–years later, when you look back at a college transcript, that A- you earned in English 271 doesn’t say anything about you, what you learned or what you can do.

I started digging a little bit to find out more about how organizations can get involved and start implementing badges. Mozilla is really at the epicenter of the badges movement with their Open Badge Infrastructure, so that seemed like a logical place to start. It’s actually quite easy to create your virtual backpack as a learner who wants to start collecting and accepting badges. As someone who’s not very good at programming and customizing Open Source tools, like WordPress, the process of creating a badge seemed a bit over my head. I worry that unless I start working on that deficiency (Hey! Maybe there’s a badge out there I could earn…), I might not be very useful to a library as a tech-savvy young person after all.

Canton Public Library and Ann Arbor District Library‘s use of badges in their summer reading programs was incredible. (Thanks to Greg Landgraf for showcasing their programs in his article, “Summer Reading Levels Up!”) But again, I got to thinking, “Well, crap! Where on earth am I going to get the skill-set to help bring something like that to a library or school near me?!”

Librarygame is cool and seems to be ready-made (no skill-set necessary), but for something as experimental as the OBI movement, it seems a little risky to invest the money into quite yet–better to mess around with open source…

I continued my research though and came across a couple of points of light. BadgeStack is an open-source platform that organizations can use to launch a badge program. I actually think I can handle this one. I could at least play around with demo. I also ran into MouseSquad. It’s a 21st Century Skills Training system that prepares and supports students to establish their own technical support help desks in their schools. Basically, kids join and they can earn badges and receive the training they would need for a pretty meaningful project (all the while, they are stealthily being taught a bunch of information literacy!)

Now, this is something that I can advocate for–now! Standards-based assessment doesn’t look so daunting when it’s gamified. Yeah, yeah, there are arguments that gamification is really bad for motivation, but really, for a lot of students, once they hit high school, education is about playing the game and working the system. I like the idea of students pursuing their education by earning badges for achievements and skills that they deem valuable. Being Duchess of the Kohl Center might not be real useful, but Duchess of the Tech Support Desk could be!