Alternatives to grades: XP, currency, and honorable mentions

I have already covered how grades are a terrible system of assessment, and how a mastery-based system using skills trees for illustrating and enabling progress is superior. There is more to be written on mastery based learning, certainly, and I am likely to come back to that at some point. But first I want to cover some other assessment techniques where technology can help you - the teacher - and your students. I will only briefly cover them, and I might come back to tackle them more thoroughly later.

Experience points and class currency

Image credit: Technoccult.net

XP - experience points - is something most kids and many adults understand very well these days. They are a staple of video games, beautifully and simplistically illustrating progress and giving immediate positive feedback when deserved. In its most simple version, it is simply an open-ended points scale, ever-increasing as the player (student) performs actions and demonstrate skills that fit the goals of the game (lesson/class/subject). In this form, it provides nothing more than a simple Skinner Box mechanism, providing a rapid-fire external motivating force for learning (ideally).

This, of course, is neither an effective nor an efficient use of XP in a learning context. If you look at the games available out there today, only the most simplistic of them use this form of points system. The reason is simple; there is no way for the system to reflect any complexity beyond "good, keep going". This is why it works for games like Flappy Bird, which consists entirely of tapping the screen to flap a bird's wings to pass obstacles, but it would likely not work for even minimally more complex games. So what would be a better approach?

When the game becomes more complex, the scoring system should reflect that complexity. Tiered rewards, collective rewards, level systems (0-999XP = level 1, 1000-1999XP = level 2 and so on) and hidden bonuses are some mechanics you can use to improve a basic XP system. I'm a particularly big fan of collective rewards, because they enable you to use individual and class academic achievements as a tool for managing collective student behaviour. Another mechanic I'm a big fan of is the concept of spendable XP, or "class currency". This could be used to reward pro-social behaviour, cooperation, achievements (both individual and collective) and so on. Three possible outlines: Individual savings only, individual and collective savings, individual savings with collective buy-in rewards. It is important to remember that this is a type of mechanism, not a static system. It is very customisable.

Keeping track of all of these points could be a challenge, but it would be a very simple matter to set up a computer system to do it. You could do it in Excel, for example, and it wouldn't be particularly hard. Check back here again for an example worksheet I have planned to make for a while...

Prof. Lee Sheldon's simple XP scale with levels matching grades.

One final point about XP is that it is potentially very easy to implement. To some extent, most teachers already use a points system in their assessments. At the end of an assessment, this point score is translated into a grade. As I have already covered, this is a dumb idea. In the simplest XP implementation, all you would have to do is NOT convert to a grade and just add this point score to the student's XP - which would just be the sum of all points from all assessments.

This type of assessment has been implemented successfully by teachers in both simple and complex implementations. Many teachers also decide to separate their XP scales from their grading, which does present some significant benefits (as mentioned in the link to the complex implementation above). Keep in mind that many - maybe even most - teachers are forced still to use grades for their final assessments, regardless of their flaws, and any alternative system that isn't adopted on a regional or national level would then have to accommodate for that. It is unfortunate, but it is the reality.

E-portfolios, checklists, iterative grading

These three approaches are often championed by edtech evangelists like myself, and while I do see the benefit for some subjects I don't think they are as widely applicable as many thinks. Moreover, I don't thinkthey are good enough to compete with gamification approaches. There is probably a good case to be made for implementing one of them in combination with some level of gamification, however.

E-portfolios

An e-portfolio, or an electronic portfolio, is an electronically kept portfolio of a student's work. This could be written work, tests, presentations, multimedia content (video, audio, pictures, multimodal texts), hyperlinks, and whatever other work may be relevant to the class/subject. Most LMS software supports student e-portfolios, some are even geared specifically at this form of assessment (see Digication). Assessment of e-portfolios vary, but usually the student will produce more work than is required for assessment and then is allowed to choose what work is used for the assessment.

Immediately, my problem with the concept is apparent: This is not a replacement for grading, it is merely a way of - arguably - improving it by making it more transparent and goal-oriented. But it still carries the inherent problems of whatever final assessment method is used, and that method is usually grading.

Checklists

Checklist assessment can be used for self- and peer assessment, and for clearly showing to your students what exactly they are expected to learn about any given subject/in any given class. I'm generally very much in favour of using checklists in your assessments, regardless of what exact method of final assessment you use. They make it easier to track your students' progress, they help your students get a sense of direction and purpose in their studying, and they help enable peer assessment - which I will almost always be in favour of. However, there is still the problem of checklists not really being a solution for final assessment - unless your school belongs to a system that actually assesses according to learning goals rather than curriculum, in which case the "top layer" of checklist could simply be the learning goals of the given subject.

Still there would be the problem of checklists being black and white, and reality being... well, not. A filled checklist can be thought of as a car; if all the boxes are ticked, you know you have a car, but you have no idea if it's a well used Skoda or a brand new Ferrari. Checklists fill in some of the information gaps that regular grades are missing, but at the same time they lack the quality information that grades actually do - to some extent - contain. This could be fixed by gamification measures, for example a set of Quality Badges (possibly tiered) for each checklist.

Iterative grading

Iterative grading is better than regular grading, in that the process in itself actually teaches some valuable lessons. A student hands in a piece of work, that piece of work is graded and other feedback is given. The student then takes this feedback and is allowed to revise the work before handing it back in. This could repeat a few times, until either the teacher accepts it, says it's enough, or the student decides to not revise it again. Using collaborative writing platforms like Google Docs makes this sort of revision system a simple thing to do, but it applies to any type of revisible work. In the end, however, this method still suffers from most of the issues with regular grades.

There is one part of this method that I think is well worth keeping in mind: Comprehensive feedback. It is a core part of iterative grading that the work is commented on thoroughly so that the student knows what to improve and how, and I honestly believe that these comments are far more valuable and informative than the grades are.

My next post will be a quick summary in the form of an outline of a hypothetical classroom employing some of the mechanisms I have covered. Expect the posts sooner rather than later, since I missed my scheduled post last week!

Alternatives to grades: Skills trees

Last week, I went over a few reasons why traditional grading is a terrible system. It fails on its most crucial aims: Indication of level, progress and skill, and motivation for progress. There are better systems already out there, and beginning this week I will cover some of the alternatives.

Skills trees

Breaking down each subject into distinct parts - for example, maths into Basic Division, Basic Addition, Division 1-3, Addition 1-3, Advanced Division etc - is not a particularly new idea. In fact, it is commonly done already in most schools around the world. It's just that this isn't really given much weight outside of the curriculum itself, and the way it's done leaves a lot of students lagging. Currently, subjects are taught by time periods. If you haven't mastered basic division in the time the whole class is given - say, a week - then tough luck, next week the subject will be different.

This is a shame, because it loses track of what the point of this breaking down of subjects is all about: Actually teaching the students those skills. Instead, the focus falls on lecturing the students on those skills, and then testing them for what they've learned. Rarely does the teacher get an opportunity to revisit what they know the students haven't learned.

There are schools out there who are using a different approach, however. It's called the mastery based approach to learning, and it tends to necessitate this sort of breaking down subjects into skills and levels. Khan Academy's knowledge map is an excellent illustration of how this could work.

Khan Academy's Knowledge Map

The map is a skills tree, a simple tree structure consisting of nodes and edges. Completing a node indicates mastery of the skill taught in that node, and edges open to other nodes the student now qualifies for. Completion is unlocked through successfully solving a set number of problems within each node in a row without asking for hints or help. In other words, once a student has mastered a skill, they can move on to work on the next one.

It is a way of thinking that is very familiar to many students, who see this way of structuring progress in the video games that they play. It also lends itself very easily to certain ways of rewarding and motivating progress, particularly those that belong to the category of "gamification" techniques. This is a topic I will return to frequently in this series of blogs, as it is highly relevant and more than worth mentioning.

Another advantage of using skills trees to "grade" progress is that they give a far more complete picture of the students' skill levels. Just by looking at the map, you know what the student knows. An advantage not at all shared with traditional grading.

This type of progress indicator is not without its problems, however. More abstract and less discrete subjects, where the students' skills and knowledge are not as easy to delineate, can be problematic to map on a skills tree.

Problematic, but not impossible.

Most subjects have parts of it that can be delineated pretty clearly. In English, for example, grammar is such a part. It is clearly structured, and can be broken down into sub-parts just like maths. Writing, on the other hand, is different, and it can be argued that it would be difficult to break it down in the same way as maths or grammar. But with a little thought you can get close. All teachers know this, because they have to do it when grading: Break a text down into structure, arguments, creativity and so on. Though you can do this, it is important to keep in mind that you should not think of mastery or skills trees as any sort of silver bullet for assessment. Differing situations may call for differing methods. But it is certainly one more bullet for your chamber.

Grades Are Terrible

Neil deGrasse Tyson nails it, as per usual.

I've been planning this post for a while, but as I've been researching and writing it I realised there was way more to it than what would fit in a single post. So I've decided to split it up, and cover different aspects of the issue in separate posts. This one is about the fundamental problems with classic grading practices, and a short introduction to how tech could begin to solve those problems.

One of the two primary problems with grades is that they are fundamentally demotivating. In the best case scenario, a student will start out with an A (or its local equivalent; in Norway it would be a 6). The best outcome from the next grading would then be another A. Can you spot the progression here? No? Good, because there is no clear progression. Going from an A to an A is in reality an indicator of excellent progress, but it is illustrated through stagnation. The worst case is a student that starts out at an F level, and never goes much beyond that (if at all). They will possibly get some E's (or D's, depending on the system) from teachers who want to give them a bit of motivation, but this will probably demotivate further. Basically, any slight improvement runs the risk of being seen as "pity points" and any dramatic improvements are in danger of being shrugged off as anomalous.

This is a terrible state of affairs. I don't think it's a stretch to declare that students learn best when they are motivated to do so, and from this it follows that we need a feedback system that actively motivates learners. Traditional grades clearly don't. But as I said, this is just one of the two primary problems with them. The other? Grades just don't do what they are supposed to do.

The idea behind grades, if you distill it down to its core, is that they should be indicators of skill and knowledge at the end of a term, and progress during the term. I've already covered how they don't show progress, but to the outside world - outside of school - the indication of skill and knowledge is the primary function of the grade. An A student should be an excellent student. A B student should be a great student who is probably hard working (or in some cases an excellent student who is maybe a bit lazy). But realistically, grades tell you nothing about this. In a perfect, standardised grading system, a grade will tell you a student's level relative to a set of goals. It says nothing about work ethic, intelligence, learning ability, specific skills or strengths and weaknesses in the field to which the grade applies. Grades are an oversimplified metric of ability only understood in the context in which the grade was determined - information not always available or understood by those who read them - and even then they are not particularly useful.

Grades demotivate, and they don't fulfil their function. Why do we still insist on using them? In a word: Tradition. We've "always" used grades, and tradtionally they've been simple to both determine and read. They are less so now, but it is difficult to change tradition. Particularly in areas where teachers are actually required by law to use grades for feedback. Grades are, and have been for a very long time, all but synonymous with education.

All of this is hugely problematic. When a core component of a system designed to educate is both actively hindering learning and making learning outcomes opaque while pretending it isn't, that represents a fundamental flaw that permeates the entire system. The worst thing is that it is completely unnecessary, because we have a lot of systems for feedback and progress tracking that are inherently motivating and far more useful by pretty much any given metric. And most of these systems are found in video games.

In the next post, which should be out by the end of the week or before, I will cover some possible solutions to these problems. And big surprise: Technology plays a huge part.

Duolingo Versus One-on-One Tutoring

Luis von Ahn, CEO and Co-founder of Duolingo

I recently found a video in my Youtube subscription feed that I thought, from the title and description, would be brilliant. The video was from Big Think, and it was titled “The Future of One-on-One Education, with Duolingo’s Luis von Ahn”. I know a great deal about von Ahn, and about Duolingo, as I researched them both during my technology in education studies. At the time, Duolingo had just been announced but not yet launched, and von Ahn was mostly known for his invention of the Captcha system for distinguishing actual people from machines and algorithms.

The idea for Duolingo is strongly linked to his Captcha project. In the TEDx talk at Carnegie Mellon University where he announced the service, he talked about how he felt bad about having wasted “500,000 hours every day” of humanity’s time by making people solve 200 million captchas per day. So with the next version of the Captcha system, later dubbed re-Captcha, he set about to solve that. And the solution was ingenious: With re-Captcha, the solutions that people give help computers digitise texts by, essentially, putting unresolvable snippets of text up to a “vote”. A re-Captcha has two parts; one word picture that is known, and one that is not. The user doesn’t know which is which, so they have to solve both. Get the known one right, and you’re human – and by guessing on the unknown one, you’re helping computers interpret it.

Duolingo works along similar lines of reasoning. With it, von Ahn wants to “translate the web”. Doing this in the traditional way – by hiring professional language translators – would be prohibitively expensive. But von Ahn realised that there are millions, probably billions, of people out there who are actively trying to learn a second (or third etc.) language. He cites a figure of 1.2 billion people. Many of them are willing to spend a lot of money on software to help them achieve their goals, with over 5 million Americans having paid over $500 USD for common software solutions like Rosetta Stone. With Duolingo, von Ahn believes he has found a better way. The idea is that the translation parts – the “work” that they want the translators to do – is woven into a learning experience. With Duolingo, the users learn a language and the company gets paid by having the learners perform translation work for them.

From a language learning perspective, Duolingo is an excellent tool. It is heavily gamified, which I like very much, and it has been shown to be very effective. As is mentioned in the Big Think video, there are studies that show Duolingo is “as good as a classroom”. This isn’t particularly surprising, for several reasons: First, because classroom teaching tends toward ‘teaching to the middle’, with the strongest and the weakest students left behind. This is a natural consequence of trying to fit X students to a single set of lessons. It is also why all good language teachers will tell you that the most important work you can do to improve is done outside the classroom. This, in my opinion, is where Duolingo shows its value. However, Luis von Ahn wants it to be more than that.

It is well established that one-on-one tutoring is dramatically superior to classroom learning. The famous 2-Sigma studies he mentions early in the video are real, and clear proof of this. It also just makes sense, rationally: One-on-one tutoring gets rid of the teach-to-the-middle problem, because the single student is the middle. In von Ahn’s vision, Duolingo solves the scalability problem with one-on-one tutoring by making the service as good as the human-to-human variant. This is a great idea, and a good goal to have. There are a few problems with the idea, though, and the biggest problem with it, is that it’s never going to happen. Let me explain why.

Machines can do a lot of things, and they will certainly take over a lot of our jobs even in the relatively short term. However, they cannot replace the human element – and that is something that language learning requires. Language exists solely to facilitate communication between interpreting entitites, normally between humans. The properties of the language used changes based on the communicating participants. There are too many factors to count: Shared language basis, culture, gender, geography, relative age, context… the list goes on. And this is why human language teachers have an advantage that no computer is ever likely to have (until they, too, become “human”): They adapt to these factors instinctively and continuously in the same way as the student does, and can adapt their teaching to fit.

There is great learning potential in interaction, particularly in language learning. As a language teacher, I can tell you that it is exceptionally easy to spot the difference between a learner who interacts frequently in the L2 (the target language) and one who does not. Even if they are otherwise of similar levels, even if the one who does not use the L2 much has otherwise excellent learning strategies, the one that interacts is likely to have a much better progression than the other. And this is what makes the very idea of software replacing the human interaction element in language learning a no-go.

I am a big proponent of embracing technology and technological solutions in most teaching environments. Language learning is not an exception. However, I also feel it is both dangerous and counter-productive to think of any technology as a panacea. I have used, and will continue to use, Duolingo in my teaching and in my personal language learning endeavours. The benefits are great and many, and it is hugely effective as a language learning tool. But a replacement for human interaction it is not.