Cognitive Daily

In education school, I was taught that the purpose of grading was to rank-order students — to create a system whereby the highest-achieving students were ranked at the top and the lowest-achieving students were at the bottom. But recently there have been worries that grade inflation is making it difficult to use grades to rank students.

At most Ivy League schools, nearly 50 percent of grades given are A or A minus. When dozens of students have perfect GPAs, how do you determine who is best? If the average GPA at a school is 3.4, then what’s the point of having a four-point scale: half the students are being ranked on the basis of 1/6 of the grading scale.

Worse, some professors give As to nearly every student, compounding the difficulty of determining who did best in their class. When other professors grade tougher, there’s a disincentive for students to take their class, no matter how worthy the subject matter.

Amy Perfors has identified a better system for calculating GPA, one which attempts to mitigate these problems:

The model, which is Bayesian, calculates “achievement index” scores for each student as latent variables that best explain the grade cutoffs for each class in the university. As a result, it captures several phenomena: (a) if a class is hard and full of very good students, then a high grade is more indicative of ability (and a low grade less indicative of lack of ability); (b) if a class is easy and full of poor students, then a high grade doesn’t mean much; (c) if a certain instructor always gives As then the grade isn’t that meaningful — though it’s more meaningful if the only people who take the class in the first place are the extremely bright, hard-working students. Your “achievement index” score thus reflects your actual grades as well as the difficulty level of the classes you have chosen.

Makes some sense. And, of course, ultimately when professors realize that the difficulty of their class is factored into students’ GPAs, it becomes in their best interest to grade tougher — that will be the way to attract the best students to their class, since getting an A in an easy class won’t do much to help a bright student’s GPA.

The program is currently under serious consideration at UNC-Chapel Hill, but it has aroused some controversy. (Perfors links to an article in the faculty newspaper about the controversy, but the link is broken — I think because the UNC site doesn’t support permanent links. I found the article using Google’s cache, and I’m reprinting the whole thing below the fold.)

Perfors can’t understand why students are objecting to the system, but I have one guess: Students believe they have figured out a way to work the system, so any change in the system will be viewed as harmful. If tried-and-true tactics such as taking easy classes to boost grades won’t work any more, then they’re concerned that they’ll fail under the new regime. It’s not really a fear of the new system, it’s simply fear of any change.

As promised, the UNC article is below.

Faculty members and students aired their questions and comments about the proposed Achievement Index, a statistical measure of student performance, at a special April 13 forum of the Faculty Council.

The forum was held to provide additional information before the council votes on the Educational Policy Committee’s proposal to use the Achievement Index as a way to address variations in grading practices across the University. The index, designed to supplement GPA, measures each undergraduate’s performance relative to his or her classmates. If adopted, it could be listed on a student’s transcript and used in designating honors of distinction and highest distinction.

Faculty Chair Joe Templeton welcomed the group of faculty members and students. “You’re here, which suggests a commitment to excellence in undergraduate education at Carolina. We’re all here because we want Carolina to be as good as it can possibly be,” he said. “It’s helpful to recognize that both as students and as faculty so that when we get down to the details of an achievement index — about which we might disagree violently — we’ll recognize that we’re all trying to get to the same point, although there might be different pathways to get there.”

A panel of six fielded questions and offered comments. Panelists were Beverly Foster, nursing, and chair of the Educational Policy Committee; Suzanne Gulledge, education; Peter Gordon, psychology, and chair of the committee’s Subcommittee on Grading; Mike Radionchenko, the committee’s undergraduate representative; Andrew Perrin, sociology, and committee member; and Bill Balthrop, communication studies.

Initial questions focused on the impetus for the Achievement Index and how it would address grade inflation and perceived inequities in grading.

“There has been a long-standing faculty discussion on grade inflation and an exploration of possible solutions,” Foster said. “The committee’s intent was to get an issue of concern to many — and one possible approach — on the table and out for discussion.”

The measure addresses both grade inflation and inequities in grading, which are inexorably linked, explained Perrin. “The index says that when we’re comparing students to each other, which we do anyway, we attempt to factor out differences in grading practices,” he said. “Our hope is actually to reduce the pressure on grades.”

Gordon explained that the index recognized the use of grades for different purposes and didn’t negate the value of standard-based evaluation. “The Achievement Index is complicated, and that’s unfortunate, but it stems from a desire to make sure the method doesn’t penalize a student who does well in a class in which a high percentage of students do well.”

Gulledge described the Achievement Index as a tool that should help correct some of the imperfections of GPA assessment. “My sense is that the index shines light in some murky places, because it invites us to think about teaching for mastery. It is not a panacea and to suggest that is a mis-intent of the proposal,” she said. “I would be opposed to it if we don’t have an oversight committee to look at the issue of competition or how grades can be poorly used. I’m not afraid to try it, but I don’t want it to be the sum total of our assessment.”

Students were concerned about the extent to which the index would foster competition among students and whether it would discourage students from taking courses in disciplines in which they thought they might not perform well.

Student representative Mike Radionchenko said the Achievement Index had the potential to change Carolina’s campus climate by providing a disincentive for students to work cooperatively.

Eve Carson, student body president, agreed. “It takes license from the students to control our academic performance because we would be evaluated not only by our personal performance but also by the performance of other students at Carolina,” she said. Currently, Carolina emphasizes a culture of encouragement and initiative, she explained, not cutthroat competition.

Students also were concerned about the unknown impact on their acceptance into graduate and professional schools and future employment. “Would you give your child a medicine if you didn’t know its side effects?” Radionchenko said.

An audience member asked about the impact on leakage in honors courses. “When Peter started down this path, I was skeptical, because the impact on honors courses was one of my concerns,” Balthrop said. “But the committee found compelling evidence that for students who take honors courses, the Achievement Index in fact works to their benefit.”

Others asked how the index would affect courses that were based on participation, such as music ensembles and dramatic arts classes, or on small collaborative seminars, such as first-year writing classes.

A more radical proposal than this one was considered at Duke University about 10 years ago, Perrin said. That proposal called for replacing GPA with an achievement-adjusted GPA, but the idea was rejected. “If we adopt the Achievement Index, it is crucial that we evaluate the impact on the campus culture within the next few years,” he said.

While the students were largely opposed to the new measure, faculty members were split in their support.

Bobbi Owen, dramatic art, described the formula as a little mysterious, but something that probably could be addressed through implementation.

Course selection, however, was a question mark. “Some people are concerned that there might be a perverse incentive toward the larger low-level courses rather than smaller high-level courses,” she said.

Alice Poehls, University registrar, raised the issue of cost in incorporating the index in the University’s new student database system, which already included a GPA calculation. “If we go with the Achievement Index in addition to GPA, we could either ask for expensive modifications to the software or we could run it as a side system with interfaces,” she said. “I hope we recognize that if we choose now to invest in the Achievement Index as a modification, we might preclude other modifications that might be beneficial to a broader student population.”

Gordon said that he was concerned about the issue of cost. “I don’t expect this to be implemented outside the registrar’s regular budget,” he said. “But the cost per grade or cost per student is miniscule compared to anything else we can do to affect how the University evaluates its students.”

Frank Wilson, orthopaedics, called the Achievement Index the best thing to come along in addressing grading problems in the last 30 years. “If we’re going to be the leading public university, we have to lead,” he said. “We can’t develop a single perfect answer to all the inequities involved in the GPA. Complex problems have complex solutions. This first step will get better with time.”

At its April 27 meeting, the council will vote whether to adopt the resolution. If the vote is positive, a special implementation task force will be appointed to study the feasibility of incorporating the Achievement Index at Carolina.

“A positive vote will endorse the concept of the Achievement Index but will not make it mandatory to put it into effect at any time,” said Council Secretary Joe Ferrell. “A negative vote will prevent it from going forward.”

For more information about the Achievement Index and how it is calculated, refer to www.unc.edu/faculty/faccoun.

Comments

#1 Josh
May 10, 2007

I went to a very small school (Simon’s Rock College of Bard) and the tougher the professor, the tougher it was to get a spot in their class. One professor, who would only give out A’s once or twice a year, had no problem filling his classrooms. I’m sad to hear that is not the case at the Ivy League schools.
#2 Jason
May 10, 2007

Grade inflation is a serious problem, both in high school and undergraduate education. Whatever happened to a “C” being “average”?
I clicked through to the “AI Primer” (http://www.unc.edu/~pcg/grading/AIPrimer.pdf), but I still don’t fully comprehend how the difficulty of a given class would be judged. I think it would be more straightforward to utilize professorial oversight to make sure that the teachers aren’t simply acting as a rubber stamp to the next level.
#3 RPM
May 10, 2007

A much simpler system is employed by some high schools: an A in an honors or AP class counts as 5 grade points (B is 4, C is 3, etc), while non-honors classes reward a 4 for an A, 3 for B, etc.
#4 sasha
May 10, 2007

The AI system is designed to fix grade inflation, so I want to know how big a problem grade inflation is and if an attempt to fix it could possibly create other (possibly worse) problems.

To what degree can professors determine ranking in a meaningful way? The problem with grade inflation is that there are so many A- students, some better than others, and the scale has shrunk too much to differentiate between them. But I wonder if a lot of those A- students actually display quite similar mastery of class material. I also wonder if an attempt to re-expand the grading scale would introduce more arbitrariness when trying to differentiate between those very similar students. It seems like that might be a particularly hard problem at Ivy League schools (or other top schools, like Davidson) where essentially the entire student body is filled with very smart achievers.
#5 Beren
May 10, 2007

I don’t think the system of using 5 points for honors courses helps much: it still ends up with students in an unnaturally compressed range of high grades. It also gives students who don’t have access to five point rated honors students a disadvantage, even if their actual course material was as difficult. (I ran into the problem, coming out of a private school that happened not to use that system. I didn’t even hear about it until I was in college.)

I think the problem of competing amongst your classmates is a valid point: at the very least, the sample size of a single class of 10-30 students is probably not enough to determine how difficult a class is. On the other hand, many courses are taught differently by different professors, and an individual professor will likely vary the course from year to year. It seems difficult to produce a large enough sample size to get a valid comparison between a student’s performance and the performance of an “average” student. What if you’re the fourth brightest student at the university, and the other three are in all of your classes? :p

On the other hand, if these problems could be solved and a student could be told that she is competing against a large sampling of the students, rather than her current classmates, that could probably go a long way toward defusing the social tensions.
#6 Ted
May 10, 2007

Is it unusual to expect an A for A-level work? Or does the standard for A-level work constantly change relative to the competitive pool?

What is the point of grades — to identify mastery or to rank order the participants?
#7 Jeff
May 10, 2007

Grades continue to be a terrible metric for ranking students, especially since students first learn how to get graded well and only later their subject at hand. Even a difficult course can be crammed into one’s short-term memory for testing purposes without absorbing any of its essence. Grades cannot possibly fairly compare one student to another, since their backgrounds, drives and resources are not comparable.

Can one include factors such as: Students who strive to take hard courses for them in order to learn faster (though they may then have lower grades in that class)? Students who overcome lesser backgrounds than their peers with better grades but who have overcome less? Family/relationship problems? Creative students who often are not as concerned with missing homeworks, but who will probably have more to contribute to the field later on? Etc.

I’ve seen no strong correlation between grades and how that student would be as a professional in their chosen field. And there is another thought: Most people do not work in the field of their chosen major, so often their grades in that subject are less relevant, but how does one tease out such information?

I’m against grades in college at all levels. Make comprehension and testing requirements tough, but pass/fail. Let students try as often as they like. When they’re done with their degree there is no student ranking. One must evaluate the student as they are when they interview toward their next step in life. It’s a shame we must find anonymous means of weeding out people before they have a chance to come into their own.
#8 Banerjee
May 10, 2007

If the aim of grades is identify the very best and the very worst students, all you have to do is to use a larger scale with finer distinctions and include a median grade of the class. That’s quite easy to do in the sciences and engineering. For example, one could get grade points ranging from 1 to 100. If the median is 99 we know it’s probably an easy class etc.

I feel grades are meaningless as far as the students in middle of the distribution are concerned. The very best students should be identified by other measures than grades and grades should be used only to find the absolute slackers.
#9 roseindigo
May 10, 2007

Jeff made an interesting comment when he said: “Creative students who often are not as concerned with missing homework, but who will probably have more to contribute to the field later on?” — Creative people are often considered “difficult” because they don’t follow the rules of the majority and can therefore be quite an irritation to a professor with a large ego; however, when creative solutions are needed to problems creative nonconformists are absolutely necessary. Neither the educational system nor the corporate system seem to have a good grasp on that. What is rewarded instead, is the parroting back of already known information instead of coming up with new solutions. I think that’s probably fine while a young person is learning the basics, but by the time of university studies there should be room for new ideas and concepts and creativity. I also agree with Jeff that the “weeding out” orocess before some people come into their own is a futile effort, especially for those who have very creative minds. Think of Einstein and Edison, neither of whom had particularly good grades.

On the other hand, the current politically correct system of grading so that everyone “feels good about themselves” is a wash-out. I’ve seen graduate papers that received an “A” for which I would have given at the most a “C”, and then the whole concept of reaching for higher attainment falls flat on its face because an “A”, even if not truly earned, is “good enough”.

Not sure how to measure this sort of thing, because no matter what system is instituted, somehow the truly creative genius is often shunted aside just when our society needs creative answers.
#10 Celeste
May 10, 2007

Grading on a pass/fail basis, as some small colleges do makes it difficult when they are competing for a place in a graduate program against students with high GPAs.

As far as highschools using a 5point system for honors classes, that is mostly helpful in making sure the highschool’s top 10 students are those taking AP and honors courses and not just getting straight As in regular-levels. Also the transcript includes the ‘unweighted GPA’ and a list of courses taken, which helps to even the playing field for college applicants whose schools didn’t use the 5 point system. But hurts students who didn’t even have access to AP courses.

The line between grading a student on achievement, and grading a student against their classmates is difficult. The fact that some courses are simply easier is well defined by the 100-level to 400-level system in place. Yes, some students try to take as many low-level courses as allowed to boost their GPA, which I don’t see as a large problem, because those aren’t usually the top students. The brightist students are taking difficult courses and difficult professors because they want to learn the most useful material and they are doing well in them. These students are allow persuing double majors and minors that will make them stand out on graduate school and job applications. I have found that required general studies courses have the easiest grading, but since everyone takes them, it has few effects in a students ranking.

The proposal is interesting and I would be interested in seeing how it develops, but from my experiance as a student at Berea College, I do not see a need for it.
#11 Eugene Ray
May 10, 2007

Grade system (pretty much the same all over the world) fails to produce any meaningful result because it is meaningless. It is same as to judge quality of the produced work by the number of the written words.
Firstly, achievement itself can vary a lot, the world talent is not a recent invention. And talented students put less effort to achieve the same goal as the rest. Which brings us to the second problem, judging effort. How can we do it excluding student’s self-estimate? Which shows the next problem – honesty. We can probably test truthfulness of a person in general, but this brings us back to the talent (to lie) and finishes the circle, or rather a spiral and there is no way out, just up or down.

It’s all happening because system of gradation mainly unrealistic. If we face a challenge in life we either overcome it or not, half-efforts don’t count (just imagine someone being half-healthy, half-not-obese or half-conscious). If we can’t split it in two, how there can be 5 grades or a hundred?

Why grade should be any different than pass/fail?
#12 Lord
May 10, 2007

What? There is something wrong with the current system of tossing finals down the stairs and grading them on where they land?

that will be the way to attract the best students to their class, since getting an A in an easy class won’t do much to help a bright student’s GPA.

If they are there for a reason other than learning, it is misguided indeed, and grades have nothing to do with learning and everything to do with ranking.

There are schools which don’t even credit grades below B towards a degree. How’s that for inflation?
#13 Michael
May 10, 2007

So, instead of grading each class on a curve, you curve the entire grade system?

Under the new system, if you can make the rest of the people in your class do badly, then the class looks harder and your grade in it will look better. If the class is curved, you’re already motivated to make your classmates do badly, though, since a low average increases your relative grade.

I think the net effect will be very positive. Currently, to game the system you take classes that tend to give As, and there’s a disincentive to take hard classes. Under the new system there would be a noticeable incentive to take the hardest class you can get an A in. Obviously a lot of lazy students are going to scream bloody murder, you’re forcing them to take harder classes to maintain their standing.

I don’t think that grading should be intended to rank order students, precisely. Grading should test mastery. The problem is that designing a rigoroous, difficult exam that also provides the proper average and grade distribution is really hard. I’ve seen professors fail to do it often enough to know. So then you have to rely on curving the class… at which point you’re just rank ordering. I don’t see any real solution to that, and i worry about the interactions with this proposed GPA weighting scheme. The number of As can have very little to do with the difficulty of the material, if the professor simply decides that 5% of his class will get As as opposed to 10 or 20%.
#14 Nisarg Kothari
May 10, 2007

I don’t think the complex body of classes a student takes should even be reduced to a single number. Far too much information is lost. For example, under traditional GPA systems, a student who takes honors math team in addition to 7 AP/IB level classes is actually _penalized_ compared to his peer who took only 7 AP level classes, even though the first student did unconditionally better than the second (yes I’m bitter ). Furthermore, at the highest level of the scale GPA differences are due to random events like students who transfer from other districts and have to take different classes or students who deliberately choose classes to maximize GPA rather than take what they are good at / interested in. People are really only interested in ranking the very top of the scale, the place where ranks make the least sense. I think a student’s report card is digestible and at the same time informative enough to count anywhere GPA would now.
#15 roseindigo
May 12, 2007

Can’t say that I blame students for taking “easy” classes so their GPA is higher. My brother had a major in physics and a minor in math, and worked his butt off. When he went out into the corporate world no one ever asked him what his degree was for; they only wanted to know if he had a degree (any would do), his grade point average, and from which university. In fact, he frankly stated if he had known then what he learned after he entered the corporate world, he would have taken all the easy classes just to get a degree, and be done with it, without working his tail off. Of course, when one enters certain professions, such as medicine, it’s a different ball game, but for most people who enter the corporate work world working your butt off for a degree seems a bit futile, so why not take the easy route? There isn’t any incentive to take the more difficult courses or the more difficult professors for because most employers never even ask such a question.
#16 Ted
May 14, 2007

There isn’t any incentive to take the more difficult courses or the more difficult professors for because most employers never even ask such a question.

In a globalized economy the stability and value of a degree is diminished, particularly of a ticket-punching degree designed for near term employment. Whether people realize it (some do) or not, there are people that attempt to maximize their education for themselves and there are people that view the education as a stepping stone to a job (which is fine because if we didn’t consider employment we’d all be art history majors).

The value of consciously taking difficult courses is that you take them for yourself, not for your employer. The fact that your brother worked his ass off is exemplary, even if he didn’t realize he was doing it for the right reason. Give it 20 years then go back and ask if he still feels the same way.
#17 roseindigo
May 14, 2007

Sorry Ted, but my brother is now retired. Maybe things have changed in this global economy. I have no idea since I’m also retired and don’t have to deal with that anymore. I do believe that young people have a harder time of it out there in the corporate world than we did, even if they have an easier time in school because the courses are less difficult. At least most of us had job security, but I see how hard my children are working just to stay even these days, sometimes even working while they are on vacation because they are tethered to their companies by cell phones and computers and it’s expected. When we were “OFF” we were really off and were free to enjoy ourselves. Corporations expect a form of slavery these days, and I think it’s the pits.
#18 Ann Nunnally
June 12, 2007

When my son started high school, I got a little shock. I believe his school practices grade deflation. He is in the 9th grade. In order to get a 4.0 A in a class, he has to have a 97 average in the class. If the class is an honors class, he can get a 4.5 for the 97. If the class is a pre-AP or pre-IB class, he can get a 5.0 for the 97.

For a 93 to a 96, he gets a 3.8. For a 91 to a 92, he gets a 3.6. And so on. GPAs are so complicated, the school has to calculate them by computer and hand them out each semester. By the time this kid gets to college, he will have a PhD in working a GPA system.