Worst. Evaluation Scheme. EVER.

Speaking of teacher evaluation schemes, as we were, Doug Natelson draws my attention to a new proposal from Texas A&M:

[Frank] Ashley, the vice chancellor for academic affairs for the A&M System, has been put in charge of creating such a measure that he says would help administrators and the public better understand who, from a financial standpoint, is pulling their weight.

A several-inches thick document in the possession of A&M System officials contains three key pieces of information for every single faculty member in the 11-university system: their salary, how much external research funding they received and how much money they generated from teaching.

The information will allow officials to add the funds generated by a faculty member for teaching and research and subtract that sum from the faculty member’s salary. When the document — essentially a profit-loss statement for faculty members — is complete, officials hope it will become an effective, lasting tool to help with informed decision-making.

Debates about education policy, like every other political topic in modern America, have a remarkable ability to keep going long past the point where you would think they had to stop. Just when you think you’ve seen the most idiotic proposal possible, somebody will put forth something even dumber. Usually, somebody from Texas.

This is going to be a tough one to top, though.

Which is not to say that I don’t expect to find people speaking in favor of this idea, even in comments here. This is a notion that will appeal to the Teenage Objectivist faction that is depressingly common in science and engineering circles– after all, the sciences and engineering are vastly more likely to bring in grant money, and thus end up on the positive side of the balance sheet. Humanities faculty have many fewer chances to bring in grants, which means that unless they teach lots of huge sections, they’re likely to end up negative, but then a good Randroid disdains modern humanities scholarship, anyway.

This is really impressively cynical, though, as it makes no pretense at measuring quality, just quantity. It doesn’t matter how well you teach students in this system, just how many of them you teach. So load up on the big intro sections, turn everything over to the TA’s, and spend all your time writing research grants. You don’t even need to publish in this system– just bring money in, and you’re gold. Which, as it happens, is the only thing Frank Ashley seems to care about.

Comments

  1. #1 Paul
    September 4, 2010

    If it’s not actually promoted as a tool for evaluating *teaching* performance, or as a sole evaluation on which staffing decisions will be made then I don’t see what is so wrong here.

    Aren’t financial people taking precisely those factors into account anyway? Wouldn’t you if you were in their position?

  2. #2 Anton P. Nym
    September 4, 2010

    Aren’t financial people taking precisely those factors into account anyway? Wouldn’t you if you were in their position?

    No. You don’t measure academic returns by dollars raised; universities should not be profit centres in the financial sense. They’re responsible for training and R&D, to put it in biz-speak… always pure loss on a balance sheet but essential to the business as a whole.

    Also, the criteria as stated don’t measure how well students are taught… just volume; hence Chad’s commentary on adopting huge intro courses and turning them over to an army of TAs. (Is a prof actually teaching in that case, or just sharecropping?)

    Measuring academic ROI solely or primarily in terms of dollars is just a bad idea, skewing the performance away from research and education and instead towards grant-farming and diploma-milling.

    — Steve

  3. #3 CCPhysicist
    September 4, 2010

    It isn’t hard to top, because this spreadsheet approach didn’t start in Texas. One university closed departments and fired tenured professors allegedly based on a similar metric. The story even made Science because of the fame of one of the people who got fired. Hint: They were not in the humanities.

    The only difference is that, despite claims to be a research university, that other school ignored research productivity. Texas should be proud that their bean counters know that the university runs on “overhead” from research grants.

    But has this lazy administrator considered the risks of using only a single, purely objective, measure of productivity? After the faculty sue to get access to this public information at a public university, how many will demand substantial raises? How many will provide uncompensated advice to someone else’s students? How many will serve on committees?

  4. #4 CCPhysicist
    September 4, 2010

    Under that system, there is a disincentive for quality teaching. Your sections would always be full if you just passed everyone. Looks good on all measures, except the one that matters (how they do the next semester). However, in the long run, you might get more students enrolled if they have to repeat the next class — unless that prof does the same thing.

    How long before the university says they are losing money because too many kids are passing the first time …

    Memo to Steve @2:
    They long ago gave up on teaching personalized classes at the Enormous State Universities I know about. Now if each TA gets credit for their share of the income, guess who looks exploited on those spreadsheets?

  5. #5 Rob Knop
    September 4, 2010

    This makes me want to cry.

    Every time I hear somebody making a cry that universities need to be “run like a business”, I cringe. People will point out that yes, they are a business. But the problem is, when people say “run like a business”, what they’re really saying, even though they don’t know it, is “run like a poorly-run business”. I.e., run by people who think that “running a business” is a skill that doesn’t require any knowledge of what the business done. People who think that “the goal of a business is to make money for the shareholders”, and who don’t think that the real goal of a business is to do what the business is doing.

    But, even if you are one of those cynical types who thinks that businesses exist only to make money, and the products and services they provide for civilization are a mere side-effect, this is a particularly stupid approach to running a business. The whole notion that each individual must be carrying his own weight when easily measured financial metrics are used is flat-out stupid. How much money to janitors directly bring into universities? Zero. Should we conclude that universities will be better off firing their janitorial (and, while we’re at it, maintenance) staff? I mean, DUH.

    The real question is how pieces of work like this get into high-level administration positions. Evaluating teachers is really hard. I suspect evaluating administrators is really hard too. But when somebody comes up with something like this, it should be a pretty clear signal that no university or college that has a tenth of a clue about what their mission and goals are should allow this person anywhere near a decision-making position.

  6. #6 agm
    September 4, 2010

    Um, one important piece of knowledge. If it’s from the A&M system, it is NOT from A&M. In Texas “System” means umbrella organization over semi-independent universities, not the universities themselves. There are at least 3 – the A&M system, the University of Texas system, and the UH system, with the UT system being pre-eminent, as least in terms of the way interest income is directed from the trust fund held by the state.

    That being said, A&M is so much bigger than the others that within the A&M system is has a place similar to the Texas system, sometimes described as “UT and the Seven Dwarfs”. Meaning that to a large extent what the system does is synonymous to what A&M is doing. But the difference is important in that it adds more personalities into the situation in addition to those that would be there if it was just Spaceballs The University.

    I can see the motivation for this proposal. Education in Texas isn’t the highest priority, particularly as we have a multi-billion shortfall for multiple years. So a lot of money-oriented things of the sort you see Dean Dad blogging about happen

  7. #7 Rhett
    September 4, 2010

    You say “worst ever”. I say “worst so far”. Do not underestimate the creative stupidity of some administrators somewhere.

  8. #8 Paul
    September 5, 2010

    Me: “Aren’t financial people taking precisely those factors into account anyway? Wouldn’t you if you were in their position?”

    Anton P. Nym: “No. You don’t measure academic returns by dollars raised; universities should not be profit centres in the financial sense.”

    I am not talking about measuring academic returns, I am talking about favoring faculty who bring in grants.

    Let me put another way: you have two *otherwise comparable*(critical detail!) faculty members but one of them brings in much more funding in grants then the other, financial difficulties hit your university and now you have to fire one of them, which one would you retain if it was your job to decide?

    I would think it’s pretty obvious that in such circumstances you retain the one who brings in much more in grants, am I wrong?