Kevin Drum commented on a charter school study a couple of days ago, which made me go look through the report (available from this ultra-minimalist page-- seriously, you can't even be bothered to cut and paste some of your introductory boilerplate into an HTML file to give people an idea of what's behind those PDF links?).
The summary message is kind of bleak. From Drum's post, quoting the LA Times:
The study of charter schools in 15 states and the District of Columbia found that, nationally, only 17% of charter schools do better academically than their traditional counterparts, and more than a third "deliver learning results that are significantly worse than their student[s] would have realized had they remained in traditional public schools."
Looking closer, there are a couple of weird things in the executive summary, most notably this:
Charter schools have different impacts on students based on their family backgrounds.
For Blacks and Hispanics, their learning gains are significantly worse than that of their
traditional school twins. However, charter schools are found to have better academic
growth results for students in poverty.
That seems like a really odd combination of results. Given that Blacks and Hispanics are disproportionately likely to be poor, this would seem to imply that middle-class Black and Hispanic students are absolutely crushed by charter schools, or that poor whites and Asians kick butt in charter schools. Or some linear combination of the two.
Of course, either of those seems pretty damn strange. So what's really going on? Looking at the full report, my inclination is to say that this is one of those cases where averaging results together hurts more than it helps.
If you look at Tables 5 and 6, giving the lists of state-level effects for Black and Hispanic students, respectively, you'll see that the results are kind of all over the map. Georgia and Texas are bad across the board, Missouri is good across the board, and the rest of the states jump around. Florida is bad for Blacks, Illinois, New Mexico, and Ohio are bad for Hispanics. Louisiana and Minnesota are good for Blacks, while Arkansas is good for math but indifferent for reading. California charter schools manage to have a statistically significant positive effect on reading scores for Blacks, but a statistically significant negative effect on math scores for Hispanics.
Turning to Table 7, giving the same information for poor students, we find that Missouri and Louisiana, whose charter schools did well by minorities, do a lousy job with poor kids. Meanwhile, Georgia and Texas, which were awful for minorities, are good for poor kids.
I think the technical term for this is "a hopeless muddle."
I suspect that if this were broken down further (which it probably is in the state-by-state reports, but there are limits to how many PDF's I'm willing to wade through), the school-by-school results would be as erratic as the state-by-state results. Which suggests that trying to report nationwide demographic trends is kind of foolish-- I mean, sure, you can average everything together and get some sort of composite result, but it's not clear that it means anything. The benefits or costs of charter schools appear to be highly contingent on local factors, which is not that surprising, as education is a complicated business.
Really, all that I'd be willing to say, looking at the report, is "Answer unclear, try again later." There's no clear, large positive benefit to charter schools, and there's no clear, large negative effect, either. The averaging process leaves you with a smattering of results that appear statistically significant, but taken together, they don't form any kind of coherent picture. Which means they're probably just noise.
But, if you're going to write a report, you need more than "It's complicated..." So you get demographic breakdowns reported, whether they make sense or not.
- Log in to post comments
As you said, the problem is averaging. Lumping all charter schools together doesn't make sense. Charters do different things. I have no problem with charters per se, the problem comes when people think it's a magic bullet for education.
In my area we had a bunch of rich parents revolt when they re-drew the district lines. Their kids would have to mix with the chattle. They flipped out and got the district to create a charter school so they could all be together. There's nothing special about this school. It draws from the same pool of teachers and the year before it was a regular public school. Clearly this charter school isn't doing anything better than when it was public.
As you said, the problem is averaging. Lumping all charter schools together doesn't make sense. Charters do different things. I have no problem with charters per se, the problem comes when people think it's a magic bullet for education.
In my area we had a bunch of rich parents revolt when they re-drew the district lines. Their kids would have to mix with the chattle. They flipped out and got the district to create a charter school so they could all be together. There's nothing special about this school. It draws from the same pool of teachers and the year before it was a regular public school. Clearly this charter school isn't doing anything better than when it was public.
"a hopeless muddle" 1836-1970 US education absent psychopharma and jackbooted State compassion worked on every variety of immigrant scum effectively, efficiently, and with minimal budget. Even the Irish were rendered English-speaking and productive.
1960s NASA went to the moon starting with nothing but slide rules and Nazis; 2010 NASA is utterly clueless with ultrahypersupercomputer Columbia plus a crib sheet. 130 years of US education brought forth a manure pit of psychobabble, professional management, and social avocacy. Screw desks into bare wooden floors, set standards, and bring back the Nazis. It would work for US education, too.
uncle al is an ass if he really thinks education mid 1800s to mid 1900s turned out better products than today's.
and I am an uncontrollably twitchy dweeb for hitting the post key twice.
Sweet. An imaginary solution to an imaginary problem.
I have four kids in a charter school. The reason for the discrepency is this. When they were in regular school, their grades (ie. the way learning is measured) were better, but they were learning less. The charter school is more rigorous, for example making ALL the 8th graders take Algebra, their grades are much worse than in regular school, but they are learning more. The LA times needs to come clean on how they got the study data. If they used standardized test scores, it is hard to see most charter schools doing worse than their regular curriculum types, though it is possible. In the case of my kids, their standardized test scores are above the average compared to the normal school averages even though their grades are not.
I think anybody can do that, there is nothing interesting, no valuable information obtain from there.
I think anybody can do that, there is nothing interesting, no valuable information obtain from there.
I'd be interested to see how the data plays out if you could sort charter school by type of curriculum. DC has lots of charter schools with a huge range of models, approaches to teaching and themes. Some of the them seem flakey to me. Some of them seem great. Some seem like they would be wonderful for some types of kids and torture to others. One of the ideas was that charter schools would be free to experiement with different approaches. Are certain kinds of charter schools doing better with certain kinds of kids? If they are how can that be replicated.
File this one under "well, duh!" The whole point of charter schools is to experiment with techniques and curricula that depart from what the regular, allegedly bureaucracy-choked public schools use. Educational methodology is at a largely pre-scientific stage: we know that certain things sometimes work and that certain other things are inherently bad ideas, but there are too many variables and not enough data to predict that any given technique will be better for all students (or even an overwhelming majority of them) than what we have been doing. Ultimately, we will find that some charter schools work under the conditions they have, and others will not. Proving which ones are which, and drawing systematic conclusions from those results, will take quite a bit more data than we have now.
Proving which ones are which, and drawing systematic conclusions from those results, will take quite a bit more data than we have now.
*************
It will also be telling what sort of selection biases are happening as well. Not always the easiest thing to control for.
All of the comments ignore the elephant in the tent. What is the average cost per student of charter schools? What this stuff shows to me is that they are no worse than public schools in general. Now they could be cheaper on the margin and wreck public schools that way or not.
I am told by a high school teacher at a good exurb public high school that the non-teacher costs have quintupled since he started 20 years ago, so something is going to break. They have an onsite cop for goodness sake, full time, AND a disciplinary officer (full time) and a vice-principal in that area and a "compliance" officer and more and ... they have less than 1000 kids. The school has zero option for those positions. They are mandated. When funding cuts come - the teachers and teachers aids and activity budgets get hit. I am not not a big fan of charter schools, but this kind of thing is what leads taxpayers to support them.
Orlando @8: I'm pretty sure that the reference to "learning" in Kevin Drum's article means they used standardized tools, probably the NCLB mandated ones that are used in my state. These vary from state to state but provide a baseline for comparing schools in the same state.
Chad, thanks for looking at the article so I don't have to! Your analysis makes sense to me, for the following reasons, but first a comment on race and demographics (and a reminder to read about the Green Dot program in LA in "The New Yorker" of a few weeks back).
It is no longer the case that Black equals Poor, and it was never the case that Poor equals Black. Although black students are more likely to be poor, there are so many more white kids that many more of them are poor if you compare numbers rather than percentages. It could very well be that charter schools deal with the effects of poverty better than public schools, in a race-neutral fashion, but I also found those results interesting. Are middle class black students hurt more by the "everyone is above average" and "you must be a visual learner" dreck that middle class kids are being fed in too many schools today? But, as one commenter on Kevin Drum's blog noted, they quote the mean but not the standard deviation. It is quite possible that none of those differences are significant.
Now to why I believe the main result:
First, in our state charter schools (unlike private schools) are required to administer the State's NCLB "high stakes" standardized evaluation, and the results are reported each year. Our local paper publishes a "compare the schools" shopping list each fall that shows those results on a school-by-school basis along with other data mentioned in the study (% with free lunch, i.e. poverty, and race). It's pretty clear that some of the charter schools don't do very well, even in comparison to public schools that don't do very well either. [In some cases, it might be a comparison of bad with awful.]
It is true that what I see is not the result of a "longitudinal" study where you can track an individual student from year to year, but it does show remarkable differences between schools that have similar demographics.
Second, although private school students don't take that State test, they do take the placement test at our CC when they come here. Now I only have my own observations of placement results -vs- high school attended from years of advising these incoming students, not statistical data, but there are patterns that indicate that certain private schools and certain charter schools do well, and that others from both categories are awful, just as is the case for the larger public schools.
BTW, awful means student is taking Algebra 2 in HS but places into Basic Algebra rather than College Algebra at the CC. I can only characterize that as a teacher / standards effect.
Finally, @4: Uncle Al may be an ass, but he might be comparing 1950-1970 to 1990-2010. There are some differences on that time scale, some good, some really bad.
Hey check out Privateaircharterjets.com they have a lot of different options at affordable prices.