Forget about measuring student outcomes. Can we even measure student numbers? A couple of weeks ago, I started pulling data from the NY Times website that displays the citywide testing scores (I was interested in exploring the relationship between poverty and test scores at a finer resolution than I had previously).
Here's the problem: the No Child Left Behind (NCLB) numbers--the ones the federal government uses--and the state numbers don't agree. I'm not referring to educational outcomes: they don't even have the same number of students. Let's look at New York City. The NCLB numbers* state that NYC has 939,317 students. But if you look at the K-8 test data, NYC has 1,116,427 students. At best, only one of these numbers can be correct. If anything, we might expect the numbers to be the other way around: some students won't take the exams, or aren't counted in the aggregate scores (e.g., learning disabled students).
But we shouldn't have more students taking tests than are enrolled.
What's insane about all of this is that we are using these data to determine if teachers and principals should be fired, if schools should be closed. And we can't even get the same (or even similar) student numbers.
This is some educational reform, I tells ya.
*As best as I've been able to determine, these are the NCLB data.
- Log in to post comments
Are private schools and homeschooled children tested and thus in the aggregate NYC data but excluded from NCLB numbers? I think only public schools are subject to NCLB.
I don't know about NY but in KS (should be the same) home schools are not subject to NCLB as far as regulating what is taught or requiring testing. However I don't know about private schools.
When I was homeschooled, I went in once a year for standardized testing.
This was before NCLB, but it's certainly possible those schools got to count my scores in their aggregate.
That said, it would be a large number for such students to be a significant part of the explanation. Could there be some weird reason some students take multiple tests?
If you look at the breakdown by grade, you get approximately the 1,116,427 figure. It's only the 939,317 on the top that seems weird. Maybe that's the subset for which they have poverty + race data, and thus is the N for those statistics?
Which is why educational researchers cite the number they use. In Iowa, we have 1) Average Daily Attendance, 2) Enrollment, 3) Weighted Enrollment, and 4) Total Certified Enrollment - all different numbers, all counting students differently.
There is no error.
The students were counted by Diebold.
I think the problem is that they say " students tested" as opposed to " tests taken". The results the Times shows is an aggregate of test results. It isn't clear that it is from a single year. They are doing some kind of meta-analysis to compute their performance index, not simply showing a snapshot of the NCLB results. They might be including test scores for students after remedial summer school or compensating for students who have been left back, or tests not taken for a particular grade. The methodology discussion is rather vague, but does mention that the test structure is not as straightforward as the results they present. That they discuss changes years back suggests that they have restructured the data somewhat. Given this, it isn't surprising that there were more tests taken than students registered.
I know that the NCLB tests include charter school results, but that private schools in New York City do not use the same standardized tests. They tend to use the ERB to track their student results, particularly in the earlier grades. Their high school students do not take the Regents examinations, though some take the International Baccalaureate (IB) tests which are useful for the children of ex-pats stationed in New York and the children of New Yorkers who might become ex-pats on assignment.
I thought I would just check, in case I have gotten misinformation elsewhere. I have heard that the NCLB test compares apples and oranges. I. e., they compare this year's nth graders with last year's nth graders. That does not tell us how much children have learned. For that we need to compare children with themselves with before and after tests. Is that actually what happens? Thanks. :)