I had been agonizing over what would be my first "substantive" post for this new venue. My husband (who will be properly introduced in another post) suggested I make a list of a bunch I'd like to write so I have fodder for the fire when I feel otherwise tapped out.
Ah, me of little faith. I made such a list, verily I did. But instead of drawing from the well of better-intentioned ideas, I will blog about what I'd like to see in conference papers while I'm still all hot under the reviewing collar.
One of my professional societies is ASEE - the American Society of Engineering Education. It puts on a national conference every year (of course - last year it was in Hawaii, which was nice :-) ) and I review conference submissions for several of the divisions, have done so for probably at least 5 years now.
The conference is a little schizophrenic - there are a few pretty good papers every year, but many seem to be submitted by people so that they can get funding to go to ASEE. (Are other conferences any different? I suspect not...)
I admit that what I find most valuable at conferences is chatting with people in the hallways between sessions (and sometimes during sessions) and less sitting in the sessions themselves - I suspect it may be the same for many folks. Understandably too, particularly in the context of this conference - the people who come to ASEE are people who care about teaching in some form or another, and they often seem to feel pretty alone in their departments. They come to ASEE to find kindred spirits, to find validation, to become rejuvenated and inspired before returning home. That's all cool. I feel this way about a bunch of different conferences I go to.
But there is a problem brewing here. The vast majority of attendees are first and foremost engineers. ;-) Not education researchers. Engineering is their day job, and education conferences are things to sneak in if you have time outside the rest of your "real" research.
By the way, most engineers have a set of analytical tools that are pretty significantly different than education researchers. Engineers also have limited traditions for writing up data (and - acknowledged snark - seem to hope that everything can be understood through surveys).
This problem emerges in the papers themselves. While there is (duh) great value in people spending their analytical skills and intellectual CPU thinking about what they are teaching, and trying to figure out whether students are really learning it, and while people who study their own practice can something valuable to say to others, sometimes we have to learn some other tools to make sure that what we've learned is what we think we've learned, let alone whether it can help others learn. And there doesn't seem to be much of a structure to help authors trained as engineers to improve their education-oriented research on their practice.
Well, let me provide a couple of tips.
The three related things I find regularly lacking in papers are:
- a clear, internally consistent, and informed theoretical framework that goes a long way toward explaining why the methodology the authors propose - both the research methodology, and, if applicable, the intervention/program methodology - matches what they want to figure out. An example of the latter: if you want to increase the number of young women who choose engineering as an undergraduate major, why will your program on math skills help? Your implied theoretical foundation is that you think young women don't go into engineering because they are having a hard time with math. What is your evidence for this? (Limited, actually.) How will you know if your program is what made the difference, as opposed to a bunch of women developing a support network to choose engineering as a major? Etc. etc. etc.
- a consistent story from beginning to end. Lots of papers seem to set out good arguments at the beginning for trying to figure certain things out, but then never return to that set of arguments in the methodology and the conclusion. They get distracted along the way, they mention characteristics that are not connected to their theoretical framework, they forget to signpost their argument as it develops, and then you get to the end and you've forgotten what they wanted to know in the first place. Perhaps so have they.
- a "so what?", as my 12th grade English teacher (hi Mr. Keys!) used to say. So often, the authors crunch their way through a bunch of quantitative data, perhaps salted with a few illustrative quotes, and they get to the "conclusions" section and completely run out of gas. Tell me, please, what you learned after doing all this work?? Or what I should now know that I have followed you through all these pages? Not just that one group is statistically significantly different from another, but what that means in terms of what you set out to do in this research more broadly.
Focusing on addressing these three limitations would go a long way to making these papers not only a better read, but actually figure something out that would be important to share with others. After all, isn't that the purpose of these get-togethers? Okay, except for the ones in Hawaii?
Furthermore, these points are important to think about in whatever papers you're writing, not just education-oriented ones where some of us may be moving out of our disciplinary traditions just a wee tad. But perhaps the holes in our theoretical foundations and our writing become more visible in places where we lack disciplinary crutches such as journal section headers? And maybe writing educational papers isn't just a walk in the park? Maybe...
What are your thoughts on what would improve conference papers? Constructive comments, if you can, please - I had to edit this post a lot. ;-)
- Log in to post comments
The main problem here seems to be that engineers interested in teaching and learning do not have access to the language and traditions from the education field. Maybe they don't realize how much they need a theoretical framework for the points they are trying to make and they don't have the "technical vocabulary" needed to talk about engineering education in a professional and coherent way. Please, correct me if I've misunderstood that, but I'd that issues like these would form major barriers to talking professionally about education within any (science?)field. I'm not sure how to go about this exactly, but I get the impression that the issue is not so much about how to get people to write better conference papers, but rather how to educate people in engineering/"any field" education.
I think saxifraga raises a good point. When I've asked about doing education research in -ology, the main thing I've been told is that I must get consent from my students before using data from surveys or assessments. As if informed consent were the only thing necessary to research science education.
Too true, too true, and perhaps I myself didn't make a consistent story from beginning to end in this post. (That'll learn me to post late at night! Bad dog...)
I guess what I was frustrated at was the sense that some scientists and engineers seem to think they can do educational research because they do their own disciplinary research, without realizing that there are other tools out there that might be critical to their doing educational research successfully. And that it is their (our!) responsibility to go in search for those tools.
However, you're right that this point was confounded by the general suckyness of conference papers. ;-) I guess reading less-good ASEE papers can make it hard to differentiate the two.
So for those folks out there still reading this - what do you want to know about doing educational research in your STEM classrooms but were afraid to ask? Maybe I can make a post out of it...
I think a major problem is, that it is not widely accepted to spend a lot of time on doing something, that is not strictly connected to one's own field of research. Spending time on the big picture, the wider aspect of the field, or god-forbid something as airy as education theory is, if not directly frowned upon, at least punished because it leads to fewer publications/grants/whatever in the area that matters to the department. It may not be like this everywhere and it might a European thing or a geology thing or a research university thing, but I think it's true for the academic environments I'm familiar with. I think this is sad, because thinking about the wider perspectives including the educational part of my field is what makes me excited about it. I would love to see departments like yours appear at more universities, not only for engineering but also for other STEM fields. I would love to see education (including the theory behind it) being taken seriously at research universities and to see a much wider accept for cross-disciplinary research whether it's a full time engagement like yours or the average science faculty memeber, who wishes to publish a bit on education as well.
How about classes in educational theory/background in grad school (I would have signed up for one if it had existed)? Or how about courses being available to faculty members?
I think many people (including myself) wouldn't know where to start. I'm interested in geology education and teaching and I've met people "on the net" who have pointed me to ressources for my field. I teach at a place that prides itself of it's innovative teaching methods, but yet I have never heard about anyone there being involved in education research. I think you are right, that people who are trying to be cross-disciplinary shouldn't be amateurs, but people need to be able to find the tools to become professionals, and their employers need to respect that this will take up some of their time.
So basically the problem is that Engineering education researchers are trying to reinvent the wheel when it comes to techniques and methodologies in education research. This is nothing new! And happens in research all the time: biomed engineers, biologists and materials scientists try to reinvent the chemistry wheel, pharmaceutical scientists, biomed engineers, materials scientists etc try to reinvent the biology wheel...and on and on.
The problem is that each field of research is rather like a locked room - it is very difficult to break in and gain the knowledge contained within. It is often easier to pick up a text book or address the problem from your existing skill set than to try and hack into a new area of research.
Knowing the vocabulary as saxifraga points out is critical in being taken as a legitimate researcher in a new field, but it is near impossible to pick up without a collaborator or close mentor who is already in that field.
If an educational technique or technology is to be picked up across STEM subjects then it must meet a certain standard. The grasping point for the instructor must be fast and straighforward. The technique/technology must be cheap, efficient and easy to implement and give clear benefit over traditional methods. It must not add prep time and extra work to the already swamped academic, and must not require massive infrastructure investment by the institution. So a website with good practice advice using conventional techiques (powerpoint, tutorial, discussion, assignments) will have a higher uptake amongst educators than a website selling the latest second life interfaces so we can all go teach there.
I think one thing you might consider for these "STEM researchers who are interested in education research" would be a website-forum where people can post their methods and ideas and interact with otherlike-minded researchers. You could have a glossary of the right vocab, introductions to appropriate techniques in education research and target it entirely towards engineers or geologists or chemists, or whatever field you like. Perhaps your conference should have some open panel sessions where methodologies are discussed (with strong moderation of course).
Anyway, nice to see another blogger who is interested in this kind of stuff ;-)
I know what you mean about ASEE papers. Reviewing them can be tedious. There are alot of "so what?" moments (my English teacher used that phrase, too). I do like seeing a new innovative idea for a lab or a piece of hardware, but those papers don't always fit into the "research' mold thet you're talking about.
My favorite part of ASEE this year (and I wasn't able to stay long) was sitting with other WIED members at the picnic.
I'd echo most of Propter Doc's comments with this addition: most biomed researchers, say, know that a field called chemistry exists. They may not go to it looking for tools they need and thus go reinventing some stuff, but that may be because they don't know how or where to look; the energy barrier is too high.
With things like educational research and feminist theory, most engineers (at least in my acquaintance) aren't even aware that such things exist, much less that they could have some application to engineering.
If there is someone in the college to champion and lead exploration into better teaching (and this needs to be someone in a traditional authority figure, preferably dean and/or assistant dean) then engineers will pay attention and adapt/adopt some tools from these disciplines. I've seen it happen on a modest scale at K-State within the framework of a formal program. But there was leadership at the dean level, and department heads supported faculty participation. Even so it takes a lot of work and time for engineers to pick up and become modestly proficient in the necessary tools. This is compounded by having this stuff constantly described as "soft skills" which automatically puts it in a lower hierarchical position next to technical expertise, or "hard skills". (N.B. that I loathe beyond belief the use of hard and soft skills in the science and engineering community. My use thereof in this comment does not constitute an endorsement.)
How will people know what to ask for if they don't even know there's something there TO ask for? When you take over the engineering universe (and have plenty of spare time:) ) maybe you can get the ASEE conference paper guidelines modified to include just the sorts of stuff you outline above, along with a few handy references.
It has always seemed to me that professors in STEM could use a course on education and that the people who needed such a course the most were the very people who were not interested, likewise with advising courses. Otherwise I feel that many students are poorly served particularly by the subset that really don't want to teach but must.