To recap, I'm prepping a new graduate level course on experimental design and data analysis (EDDA) that will serve MS and PhD students from geosciences and civil and environmental engineering. I've been working through the SERC course design tutorial, and so far I've figured out context and constraints, over-arching goals, and ancillary skills goals. It's finally time to add content to my course.
(Note that the material that follows is a bit outliny, because I'm thinking out loud (on blog paper) here and looking for feedback)
First, according to the good folks at SERC, I need to bring it back to those course goals:
Task 1.4: Choose broad content topics, and flesh those broad topics out.
List the overarching goal(s) for your course. For each goal, list the broad content topics that you could use to achieve the overarching goals of your course. Below each broad content topic, list the more specific content items that are imbedded in each broad topic and that students must master to achieve the goal.
1. I want students to be able to evaluate the connections between: (a) knowledge of existing literature and/or preliminary data; (b) research question and hypothesis generation; (c) experimental design; (d) quality of the collected data; (e) methods of data analysis; (f) ability to answer the posed research question.
- Scientific method discussion (Chamberlin's Method of Multiple Working Hypotheses, Platt's Strong Inference, traditional hypothesis/null hypothesis approach)
- Literature search tools (Web of Science, Compendex)
- Basic experimental design (What are the variables? Controls? Sampling size? Replication? Sampling approaches (stratified random, etc.) Power analysis)
- Linkage between experimental design and statistical data analysis
- What happens when we don't have a good experimental design to start with?
- Data quality (QA/QC, duplicates, blanks, spikes, ground-truthing, standards, precision versus accuracy) Things go wrong. The importance of contingency plans and well-trained helpers. Ethics of data analysis
2.I want students to be able to work in teams to formulate a research question, design a study to answer the question, and analyze the resulting data using appropriate statistical techniques.
- What goes into a proposal? Ask students to look at NSF guidelines for their program of choice. Ask advisors/committee members for example proposal (+ reviews if possible).
- Preliminary data. What sort? How much? What sort of analysis should we do on it?
- Statistical analysis (selected topics based on student interest, possibly including statistical hypothesis testing; ANOVA/ANCOVA; Repeated Measures Analysis; Time series analyses (ARMA models); regression, multiple regression, stepwise regression; Jack-knifing and bootstrapping; Geo- and spatial statistics; Principal components analysis and other multivariate methods
- Broader impacts
- Putting the pieces together in a proposal, revising, and polishing
3.I want students to be able to critique experimental design and data analysis techniques that appear in proposals or the published literature of their field.
- The peer review process and how it works
- Many pieces of content from above are applied here too
- Writing a thorough and constructive review
That done, the SERC tutorial asks me:
What order might you put the broad topics in that would allow students to build the complexity of ideas and applications over time and to revisit concepts or topics in an appropriate way?
- Identifying and refining the research question
- Literature Review and preliminary data
- Scientific Method discussion
- What goes into a proposal?
- The peer review process and how it works
- Basic concepts in experimental design
- Linking experimental design and data analysis
- Data Quality
- Statistical Analysis
- Writing a review
- Broader impacts
- Putting the pieces together in a proposal
While the above list certainly seems like a reasonable sequence to teach the topics, I'm a bit concerned when I think about how it will work in practice, since my goal (2) is to have the students actually write a proposal over the course of the semester. It seems like there's a lot that has to happen early in the semester so that they have the tools to put the proposal together before the last minute.
Fortunately, some of those earlier topics won't be covered in as much detail as the later ones, so maybe the timing won't be as bad as it looks in that list. For example, I envision being able to talk about literature searches within the first course period or two, and having teams identify their research questions by the third week. But in this part of the tutorial I was having trouble not jumping ahead to the next section: Part 2.1 Developing a course plan in the context of goals and content topics.
I think I'm also jumping ahead to thinking about teaching strategies and assessment, and I'm having a hard time not digging out a calendar and mapping content with dates. But perhaps this is a strength of sticking to this tutorial, I am forcing myself to follow a better course design protocol than if I just came up with something ad hoc. After all, that's exactly the sort of take-home message I want my students to learn about experimental design and data collection.
- Log in to post comments
This sounds like it will be a really good class. I wish I had had one like it.
I had an experimental design class with Hurlbert at SDSU... best/most useful class ever... good luck! Stu's class was a bit more, let's say aggressive... but really allowed me to critically dissect published methods and review common types of errors (and highlight what to avoid doing myself). I hope to put together something similar at some point, thanks for sharing our outline and thoughts. I'd love to see your recap of what worked and what didn't afterward!