In Research We Trust

On June 14, 2019 By teachingcyborgIn Education Reform, Research, Statistics, STEM and Society, STEM EducationLeave a comment

“Facts are stubborn things, but statistics are pliable.”
Mark Twain

Anyone that knows me knows I believe in research and data backed decisions in education. Successful research is a balancing act between skepticism and an openness to new sometimes radical ideas. To avoid the possibility of bias, we have developed methodologies and techniques to determine the validity of an experiment. Experimental validity falls into two categories: internal, experimental design, data collection, and data analysis. The second is external, the progression from hypothesis to theory, and finally to the fact. Research drives the progression from hypothesis to fact with supporting evidence and replication.

Considering how vital replication is to research, there appears to be very little direct replication. Makel and Plucker showed that only 0.13% of educational research is replicated (Facts Are More Important Than Novelty: Replication in the Education Sciences). Compared to a rate of 1.07% in psychology and 1.2% for marketing research. However, the rate of replication does not tell the whole story. After all, to publish research, you need to conduct an experiment, submit it for peer review, make changes, and then have your article published. Perhaps we can accept published results.

Looking at actual replication studies suggests that publication is not enough. One study in psychology, Estimating the reproducibility of psychological science, was only able to replicate 63% of the studies they examined. Replications of clinical research are even worse. A group from Amgen attempted to replicate 53 research studies in cancer research they only replicated 6 of them. Additionally, a group with Bayer Health could only replicate 25% of the preclinical studies they tested (Drug development: Raise standards for preclinical cancer research).

So how do we resolve the replication crisis? We need to reproduce previous research and publish the results. The problem is that professors, postdocs, and graduate students don’t benefit from replication studies. Even if researchers get the articles published, they don’t carry the same weight as original research. One possibility would be to have graduate students replicate experiments at the beginning of their graduate study as part of their training. However, this is probably not a workable solution as it would likely lengthen the time to degree.

So, who would benefit from reproducing research? The answer is undergraduates. Conducting replication studies would more effectively train students in research methodologies than any amount of reading. Why would conducting replication studies help students with research design? The reason is that if you replicate a study perfectly (exactly as undertaken previously), you might have the same problems the original researchers had. After all, most of the issues in research are not intentional but unintentional and probably unidentifiable problems with data collection or analysis.

Statistical analysis of most data involves a null hypothesis. When the data is analyzed, the null hypothesis is either accepted or rejected. Errors analyzing a null hypothesis, are classified as Type I (rejecting a correct null hypothesis) or Type II (accepting a false null hypothesis). The critical thing to keep in mind is that it is impossible to eliminate Type I and II errors. Why can’t researchers eliminate Type I and II errors? Think about a P value, P < 0.001, what does the number mean. Written in sentence form as P value < 0.001 means: the likely hood that these results are the product of random chance is less than 1 in 1000. While this is a small number, it is not zero, so there is still a tiny chance that the results are due to random chance. Since P values never become P < 0, there is always a chance (sometimes ridiculously small) that results are due to random chance.

In addition to Type I and II errors, there could be problems with sample selection or size. Especially early in the research were influencing and masking factors might not be known. Alternatively, limited availability of subjects could lead to sample size or selection bias. All these factors mean that a useful replication study looks at the same hypothesis and null hypotheses but uses similar but not identical research methods.

Beyond the benefits students would gain in experimental design, they would also learn from hands-on research something that many groups say is important for proper education. Additionally, replication research is not limited to biology, chemistry, and physics. Any field that publishes research (i.e., most areas of study) can take part in undergraduate replication research.

Of course, these replication studies will only benefit research if they are published. We need journals to publish replication studies, how do we do that. Should a portion of all journals be devoted to replication studies? The Journal Nature says it wants to publish replication studies; “We welcome, and will be glad to help disseminate, results that explore the validity of key publications, including our own.” (Go forth and replicate!). Hay Nature how about really getting behind replication studies! How about adding a new Journal to your stable, Nature: Replication?

However, if we want to disseminate undergraduate replication studies, it may be necessary to create a new Journal, The Journal of Replication Studies? With all the tools for web publishing and e-Magazines, it should be straight forward (I didn’t say free or cheap) to create a fully online peer-reviewed journal devoted to replication. Like so many issues, the replication crisis is not a problem but an opportunity. Investing in a framework that allows undergraduate to conduct and publish replication research will help everyone.

Thanks for Listing to My Musings
The Teaching Cyborg

There is a Lot of Pressure, Partially, Involved

On March 14, 2019March 14, 2019 By teachingcyborgIn Ed Tech, Pedagogy, Research, STEM EducationLeave a comment

“Gases are distinguished from other forms of matter, not only by their power of indefinite expansion so as to fill any vessel, however large, and by the great effect heat has in dilating them, but by the uniformity and simplicity of the laws which regulate these changes.”
James Clerk Maxwell

When learning chemistry gasses get a lot of attention. There are a lot of laws and formulas that relate to gasses. Here is a list of gas laws:

1) Avogadro’s Law
2) Boyle’s Law
3) Charles’s Law
4) Gay-Lussac’s Law
5) The Ideal Gas Law
6) Dalton’s Law of Partial Pressures
7) van der Waals Equation (Non-Ideal gases)

I want to talk about Dalton’s Law of Partial Pressures. If you look up Dalton’s law like most students would Wikipedia via Google we see that the definition of Dalton’s law is: “Dalton’s law (also called Dalton’s law of partial pressures) states that in a mixture of non-reacting gases, the total pressure exerted is equal to the sum of the partial pressures of the individual gases.” Which comes from Silberberg, Martin S. (2009). Chemistry: the molecular nature of matter and change (5th ed.). Boston: McGraw-Hill. p. 206. ISBN 9780073048598.

I have not thought about Dalton’s law in years. The last time was when I was helping develop some chemistry labs. Then two weeks ago I ran across a YouTube video from Cody’sLab called Demonstrating The Law of Partial Pressures.

What I found interesting about this video is that Cody built a physical device to demonstrate the law. The device is two pressure chambers made out of copper plumbing parts and glass tubs attached with a simple valve. A quick search online suggests that one of these devices could cost less than $100.00. He uses the device to demonstrate several principles of Dalton’s Law. What I find fascinating about the device is that in many ways this device did a better job demonstrating Dalton’s law then any device I encounter in my high school or early chemistry classes.

With a small amount of work, it should be possible to build a device that was composed entirely of parts that could be screwed together allowing assembly into multiple configurations. A device students use in multiple configurations would expand the options for open-ended inquiry. Multiple configurations would let chemistry students conduct inquiry-based labs. Students could assemble the appropriates so that they could combine 2, 3, 4 or even more samples.

We know that hands-on experiences improve student learning. In the article Physical Experiences Enhance Science Learning, the authors show that physical experiments lead to increased test scores. Additionally, they showed that later recall of the information activated the brains sensorimotor region. Which suggest a mechanism by which hands-on teaching can improve learning. Since hands-on learning enhances science education, we could argue that the current model where we have a 3-4 credit lecturer class and a one-credit laboratory class is backward and we should be running 3-4 credit labs with one credit lectures or recitations. That, however, is an argument for another day.

Since we know the value of hands-on learning lets takes this idea to the next step. Suppose the students not only ran an experiment to confirm and explore Dalton’s Law but they also built and designed the equipment to do the experiment. Would this enhance learning even more? I could see a couple of ways building your equipment could enhance learning. One, this would be a way to expand the amount of hands-on time. Two, building the test apparatus could give the students a better understanding of how the device works. Three, the designing and building process could potentially lead to enhanced ownership in the experiment. The impact of building your research equipment is an area where more research is needed.

However, even with the evidence that shows hands-on learning enhances science education laboratory classes are under increasing attack. One of the most common arguments I hear against hands-on science education is the cost. The device Cody built for his video is relatively cheap if you were to build it for less then $100 this is less then some microscope slides or chemical reagents. There are of course questions about the equipment used in labs. There are many arguments that the lab is the place for students to learn about research equipment. I have had many discussions concerning the design of laboratory activities that started with “my students need to know how to use X piece of equipment.”

While there are some types of equipment that students should have a familiarity with the idea that students need to learn a specific piece of equipment is ridiculous. First, what is the likely hood that a specific piece of equipment is still going to be in use when they end up working in a research laboratory? Second, what are you trying to teach the students? As an example suppose I design a lab to demonstrate the Mendelian laws of inheritance. The students will need to use a dissecting scope to determine the sex of their fruit flies. Should I devote half the laboratory activities and time to the use of the dissecting microscope? Of course not, the microscope is not part of the principle of inheritance.

Beyond the fixation on specific pieces of equipment, there is also a belief that low-cost equipment is unusable. Whether the cost of equipment affects its educational value is an interesting question. Generally, the cost of equipment is directly related to precision, how much precision do we need. If the purpose of a laboratory activity is to show that acceleration due to gravity is independent of mass, do the student need precision out to 10 decimal places? As long as the equipment meets the need for the activity, we do not need to go with the most expensive thing. When we design STEM activities, we need to focus on the learning goals what has the best chance of enhancing the students learning. In regards to student learning, learning is not proportional to the cost of the equipment, and it is not dependent on a specific piece of equipment. Our instructional design needs to be informed and based on what the research says not ideas about the “best” piece of equipment.

Thanks for Listing to My Musings
The Teaching Cyborg

Researching Prototyping and STEM Education

On February 28, 2019February 28, 2019 By teachingcyborgIn Ed Tech, Research, STEM and Society, STEM EducationLeave a comment

“The visionary starts with a clean sheet of paper, and re-imagines the world.”
Malcolm Gladwell

Microscopes are an essential piece of scientific equipment they gave us the ability to view parts of the world that we can’t see otherwise. The invention of the microscope lead directly to germ theory which revolutionized healthcare. Throughout my career I’ve done a lot with microscopes; research, teach, maintenance, and I’ve even worked with a group to make them remote controlled.

Microscopes can also be extremely expensive, I worked with a microscope that cost a million dollars, and some microscopes cost more than that. Microscopes are particularly crucial in pathology and medical diagnostics. Which in some cases can be a problem; the cost of microscopes can be limiting in some areas of the world.

Take for instance sub-Saharan Africa; malaria is one of the most common causes of death due to illness in this region. According to the CDC 90% of all the worlds malaria-related deaths are in sub-Saharan Africa. Which is sad because malaria is completely treatable especially if identified early. The problem is malaria can present like the flu. Without going to it all the reasons the only way to conclusively diagnose an active malaria infection is by a stained blood smear observed under a microscope.

In the United States, this is not a problem if your local medical office doesn’t have a diagnostic lab; one is available within a few hours by medical courier. However, in places like sub-Saharan Africa diagnostics labs can be prohibitively expensive and far out of reach. A basic diagnostic microscope is going to cost several thousand dollars; a clinical centrifuge will also cost a couple of thousand dollars. In addition to the cost, this equipment can be difficult to transport and set-up. The diagnostic equipment also requires electricity something that is not commonly available. So, you also need a generator and fuel.

In addition to malaria, poverty severely impacts sub-Saharan Africa. According to the World Bank in 2015, 66.3% of the population live on $3.20 a day or less $1160 a year, 84.5% lived on $2007.50 or less a year. One of the effects of poverty is a lack of infrastructure which makes it difficult to access many areas.

A potential solution to this problem came from Dr. Manu Prakash an associate professor of bioengineering at Stanford. In 2014 his group developed the Foldscope a small microscope built from paper, an LED, watch battery, and spherical lens, it has magnification from 140X to 2000X. The Foldscope cost less than a dollar to make.

In 2017 his group developed the Paperfuge a hand-powered centrifuge with speeds of 125,000 RPM it costs about $0.20.

The Foldscope and Paperfuge don’t require power they’re small and easy to transport and we can easily replace them because of their low-cost. These pieces of paper can change diagnostics in remote regions drastically.

So, what do the Foldscope and Paperfuge have to do with STEM education? Historically building, prototyping, and testing a new device was a long and expensive process. The cost limited the development of products to a few high-end research institution and large companies. In today’s world of desktop manufacturing and prototyping, the cost to prototype has come down and is readily accessible to most schools and institutions.

With desktop tools available you can imagine building research/teaching programs around social and educational problems. On the educational side tools like the Foldscope and Paperfuge can be used by groups of students to do fieldwork. Imagine taking groups of students out to a field site and giving all of them a microscope and centrifuge to do examinations.

Alternatively, we could use the Foldscope and Paperfuge as a model. Schools and classes could partner with a community organization to develop tools to deal with problems and issues these organizations are facing. Students will start by learning the science behind the issues and the existing solution if there is one. Then as a laboratory component, students would use modern desktop manufacturing tools to design, prototype, and test solutions. We could adapt this type of program to any level of school. Additionally, they would combine science, engineering, and community service in one class.

Thanks for Listing to My Musings
The Teaching Cyborg

What the Moon Can Teach Us About Science

On February 21, 2019February 20, 2019 By teachingcyborgIn STEM and Society, STEM EducationLeave a comment

“I still say, ‘Shoot for the moon; you might get there.’”
Buzz Aldrin

Last month on January 21, 2019, I stood in the snow in below freezing temperature to photograph the lunar eclipse.

January 2019 Lunar eclipse, photography by PJ Bennett

Almost as much as the lunar eclipse itself, I enjoy the discussion leading up to the eclipse. The news seemed to focus on the name of the eclipse, the super blood wolf moon eclipse. I will admit it’s a great name and each part of it means something. However, what if I told you that all total lunar eclipses have names.

A total lunar eclipse can only occur when there is a full moon. The full moon is essential because every month’s full moon has a name. February’s full moon (Feb 19, 2019) is the full snow moon. February is also a super moon the second of three super moons in a row March will also be a super moon. So, using the pattern from January Februaries full moon is a full super snow moon.

February 2019 Full Super Snow Moon, photograph By PJ Bennett.

Our fascination with eclipses is interesting. After all its not like they surprise us anymore, for instance, there will be a total Lunar eclipse in Denver on Feb 13, 2101, with its maximum at 7:46:33 pm. The precision of this prediction is, of course, dependent on the model of the solar system and our observations of the positions of the plants. I suspect our fascination with eclipses has to do with the fact that there are very few things that let us observe the workings of the solar system.

Regardless of why its fascinating astronomy is an excellent way to both increase interest in the STEM fields and teach research methodologies. Using astronomy to promote an interest in STEM is rather simple. Anytime there is an astronomical event it gets covered in all the media. Schools and organizations that promote STEM education should hold viewing parties. In addition to helping people get a good view of the celestial event having experts present to talk about the event and science, in general, helps stir interest in STEM fields.

While I have seen some schools, observatories, and planetariums hold viewing parties it has defiantly not been all or most schools. Additionally, these viewing parties would make a great cornerstone for a larger event that involved multiple STEM fields. Helping participants understand that all the STEM fields are related and accessible will only help improve interest in the STEM disciplines.

Beyond promoting general interest in STEM, the history of astronomy makes a great teaching tool for the scientific method. Anytime there is an eclipse especially a total solar eclipse someone always talks about how terrified this event must have been for early peoples. We take for granted that we can predict eclipses.

In the media, we tie our ability to predict eclipses to our understanding of the plant’s motion around the sun which was first formally proposed by Copernicus in the 1543 publication of On the Revolutions of the Heavenly Spheres. The only significant flaw with Copernicus’s model is that he thought the orbits had to be perfect circles.

Before Copernicus, the astronomic model of the solar system was dominated by the Ptolemaic model which had the earth at the center of the solar system (the center of the Universe). This model lasted for about 1400 years. However, even with incorrect or incomplete models of the solar system, the ability to predict eclipses has existed for at least 2000 years probably longer. For instance, the Dresden Codex is a Mayan book written sometime in the 13^th or 14^th century; the authors based the codex on a Mayan book several centuries older. The codex contains calculations on astronomy including accurate predictions of eclipses for both the sun and the moon.

Six sheets of the Dresden Codex (pp. 55-59, 74) depicting eclipses, multiplication tables and the flood. Auther is unknown, This work is in the US public domain. — Six sheets of the Dresden Codex (pp. 55-59, 74) depicting eclipses, multiplication tables and the flood. Author is unknown, This work is in the US public domain.

Using the information in the Dresden codex anthropologists Harvey and Victoria Bricker were able to predict the Central American solar eclipse of July 11, 1991, to within a day in 1983 (If you’re interested the full paper is here.) Considering that we must convert the Mayan calendar to match our calendar that is amazingly accurate for something written hundreds of years before Copernicus published his model of the solar system.

We also know that the Mesopotamians and ancient Greeks predicted eclipses perhaps as far back as 2000 years. (Griggs, M.B. (2017, August 18). We’ve been predicting eclipses for over 2000 years. Here’s how. Retrieved from https://www.popsci.com/people-have-been-able-to-predict-eclipses-for-really-long-time-heres-how) If the correct understanding of the motion of the plants in the solar system is a relatively new thing how did older cultures predict eclipses and how does this help explain why the scientific method is essential?

Older cultures were able to predict eclipses because they follow a repeating cycle called the Saros Cycle, which is approximately 223 months long. If a civilization lived long enough and its records were accurate enough deriving the Saros cycle is possible. Information on the periodicity of celestial events and observations of the night sky let individuals like Aristotle and Ptolemy developed the first models of the solar system with the earth at its center, also known as a geocentric model.

The Solar System according to the geocentric model of Claudius Ptolemaeus. By Andreas Cellarius. This work is in the US Public domain.

So how did this Ptolemaic model and its decedents last for almost one and a half millennia? The biggest reason is that the model fits all the relevant data and for most of this period the scientific method as we know it didn’t exist.

If the modern scientific method had been present at the time of Ptolemy, his geocentric model would have been a hypothesis, a prediction based on observation. Again, using the scientific method astronomers would have tested the model by either trying to disprove it or by trying to disprove an alternative hypothesis. Nowadays we understand that the best experiments are the ones designed to either disprove a hypothesis or distinguish between competing hypothesis. At the time of Ptolemy, astronomers did not challenge the model because it matched the observations and social beliefs.

Using the models of planetary motion from Ptolemy to Kepler makes an excellent background for discussions of the scientific method. For the average person, all three models appeared to work and could predict celestial events. Because they lacked our modern approach to science, several of these models persisted much longer then they could have. Linking education to current events that capture people’s attention and excite them is one of the best ways to motivate a student. Next time a science-related story catches peoples attention think about how you might use it for education or motivation.

Thanks for Listing to My Musings
The Teaching Cyborg

My Students Need to Turn Knobs in Labs

On January 31, 2019January 31, 2019 By teachingcyborgIn Pedagogy, Research, STEM EducationLeave a comment

“In general, obsolete technology is obsolete for a reason. Monocles are no exception.”
Neil Blumenthal

Many science faculty view laboratory classes as a central component of science education. Many groups have come out in favor of the laboratory class. According to the America Chemical Society (ACS), “Hands-on laboratory science experiences are critical to the learning process across all areas of study, beginning with kindergarten and continuing through post-secondary education.” (Public Policy Statement 2017-2020) The National Science Teachers Association says “For science to be taught properly and effectively, labs must be an integral part of the science curriculum.” (NSTA Position Statement)

What is the laboratory class? According to America’s Lab Report: Investigations in High School Science “Laboratory experiences provide opportunities for students to interact directly with the material world (or with data drawn from the material world), using the tools, data collection techniques, models, and theories of science.” (NRC 2006 p. 3) while the ACS says, “well-designed laboratory experiences develop problem-solving and critical-thinking skills, as well as gain exposure to reactions, materials, and equipment in a lab setting.” (ACS Public Policy Statement 2017-2020)

While these definitions have some similarities, they also have differences. I know science faculty that think we should get rid of science labs and faculty that believe we can’t teach science without them. The thing that surprises me the most is that a many science faculty tell me that one of the most important aspects of laboratory science is learning to use the equipment.

I was involved in a redesign of a physics laboratory course; this course had not been reviewed or updated in, let’s just say “a really long time.” We were discussing an acceleration due to gravity lab. The main goal of this lab was to understand that acceleration due to gravity is independent of mass. This experiment is often run using an air track which is a device that uses air to produces a relatively frictionless surface for a “car” of different masses to run on. I won’t go into the reasons but setting up the air track to get accurate readings can be difficult.

Several of us proposed some changes to make the set-up easier so that students could collect more significant amounts of data; this would give us more opportunities to build analysis and data testing into the lab report. One of the faculty members argued that he set up his lab so that the students had to spend 80+% of their time setting up the equipment because the most important thing for the students was to “learn” how hard it was to collect accurate data. Ask yourself what does this have to do with the learning goal?

While developing biology labs, many faculty members have told me “my students have to learn to twist the knobs on a microscope.” I graduated from graduate school in 2006 even then every microscope I used was connected to a computer and most of them could not be run without a computer. I rarely twisted knobs. Additionally, most of these labs had learning goals associated with learning to identify cellular organelles or the differences between different types of muscles. Even if the students end up using a non-automated microscope what does twisting knobs have to do with the learning goals?

Beyond an incorrect aliment with learning goals in a world where technology is rapidly evolving it is almost impossible for student labs to teach the use of equipment that will not be obsolete by the time they graduate. As Hofstein and Lunetta said “It is unreasonable to assert that the laboratory is an effective and efficient teaching medium for achieving all goals in science education” (Review of Educational Research Vol. 52, No 2 pp. 201-217) They do suggest that laboratory activities can be used to develop inquiry, problem solving, and observational skills.

Over the last few decades, all this mixed information has allowed laboratory education to come under increased attack. Several years ago, I worked with an assistant dean of engineering to develop an assessment tool he could use to reinforce the value of lab classes because the college wanted to cut back on lab classes. Beyond this example lab classes have been subject to a lot of attack over recent years. From an administrative point of view, there are questions about the cost; laboratory classes are the most expensive classroom on campus.

Beyond cost laboratory classes are often assigned the same learning goals as the lecture classes. Some argue if the two classes are doing the something couldn’t the extra time be better used on additional material? Especially since there are countries that don’t have lab courses in their curriculum. (Science Education Vol 88, #3, p. 397-419)

So, what does this mean for faculty members and instructional designers in science? First when it comes to laboratory classes making sure we have clearly defined learning goals may be even more critical than it is in lecture classes. Making sure that the activities in the lab support the learning goal are a must. Lastly, we need to spend more time thinking about why we use labs, what labs can be used for that other forms of education can’t and focus on them. If we want lab science courses to last, we need to start fighting for them now.

Thanks for Listening to My Musings

The Teaching Cyborg