Author: teachingcyborg

In Research We Trust

On By teachingcyborgIn Education Reform, Research, Statistics, STEM and Society, STEM EducationLeave a comment

“Facts are stubborn things, but statistics are pliable.”
Mark Twain

Anyone that knows me knows I believe in research and data backed decisions in education.  Successful research is a balancing act between skepticism and an openness to new sometimes radical ideas.  To avoid the possibility of bias, we have developed methodologies and techniques to determine the validity of an experiment.  Experimental validity falls into two categories: internal, experimental design, data collection, and data analysis. The second is external, the progression from hypothesis to theory, and finally to the fact.  Research drives the progression from hypothesis to fact with supporting evidence and replication.

Considering how vital replication is to research, there appears to be very little direct replication.  Makel and Plucker showed that only 0.13% of educational research is replicated (Facts Are More Important Than Novelty: Replication in the Education Sciences).  Compared to a rate of 1.07% in psychology and 1.2% for marketing research.  However, the rate of replication does not tell the whole story.  After all, to publish research, you need to conduct an experiment, submit it for peer review, make changes, and then have your article published.  Perhaps we can accept published results.

Looking at actual replication studies suggests that publication is not enough.  One study in psychology, Estimating the reproducibility of psychological science, was only able to replicate 63% of the studies they examined.  Replications of clinical research are even worse.  A group from Amgen attempted to replicate 53 research studies in cancer research they only replicated 6 of them.  Additionally, a group with Bayer Health could only replicate 25% of the preclinical studies they tested (Drug development: Raise standards for preclinical cancer research). 

So how do we resolve the replication crisis?  We need to reproduce previous research and publish the results.  The problem is that professors, postdocs, and graduate students don’t benefit from replication studies.  Even if researchers get the articles published, they don’t carry the same weight as original research.  One possibility would be to have graduate students replicate experiments at the beginning of their graduate study as part of their training.  However, this is probably not a workable solution as it would likely lengthen the time to degree. 

So, who would benefit from reproducing research?  The answer is undergraduates.  Conducting replication studies would more effectively train students in research methodologies than any amount of reading.  Why would conducting replication studies help students with research design?  The reason is that if you replicate a study perfectly (exactly as undertaken previously), you might have the same problems the original researchers had.  After all, most of the issues in research are not intentional but unintentional and probably unidentifiable problems with data collection or analysis.

Statistical analysis of most data involves a null hypothesis.  When the data is analyzed, the null hypothesis is either accepted or rejected.  Errors analyzing a null hypothesis, are classified as Type I (rejecting a correct null hypothesis) or Type II (accepting a false null hypothesis).  The critical thing to keep in mind is that it is impossible to eliminate Type I and II errors.  Why can’t researchers eliminate Type I and II errors? Think about a P value, P < 0.001, what does the number mean.  Written in sentence form as P value < 0.001 means: the likely hood that these results are the product of random chance is less than 1 in 1000.  While this is a small number, it is not zero, so there is still a tiny chance that the results are due to random chance. Since P values never become P < 0, there is always a chance (sometimes ridiculously small) that results are due to random chance.

In addition to Type I and II errors, there could be problems with sample selection or size. Especially early in the research were influencing and masking factors might not be known.  Alternatively, limited availability of subjects could lead to sample size or selection bias.  All these factors mean that a useful replication study looks at the same hypothesis and null hypotheses but uses similar but not identical research methods.

Beyond the benefits students would gain in experimental design, they would also learn from hands-on research something that many groups say is important for proper education.  Additionally, replication research is not limited to biology, chemistry, and physics.  Any field that publishes research (i.e., most areas of study) can take part in undergraduate replication research.

Of course, these replication studies will only benefit research if they are published.  We need journals to publish replication studies, how do we do that.  Should a portion of all journals be devoted to replication studies?  The Journal Nature says it wants to publish replication studies; “We welcome, and will be glad to help disseminate, results that explore the validity of key publications, including our own.” (Go forth and replicate!).  Hay Nature how about really getting behind replication studies! How about adding a new Journal to your stable, Nature: Replication?

However, if we want to disseminate undergraduate replication studies, it may be necessary to create a new Journal, The Journal of Replication Studies?  With all the tools for web publishing and e-Magazines, it should be straight forward (I didn’t say free or cheap) to create a fully online peer-reviewed journal devoted to replication.  Like so many issues, the replication crisis is not a problem but an opportunity.  Investing in a framework that allows undergraduate to conduct and publish replication research will help everyone.

Thanks for Listing to My Musings
The Teaching Cyborg

Common Core Math Does it Work?

On By teachingcyborgIn Education Reform, Pedagogy, Research, STEM and SocietyLeave a comment

“Do not worry about your difficulties in Mathematics. I can assure you mine are still greater.”
Albert Einstein

A friend of mine sent me a YouTube video comparing common core math with “old math.”

https://youtu.be/Nlfa5v3ullg

My first thought was this is the dumbest thing I have ever seen. Now let’s be clear my reaction was not because the old math was so much faster. After all, the person doing the old math is merely solving an equation. The teacher is instructing the students in a common core mathematics process which takes longer. So it was not the length, it seems to me that the process is complicated, off track, and fails in several cognitive theories.

However, I believe in letting the research speak for itself, which means double checking your opinions with the literature. Most of my work is at the college and university level with a focus on STEM education. So what effect has the common core had on college students, primarily STEM students?

Before we look at the effect of the common core standards lets review what the common core is. The common core standers are a guideline of what students should learn each year of K-12 education. The standards are meant to be rigorous and meet the need of colleges and employers. According to the criteria for the working group, each standard should have:

“Goal: The standards as a whole must be essential, rigorous, clear, and specific, coherent, and internationally bench marked.

Essential: The standards must be reasonable in scope in defining the knowledge, and skills students should have to be ready to succeed in entry-level, credit-bearing, academic college courses, and in workforce training programs.”

The publishing of the full common core standards was in 2010. As of 2017, 46 states have adopted the common core standard to some degree. Eleven of the states have announced they are undertaking rewrites and changes to the standards.

Even with 11 states announcing rewrites or changes, this is still a high adoption rate. The adoption rate does not tell the whole picture. In K-12 education a lot is left up to the local school districts. While states have adopted the standards, it is not clear how consistent implementation is. It will likely get even harder to study the common core standards, as many states are renaming and modifying the standards. Many of these changes may be cosmetic as Tom Loveless says:

“A lot of states have simply re-branded the standards, changing the name or slightly tinkering with them without making any great change in substance” Loveless says. “That to me suggests that it’s more a political response than anything else.” (Common Core no more? New York and 21 other states revise or rename K12 standards, District Administration, By Alison DeNisco | October 9, 2017, retrieved June 6, 2019, from https://districtadministration.com/common-core-no-more-new-york-and-21-other-states-revise-or-rename-k12-standards/)

How do teachers view the standards? According to a report by the Center for Educational policy: “Across the five focus groups, most elementary school teachers expressed positive views of the Common Core State Standards. … Teachers said the Common Core had changed instruction in positive ways, such as teaching for conceptual understanding and developing students’ thinking and problem-solving skills.” (Listening to and Learning from Teachers: A Summary of Focus Groups on the Common Core and Assessments Key Findings and Policy Recommendations, Center on Education Policy, By Diane Stark Rentner, Nancy Kober, Mathew Frizzell, and Maria Ferguson, October 12, 2016, Retrieved June 6, 2019, from https://www.cep-dc.org/cfcontent_file.cfm?Attachment=RentnerKoberFrizzellFerguson%5FSummary%5FListenLearnTeachers%5F10%2E12%2E16%2Epdf)

So why don’t I like the method of mathematics presented in the video? Let’s look at the steps the students are being asked to do when answering, 35 x 12. In the first step the students break the numbers down into their components 35 = 30 + 5 while 12 = 10 +2. Students then plug the numbers into a grid and multiplication is done by multiply the rows by the columns. The multiplication produces four numbers which are added to get the final answer.

I have heard several arguments about why this method is better. First, it teaches students how to manipulate numbers. Second, by breaking the numbers apart, it is easier for students to remember and do the math in their head. The grid is a rectangle some instructors use area equations to represent the multiplication, height x width = area. By using this representation, students get a feel for the real size of numbers.

While I agree learning to manipulate numbers is essential for students, I am not sure this method teaches students that. I think it is more likely that students are viewing this as a trick or formula. We know from research that students are good at plugging numbers into formulas without understanding what they mean. Just look up the original research on the Force Concept Inventory Test.

The idea that this method makes it easier to do in your head sounds intuitively correct. However, it might fall short of our research on how memory works. Again we know that working memory has a capacity limit (I wrote about it here).

So when multiplying 35 x 12 in your head, you have to remember two numbers. When you separate the numbers, you need to remember four numbers; 30, 5, 10, & 2. Additionally, as I do the math, I need to remember more numbers 30 * 10 = 300. I need to remember; 30, 5, 10, 2, & 300 additionally, I need to remember that 300 is different than the other four. Using this method, it is more likely that a student will run out of working memory.

Lastly, I have two problems with using the grid to represent the actual size of the number. There is a counter argument of numerals being symbols so we can deal with numbers that we can’t intuitively grasp. However, that is not the biggest problem; the real issue is transference. Transference is the ability of students to take the information they learned and use it in new situations. If students get to fixated on numbers representing fiscal shapes and physical quontites, they may have trouble with things that are difficult to see or understand.

So what does the research say about college students that were taught using the Common Core standards during their K-12 years? According to a 2016 study, there is disagreement about what math standards college students need. “Mathematics finding 4 indicates that although middle school and high school teachers generally agree about what mathematics skills are important to success in STEM courses and careers, college instructors or workforce respondents ascribed much less importance to those skills.” (ACT National Curriculum Survey 2016, ACT, Inc, retrieved June 6, 2019, from http://www.act.org/content/act/en/research/reports/act-publications/national-curriculum-survey.html ) At least part of this discrepancy comes from colleges and universities have different views and requirements. The 2015 Brown Center Report on American Education (https://www.brookings.edu/research/2015-brown-center-report-on-american-education-how-well-are-american-students-learning/) shows small gains in student performance in states that fully implemented the common core standards. Unfortunately, these difference are below or borderline concerning statistical significance.

Sadly it appears there is not a lot of research, at least yet, on the common core standards. What research exists seems to be leaning in the direction of the standards not living up to its goal. Whether this is the results of implementation or the standards themselves, it is not clear. For the time being, I will have to live with my dislike while trying to keep an open mind. What is defiantly clear is that more research, mainly that focused on learning gains, is desperately needed. Also, colleges and universities frantically need to work with K-12 so that everyone knows what is the need and expected of students perusing higher education.

Thanks for Listing to my Mussing
The Teaching Cyborg

Is Free Enough?

On By teachingcyborgIn Education Reform, ResearchLeave a comment

“Our progress as a nation can be no swifter than our progress in education. The human mind is our fundamental resource.”
John F. Kennedy

The history of free vs. tuition-based education is a long a twisted tail for the United States, one we are still debating. President George Washington encouraged public education as part of his farewell speech.

“Promote then, as an object of primary importance, institutions for the general diffusion of knowledge. In proportion as the structure of a government gives force to public opinion, it is essential that public opinion should be enlightened.”
George Washington, 1796

When the United States formed, there was no public education system. Children were educated at home by either parents or privet tutors.

The first schools in the US were established by religious groups to teach literacy for bible study. The first public school systems (covering areas more than a single town) began to emerge in New England in the 1840s. In 1852 Massachusetts passes a law making elementary education compulsory. Something that the rest of the United States would not do until the 1900s.

Today we have a system of publicly funded compulsory education from Kindergarten to High School. While there has been disturbing decrees in government funding for public higher education, historical governments have been involved there as well. Examples of this are land grant colleges and university, the GI bill, and Pell grants.

In recent years there has been a lot of discussion about free public education at the college and university level. The current debate over free college education was kicked off by President Barack Obama’s 2015 State of the Union address. He proposed to cut the cost of community college.
“That’s why I’m sending this Congress a bold new plan to lower the cost of community college — to zero.” Since then, there have been arguments at the national level about the validity and cost of a free community college education. However, just like with elementary and high school education states are leading the way while the federal government debates.

There are currently 20 states that offer some form of free community college (College “Free for All” in Almost 20 States!, by Susan Dutca-Lovell, Scholarship, January 8, 2019 4:15 PM, retrieved from https://www.scholarships.com/news/college-free-for-all-in-almost-20-states on May 29, 2019) New York is even offering its program to 4 year college students. Most of these programs are last dollar programs; they cover whatever is left over after financial aid is exhausted.

While these state-level programs will undoubtedly make a college education more accessible, I wonder if free is enough. Higher education and its interactions with society can be a complicated process with a large number of pitfalls. However, it may be even more complicated than we ever thought. While earning a degree increases your earning potential. According to the Association of Public and Land Grant Universities, lifetime earning potentials are:

  1. High school diploma is $1.3 million
  2. Associates degree is $1.7
  3. Bachelors degree is $2.2 million
  4. An advanced degree is $2.7 million

How does a college degree improve graduates’ employment and earnings potential?

Research suggests earnings potentials might not be as clear cut as these numbers indicate. All of us have heard the comment; “It doesn’t matter where you start your undergraduate education. All that matters is where your final degree is from.” Vanderbilt Law professor Joni Hersch published a paper, Catching Up Is Hard to Do: Undergraduate Prestige, Elite Graduate Programs, and the Earnings Premium that challenges the comment that only the final institution matters.

In her research, she compared students that earned their bachelors degrees from a lower tier (Carnegie Classification system) school than their graduate or professional degree. With thoughts that got both their bachelors and graduate/professional degree from higher tier schools. She found students who moved up to a higher tier school for their terminal degree had a salary that averaged $52 thousand less than graduates that started at a higher tier school. That salary difference works out to nearly $1.6 million over a 30-year career.

Additionally, it appears to be challenging to move up in from a lower tier school when applying to graduate school. Nearly 33% of all tier 4 bachelors recipients go on to earn a graduate degree. However, only 7% of these tier 4 bachelors students earn their graduate degrees from a tier 1 institution. Nearly 66% of all tier 4 bachelors students that pursue an advanced degree earned their degree from a tier 4 institution. The low student transfer rate suggests that it is difficult to move up in tiers for graduate degrees. Even if students do move up, they don’t have the same earning potential.

While making college free is a big step in making a college education accessible, several other questions need to be asked and addressed. Why do students that transfer from lower tiers to higher tiers still earn less? Even if there is a difference in rigor between different tiers the students received their final degree from the higher tier, there should be no difference. Professor Hersch suggests that the difference might come from things outside academics, like networking, family connections, and job/career assistance. Additionally, Why is the transfer rate up to the school tiers so low? How does the addition of an associates degree or community college effect these issues?

Therefore, merely making community college and even four-year state schools free will not completely level the playing field. We probably need to invest in accessory programs to help students make connections, network, gain real mentors, and gain an understanding of what their career will require. But most importantly, we need to do research and determine if problems are internal to academia, dependent on society or more likely both. Then we need to find a way to fix them. Free is part of the solution; however, we need to remember free is not all of it. If we don’t address the rest, we will still be wasting all that human potential.

Thanks for Listing to My Musings
The Teaching Cyborg

Pixels or Ink

On By teachingcyborgIn Pedagogy, Research, TextbookLeave a comment

“The technology itself is not transformative. It’s the school, the pedagogy, that is transformative.”
Tanya Byron

In an earlier blog post (To Be Digital or Not to Be Digital) I discussed how governments and schools are encouraging the adoption of digital media especially etextbooks (etexts) as a means of controlling cost. However, in most cases, these groups have failed to look at whether or not students want to use these etexts. Because of multiple issues, current students are not adopting etexts at a high rate. However, even if institutions deal with the problems of etext adoption, should we be using etexts?

While cost is an import factor in making education accessible, we need to be careful that cost does not run roughshod over pedagogy. In addition to the questions of whether students will use current etexts, we also need to ask to do etexts work, are etexts a legitimate pedagogical tool? Does it make any difference if words are on paper or screen to the learner? I don’t think written language has been around long enough for the brain to evolve to function exclusively with print.

Modern human evolved between 200,000 to 300,000 years ago in Africa (Smithsonian and UK National History Museum. Around 5200 years ago, humans invented writing. Writing gave us the ability to record complex ideas, theories, and information. Writing has only existed for 2.6% – 1.7% of human existence.

For most of writings existence information was recorded on things like; clay, stone, and paper. Today our writing can also be displayed on screens. While it might not be obvious, there is some evidence that the method of reading on paper vs. electronic screen might make a difference in learning.

Digital reading or more specifically reading on a digital device has many advantages. As Singer & Alexander say in their paper Reading on Paper and Digitally: What the Past Decades of Empirical Research Reveal “These paperless classrooms allow the reader to alter the size of the text, highlight important passages, and search related terms outside of the text with the click of a button.”

One of the outcomes from Reading on Paper and Digitally: What the Past Decades of Empirical Research Reveal are; students had better comprehension when reading a printed book when the text length was more than one page or screen. Researchers have proposed that scrolling text leads to increased cognitive load, which would negatively impact comprehension.

Interestingly when it came to general comprehension (general or broad topics), there was no difference in basic comprehension between digital and paper texts. However, when it came to specific questions (specific facts, comparisons, and deep understanding), there was better comprehension from paper texts over digital. It is possible that this difference between general and specific content has to do with differences in how students read in a digital vs. paper environment.

When the research was designed to study comprehension between digital and paper reading while reducing cognitive load, there were no significant differences in comprehension between digital and paper. However, it did appear that there was a broader deviation in the digital group then the paper group. (The Effects of Reading Mode on Recall and Comprehension). Why is there a difference between different types of comprehension?

Research into student reading has shown that digital readers have developed different reading habits. In Reading behavior in the digital environment: Changes in reading behavior over the past ten years it is shown that digital readers make greater use of short cuts like skimming for keywords, bookmarks, and skimming the text then paper readers. Additionally, it has been shown that it is much easier for students to become distracted by multitasking when using digital texts (David B. Daniel and William Douglas Woody, “E textbooks at what cost? Performance and use of electronic vs. print texts,” Computers in Education, Vol. 62 (March 2013): 18-23)

In addition to a lower rate of adoption of etexts, the current research suggests that etexts, in general, might hurt learning especially deep learning. However, except for the cognitive load because of scrolling, there is little information that shows etexts have a significant (cognitive) adverse effect on learning. I would be interested in variations on this cognitive load of scrolling.

While I have encountered a lot of etexts that require scrolling, especially on the web, I have encountered other types of digital page turning. On my tablet (Samsung Galaxy), I have read books that have the same layout as print books where you flip pages from the edge of the screen. I also have, an e-reader (A dedicated ebook reader) again, the reader uses the same basic layout as a print book, and readers turn pages with small buttons. I would be interested in seeing if these methods of digital reading had the same effect on cognitive load and comprehension as scrolling.

The issues of digital devices leading to more significant amounts of skimming versus in-depth reading and increased multitasking is an issue of training. Somewhere along the line, we forgot that part of a students educational training is teaching them how to study and learn. It may well be that due to external forces, education will convert to the effective use of etexts. While there might be problems with etexts, many of the issues could be dealt with by teaching students how to study and read using etexts. Almost all of the ereaders (both physical and app-based) give students the ability to highlight, take notes, bookmark, and link to additional materials. If we teach students to use these tools, it might be that the comprehension differences will go away. Anyone care to look into it?

Thanks For Listening to my Musings
The Teaching Cyborg

Polymaths Everywhere

On By teachingcyborgIn Education Reform, Pedagogy, ResearchLeave a comment

“I had a terrible vision: I saw an encyclopedia walk up to a polymath and open him up.”
Karl Kraus

Diploma_in_Acting by Fahadseo [CC BY-SA 4.0] A student shakes hands while receiving his diploma.
Diploma_in_Acting by Fahadseo [CC BY-SA 4.0] A student shakes hands while receiving his diploma.

It seems like people have been talking about reforming college degrees “forever.” Obviously, this is not true, or maybe it is, academics are always trying to design something new. The redesigning of degree programs usually fall into one of two categories; time to degree, and employability after graduation. Concerns over cost is driving discussions about time to degree and employability. The argument is if it takes less time to earn a degree, it will cost less, therefore, make it more accessible and affordable. The case for employability is that degrees should focus on skills that employers want so that degrees are a better investment.

Just recently, I read about another degree idea, the polymath degree. Project polymath run by The Polymath Foundation offers such a degree. The idea behind the project is to create a school that trains polymaths, individuals that “think” like da Vinci. According to Merriam-Webster, a polymath is “a person of encyclopedic learning,” according to Wikipedia a polymath is “a person whose expertise spans a significant number of subject areas, known to draw on complex bodies of knowledge to solve specific problems.”

In the polymath project, projects are the center of the educational experience. It seems to me that a lot of these projects are focused on industry partners and creating startups; in fact, the school lists professional applications as the most critical metric in their educational system. The degree will be the standard four years in length but student, with the help of mentors, follow education units rather than courses.

In addition to Project Polymath, there is also the London Interdisciplinary School (LIS) which offers a similar program to Project Polymath. Both programs talk about needing a new approach to solve today’s complex problems. They argue that issues stretch across disciplinary boundaries, so education needs to as well. LIS describes their coursework as,

“We believe that real-world problems require an interdisciplinary approach. This is why we offer one course that cuts across disciplinary boundaries. Our course takes the most fundamental theories and knowledge areas from across the arts, sciences, and humanities and applies them to real-world problems.”

In addition to employability, polymath degrees are an obvious counter to the increasing number of specialized college degrees. I am a little concerned with over specialization. When I graduated with my BS in Biology in ’97, the only biology degree you could get at my school was Biology. In 2019 that same school offers five specialized, or as they call it areas of emphasis, Biology degrees.

However, the fact that I earned BS degrees in both Biology and Biochemistry also shows that I do have a strong belief in cross-disciplinary training. However, unlike these polymath programs, I graduated with the additional credits for the two degrees 240 instead of 180. However, the real question is what is the balance needed between breadth and depth in a Bachelors degree.

The question of depth versus breadth is not a simple one. In addition to what you need for a successful degree, there is the fact that information is continually growing. Buckminster Fuller proposed the knowledge doubling curve in his book Critical Path. The knowledge doubling corve shows the rate at which knowledge doubles from year 1 to about 1945. If we took all the information created by the human race until year one as one unit it took till the year 1500 to double it, the next doubling occurred in 1750. By the end of World War II knowledge was doubling every 25 years. According to IBM’s Toxic Terabyte by the mid-2010s knowledge doubling time will decrease to hours.

The increasing amount of knowledge is one of the driving reasons for the growing number of specialized degrees. As the amount of knowledge increases the information an individual has to learn in their “field” increases. Eventually, it starts getting difficult to fit everything into four years. When this point is reached rather than change the structure of the degree like adding a year (after all life expectancy increased by 34 years from 1900-2000) schools created degrees with a narrower focus. It is indeed legitimate to ask; when do degrees get to narrow to be useful?

One of the counters to specialized degrees is a Polymath Degree. However, do polymath degrees even work from an educational point? Only time and hopefully, research will tell. Additionally, even if a Polymath degree works are they the correct solution to specialized degrees? A Polymath degree covers the fundamental theories and analytical methods across multiple disciplines. However, can students learn the information Polymath programs teach without learning the foundational information?

Alternatively, can you learn to properly use higher order thinking skills without first learning the lower order thinking skills? I don’t think so. We often forget a large part of first and second-year courses is learning the language of the field. As I have written about previously (The Language of the Field), the same words have a different meaning in different fields. If students don’t understand the meaning of the words, they can’t understand the nuances of theories and methodologies in a field.

The Polymath degree is designed to deal with what some see as problems in higher education. While I think the growth of highly specialized degrees, especially at the undergraduate level, are concerning I don’t think the solution is to create degrees without depth. I would like to see time added to undergraduate degrees to take into account the growth in knowledge. However, until we get a grip on the rising cost of education, lengthening the time of a bachelors degree will not happen.

Polymath degrees are an idea to reform higher education. One way or another, I look forward to seeing real research about student learning in these polymath programs. Maybe I will be wrong, and Polymath degrees will work, I’m not holding my breath. Even if polymath degrees work, there will still be a need for degrees with greater depth, traditional Bachelors degrees.

Thanks for Listing to My Musings
The Teaching Cyborg