We Can Rebuild It, Better, Stronger: The Augmented Textbook

“VR and AR will eventually converge, and smart glasses will take over our digital interactions.”
Carlos López (Founder @ Oarsis)

Augmented Reality (AR) is a process that uses technology to overlay digital content on real-world objects.  The digital content can be provided by, smartphones, glasses, and screens.  While AR is still an emerging technology, the buy-in from major companies like Microsoft with the HoloLens, WebAR support in Google Chrome using ARCore, and Apple’s augmented reality development kit ARKit, likely mean this technology is here to stay.

While the form factor used in AR will undoubtedly go through multiple iterations the primary function overlying digital content will remain constant.  AR is a great place for higher education to embrace technology and stay current rather than playing catch-up.  While wearable AR tech is not yet coming place, we can use the near ubiquitous smartphone with augmented reality.

There are already educational AR tools developed both inside and outside of education.  The Dinosaur 4D+ flash cards by octagon studio bring Dinosaurs to life.  Using an app installed on an Android or Apple device the flash cards allow you to explore and interact with the cards, as you can see here.

The International Society for Technology in Education (ISTE) has a blog post by Larysa Nadolny Worksheets for the digital age: AR interactive print.  The author gives a brief overview of the creation of these AR worksheets using existing technology.  Case Western Reserve is using AR to help teach anatomy, using the Microsoft HoloLens.  Students can see the anatomical process in active 3D.

Publishing companies are also starting to use AR in their books.  Carlton Books has two categories of AR books an educational category including titles like Explore 360: The Tomb of Tutankhamun and iExplore – Bugs that use AR apps to bring the content to life and let the readers interact with it.  They also have a new category of fiction novels they are working on; the first is The Ghostkeeper’s Journal and Field Guide a book that uses AR to enhance the story and engage the reader.

Many companies are producing AR books. Currently, the publishers are mostly focusing on the children and youth market.  These books have evolved from some simple animations like moving gears and simple 3D animals to full multimedia that include animations, sound, and interactivity.  Some of these books like the previously mentioned The Ghostkeeper’s Journal and Field Guide were written to include the book and its AR content as part of the story.

I have previously discussed how storytelling is a powerful educational tool (you can read about it here), I wish it was used more in textbooks.  If AR can enhance storytelling like these publishers are suggesting it should also enhance learning. While some people think the AR in books is gimmicky, I think anything that increases engagement with books is good.  Also, with regards to AR being gimmicky while Arthur C Clark said: “any sufficiently advanced technology is indistinguishable from magic” technology doesn’t have to be “magical” to be effective in learning.

This discussion of AR and books brings me back to the idea of textbooks.  The addition of augmented reality to textbooks can enhance education.  Let’s start by thinking about the basic content in a textbook.  We could add something simple like sound.  Imagine a music appreciation course; the textbook could describe techniques used in improvisational jazz.  Say for instance arpeggio, where the musician plays the notes of a chord one after the other instead of together.  Think how much easier this would be to understand if the textbook could play clips of music with and without arpeggio.

In biology, we often talk about how seasonal changes affect the local ecology and behaviors of organisms.  A great example of this is the Amazon Floodplain forests.  A large area of the Amazonian forest that is flooded every year in the rainy session when the Amazon river is overflowing its banks. Textbooks will often show flooded, and dry pictures to show the effects of the flooding.  With AR you could show a time-lapse video of the flooding and retreating water to get a better idea of how the water affects the landscape.

Something I remember from my days as an undergraduate in chemistry and biology is the difficulty students have learning to translate a 2D model into 3D.  Molecules are 3D objects when writing about them; we need to represent them on paper.  A simple model would be the wedge and dash model used for methane below.

Structure of Methane By NEUROtiker Downloaded from Wikimedia Commons.
Structure of Methane By NEUROtiker Downloaded from Wikimedia Commons.

In the diagram, the solid wedge means the atom is projecting out of the paper towards you while the dotted wedge means the atom is projecting away. I was one of the lucky students I have always been able to picture the 3D shape of from these drawings rather easily. However, I have known a lot of people that have real trouble seeing the 3D form.

Now imagine if the textbook had AR we could design interactions that not only projected the molecule in 3D but let the students manipulate, rotate, and zoom in and out to examine them.  AR projections would be especially useful when you get into more complex structural issues like stereochemistry, were molecules have the same formula but differ in their shape.

A textbook on public speaking could include actual audio and video clips of famous speeches.  A math book could include video clips were professors solve example problems with explanations.  We already know that publishers are taking advantage of AR especially in the case of books for young audiences.  However, AR textbooks are starting to appear, Introductions to Graphics Communication is a college-level textbook using Ricoh’s Clickable Paper.  Publishing companies in Japan have released textbooks with AR; you can read about them here.

Even with the availability of many AR platforms some of which are Augment, Blippar, HP Reveal, Daqri and Layar that offer educational pricing.  I have not seen any Open Educational Resource (OER) textbooks with AR content even the textbooks developed with large federal or privet grants.  In addition to whether governmental and privet organizations will be willing to pay to update these OER textbooks in a few years, are we also going to end up in a situation where we have different classes of textbooks? Is there going to a case where if you can afford it you get a different textbook?

Augmented Reality is a technology that higher education needs to embrace.  We need to develop not only resources using AR but the tools, preferably in a free and opensource platform, we can use to incorporate into any resource where it makes sense.  Textbooks are a resource where AR makes a lot of sense.  Like I have said before we are in the middle of a revolution regarding textbooks it is critical that we don’t focus on just one aspect of the textbook.  We need to think about what we want a textbook to be in total, and one of the things we should add is AR.

Thanks for Listing to My Musings

The Teaching Cyborg

Researching Prototyping and STEM Education

“The visionary starts with a clean sheet of paper, and re-imagines the world.”
Malcolm Gladwell

Microscopes are an essential piece of scientific equipment they gave us the ability to view parts of the world that we can’t see otherwise.  The invention of the microscope lead directly to germ theory which revolutionized healthcare. Throughout my career I’ve done a lot with microscopes; research, teach, maintenance, and I’ve even worked with a group to make them remote controlled.

Microscopes can also be extremely expensive, I worked with a microscope that cost a million dollars, and some microscopes cost more than that. Microscopes are particularly crucial in pathology and medical diagnostics. Which in some cases can be a problem; the cost of microscopes can be limiting in some areas of the world.

Take for instance sub-Saharan Africa; malaria is one of the most common causes of death due to illness in this region. According to the CDC 90% of all the worlds malaria-related deaths are in sub-Saharan Africa. Which is sad because malaria is completely treatable especially if identified early. The problem is malaria can present like the flu. Without going to it all the reasons the only way to conclusively diagnose an active malaria infection is by a stained blood smear observed under a microscope.

In the United States, this is not a problem if your local medical office doesn’t have a diagnostic lab; one is available within a few hours by medical courier. However, in places like sub-Saharan Africa diagnostics labs can be prohibitively expensive and far out of reach. A basic diagnostic microscope is going to cost several thousand dollars; a clinical centrifuge will also cost a couple of thousand dollars. In addition to the cost, this equipment can be difficult to transport and set-up.  The diagnostic equipment also requires electricity something that is not commonly available. So, you also need a generator and fuel.

In addition to malaria, poverty severely impacts sub-Saharan Africa. According to the World Bank in 2015, 66.3% of the population live on $3.20 a day or less $1160 a year, 84.5% lived on $2007.50 or less a year.  One of the effects of poverty is a lack of infrastructure which makes it difficult to access many areas. 

A potential solution to this problem came from Dr. Manu Prakash an associate professor of bioengineering at Stanford. In 2014 his group developed the Foldscope a small microscope built from paper, an LED, watch battery, and spherical lens, it has magnification from 140X to 2000X. The Foldscope cost less than a dollar to make.

In 2017 his group developed the Paperfuge a hand-powered centrifuge with speeds of 125,000 RPM it costs about $0.20.

The Foldscope and Paperfuge don’t require power they’re small and easy to transport and we can easily replace them because of their low-cost. These pieces of paper can change diagnostics in remote regions drastically.

So, what do the Foldscope and Paperfuge have to do with STEM education?  Historically building, prototyping, and testing a new device was a long and expensive process. The cost limited the development of products to a few high-end research institution and large companies.  In today’s world of desktop manufacturing and prototyping, the cost to prototype has come down and is readily accessible to most schools and institutions.

With desktop tools available you can imagine building research/teaching programs around social and educational problems. On the educational side tools like the Foldscope and Paperfuge can be used by groups of students to do fieldwork.  Imagine taking groups of students out to a field site and giving all of them a microscope and centrifuge to do examinations.

Alternatively, we could use the Foldscope and Paperfuge as a model.  Schools and classes could partner with a community organization to develop tools to deal with problems and issues these organizations are facing. Students will start by learning the science behind the issues and the existing solution if there is one. Then as a laboratory component, students would use modern desktop manufacturing tools to design, prototype, and test solutions. We could adapt this type of program to any level of school. Additionally, they would combine science, engineering, and community service in one class.

Thanks for Listing to My Musings
The Teaching Cyborg

What the Moon Can Teach Us About Science

“I still say, ‘Shoot for the moon; you might get there.’”
Buzz Aldrin

Last month on January 21, 2019, I stood in the snow in below freezing temperature to photograph the lunar eclipse. 

January 2019 Lunar eclipse, photography by PJ Bennett
January 2019 Lunar eclipse, photography by PJ Bennett

Almost as much as the lunar eclipse itself, I enjoy the discussion leading up to the eclipse.  The news seemed to focus on the name of the eclipse, the super blood wolf moon eclipse.  I will admit it’s a great name and each part of it means something.  However, what if I told you that all total lunar eclipses have names.

A total lunar eclipse can only occur when there is a full moon.  The full moon is essential because every month’s full moon has a name.  February’s full moon (Feb 19, 2019) is the full snow moon.  February is also a super moon the second of three super moons in a row March will also be a super moon.  So, using the pattern from January Februaries full moon is a full super snow moon.

February 2019 Full Super Snow Moon, photograph By PJ Bennett.
February 2019 Full Super Snow Moon, photograph By PJ Bennett.

Our fascination with eclipses is interesting.  After all its not like they surprise us anymore, for instance, there will be a total Lunar eclipse in Denver on Feb 13, 2101, with its maximum at 7:46:33 pm.  The precision of this prediction is, of course, dependent on the model of the solar system and our observations of the positions of the plants. I suspect our fascination with eclipses has to do with the fact that there are very few things that let us observe the workings of the solar system.

Regardless of why its fascinating astronomy is an excellent way to both increase interest in the STEM fields and teach research methodologies.  Using astronomy to promote an interest in STEM is rather simple.  Anytime there is an astronomical event it gets covered in all the media.  Schools and organizations that promote STEM education should hold viewing parties.  In addition to helping people get a good view of the celestial event having experts present to talk about the event and science, in general, helps stir interest in STEM fields.

While I have seen some schools, observatories, and planetariums hold viewing parties it has defiantly not been all or most schools.  Additionally, these viewing parties would make a great cornerstone for a larger event that involved multiple STEM fields.  Helping participants understand that all the STEM fields are related and accessible will only help improve interest in the STEM disciplines.

Beyond promoting general interest in STEM, the history of astronomy makes a great teaching tool for the scientific method.  Anytime there is an eclipse especially a total solar eclipse someone always talks about how terrified this event must have been for early peoples.  We take for granted that we can predict eclipses.

In the media, we tie our ability to predict eclipses to our understanding of the plant’s motion around the sun which was first formally proposed by Copernicus in the 1543 publication of On the Revolutions of the Heavenly Spheres.  The only significant flaw with Copernicus’s model is that he thought the orbits had to be perfect circles.

Before Copernicus, the astronomic model of the solar system was dominated by the Ptolemaic model which had the earth at the center of the solar system (the center of the Universe).  This model lasted for about 1400 years.  However, even with incorrect or incomplete models of the solar system, the ability to predict eclipses has existed for at least 2000 years probably longer.  For instance, the Dresden Codex is a Mayan book written sometime in the 13th or 14th century; the authors based the codex on a Mayan book several centuries older.  The codex contains calculations on astronomy including accurate predictions of eclipses for both the sun and the moon. 

Six sheets of the Dresden Codex (pp. 55-59, 74) depicting eclipses, multiplication tables and the flood. Auther is unknown, This work is in the US public domain.
Six sheets of the Dresden Codex (pp. 55-59, 74) depicting eclipses, multiplication tables and the flood. Author is unknown, This work is in the US public domain.

Using the information in the Dresden codex anthropologists Harvey and Victoria Bricker were able to predict the Central American solar eclipse of July 11, 1991, to within a day in 1983 (If you’re interested the full paper is here.) Considering that we must convert the Mayan calendar to match our calendar that is amazingly accurate for something written hundreds of years before Copernicus published his model of the solar system.

We also know that the Mesopotamians and ancient Greeks predicted eclipses perhaps as far back as 2000 years. (Griggs, M.B. (2017, August 18). We’ve been predicting eclipses for over 2000 years. Here’s how. Retrieved from https://www.popsci.com/people-have-been-able-to-predict-eclipses-for-really-long-time-heres-how) If the correct understanding of the motion of the plants in the solar system is a relatively new thing how did older cultures predict eclipses and how does this help explain why the scientific method is essential?

Older cultures were able to predict eclipses because they follow a repeating cycle called the Saros Cycle, which is approximately 223 months long.  If a civilization lived long enough and its records were accurate enough deriving the Saros cycle is possible. Information on the periodicity of celestial events and observations of the night sky let individuals like Aristotle and Ptolemy developed the first models of the solar system with the earth at its center, also known as a geocentric model.

The Solar System according to the geocentric model of Claudius Ptolemaeus. By Andreas Cellarius. This work is in the US Public domain.
The Solar System according to the geocentric model of Claudius Ptolemaeus. By Andreas Cellarius. This work is in the US Public domain.

So how did this Ptolemaic model and its decedents last for almost one and a half millennia? The biggest reason is that the model fits all the relevant data and for most of this period the scientific method as we know it didn’t exist.

If the modern scientific method had been present at the time of Ptolemy, his geocentric model would have been a hypothesis, a prediction based on observation.  Again, using the scientific method astronomers would have tested the model by either trying to disprove it or by trying to disprove an alternative hypothesis.  Nowadays we understand that the best experiments are the ones designed to either disprove a hypothesis or distinguish between competing hypothesis. At the time of Ptolemy, astronomers did not challenge the model because it matched the observations and social beliefs.

Using the models of planetary motion from Ptolemy to Kepler makes an excellent background for discussions of the scientific method.  For the average person, all three models appeared to work and could predict celestial events.  Because they lacked our modern approach to science, several of these models persisted much longer then they could have.  Linking education to current events that capture people’s attention and excite them is one of the best ways to motivate a student. Next time a science-related story catches peoples attention think about how you might use it for education or motivation.

Thanks for Listing to My Musings
The Teaching Cyborg

The Raven Paradox and Science

“Anything that thinks logically can be fooled by something else that thinks at least as logically as it does.”
Douglas Adams, Mostly Harmless

The democratization of knowledge is a tremendous and empowering idea. The internet plays a huge role in this democratization. The growth and expansion of the Internet are almost unfathomable. The growth of online video is an example of this. In the early stages of the Internet, one small picture could slow your website to a crawl. Now we’re watching 4K YouTube videos at 60 frames per second.

You can find almost anything on YouTube. Need to paint a room in your house there’s a video for that. Want to listen to your favorite band they probably have a channel. Want to know how to build an electric guitar there is a playlist for that. There are even channels focused primarily on teaching science. Some of my favorites are Dianna Cowern’s Physics Girl, Derek Muller’s Veritasium, Michael Stevens’s Vsauce, and Brady Haran’s Numberphile.

However, there are also other channels on YouTube presenting pseudoscience or even outright falsehoods. Did you know that the Flat Earth Society has its own YouTube channel? (No, I’m not linking to it!) As much as we might like the idea of deleting them if we support an open and free Internet and the democratization of knowledge we can’t.

Fortunately, a lot of them are easy to spot. However, what about videos that make a mistake or fall into a logic trap. What about videos recommended by YouTube? Does a YouTube recommendation increase the validity of a video?

The other day a video popped up in my YouTube recommendation feed the title intrigued me “The Raven Paradox (An Issue with the Scientific Method)” the video is by a channel TritoxHD which is a channel about “science, theory, and history!” The video concludes that scientists shouldn’t make overly broad generalizations.

The video centers around the Raven Paradox, which is an argument in inductive reasoning first presented by Carl Gustav Hempel and how it impacts on the scientific method. The raven paradox is interesting from a logical standpoint. The paradox is dependent on logical equivalents from a logical point of view; all A’s are B’s is equal to if not B then not A.

The paradox uses these two statements.

  1. All ravens are black.
  2. Something is not black; then it is not a raven.

Since these two statements are logically equivalent observing one is support for the other. As an example, the flower in my front yard is pink, this flower is not black, and it is not Raven, so this pink flower supports all ravens are black. If you are like most people, your response was just “WHAT!” The idea that dissimilar things can be used to prove each other is where the paradox comes from how can an observation of a flower have anything to do with ravens. Fred Leavitt does an excellent job of explaining how this works in his article Resolving Hempel’s Raven Paradox in Philosophy Now; my interest is in the description of the scientific method.

How does The Raven Paradox relate to the scientific method? Our YouTuber and others have suggested that many if not most hypotheses are of the format all A’s are B’s. In this case, the YouTuber makes his first mistake when he takes All ravens are black as a hypothesis.  The video states that the hypothesis is the first step in the scientific method, this is not true.

I like to think of the scientific method is a cycle that we can enter from any point, so there isn’t a first step. However, if you think of the scientific method linearly the first step is to ask a question.

Two representations of the Scientific Method one circular the other liner.
Two representations of the Scientific Method one circular the other liner.

Following in the raven example the question would be “Is there a trait that all ravens share?” Then you’d go out and observe ravens. This step is necessary because a hypothesis is a prediction based on observation. So, if you need observations to make a hypothesis, the hypothesis can’t be the first step. Our YouTube author even states that a hypothesis is a prediction based on observation.

An important thing to know about the hypothesis all ravens are black is that while very rare there are white (albino) or cream (leucistic) colored ravens.

Modified from Raven by Marcin Klapczynski, licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.
Modified from Raven by Marcin Klapczynski, licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.

The argument concerning the paradox is twofold one, to “prove” the hypothesis you must observe every single raven, I’ll come back to this latter. Two, you can observe hundreds even thousands of ravens and never see a white raven and therefore conclude that all ravens are black incorrectly.

Let’s suppose you examine 50 ravens and they were all blacks you come up with the hypothesis all ravens are black. You then go out and examined 5000 ravens, and they are all black. What is the problem, while we don’t know the exact numbers there are 4 or fewer albino ravens worldwide out of a total population of 16 million. That means that your probability of seeing an albino raven is 0.000025%. 

Beyond the small chance of seeing a white raven, there is another problem with the approach. Observing Ravens to see if they are black is an experiment that is designed to prove the hypothesis.  Specifically observing 5000 black Ravens is a result that is consistent with the hypothesis, this type of research doesn’t provide any information on alternative hypotheses.

With science, supporting or consistent data is of a lower value. Experiments that focus on disproving a hypothesis always have a higher value. They have a higher value because they eliminate alternative ideas which strengthen the validity of the remaining hypothesis. Additionally, a hypothesis is only scientific if it can be disproven.  Which means if you try to disprove a hypothesis and can’t the likely hood that the hypothesis is pointing at something real is stronger.

Let’s briefly get back to the issue of testability, since all ravens are black requires an examination of all ravens something that is impossible the hypothesis is untestable and is therefore not a scientific hypothesis. 

In the end, this the video uses the Raven Paradox to say that scientists can overreach and should be careful of generalizations. However, this argument is problematic because it is dependent on the definition of a hypothesis which is not complete.  The hypothesis all ravens are black is not a valid hypothesis. The author states a hypothesis is a prediction based on observations. I would say a prediction that is consistent with observations. Additionally, a hypothesis must be testable. Lastly, a hypothesis must be falsable or able to be proven incorrect to be a scientific hypothesis.

While I would like to see the YouTube logarithm not suggest things that are incomplete or oversimplified beyond usefulness, I suspect that will not happen.  Like I have stated before we need to focus on teaching students how to evaluate information.  I suspect most of the problems with the video come from things being oversimplified. As Einstein said, “Simplify everything as much as possible but no further.” Concerning basic education, I think we’ve taken the scientific method further. We tend towards being very simplistic in how we present the scientific method. We need to do a better job of teaching the basics if our students don’t know the foundation how can we hope to teach them the specifics.

Thanks for Listening to My Musings
The Teaching Cyborg

Have Textbook Chapter Review Problems Outlived Their Usefulness?

“Your mind will answer most questions if you learn to relax and wait for the answer.”
William S. Burroughs

While I don’t remember the specifics, one thing that I remember from many of the course outlines I had as a student is, read pages x-xx or chapter X and then answer the problems at the end of the chapter. The presence of problems in textbooks is in some way directly related to the creation of the textbook.

Homework - vector maths.jpg, Me and my homework, by Fir0002, From Wikimedia commons, published under Creative Commons license: Attribution NonCommercial Unported 3.0
Homework – vector maths.jpg, by Fir0002, From Wikimedia commons, published under Creative Commons license: Attribution NonCommercial Unported 3.0

The use of problems and answers in the “Textbook” predates the use of the word textbook.  According to the Merriam-Webster Dictionary “The first known use of textbook was in 1779.” However educational books have been in use far longer.  During the 4th century AD, Aelius Donatus wrote school books about grammar one of them Ars Minor is written entirely in the format of problem and answers (Encyclopedia Britannica).

The Ars Minor included both the problem and answer.  The instructor used the Ars Minor in a recall method where an instructor would ask a problem, and the student would recite the answer.  In most modern textbook’s problems are found at the end of chapters or units.

Modern textbook present review problems in one of three methods, first the textbook will contain all the answers to the review problems. In the second method, the textbook will provide the answers to half or just some of the problems.  Lastly, the textbook does not include the answer to any of the problems.

The purpose of review problems in textbooks has historically also had multiple uses. One, students can check their understanding of the material with the problems that have answers. Additionally, if the author presents the solutions in enough detail, they can be used to model problem-solving.  The faculty member is meant to use the problems without answers for quizzes and homework assignments.  While many publishers are starting to provide access to problem banks, I have seen the addition of “new” problems used as an argument for adopting a new version of a textbook.

In this day and age of interconnectivity and the internet does it even make sense to include problems in a textbook?  The concern is students will look up the answers online.  The availability of answers online makes the included problem useless for homework and quizzes. Upgrading to a new edition of the textbook ever 3-4 years will probably not help.  After all, how long do you think it takes to post answers online? While I have not tested this, I suspect all the answers are on the internet in a couple of days to a few weeks from the publication data of most textbooks.

The availability of so many answers online causes several issues.  One, especially when we are dealing with problems at the introductory level plagiarism can be difficult to identify.  Even if I ask the students to write out a paragraph, for example, explaining Mendel’s Law of independent assortment how many ways are there to write that paragraph?  While I’m not sure how many ways there are to write that paragraph, I suspect many generations of students have already written them.

I know some faculty that say education is ultimately the student’s responsibility and if they choose to shortcut the process, they will only harm themselves in the end.  While I think most of this is true, I also think it is the responsibility of the instructor and the institution to hold the line on ethical behavior in the learning environment.

There is a lot of arguments about students using “Google” to answer problems.  I have heard a lot of faculty say that it is beneficial for the student to struggle with the answer to problems.  While that is true to some point, it is also important that the students have a reasonable starting point.  Providing a starting point is where a problem with “complete” answers that model problem solving are useful.  Additionally, the problems need to be solvable; if students can’t solve the problems, it can get discouraging.

I also think the issue of looking up problems on the internet touches on another point.  Most schools state that part of their educational goal is to foster lifelong learning. When the students graduate and leave the school how are they going to engage in that lifelong learning?  They’re going to use the internet.  It is desperately important that we teach students how to use the internet, how to evaluate the validity of information, and how to determine credible sources.  We need to embrace the internet and start including it as part of our educational process instead of just saying “ITS BAD!”

Lastly, the proper use of problems doesn’t only benefit the students but also the instructor.  Problems and their answers should be used to provide feedback on pedagogy and teaching in the classroom.  The solutions to the problems should inform revisions and changes to the course based on student difficulties and misunderstandings.

If problems are essential but the internet makes the usefulness of textbook problems suspect what are we going to do about it?  One, make it clear what is and isn’t allowed as far as “help” is concerned and do your best to enforce this policy.  Help students learn to use research tools, which includes the internet, correctly.  Lastly, concerning textbooks, we should stop including and using problems in the textbook itself.  We should include problems in a separate workbook that we can change every semester or at least every year. Workbooks will let instructors change problems not only to “try” and keep ahead of the internet but to meet the changing needs of the course without having to change the textbook.

Thanks for Listing to My Musings
The Teaching Cyborg