The Importance of Understanding and Evaluating Research
Losing the ability to understand and value scientific research can be a matter of life and death. Recently I’ve heard the word “data” and “research” batted around many situations. Education calls for data to evaluate schools. Librarians call for data to support programs and budgets. Coming from a social science background in which I implemented research projects and taught research methods courses, I am often concerned the “data” being referenced is at best, being mined without consideration of scientific research methods and at worst, merely numbers presented only because of their quantitative nature.
People with little understanding of the process and seemingly no knowledge of the reasons behind research methods toss around terms and numbers with great pretense. Worse, others with potentially less knowledge are making policy decisions based on these numbers. With this lack of understanding, it is no wonder we have arrived at a world confused by “alternative facts,” “fake news,” and the disregard of scientific institutions and data.[1]
I am reminded of the popular belief “people lie with statistics.” As I told many a class, people can only lie with statistics to those that don’t know statistics. The same is essentially true of all information. People can lie about history, but only to those who don’t know history. People can lie about anything to those that do not know about the topic. The way to prevent this is to learn about the topic in question. In the case of trusting research and data, one needs to understand research methods and evaluate.
As librarians we are dedicated to seeking out information. While we generally do not engage in original scientific research ourselves, we refer people to such research and help people to evaluate the materials they find. Academic librarians may do this with greater depth and frequency, but I would argue the responsibility is even greater for the public librarian who serves a population that comes to them for information without any expectation of foundation and background knowledge. For the average public library patron there is no expectation that they possess a background in even website evaluation, let alone scientific research methodology.
For the public librarian we (rightfully) take pride in our role of evaluating material and educating others to do the same. As the evaluation process, not subject matter, is generally our expertise, we depend on other factors such as knowing the credentials of authors and publishers. A simplistic example is that we refer people to information produced by a journalistic news source rather than a tabloid. In the past, the expectation was that the most trusted information could be obtained from the juried professional journals of an academic discipline. If a cure was touted by the American Journal Of Medicine, we knew we could trust this cure more than if presented in The National Enquirer. The actual reason behind this had little to do with the publisher, and everything to do with the process. The material presented in the journal we knew was going to be based on data gathered by scientific methods and reviewed by experts in the field.
This week, I learned the American Psychological Association (APA) asked a journal editor to resign when he asked to see the data of the research he was evaluating for publication.[2] This is akin to the restaurant firing a chef when he asks for proof his ingredients are not spoiled or the hospital firing a surgeon when he asks to see a patient’s record before operating.
What began within me as minor annoyance has now developed into what can only be described as fearfulness. As librarians we are not only on the front line of information sharing, we are also its guardians. I believe we need to hold creators accountable. If you don’t know or understand research methods – learn them! If a source or organization will not provide or support the process, don’t support it. We need to start treating data with respect or all information will soon become meaningless.
References
[1] http://ngm.nationalgeographic.com/2015/03/science-doubters/achenbach-text
[2] http://boingboing.net/2017/03/02/psychology-journal-editor-aske.html
Resources
http://www.nature.com/news/peer-review-activists-push-psychology-journals-towards-open-data-1.21549
Tags: alternative facts, American Psychological Association, APA, data, fake news, journalism, research, research methods, sourcing
