In Downsides of the internet, I’ve briefly mentioned how pre-internet writers cited the works of other authors less often in comparison to the modern wordsmiths. This observation in turn reminded me of an interesting philosophical question that someone asked while we were working on a “research” project back then when I was still in school.
At some point during the project, a librarian visited our class in order to explain the importance of citing your sources. Since kids are an insightful bunch, someone came up with the following question:
If I don’t use an external resource since I came up with the text on my own, should I add the words “my head” to the references section?
The librarian’s face lit up, and she used that question as an introduction to scientific integrity and telling us how you should always refer to reliable sources as opposed to making your own facts on the spot. I didn’t think much about this topic until a decade later when I realized that just because someone came up with a bunch of words and published a book, that doesn’t automatically mean that what they had written is actually correct. If you check out criticisms of a random contemporary popular psychology book, you’ll be amazed by the amount of nonsense that is being published these days.
If we strictly follow the citing rules of the aforementioned librarian, then the author of an article that I am referring to should also refer to the works of earlier authors. The works of earlier authors should also refer to the works of their predecessors, which should once again refer to the works of authors before them. In other words, it’s books and articles all the way down.
Obviously the world doesn’t work this way. At some point in this citation chain, someone will have to come up with original thoughts or measurements with nobody else to refer to. Since they are the pioneers of their field with no other authority to refer to, how can you be sure that whatever they have published is actually correct?
If you aren’t intimately familiar with their field, the answer is quite simple: you don’t and you can’t. If the research findings could be measured, then you could try to replicate the process and compare the results. For an outside observer, however, most of the time that is not feasible. For example, how can you verify if the actual number of people living in your capital city matches with the number that was reported by the government? Will you go door to door and count them on your own? What if performing research demands access to expensive equipment, such as a particle accelerator, that you can’t possibly afford?
Considering the questions above, we can safely conclude that an ordinary person outside of their field of expertise can do very little besides trusting the process. If the scientists are supposed to do good in this world, then surely they are not going to falsify the data. After all, there is peer review, conferences and other scientists that would detect and prevent the fake research from spreading.
And yet, fake news backed by “scientific research” is still spreading. How else can you explain all the diet fads that are backed by various papers stating how this diet surely works unlike the ones from yesteryear. Evolution is a slow process and the human body is definitely not changing every year to demand different treatments. By now we had enough time to figure out what kind of diet works or doesn’t work, but somehow new drastic ways to change your lifestyle are still appearing only to be denounced a year down the road.
When a survey was made that asked the scientists if they could reproduce each other’s papers, the results were quite surprising; 60-80% of participating scientists mentioned that during their career they had problems reproducing the results of one or more papers (see Replication crisis). These papers passed the peer review and were probably also cited by other scientists that used them as a source of knowledge when writing their papers.
The problem, however, is that if you can’t reproduce the results of a certain paper, all the other work that was based on it will also become invalid. Sometimes such papers can still push the conversation forward so the effort is not entirely wasted, but this situation introduces another question. If you can’t even trust the science and the scientific rigor to produce correct observations of our environment, who can you trust?
I am not claiming that scientists are untrustworthy, although their publish or perish environment is certainly pushing them into that direction, but we should at least be aware of the fact that not everything they write about should be taken as the holy gospel because mistakes, both honest and on purpose, do happen. The days of Isaac Newton, who was reluctant to publish his theories for years, are unfortunately long gone as we live in the era where being first is more important than being correct.
So far I have only tackled the type of knowledge that could be measured. But, there are other pieces of knowledge that are based on personal experience, and as such they can’t be measured and you might have a hard time replicating them. What works for someone living on this side of the world, might not work for somebody living on the other side of the world due to certain cultural differences. To be more concrete about the meaning of the word “non-measurable” knowledge, let me give you a simple example by asking you another metaphorical question.
Should you lie in a job interview?
If I asked you this question in public, I suppose most people would say no. If you get caught lying during your interview, that’s a sure way not to get a job. Besides lying is wrong and against whatever moral values we got instilled in our youth. That is the theory at least, however in practice things are a bit different. Maybe you really need this job, as starving to death doesn’t sound much fun, and on such occasions the preservation instinct kicks in and bends the moral values as necessary.
If we generalize a bit and imagine an average interview that happens in a large company, it usually goes like this: an interviewee pretends to be competent, while the company pretends they are not a toxic work environment where the applicant is going to produce useless reports.
Both sides are in a way lying or, should we rather say, presenting themselves in a better light by omitting the unpleasant details. A lot could be said about evaluating competency, but opening that can of worms demands its own article, therefore I will focus on just a small part of the interviewing process; the so-called personality test. For example, companies in general are trying to avoid hiring assholes, but there are lots of assholes in this world and they all end up being employed somewhere. Considering they ended up being hired, they have either managed to conceal their true colors or the people hiring them didn’t mind their behavior due to being assholes themselves.
Employees in general are trying to avoid the toxic meat grinders, but regardless of that, there are lots of companies that keep hiring and losing people due to their internal organizational issues. How do these companies manage to hire people if they are such an awful place to work? How else than by bending the truth about what kind of company they are.
It’s therefore common wisdom to avoid being entirely honest during an interview for both sides of the table, but if somebody asked me to provide a citation for such a wild claim, I wouldn’t be able to provide any. The ancient Greek philosophers have probably already addressed the “should I lie to a liar” moral dilemma, but I am not aware of any more recent insights related to lying in job interviews.
For this reason I perk up my ears every time somebody starts talking about citing “reliable sources” of knowledge. It’s widely assumed that published books are more trustworthy in comparison to a blog post or a personal anecdote. The assumption is that considering there are more hoops to jump through before you can publish a book or a paper, the mistakes will be filtered out in the process. But is that really so? After all, I have shown you a few cases where this idea falls apart like a wet toilet paper, thus citing reliable sources of knowledge means nothing more than appealing to authority. Well, the publishing process is not perfect and it certainly needs improvements. What kind of improvements? I am not sure.
It has to be said that at least when it comes to programming, I’ve learned far more from blog posts and personal anecdotes written by non-authorities than I did from books published by academics. It may not be the most reliable source of knowledge, but it sure helped me. The original “should I cite my head” question may sound silly, but when you think about it, it actually makes a lot of sense. Knowledge may come from various places and ideas sometimes grow up in your brain without you having to rely on other sources. Sometimes your experience is the actual source.