How can we objectively judge an answer's credibility?

Question

(This is tangentially related to the question about identifying native speakers.)

At present, answers to questions appear to derive from the following sources (in my subjective order of decreasing reliability):

• Grammar texts (the good ones)
• Personal experience (as a native speaker or fluent learner)
• Statements from a Japanese teacher
• Japanese Q&A sites and other internet resources
• Old classroom memories and hearsay

Seeing as we can contribute to the proliferation of bad information just as easily as we can good information, if we wish to make this into a valuable site for students of Japanese, it would behoove us to have a high standard for accepting answers. While I don't propose we start tagging unreferenced statements with [citation needed] akin to Wikipedia, I do think we should attempt to ensure credibility in answers. A mere upvote can mean many things. We all hope it means, "Yes, I have checked this answer and can confirm it to be true," but it could easily mean, "I don't know if this is correct or not, but it makes sense to me."

Off the top of my head, here are some possible solutions:

• Ask answerers to voluntarily provide a "References" section below their answer (面倒くさい for the answerer).
• Ask high-reputation users to comment their concurrence when they upvote (面倒くさい for the upvoter).
• Implement a standard of "minimum reference credibility" and draw the line somewhere inside the above list.

I'm not sure if I'm bringing up an issue worth discussing at this point, but I look forward to seeing everyone's comments nonetheless.

Answer 1

Like my response to the other question, enforcing such guidelines will be difficult and will only serve to tick people off and discourage people from answering questions.

I think we should just maintain the status quo as-is (where users answer questions and additionally provides references if they so desire). If someone has a doubt and wishes to challenge their statements, they can leave a comment asking for more details and/or a reference.

This is pretty much how editors and readers on Wikipedia operate. If you find a statement questionable, you use the {{fact}} tag to mark it as needing a citation. This warns the reader that the statement has been challenged by someone else. Of course, on Wikipedia someone could just revert that edit or it could be a random person up to some mischief but I don't see that manifesting into a problem here on the Japanese SE (at least not enough times to make it more than a nuisance).

Edit: I think that instead of asking the people answering questions to provide references, we should instead encourage users to be bold (sorry, another Wikipedia reference :)) and question the answers that are being posted. As a teacher of mine once said, if someone has a question to ask, there's probably another person out there with the same question. If somebody is curious or puzzled about (something in) the answer, they should not be afraid of making a follow-up comment asking for further clarifications.

As Troyen pointed out in the comment below, this sort of a feedback loop ultimately ends up being beneficial to all parties involved.

Answer 2

Thank you for bringing up this topic. I have been worried about the possibility that the site may become yet another place to reproduce popular false claims, and it is important to consider this problem, even if we do not have a silver bullet for it.

A mere upvote can mean many things. We all hope it means, "Yes, I have checked this answer and can confirm it to be true," […]

I am not sure about this. My feeling is that in the ideal world an up-vote should mean that the voter verified the answer, but I suspect that for most people, there is nothing they can do to verify the answer. As someone who is asking other users to vote often, I rather argue that users should be aware that high score does not always mean that the answer is correct.

Honestly, I find this problem daunting and I am not sure what we can do.

I do not think that it is right to restrict answers to verifiable ones (even if it is possible at all). If we do so, I suspect that few people will be able to post any answers. For example, some native speakers have trouble backing up their claims :).

Personally, if I cannot back up the claims in my answers, I try to state so (at least implicitly by using phrases such as “I feel …” and “my impression is …”). But I have probably already failed to do so in some of the answers I have posted so far, and I cannot require everyone to do the same.