Judging: How should it be done?

This is a thread for people wanting to discuss how judging should be on the site as suggested by @admin

I think this could be useful.

So if people want to discuss any problems with the way judging is done on the site, want to propose new ideas for it or defend the current judging system then this is the place to do it!

I'll tag people who may be interested @cooldudebro @Tejretics @IncorrigiblePerspective @Jurisprudence @TheHouse

Bifurcations: I had about 4 goals in mind when making the current system:

- Fairness. Basically no debates should be decided on the basis of very poor judgements.
- Encouragement. As many people should judge as possible, and be incentivised to improve in a constructive way.
- Quality. The system should press judges into providing useful feedback and analysis.
- Flexibility. The system should accommodate a wide range of judges' preferences.

It's flawed but I think it's generally working pretty well at fulfilling these ends.

admin: I agree I think with the addition of the comment section it is possible to ask for more feedback which many new judges have but are sometimes unsure about posting especially after a large judgement like the ones I normally do. It is a great way to start of new judges in a comfortable environment and then encourage them to engage more in debates where they are more confident about what their decision is.

I basically like the status quo on DDO a lot with regards to voting standards. Votes that don't fit the appropriate standards are removed. I think "very poor judgements" is way too lenient, and that debaters are entitled to get a quality of RFD that matches the effort put into debates. People new to Edeb8 have often never debated formally and just bring up their own opinions (e.g. see the most recent vote on my gay marriage debate) -- they need to be taught what is expected of an RFD. They should know that if a debater spends so much time and effort on their arguments, the judge ought to respect that with quality feedback and reasoning.

Tejretics: By the same logic, would you also support banning routine forfeiters or those with very poor debating skills?

admin: At least with routine forfeiters, the debater who didn't forfeit gets a reward: the victory. With bad voters, the debater's prize: a loss.

Tejretics: Then wouldn't it be fair to assign points based on the quality of the judgment?

I actually think having inexperienced voters on the site incentivizes skills besides good debate theory, like persuasiveness and good writing. You could say it judges a fuller picture of debate skills

I think what is missing is a specific judging standard and format for the site. This means that those who are used to L-D debates would judge based on that framework and those who do BP would judge on that frame work. The problem is that normally these debates are not specified as L-D or BP or any other debating format therefore there has to be a standard set out that creates fair judging for these types of mixed debates. There is the resource that @admin put up about the way in which he would normally judge but I think something more formal than that is required. This would help novice judges feel more confident in what they have to say to provide a solid RDF and would help regulate for the fact that there is a mixed bag of debating experiences on this site.

I also think it should be encouraged that a specific debating format is used and therefore the rules and requirements of that format are followed. There should be resources on each of these formats and resources on how to judge these formats. Maybe some time of judge accreditation could be applied to this system; like a badge to show that this judge is experienced with whatever format of debate.

Bifurcations: The inherent problem with this is that there are a LOT of formats.

admin: Very true but I think that you setting a judging standard for general debates is still applicable and I think it is possible to introduce resources for the top three most widely used formats given that these ones already have good resources available online to link to.

Bifurcations: That would be a good idea, I am a novice on judging and debating and it would help to have a format to use. I think we need a judging standard for the debates here because then I( and other novices) can help learn how to judge a debate properly and adjust to that judging standard.

Bi0Hazard: Its hard to have a framework or standard because each adjudication is different and can't really be simplified into a standard format. No matter what format your using every debate will come down to different things which is why in the real world judges are taught that you should decide the winner and then allocate points. I think the solution is to weight the decisions of high quality judges higher than newbies and begginers.

A general judging standard would definitely be helpful. Like list out basic judging principles and then pragmatic ways to apply them

Leachy: I like your idea and it is not mutually exclusive to what I am suggesting. Like I said there is already a resource that admin has put up about how he would judge. This is something that he could say is the standard of judging on the site. This is really just so that novice judges understand that they cannot judge based on their own beliefs or that a mechanism might be important for some debates. This is nothing to do with the judging decisions that would take place but rather lay out the rules of judging a general debate.

Leachy: At least I would want examples, like a judging standard that you could use for reference.

admin: 1. That is not what happens on Edeb8. Length =/= quality.

2. No, it wouldn't, because if a judgement is of bad quality, it should have **zero** points. On Edeb8, a single person can vote a bad vote, and nobody else votes, so a debater loses because of poor reasoning. That isn't fair.

Voting is for the debaters, not for the voters. Voters should vote as a service, not to exercise some sort of "privilege": they think they have.

Tejretics: But if, in the second scenario, there is one bad vote and nobody else votes ... how else is the system to determine the winner? By coin toss? Is that more fair?

Tejretics: I also would challenge anyone to come up with a better algorithmic solution to assigning points to votes for "quality" than the present system. I say algorithmic because:
1. I'd rather work on features than moderate every single vote before the deadline; and
2. I'm incredibly biased especially in debates which I am involved in

admin: what if only posts that got 2+ "vote bomb" ratings, and no other ratings, were reassigned a value of zero?