Wildlife@Home: updating observations with new validator, and team stats

[RSS]

Hi Everyone,I'll be running the new validator over everything today. I'll be re-generating credit (although I don't expect anything to be very different) as well as re-calculating accuracy. Part of the reason I'm doing this is it will also make it significantly easier to generate all the team statistics (as I should have the top lists for teams done by the end of the day as well).Here's a run down of the main changes. Validation will work as folllows:1. I calculate the totals for each yes/no/unsure for each observation marking (eg., bird leave, bird return, etc) for a video that has enough for validation.2. from these totals, i calculate what we would expect to have as the canonical result (if it is possible to have a canonical result). Basically, for each observaton marking (yes, no, unsure) the canonical one would be the the total which is >= 2, and also is greater than the other two, so, if there are 3 yes, 2 no, 1 unsure, then the expected canonical marking is yes. If there are 2 yes, 2 no, 1 unsure, then there's no consensus so there's no possible canonical marking yet.3. If there is an observation which matches the expected canonical marks, then this is the canonical result. All marks that match it exactly are valid and an accuracy rating of 1.0, others get assigned an accuracy rating equal to what % of the marks matched the canonical one. Unsure matches yes/no for half credit/accuracy here.4. if there is no observation which matches the expected canonical marks, or it wasn't possible to get a set of expected canonical marks yet, the video will be shown to someone else (with a maximum of 5 views) to try and get a canonical result.The other big change here is accuracy isn't binary, and valid/invalid doesn't matter as much. Your accuracy rating will be calculated as the average accuracy rating over all your observations.--Travis

More... (http://volunteer.cs.und.edu/wildlife/forum_thread.php?id=307)