Wildlife@Home: testing new validator
Hi Everyone, I think I've tweaked the validator to get things working the way they should be now. It's going through all the observations and remarking everything. I haven't touched your previous accuracy and credit --> the validator is updating new calculations for these in different tables so in case we don't like it I can always go back and nothing is lost. You can check your new credit calculating (note they're being updated as we speak so it'll take awhile for the validator to get through everything) here: http://volunteer.cs.und.edu/wildlife...redit&offset=0 Also, you can check to see how much credit was awarded to individual observations on your watched videos page. This will let you know how partial credit is being awarded, and will help me find any inconsistencies with the new approach. So in a nutshell, the new validation scheme does: 1. gives the most priority (especially to canonical results) for observations with no unsures. 2. gives less priority to observations marked 'unsure'. 3. gives even less priority to 'video issue' markings. Partial credit will go to observations marked invalid, based on how many of the markings matched the canonical result. If the canonical result was a video issue, unsures will count as a match to it for partial credit. Please let me know if anything isn't working right. Once I have this debugged I'll update the validator to update things for reviewed videos.
More...
Re: Wildlife@Home: testing new validator
Kool
Quote:
Originally Posted by
RSS
Hi Everyone, I think I've tweaked the validator to get things working the way they should be now. It's going through all the observations and remarking everything. I haven't touched your previous accuracy and credit --> the validator is updating new calculations for these in different tables so in case we don't like it I can always go back and nothing is lost. You can check your new credit calculating (note they're being updated as we speak so it'll take awhile for the validator to get through everything) here:
http://volunteer.cs.und.edu/wildlife...redit&offset=0 Also, you can check to see how much credit was awarded to individual observations on your watched videos page. This will let you know how partial credit is being awarded, and will help me find any inconsistencies with the new approach. So in a nutshell, the new validation scheme does: 1. gives the most priority (especially to canonical results) for observations with no unsures. 2. gives less priority to observations marked 'unsure'. 3. gives even less priority to 'video issue' markings. Partial credit will go to observations marked invalid, based on how many of the markings matched the canonical result. If the canonical result was a video issue, unsures will count as a match to it for partial credit. Please let me know if anything isn't working right. Once I have this debugged I'll update the validator to update things for reviewed videos.
More...
