There needs, I think, to be some agreed way of tackling this problem.
I added the general ID, not having the skill to go further. Graham (healeygf) knew enough to go to species. But my inflated reputation held the likely ID with my broader offering.
OK, there is no easy way to code things so that the system “knows” that my reputation exceeds my knowledge (IMHO), being gained mainly through long service and the IT equivalent of verbosity.
But the only way to tip the balance is if someone “good enough” then agrees to the revision. And, with the more difficult or less popular organisms, that doesn’t always happen.
In this case, I resolved it by adding the agreement myself. I was fairly confident (as a result of “being given the answer”), but in some cases I really am not sure enough to do this.
So:
Perhaps it should be much harder to gain the higher levels of reputation,
Perhaps the user could appeal to have his/her reputation downgraded, and/or
It should be possible to “withdraw” an ID. I can understand that, for various reasons, it should still perhaps be visible; but the user could mark it so that it is effectively ignored by the system.
I recognise the problem. Martin Harvey gave me an expert badge for invertebrates but nobody can be an expert in all of them. I don’t know many moths but I’ve sometimes hazarded a guess when posting a moth obs and then found I was wrong but it is hard to shift.
One answer would be to have more nuanced expertise. Aquatic beetles are really my thing, so my expert opinion should be restricted to them.
As Amadan says, being able to withdraw an identification would also be helpful.
But in this Apion example, there is a second issue. Graham Healey has said likely, not for sure, presumably in recognition that there are other red Apion spp. But it has been established that on iSpot an agreement is a confirmation - you can’t agree that something is only probable. So unless an Apion referee says it is A. frumentarium, I think the highest level for a certainty is genus (or maybe subgenus).
Applies to me too - I know very little about UK plants and the ZA species knowledge has been from references on-line and collaborating with friends on field-trips. There is too much to learn -
One of its most remarkable – and best kept – secrets is the incredible diversity of plant species found in the Cape Floral Kingdom (or the Cape Floristic Region) on the country’s Western Cape.
One of only six floral kingdoms in the world and unique to this small area, the region supports over 9,000 plant species, 70% of which are found nowhere else on the planet. These include exquisite species such as the king protea (South Africa’s national flower) and the exotic pincushion South Africa - Fauna & Flora International
I added a lengthy comment (as always) TO the observation. I misspelled prey, as we sometimes do.
This issue has nothing to do with Reputation (except I have temporarily locked the Banner!)
It’s to do with evidence and its presentation. I will be happy to move my agreement when the evidence suggests I should.
If I should stop posting, miked has my password and permission to remove my agreement!
.
PS I do not think we should ever be able to withdraw an ID, Part of the process here is the development of the best ID… How boring would it be to just see one ID for every Observation?
See https://www.ispotnature.org/communities/uk-and-ireland/view/observation/464915/ see my Comment there
“I do not think we should ever be able to withdraw an ID”
I have to disagree. The withdrawn ID should still be visible, but should not have any “weight” in deciding the likely ID.
A case in point:
Looking at the revised ID (Bluebirdresearch’s interests and contributions elsewhere), I think it is quite likely right. But, the species is recherché: there are few images or accounts online.
Should I agree, and change the “Likely ID” on that basis? I am really not sure of the species - but this is due to my lack of expertise, no other cause.
In most cases, the identification of a subject to species level is the best possible outcome - for some users, it’s the only purpose of posting. I am also interested in the social interaction that arises, via comments and the forum (perhaps too much so): but ID remains the Holy Grail.
If the consensus is that IDs must be cast in stone, what are the other choices?
Maybe iSpot decides upon (and publicises) a “protocol”?
Maybe individual users could “appeal” their reputation score, and have it reduced (I would do so)?
The issue almost always concerns “invertebrates” - far too broad a category. But could it easily be split, without causing all sorts of taxonomic chaos? And without confusing less-experienced users?
I’m sorry to whinge on about it, but I think it’s rather important.
It is important and it is something I had hoped would be worked on as a research project at the start of ispot and several times since. Unfortunately apart from at the start we’ve never had resources to do it. Also since the pandemic suitable programmers have been impossible to get hold of given the highly inflated salaries offered in the commercial sector. I know there are a few retired programmers who are on ispot but suspect they are more interested in recording wildlife rather than trying different models of reputation even though this could be very helpful to everyone on the platform.
You are NOT whinging and it’s a very important point
But I think the example is a poor one - there will be better one I’m sure.
But the first ID is perfect - it has two agreements.
IF you are certain it is the Species, then you should agree but there is STILL the choice of adding an agreeable comment, drawing attention to the difficulty of fully agreeing.
.
For users (specially inexperienced ones) it is preferable to agree to something they (we) think is right than something you are not certain about. The only other choice is to ignore and move on. Very many ‘good’ Observations here have well over 200 views, no agreement and no comment - how bad is that?
A good example is here https://www.ispotnature.org/communities/uk-and-ireland/view/observation/14806/
,
I know WHY you’d prefer the Banner to be with Species - I guess we all would.
I have been quietly suggesting (to the point of near silence) that we should be able to agree to the ID at any level in its taxonomy.
,
OK, I know the arguments. But if iSpot was working well, there would be a lot of LOW level agreements for the Family or Genus and a few HIGH level (-icon) agreements for the species. If we had even more ‘experienced’ users then the Species will carry the Banner. If the photo is weak, then the Banner may be with the Family forever.
And if we all went though a few Other Observation when we do our own, then the banner would almost always be with the Best ID.
A case in point (I can’t find an good example) would be an ID to Sub-species but without reasonable evidence, We are forced to ignore it, or add a downgraded ID, A comment looks like a rebuke.
If we cannot see the reason for the Var suffix, like in the (topical) one here Nettle Gall | Observation | UK and Ireland | iSpot Nature then add a subjective correction or a comment.
long again…sorry
As I recall Tony Rebolo had ideas as how to tie reputation more closely to the species likely identified. But the two potential issues are can we find working and stable rules to do the job, and can the reputation be calculated at an acceptable cost.
There is however one change that I think can be made - change expert/knoweldgeable status so that it is tied to a clade rather than an iSpot group. (Not completely simple, as the code would have to work its way up the taxonomic hierarchy until it hit the specified clade.)
There are a couple of other related issues with the reputation system that I wonder about - saturation (there are now a lot of people with 5 stats) and incumbency bias (a new participant with good identification skills will be outweight by a long-termer with moderate identification skills). A way to address that it to have older observations age out - either by time period, or by last n identifications - of the reputation score.
I’m not a regular forum reader but I followed this discussion with interest - thoughtful comments on the responsible exercise of the power given by ‘reputation’. But, particularly in relation to ‘agreements’, one aspect of this issue was not mentioned.
We can all make mistakes, but I’ve no doubt that most agreements are carefully considered - maybe based on personal familiarity with the organism or by checking the evidence against authoritative sources - but one does wonder about the quality of some. It’s easy to press the button: and get trigger-happy.
Reputation, as Amadan self-deprecatingly pointed out, can be based as much on volume of production as genuine expertise. It’s easy to build up your brownie points by posting lots of robins, red admirals and popular plants, but really knowing your ichneumons, caddisflies and slime moulds doesn’t get you very far.
A problem arises - and it does sometimes - when uncritical agreement based on mediocre ID skills combines with a significant reputation.
I’m doubt whether much can be done about this by tinkering with the system. But I do think that the big hitters with genuine expertise could sometimes be more assertive in dealing with the resulting dubious ‘Likely ID’s, and not just ignore them - to the benefit of iSpot’s reputation.
That said, I recognise that people do have busy lives: and the carousel moves on….
A good point, well-made.
There’s something else to consider: the value of iSpot as a learning aid, and the pitfalls that come with it.
A number of posts I’ve ID’d or agreed to in the past have then thrown up ID “clues” that are recently-discovered/highlighted, or which have fallen from grace by being found to be unreliable. Often, it’s only by citing such a feature in an ID (or comment on an agreement) that you discover the change.
Take, f’rinstance, the idea that the wing patterns of Panorpa species can be used to identify species. This I first found here on iSpot, then tracked it down to Paul Brock’s book ‘Britain’s Insects’. But it seems to be a new concept (previously, male genitalia were said to be the key feature): is it reliable?
I think that this is the sort of thing that is best learnt through experience - accompanying skilled observers, or attending specialist courses. iSpot is an “easy” armchair alternative: however, getting it wrong can lead to false IDs, inflated reputations, and bolstering reliance on unreliable ID features.
But this was all really a bit of navel-gazing on my part. The reality is that the system is unlikely to change: there are more pressing problems with the site that ought to take precedence.
One simple answer - for me - might be to go back to square one: create a new username/profile that doesn’t carry such undue influence.
It’s generated some interesting viewpoints and discussion, though!
we are certainly considering the points made in this thread and may try to implement them on a test site. Reputation systems can become very complex which may improve how they work and at the same time of making them more difficult to explain to those using the system.
A thought that has crossed my mind is changing the system so that only identifications with at least one agreement become likely IDs, regardless of who made it. However if such an identification would have otherwise have been the likely ID it should still remove likely ID status from other identifications.
Ideally this might happen but there can be very good reasons why observations don’t get agreements. For example only an expert can make the identification and no other people are confident enough to agree or it make take a lot of research to make the ID and no one else can be bothered to repeat this.
But there are yet other observations that can be agreed with but no one has done so far - something that the ifocus projects address to some extent by sending people back to look at older observations and asking them to check/agree.
(thanks yes) the main purpose of iFocus is to review earlier Observations, but only 7 people are engaged in the process.
.
I have been quietly proposing for a few years that NO Observation gets the Likely Banner until there is an agreement. This may force us into slightly higher taxonomic IDs but with strong comment support for species. Banners should be strongly supportive of Family or Genus, But comments from experienced users may be strongly supportive of Species . Eventually a secondary Sub Panel shows Likely Species. I realise it is much too late for this concept to be considered.
The reputation system is shot to pieces as soon as you get 4 on 4 or 5 on 5 etc. It seems to be a complete lottery. On a recent post I had 42 times the reputation points of the author but still couldn’t carry the “likely ID”. Explain that one please ? It was as it nearly always is sorted by an agreement but it needs some serious work.
To be honest I cannot remember which one it was but I did the calculation at the time. As an aside in invertebrates I have nearly 64000 reputation points but if I add a revised ID against another 5 icon user I can guarantee I invariably never shift the banner. It’s not a great issue as agreements sort it out. My point is that the likely ID is not awarded on reputation points. It seems to favour those who post regularly and I haven’t really posted my own images on iSpot for 7 years.
I have quite a lot of experience of ‘watching’ Reputations and Banners.
This has actually happened to me and I have asked for considered -assistance from others which has always been accepted…
.
It is true. the Algorithm is not perfect but, generally speaking is a marvel. There are plenty of specific cases to be made for it not working properly but some of those can be explained away. For example iSpot remembers your reputation from WHEN you added an ID, so a newly added one may well displace it.
.
I suspect the Observation in question is this Chris (the above example does not apply) https://www.ispotnature.org/communities/uk-and-ireland/view/observation/845231/ Inexplicable IF the banner was given to the second ID.
.
But sometimes confirmation IS needed in the form of an agreement. IF that iIS the example then the Algorithm worked well after an initial glitch.
.
I do not think that up-to-date posters will outweigh your Icon power. It almost certainly not be part of the Algorithm.
The test site is still running Chris and you have access - we, you and I, can test this part of the algorithm there. I know that both Miked and ChrisV are interested but we all suspect that if any part of the Algorithm is modified, all Hell will be let loose!
As I say it’s not really a problem as long as agreements are applied, it is just more perplexing than annoying. That is not the observation in question, I honestly can’t remember which observation it was but if it problem arises again (and it will) I will report back.
iFocus is to review earlier Observations, but only 7 people are engaged in the process.