I have spent much time and many discussions in trying to figure out why that is so, and not only have I found that I am not alone in doubting the usefulness of the current self-assessment strategy as a learner, but also that I am not alone in doubting the usefulness as a trainer and educator.

Disrespect? Please.
Bringing these discussions out in the open is anything but a sign of disrespect ‘to those people (trainers and participants) who have been working with these instruments and have been evaluating their experiences in order to improve the tools’. I don’t want my intentions to be misunderstood or my thoughts to be misconstructed that way.

That a rigorous evaluation of these instruments is still missing—after ten years of educational practice and several educational evaluations taking a critical stance at the self-assessment approach—that is a sign of disrespect.

Complementarity of embedded and external analysis
It’s not about analysis and research being the only tools that guarantee quality, but it is their absence that delays discussion and hinders development. There is a huge difference between trainers and participants involved in particular educational processes working on improving a specific instrument for the purpose of usability improvements, and a more general approach that reviews the entire range of instruments and tests them for consistency, verifiability and rigour.

Obviously, not much would happen if trainers and participants chose to ignore and push aside self-assessment instruments, and did not work on improving them for their practice. But that is not the element of further development that is currently absent, as you rightly pointed out.

It would, in other words, be a most welcome sign of respect to all the work done if a study was commissioned—similar to Fennes/Otten and Ohana/Otten, possibly—which, at long last, picks up some of the many open questions around self-assessment.

Time versus speed
I recognise that some of the instruments have been around for some time, and quite some of these instruments were ‘discussed a lot amongst trainers and with participants.’

But somehow there is never enough time for this, is there? The development of these instruments is always embedded in the work of a team, or the context of a session… I have myself been part of too many discussions that had to be cut off—and had to cut some myself.

Not once has there been enough time to engage in sufficiently long discussions around self-assessment that are not exclusively aimed at further developing one specific instrument and that would lead to questions or conclusions that were written and shared widely.

Quality standards
I agree, the field has some clues what the job is about, and some good writing exists on quality standards in particular for European youth trainers. They are not agreed, yet, and there is plenty of discussion around what particular standards may or may not mean, but yes: enough material to provide context.

Scary research
Hehe, the sentence sounds a bit scary when taken out of context:

Research can prove what common sense and practical experience tell us: none of this is true, none of these assumptions hold, they crumble at first sight.

But within its context, it’s unfortunately not wrong:

Firstly, there is research showcasing that knowledge, skills and attitudes are not enough to capture the competence of a person, for example in the framework of UNESCO’s DeSeCo Project – in other words: a trainer with a perfectly acceptable set of knowledge, skills and attitudes may still be a crappy trainer even though the self-assessment indicates otherwise.

Secondly, there is books full of discussions around the dilemma with numerical scales and, while they suggest that the difference between levels 8 and 9 are the same for everyone, the individual judgment of respondents is miles apart – in other words: five trainers assessing their competence in the area of writing with 8=very good, will very likely display pretty diverse writing skills.

Thirdly, there is evidence already in the evaluations of our own courses in our own field that educators and learners are often not aware what a specific competence entails when doing their self-assessment – in other words: the more people learn about a competence, their assessment tends to become more self-critical and sceptical.

New instruments
Yes, indeed, what a challenge! I hope to find the time soon to polish some of my ideas around alternative instruments…

My first problem is the picture that is painted. ‘Instruments are developed at high speed’. I don’t want to go into a discussion about the definition of ‘high speed’ but the first time that I worked in a training course with a Self Perception Inventory and a Personal Development Plan was in 2001/2002, so 8-9 years ago. Since that time in training courses for/of trainers it became a kind of common practice to use these instruments. In these years both the Personal Development Plan as the Self Perception Inventory were changed, improved and new more complex and more simple versions came out. Discussed a lot amongst trainers and with participants. I think they even got better! Since the beginning a need was expressed for other and more instruments that could help people to assess themselves. ‘Not only instruments where you have to fill in a form!’. New tools were developed but also already existing methods were found. Still in the field the need for more instruments is expressed, especially after the introduction of the Youthpass in the Youth in Action programme.
Not so much ‘high speed’ anyway for me.

‘A rigorous evaluation of these instruments… is as much missing….’. A wonderful sentence but not very respectful to those people (trainers and participants) who have been working with these instruments and have been evaluating their experiences in order to improve the tools. I will be the first one to support further analysis, evaluation and research on this but the last one saying that only analysis- and research reports guarantee improvement of quality.
‘The absence of quality standards’. When that means quality standards agreed on in the European educational field, signed and stamped, I agree. They are absent. But I would not wait for that agreement to start (and continue) developing better self-assessment instruments. We are lucky in our field that many people in different constellations have been thinking about quality standards. The good thing is that they even wrote their thoughts down. It’s no problem at all to offer to participants in training courses material which give them the possibility to assess themselves against quality criteria. It’s not that the field has no clue what the job is about.
‘Research can prove what common sense and practical experience tell us: none of this is true, none of these assumptions hold, they crumble at first sight.’
First of all I get a bit scared of research that can prove that ‘non of this is true’. Especially when this is written after three statements following an assumption that I find quite doubtful.
I would be rather in favour of research that would join the forces of those people who would like to assist learners in the best possible way to take responsibility for their own learning.
So….. it’s above all the first part of your last sentence I’m supporting: taking on the challenge to develop some ideas for alternative tools!