Instruments are designed at high speed – from self-assessment forms to personal development plans, from self-perception inventories to competence improvement maps – resulting in a cacophony of abbreviations that seems only a little shy of setting new records.

A rigorous evaluation of these instruments – looking at aims, scopes and approaches as well as usage, usefulness and impact – is as much missing as a painstaking analysis of underlying frameworks and tacit assumptions.

It is clear already, however, that the entire assortment of self-assessment instruments fails to respond to some key questions, among them:

• In the absence of quality standards, what do you measure yourself against?
• In the absence of external expertise for validation, how exactly should recognition and accreditation come about?

Even when leaving all political intentions and inconspicuous ambitions in relation to validation, recognition and accreditation aside, I have trouble finding value in any of these instruments for their most palpable purpose – self-assessment.

Take whichever you want – SAF, SPI, CIM, PDP – they all start from yourself as a trainer and educator. Not yourself as a trainer and educator in a particular project or context, but rather yourself as a trainer and educator in life. Through this inherent claim of being universally relevant and the resulting decontextualisation, the self-assessment process loses most of its value for me.

Let me pick three quandaries to exemplify and justify my defiance:

Firstly, this approach implies that there is a potentially agreeable set of competences for non-formal educators. It assumes that there is a combination of knowledge, skills and attitudes that, once mastered, makes for a non-formal educator of tolerable, decent or outstanding quality.

Secondly, this approach implies that there is a universally acceptable scale along which any set of competences could and should be measured. It assumes that there is a common understanding of what it means to be moderately or exceptionally competent or incompetent in a specific area.

Thirdly, this approach implies that educators are generally aware of what specific competences entail before they have fully mastered them. It assumes that there is sufficient understanding of knowledge, skills and attitudes required to achieve basic or advanced levels of proficiency.

Research can prove what common sense and practical experience tell us: none of this is true, none of these assumptions hold, they crumble at first sight. And yet we continue to invent and re-invent self-assessment tools, defeated before we start by their envisaged universality…

How then, you ask, could a useful self-assessment instrument look like?

A very good question indeed :)

I will gladly take on the challenge to develop some ideas for alternative tools in the second part of this mini-series, but let’s first leave some time for your questions and ideas, your criticism and feedback. Fire away!

Contribution to the European Commission and Council of Europe Partnership Expert Workshop on Non-formal Learning in cooperation with the Youth in Action National Agency for the Czech Republic, Prague, 9 June 2008. Available as pdf.

A report of the workshop will be available on the Partnership website.

This workshop is being held under the title ‘Continue the pathway towards recognition’, but I would like to begin by saying that I would always prefer the plural, that is ‘pathways’. I asked myself whether perhaps the singular was used in the title to suggest that it is important that people work together to move forward on achieving specific aims, so that energies are not dissipated by going off in different directions. On the other hand we also know very well that there is more than one pathway to recognition – a phrase which you can approach at different levels and in different dimensions. I would prefer to say that there are several relevant pathways; we have to identify them and to work in a coordinated way to pursue them in coherent ways, each contributing from our own corners.

What follows is not the ‘Ten Commandments’, but two sets of five points that I would like to make. The first five points are of a general nature, in response to the task of giving the point of view of researchers. The second five points are areas of action that may be worth considering in continuing the pathway towards recognition.

In brief digression at the outset, I am sorry to say that I shall be unable to stay for the whole workshop: formal educational institutions and those who work and study in them cannot always be as flexible as we might ideally wish, so that it is extremely difficult to re-arrange one’s term-time lecturing schedule once the semester has begun. The main reason for this is because today’s university students cannot easily change their daily and weekly time schedules – at least this is so for those studying educational sciences (who will become, or already are, community education workers, youth workers, counsellors, educational administrators and similar). Many of our students take up degree studies in education after having several years work experience, either in educational and social fields or in quite different fields of work; some of them will already have begun degree studies in another subject, some will have completed a degree in a different field altogether. For a range of reasons they will have decided somewhere in the middle of their lives that they want to change their occupation, or perhaps to begin a serious career for the first time. So, many of our students are older than average and many have to juggle family, employment and studies all at once. But those who are younger and do not have family responsibilities are still very likely to have to earn money to finance their studies. Our students’ lives are busy and complicated; when they have organised their course schedule for the semester, they usually find it very difficult to shift pieces around if their professor has to go somewhere else at the time when the course is usually held.

I mention all this because it is indicative of what it means to be a young adult today – being young does not stop when you reach the age of majority at 18, after all, which is one reason researchers have begun to use the term ‘young adults’ to signify a period of life that stretches across ‘youth’ and ‘adulthood’. Juggling with many unknowns, contingencies and incoherencies is a complex game – constructing a life that puts all the pieces together, at least provisionally, is a major challenge. This has consequences for what young people experience and learn as they are growing into young adulthood – and it therefore has consequences for what young people need to learn and to be able to do in order to construct and manage a life in a very complex society. We should be thinking about the exploding discussions on competence and competences, both as a concept and as lists drawn up by experts and policymakers, from this point of view – from the perspective of the real-life conditions in which young people are positioned and with which they must negotiate and come to terms, hopefully positively and creatively.

Take five: researchers’ point of view

1 Who are the researchers in the first place?

The request to contribute with the researchers’ point of view made me feel rather helpless, because I am not sure who the researchers would be or whose point of view I am supposed to be expressing. I certainly do not know of any collective view of what researchers might think, so I can only really say what I think, which might provide some clues, having worked with lots of other researchers for many years. To be honest, the best answer I can offer with respect to researchers’ point of view on the recognition of non-formal learning in the youth field is to reply: ‘I draw a blank’. There is no clear group of researchers who address themselves to this issue. There are individual researchers working in many different disciplines in many different kinds of institutions with many different kinds of interests. Sometimes those things all come together, but most of the time they do not. You could surmise that this is an inevitable characteristic of any specialist field as it develops – it takes time to differentiate out into definable thematic sub-fields – as, for example, in the case of European youth research, whose first twenty years or so have been spent establishing the field as such. On the other hand, one would have thought that by now, the topic of non-formal learning would have become a distinct thematic specialism within youth research, rather than just a few individuals who are involved with the topic, but not necessarily as the main focus of their research and writing.

On reflection, I think one reason could be that few youth researchers are specialists in educational science; the majority come from sociology, social psychology and political science – this is also true for me, I am a sociologist who has always worked in education, but in fact most sociology of education is about formal education and training settings, processes and outcomes. On the whole, few youth researchers are centrally interested in educational questions at all, so very few are likely to place non-formal learning at the centre of their activities.

It has also proven very difficult to recruit young and upcoming researchers to specialise in this field, because it does not deliver the opportunities and rewards they need to build an academic career. For example, the SCI (Science Citation Index) is becoming an increasingly powerful indicator for evaluating where researchers stand in relation to each other – it provides a ‘points system’ for publications in specific peer-reviewed journals according to their (apparent) professional prestige, as adjudicated by senior academics. It is not relevant in all disciplines, but it is very important in some (such as psychology) and it is increasingly used in universities and ministries to evaluate individual and institutional performance and quality. These kinds of indicators can make a real difference to whether someone is able to get and keep a job or not – most young researchers are on temporary contracts and must ensure they fulfil a set of formal expectations, typically in competition with their peers, in order to stay in the game. To put it simply and clearly: a journal like Coyote is not even on the list of SCI-reputable journals – but nor are the peer-reviewed youth research journals that are the main orientation points for European youth research, whereas a publication with the Council of Europe or the European Commission may count amongst the real specialists, but is meaningless for most standard research assessment exercises, whether individual or institutional.

In effect, young youth researchers who invest in and engage with the field are unlikely to receive professional rewards for doing so; this is a demotivation to stay in the field. In a feasibility study I undertook last year for the youth affairs department of the Austrian Ministry for Health, Family and Youth, analysis of the authorship of reports and articles showed that most people had contributed to only one or two research or writing projects and had then disappeared from the youth research field altogether. My hypothesis is that the situation is similar in most, if not all countries in Europe. With the lack of a clear and stable reference group, with the lack of rewards and prospects, it is difficult to establish a stable professional community of belonging and identity – except amongst those who are already well-established and do not have to worry about the SCI or the next month’s income.

If I reflect on the four years that have passed since the first ‘Pathways’ document, then it is quite correct to identify an enormous dynamism, so much so that it is fully justified to ask whether things are happening so quickly that there is hardly time to digest and understand what is happening and what should now happen. The statement is correct with respect to policy and practice in the field of non-formal learning and its recognition. It is incorrect with respect to research into non-formal learning and its recognition. Little serious research has taken place and no coordinated research as taken place. The immediate reason is that there is little dedicated funding to do so, but the more important point is to ask why the priority attached to this topic is so low that little funding is made available. What lies behind the fact that there is little concern to establish a credible evidence base?

Read the full text here in pdf-format.

Thanks to Manfred for sharing his presentation with us (ppt 2.2 MB). For your convenience, we have also embedded the slides right here for a fast click-thru:

| View | Upload your own

“Le Portfolio européen pour travailleurs et animateurs de jeunesse vise à augmenter la reconnaissance de l’éducation non formelle et du travail de la jeunesse!”

S’il échoue, personne ne pourra prétendre qu’il n’était pas suffisamment ambitieux…

Mais répondons tout d’abord à quelques questions de base, telles que : qu’est-ce que le Portfolio, pourquoi a-t-il été conçu, qui l’a conçu, pour qui a-t-il été conçu, que peut-on en faire, que se passe-t-il en ce moment et comment pouvez-vous vous impliquer?

Le Portfolio est un outil qui permet à ses utilisateurs d’évaluer et décrire leurs compétences sur la base d’un ensemble de standards de qualité européens.

Du moins c’est ce que le Conseil de l’Europe décrit sur son site web, bien que vous ne trouverez nulle part la mention de standards de qualité dans le Portfolio…

Le Portfolio se veut être un instrument, un outil, un livre d’actuellement 52 pages qui tend à aider des travailleurs et animateurs de jeunesse opérant dans le contexte de l’éducation non formelle à élaborer un ensemble de standards de qualité européens et à:

• identifier, évaluer et enregistrer leurs compétences
• décrire leurs compétences à d’autres personnes, et
• à se fixer leurs propres objectifs d’apprentissage de développement.

C’est pour ces raisons que le Portfolio a été conçu – dans un contexte politique qui de plus en plus soutient et reconnaît l’éducation non formelle, requérant dans un même temps une approche plus structurée et plus transparente en ce qui concerne la garantie et les contrôles de qualité.

Dans le contexte du Conseil de l’Europe, cet historique politique a été exprimé par le Comité des ministres à travers sa recommandation N° 2003 (8) au sujet de la promotion et la reconnaissance de l’éducation/de l’apprentissage non formel des jeunes, dans laquelle il indique:

“Le Comité des ministres, (…) convaincu que l’apprentissage non-formel peut contribuer à garantir le savoir et les capacités dont les jeunes ont besoin afin de réussir au sein de nos sociétés contemporaines (…), recommande que les gouvernements des Etats membres (…) soutiennent la création et l’utilisation d’un Portfolio européen en tant qu’outil descriptif visant à consigner les expériences, les capacités et les savoirs (résultats d’apprentissage) acquis à travers l’éducation/l’apprentissage non formel (…)”

Basé sur cette décision politique substantielle et constitutive prise à un niveau élevé, le Conseil de l’Europe a invité un groupe d’experts à examiner la « praticabilité » de produire un tel instrument pour tous les jeunes. Après avoir apprécié les options envisagées, le group en est arrivé à la conclusion qu’il serait plus efficace dans un premier temps de se concentrer sur des travailleurs et animateurs de jeunesse. Ceci a conduit à la formation d’un nouveau groupe qui a alors défini les objectifs plus détaillés du Portfolio:

Le groupe a réalisé une analyse fonctionnelle de ce que les travailleurs et animateurs de jeunesse devraient être et devraient faire, ce qui a permit d’élaborer un cadre de compétences que des individus sont invités à a) utiliser en tant qu’outil d’auto-évaluation, et puis b) demander un feed-back de leur pairs et collègues.

Le Portfolio est destiné à être utilisé par les travailleurs et animateurs de jeunesse. Le groupe d’experts entends par animateurs de jeunesse des jeunes adultes assumant une responsabilité dans une organisation de jeunesse, réseau ou autre structure de jeunesse essentiellement sur une base volontaire. Les travailleurs de jeunesse assument le même type de responsabilités, mais essentiellement sur une base professionnelle.

Dès lors, que trouvent ces travailleurs et animateurs de jeunesse dans le Portfolio – ainsi que tout être humain intéressé, en fait ? Laissez-moi vous montrer un autre graphique qui illustre le contenu:

Plusieurs personnes ont été consultées durant le développement du Portfolio, permettant l’apport d’un large éventail d’expertise et de perspectives que le groupe d’experts (petit afin d’être plus opérationnel) ne pourrait fournir. Si ceci a mené à quelque chose d’utile et pratique ou non reste à voir dans la pratique.

Ce explique pourquoi, essentiellement, une phase test a été mise en place jusqu’en juillet 2006. Durant cette période vous tous – si vous êtes un travailleur ou un animateur de jeunesse tel que décrit plus haut – êtes chaleureusement invité à utiliser le Portfolio – en complétant les tableaux de compétences, obtenant un feed-back d’autres personnes élaborant un plan futur – et surtout, prenant le temps de fournir un feed-back pertinent au groupe d’experts. Ce qu’ils recherchent sont des personnes qui ont vraiment une volonté d’UTILISER le Portfolio – ils ont déjà eu assez de feed-back de personnes qui l’ont juste lu!

Les objectifs sont-ils appropriés? Sont-ils appropriés au niveau européen seulement ou applicables dans des contextes locaux? Le Portfolio satisfait-il une demande pratique – ou est-il simplement un outil politique? Les praticiens sont-ils motivés à l’utiliser? Le contexte est-il compréhensible? Le contenu est-il accessible? Le cadre de compétences est-il assez et générique, et spécifique afin d’être universel mais précis? L’investissement du temps et de la réflexion est-il adéquat? Les pairs peuvent-ils comprendre votre auto-évaluation? Le cadre de travail pour leur feed-back est-il utile? Qu’est-ce qui devrait être changé d’une façon générale au sujet du Portfolio, ainsi que par rapport aux parties spécifiques de son contenu?

Celles-ci sont certaines des questions auxquelles le groupe d’experts cherche des réponses. Afin d’en donner certaines basées sur votre expérience lors de l’utilisation du Portfolio, vous êtes joyeusement invité à employer le formulaire d’évaluation supplémentaire –développé dans ce but- que vous trouvez ici en format pdf ou ici en format doc – selon votre préférence.

Évidemment, tous les commentaire et pensées que vous avez peuvent également être partagés ici sur « Nonformality » en tant que commentaire à cet article, qui sera transmis.

D’une manière ou d’une autre: mille merci!

Liens et documents relatifs

Le Portfolio (pdf, 7.5 MB)

Le Formulaire d’évaluation (pdf)
Le Formulaire d’évaluation (doc)

Le site Web du Portfolio (www.coe.int/youthportfolio)

Note:

Évidemment, vous trouverez également d’autres documents sur ce site Web du Conseil de l’Europe, où ils sont librement disponibles pour le téléchargement. Il est contre nos normes éthiques, en tant que bloggers de marcher sur les plates-bandes des autres sans qu’ils le sachent, raison pour laquelle nous vous offrons le téléchargement directement de notre emplacement.

Indépendamment de cela, la propriété de et le crédit du Portfolio appartiennent seulement à la Direction de la Jeunesse et des Sports du Conseil de l’Europe – qui a une longue tradition de partage de sa connaissance avec une plus large communauté, tradition que nous accueillons, honorons et respectons.

Written by Andreas Karsten (andreas@nonformality.org) with assistance from Mark Taylor, consultant for the portfolio (brazavil.training@yahoo.com).

Many thanks to Gisèle Evrard for the translation!

“The European Portfolio for Youth Workers and Youth Leaders aims to increase the recognition of non-formal education and youth work.”

If the Portfolio fails, nobody will be able to claim it wasn’t ambitious enough I’d say…

But let’s answer some basic questions first, such as: what is the portfolio, why was it designed, who designed it, for whom was it designed, what can be done with it, what’s happening right now and how can you get involved?

The portfolio is a tool to enable its users to assess and describe their competences on the basis of a core set of European quality standards.

At least that’s what the Council of Europe describe it as on their website, although you won’t find mention of quality standards in the portfolio itself…

The portfolio understands itself as an instrument, a tool, a book of presently 52 pages which wants to help youth workers and youth leaders working in non-formal learning environments to

• identify, assess and record their competences,
• describe their competences to others, and to
• set their own learning and development goals.

This is what the portfolio has been designed for — against a political backdrop which increasingly supports and recognises non-formal education and, at the same time, calls for a more structured and transparent approach towards quality assurance and quality control.

In the context of the Council of Europe, this political background was expressed by the Committee of Ministers in its recommendation N° 2003 (8) on the promotion and the recognition of non-formal education / learning of young people, where it says:

“The Committee of Ministers, (…) convinced that non-formal learning can contribute to secure all the knowledge and capacities which young people need to succeed in contemporary societies (…), recommends that the governments of member states (…) support the creation and use of a European portfolio as a description tool aiming to record experiences, skills and knowledge (learning outcomes) acquired through non-formal education/learning (…)”

Based on this high-level substantial and constitutive political decision, the Council of Europe invited an expert group to look at the feasibility of producing such an instrument for all young people. After looking at the options, they came to the conclusion that it would be more effective as a first step to concentrate on youth workers and leaders. This led to a new group being formed which defined then the more detailed aims of the portfolio:

The group performed a functional analysis of what youth workers and leaders should be and do and from this drew up a framework of competences which individuals are invited to a) use as a self-assessment tool and then b) to gain feedback from their peers and colleagues.

The portfolio is intended for the use of youth leaders and youth workers. The expert group understands youth leaders as young adults holding a responsibility in a youth organisation, network or any other youth structure, mainly on a voluntary basis. Youth workers are, then, holding the same kind of responsibility mainly on a professional basis.

So what do these youth leaders and youth workers find inside the portfolio — and any other interested human being as a matter of fact? Let me show you another graphic to illustrate the contents:

Quite a few people were consulted during the development of the portfolio, bringing in expertise and perspectives from a wider range than the (small to be operational) expert group could provide. Whether or not this has led to something useful and practical, remains to be seen in practice.

Which is why, essentially, a test phase has been put in place which runs until July 2006. During this time you all — whether or not you are a youth leader or youth worker in the understanding described above — are warmly invited to have a go at using the portfolio — completing the competence tables, getting feedback from others and making a plan for the future — and, most importantly, take the time to provide some qualified feedback to the expert group. What they are looking for are people who really have a go at USING the portfolio – they have enough feedback from those who have just read it!

Are the aims relevant? Are they relevant for European level only or applicable in local contexts? Is the portfolio meeting any demand from practice – or is it just a political tool? Is motivation out there amongst practitioners to use it? Is the context understandable? Are the contents approachable? Is the competence framework both generic and specific enough to be universal but precise? Is the investment of time and thought adequate? Are peers able to understand your self-assessment? Is the framework for their feedback useful? What should be changed about the portfolio, both in general terms and in relation to specific parts of its contents?

These are but some of the questions for which the expert group is seeking answers. To give some based on your experience of using the portfolio, you are happily invited to use the extra-for-this-purpose-developed evaluation form which you find here either in pdf-format or here in doc-format — as you wish. Just send the form by email to youthportfolio@coe.int.

Obviously, any comments and thoughts you have can also come in here on Nonformality as a comment to this article which will be passed on.

Either way: Thanks a million!

Related links and documents

The Portfolio (pdf, 4 MB)

Evaluation Form (pdf)
Evaluation Form (doc)

COE Youth Portfolio Website (www.coe.int/youthportfolio)

Note:

Obviously, you will also find all the documents at this website of the Council of Europe, where they are freely available for download. It is against our ethical standards as bloggers to steal bandwidth from other people without them knowing it, which is why we offer you the download directly from our site.

Independent of that, ownership of and credit for the Portfolio belong solely to the Youth and Sport Directorate of the Council of Europe — which has a long tradition in sharing its knowledge with the wider community, a tradition we honour, respect and welcome.

Written by Andreas Karsten (andreas@nonformality.org) with assistance from Mark Taylor, consultant for the portfolio (brazavil.training@yahoo.com).