Using self-assessment guides : Necessary but not sufficient?

John Diamond reflects on effectiveness of self-assessment guides and tool-kits

Over the past twelve months I have read a number of self-assessment guides or 'toolkits' which promote best practice, as well as writing one myself. As a consequence I have been struck by their potential to highlight important questions and by their limitations, and I want to suggest here that their advantages often conceal their limitations. These limitations are, sometimes, quite profound. I want to explore these issues in order to stress that whilst they are helpful and necessary they are not sufficient.

Significant advantages

In reading and reviewing a number of these guides it is evident that they share certain positive features;

· Identifying core questions: The very best guides do ask participants to think about their agency or service and how it addresses its remit. The setting out of these questions often enables users to think about what they do and where the gaps may be.

· Promoting self-reflection: In posing these questions guides have the potential to promote a process of self-reflection. By recognising and then stating the gaps in their service users are part of the way to identifying how they might fill the gaps and, more significantly, why the gaps exist in the first place.

· Clarifying forms and remit: The very best guides can act as a way of defining (or restating) the focus and remit of an agency. The public, voluntary and community sector may find itself seeking to expand its focus (either to address the particular unmet needs of service users or to draw down resources from the latest funding pot), these guides can act as a way of reminding providers what it is they are supposed to do and provide a check on how they are doing it.

· Benchmarking against the sector: The need for individuals and organisations to assess the progress of their agency against other comparable agencies can be met through those guides. They provide a framework against which it is possible to place existing practice within one agency against a sector 'norm' which can then be used to stimulate discussion about where and how change can be effected as well as offering the opportunity to assess any one particular agency's efficiency.

The primary claim of such guides and toolkits is that they provide an opportunity for individuals and agencies to explore their practice and to identify potential areas for change. As a process of learning through reflection, discussion and debate they offer the potential of managing change towards an agreed set of objectives. In the language of best practice guides they help to frame the 'action plan'.

A secondary claim is that the process of self-regulation is a necessary part of managing change (and resources) in the policy and practice environment within which we all work. We could argue then, that guides legitimise (or not) our existing practice and provide the rationale to justify changes (or resources).

However I want to go onto suggest that it is the 'absences' contained in many of these guides to which we should be sensitive. In particular I want to question the claims set out above. I want to point to the assumptions implicit in such guides and to which we need to be aware of.

Significant gaps

The 'missing' elements of guides and toolkits look like this:

· Externally framed questions: All the guides I have looked at or worked with (including my own) reflect the needs of those who commissioned the guides or toolkits. This is not to argue that the questions asked are not relevant or useful. Rather it is to argue that those who frame the questions have their particular needs to be met which are not always congruent with those who then use the guides. Users of guides are, by definition, taking on a framework and way of looking at the world through the lens of those who set the limits of the guide.

· Capacity to challenge: In some instances these guides are promoted with the claim that they provide an indicator of what is necessary to achieve 'best practice'; but rarely engage in a process which is interactive or open to challenge. The potential to challenge the questions or the framework is significantly reduced for agencies or organisation which lack resources to approach a guide with an open agenda. Across the voluntary and community sector the primary pressure is one of 'keeping up' with a rapidly changing set of local and national initiatives. Retaining the capacity to respond to what is happening is difficult enough; investing in the capacity to anticipate future developments or to challenge ideas and practice is much more difficult.

· Embedding passivity: A consequence of the growth in these guides is to institutionalise passivity. Rather then encourage individuals and organisations to question critically what they do and to imagine alternative possibilities, toolkits can have the opposite effect of restricting their sense of what is possible. What I am suggesting here is that guides 'normalise' particular practice and approaches.

· Normalising practice: The risk of normalising approaches and ways of looking at the sector is present through a combination of factors. This may be an unintended consequences which the authors/promoters of guides did not wish to advocate. The identification of key questions, (discussed earlier), reflect the context within which the guide/toolkit is framed. What needs to be examined is the scope for challenging and contesting the given context. It seems, to me, that the process of encouraging users of guides to actively challenge the framework is rare. Practitioners are, on the contrary, led to examine their own practice through a named route, the questions posed have the effect of assuming a particular practice and approach which the users of the guide either confirm or fail to meet. Enhancing the capacity of practitioners to challenge the context and to engage in a process of self-reflection is, in effect, ruled out.

· Over relying on experts: The absence of alternative possibilities also reveals the over-reliance on 'expert' judgements. Most guides and toolkits have, as part of their remit, a claim to promote 'ideal' models of practice. As small organisations or agencies benchmark themselves against this 'ideal' they may fail to notice that the framework itself is not open to question. I would suggest that whilst these guides need to come with a 'health warming' they also need to have easy and accessible support through peer review or critical friends. Part of the role of these agencies/individuals would be to stand outside the received framework and work with individuals to explore how they use the processes involved in these guides to aid their learning and self-reflection.

Closing thoughts
It is important to restate that I do think that many of the guides I have looked at have value. They often provide the starting point for a series of conversations within organisations. They can point to gaps in provision and ways of working which can accentuate learning within our organisation. But I am equally struck by re-reading an essay published in 1998 by Sharon Howell (The Learning Organisation: Reproduction of Whiteness). She makes the powerful point that agencies do not sit in an isolated environment separate from the social, political, race and gender divisions present all around us. Organisational tools and management practice too reflects these divisions. We need to be both open to change and to question the basis of those claims which are about promoting value for money, learning and organisational practices. We need to be aware of positioning - that of the position of the agency in relation to other similar agencies and the positioning of all the agencies in relation to external institutions and structures. Without these, toolkits may be necessary but are certainly not sufficient.

John Diamond works for the Centre for Local Policy Studies at Edge Hill College, Lancashire.

Contact: diamondj@edgehill.ac.uktop