Portfolios have been used to showcase abilities in education, architecture, photography and the arts for many years but only began to be used in nursing schools in the early 1980s as a means to document educational attainment (Budnick and Beaver, 1984). Portfolio use in the paramedic profession, however, has a more recent history. Even as an emergency service, the ambulance service is pre-dated by the fire service and police by many years and only in 1974 did the ambulance service move from county council control into the NHS (Craggs and Blaber, 2008).
This does not diminish the need for an evaluation of the use of portfolios in paramedic practice but it does help to qualify why this particular evaluation paper places such a reliance on literature from the nursing profession. Capsey (2010) supports this position in identifying that paramedic practice has evolved from a near standing start and so there is a need for the allied health professions to guide and develop practice (Capsey, 2010). This idea is developed further by Campeau (2009) who stresses the importance of learning from other professions, but within a framework of paramedic practice that is quite unique from other health care professionals (Campeau, 2009).
The purpose of this two-part article is to evaluate, not assess, the use of portfolios in paramedic practice. One definition of evaluation is offered by Ottoson (2000) who sees it as responsible for assessing value. In accepting this definition, it is easy to see how evaluation and assessment are terms which are used interchangeably. Further clarification is presented by Wilkes and Bligh (1999) who regard assessment as a measure to determine individual performance, as opposed to evaluation which places a value on a given activity.
Whilst both assessment and evaluation can be seen to involve some form of measurement, the distinction between the two processes needs to be recognized. Drawing on the principles of evaluation within a higher education (HE) setting, the role of the Quality Assurance Agency (QAA) is key.
Role of the Quality Assurance Agency (QAA)
Established in 1997, the QAA is responsible for, amongst a wider remit, academic standards in HE institutes. The very ethos of quality assurance is in itself a form of evaluation and the six basic quality assurance questions identified by the QAA are transferable to this particular evaluation (Table 1).
| What are you trying to do? | PURPOSES |
| Why are you doing it? | REASON |
| How are you going to do it? | METHOD |
| Why is that the best way to do it? | OPTIMISATION |
| How do you know it works? | EFFECTIVENESS |
| How can you improve it? | ENHANCEMENT |
From: QAA (2010)
There is a need to define ‘quality’ before it can be evaluated so it is therefore unhelpful to note that there is no one accepted format a portfolio should take in presenting evidence of a paramedic’s competence. In the absence of a universally accepted format, confusion exists in some quarters over the strategies for portfolio compilation, not eased by the plethora of commercially produced and marketed portfolios available to buy in various retail outlets (Jasper, 2001). Having said that, the development of portfolios has undoubtedly been influenced by regulatory bodies, reflective practice, market forces and HE (Hull and Redfern, 1996).
Defining a portfolio
In its simplest form, a portfolio is a collection of material put together in a meaningful way to demonstrate the practice and learning of a practitioner (Forde et al. 2009) and it is widely accepted that the most important factor is to ensure a portfolio is compiled in a logical and well-organised manner (Redman, 1994; Karlowicz, 2000; Brown, 2001; Hayes et al, 2002; Moon, 2004; Norman, 2008; Anderson et al, 2008; Seldin and Miller, 2009).
College of Paramedics and portfolios
The College of Paramedics (CoP) is the professional body for paramedics in the UK. Since its inception in 2001, initially under the name of the British Paramedic Association, the CoP has worked with the governing body (HPC) in areas of paramedic practice such as standards of proficiency, curriculum guidance and continuous professional development (CPD) (CoP, 2010b). As part of its remit to support paramedics, the CoP has issued guidance to its members on what a portfolio should include. Accepting that it is not a definitive list of contents, there is some consistency with other literature to support the inclusion of elements such as a CV, development plan, job description, certificates, records of clinical experience and associated reflections (CoP, 2010a). To this end, and in keeping with contemporary trends of portfolio compilation, the CoP have developed an online portfolio template which its members can complete and up-date periodically as necessary.
Internet-based portfolios
Internet based or e-portfolios are seen as an innovative tool for practitioners to use in conjunction with their practice (Carraccio and Englander, 2004; Jantzi and Connie, 2005; Marcoul-Bulinson, 2006; Butler, 2006; Anderson et al, 2008). However, the transference of portfolios to the internet or electronic medium is not without its critics and is far from universally accepted as a positive development. A study by Naude and Moynihan (2009) documented the e-portfolio experience of 32 postgraduate nursing students. The findings showed that the students recognized the practical applications of having online portfolios, such as applying for jobs, but limited IT skills amongst the participants of the study proved problematic and completing and up-dating portfolios on-line was seen as time consuming and difficult (Naude and Moynihan, 2009).
The time consuming dimension to maintain an e-portfolio can just as readily be applied to the paper-based equivalent but the associated issue with both forms of portfolio is that of data protection.
Confidentiality
In 1997, a review was commissioned by the Chief Medical Officer for England over the use (and confidentiality) of patient information ‘largely due to the development of information technology... and its capacity to disseminate information about patients rapidly and extensively (Department of Health (DH), 1997).
Under the Chairmanship of Dame Fiona Caldicott, the Caldicott principles have become an integral part of record keeping for all health care professionals. The following year, the Data Protection Act (1998) was introduced and covered computer and manual records where patient information may be collected and used.
Enshrined within the eight principles of the Act, is the need for information to be held securely and confidentially and only shared appropriately within the parameters of the Act (Data Protection Act, 1998). Usefully, in 2003, the DH published a code of practice with respect to confidentiality and clearly set out what it considered patient-identifiable information (DH, 2003). Apart from the obvious inclusion of a patient’s name, address and date of birth, the code of practice extended to include photos, videos and/or audio tapes.
Portfolio templates
So while the format of portfolios presents specific challenges for paramedics and other health professionals, the unifying theme of portfolio templates has generated much debate. By their very nature, portfolios are individual and not amenable to standardization (Pitts et al. 1999). While uniformly prescribed structures may add value where paramedics are being assessed against a common framework of competence, a standardised portfolio limits the scope for paramedics to examine specific areas of their practice and to add content of special significance in the professional context (Forde et al. 2009).
The author’s own portfolio contains a number of elements which are identified as being integral to any portfolio of professional practice, such as a CV, development plan and job description. But for the most part, the portfolio is of the author’s own design and is tailored to reflect his current scope of practice.
While constructing a unique portfolio design inevitably takes longer than simply relying on some form of template model, it is undoubtedly more personal to the paramedic and could even be seen as integral to the learning process itself (Hull and Redfern, 1996). The value of a more individual style to a portfolio has been identified by Billings and Kowalski (2005) who regard it as providing ‘synthesis and interpretation’ to the meaning of the documents contained in the portfolio (Billings and Kowalski, 2005: 149) and this is supported by Alexander et al (2002) who agree that adopting a personal approach when putting a portfolio together gives it a specific identity (Alexander et al, 2002).
In the absence of an agreed format and universally accepted criteria for paramedic portfolios, it immediately presents a potential problem of inconsistency in relation to one of the fundamental purposes of a portfolio, supporting evidence of a paramedic’s competence and ultimate fitness to practice.
Role of the Health Professions Council
Formed by the Health Professions Order of 2001, the HPC is the governing body for paramedics. In 2006, the HPC’s Conduct and Competence Committee agreed a set of professional standards for paramedics which came into effect from 1st July 2008 (HPC, 2008). More specifically the HPC requires paramedics to:
‘….maintain a continuous, up-to-date and accurate record of their CPD activities….and present a written profile containing evidence of their CPD upon request’ (HPC, 2006: 2).
The HPC have identified a number of CPD activities and have divided them into categories (Table 2) (HPC, 2006).
| Work-based learning | (case studies/refective practice/secondments) |
| Professional activity | (mentoring) |
| Formal education | (research/writing academic press) |
| Self-directed learning | (reviewing books) |
| Other | (voluntary work) |
From: HPC (2006)
While the HPC encourages registrants to keep a record of their CPD activities in whatever format is convenient to them, if a paramedic is selected for an audit of their portfolio the HPC will provide a template profile for the registrant to complete. In August 2009, the HPC conducted its first CPD audit of paramedics. As part of the author’s research for this evaluation, the HPC was contacted and information relating to this audit was requested. 2.5% of state registered paramedics were asked to provide evidence of their CPD activities and this equated to 378 paramedics submitting their portfolios to the governing body.
Of those 378 portfolios, 37 profiles were returned to the registrant as they did not meet the HPC’s professional standards (HPC, 2010b). This translates into approximately 10% of paramedics audited failing to provide sufficient evidence of their CPD— the main reason cited by the HPC being ‘...no engagement in the process’ of CPD (HPC, 2010b).
This generates a conflict in the use of portfolios. Are they to be seen as a medium for documenting CPD activities or demonstrating a paramedic’s fitness to practice? Put another way, are portfolios in paramedic practice responsible for documenting a process (CPD) or measuring an outcome (fitness to practice)? This distinction has been made by Fellows (2008) who argues that there is no automatic link between CPD and competence. Fellows believes it is possible, although unlikely, for a competent paramedic not to undertake any CPD and still meet the HPC’s standards of conduct, performance and ethics. Equally, it would be possible for a paramedic who is not competent in their practice to complete a lot of CPD and document it effectively in a portfolio (Fellows, 2008).
Regulation of professional standards
The concept of professional competence and regulation of professional standards is not a new one, as professional training in the 19th century began with the formation of ‘study societies’. Members of these societies saw themselves as skilled practitioners whose professional existence was dependent on members of the public distinguishing between those who were ‘competent’ and those who were not (Carr-Saunders and Wilson, 1994). Qualifying exams were introduced for those wishing to enter into societies.
The competence of all healthcare professionals must be assessed to protect the public, ensure quality of training and maintain credibility of the professions (Hand, 2006) but it is not clear whether portfolios are suited to this end and if so, how that professional competence is measured.
Within the nursing profession, portfolios have been used to facilitate both continuous assessable learning, in response to changes and complexities in nursing practice, as well as foster personal qualities such as critical thinking (Byrne et al, 2007). This position is supported by Andre and Heartfield (2007) who argue that portfolios are a useful tool to showcase both a practitioner’s educational record and demonstrate competence (Andre and Heartfield, 2007).
Similarly, a case study relating to the introduction of portfolios at Whyalla hospital in Australia, has demonstrated that portfolios can be used to document both the process of CPD and an outcome in the form of professional competence (Emden et al, 2003; 2004).
Despite support for this dual role of portfolios, it does not reconcile the obvious shortcomings outlined by Fellows (2008). Deconstructing this further, the tension between a paramedic’s responsibility to maintain their own professional competence and the pressure exerted by external bodies who are responsible for ensuring those standards, has been identified by Lenburg (2000) who has advocated the need for a system which maximises individual accountability as well as supporting the involvement of professional regulators (Lenburg, 2000).
A possible solution lies in what represented the biggest change to terms and conditions within the NHS in more than fifty years. As part of the Agenda for Change programme, which came into effect from December 2004, the Knowledge and Skills framework (KSF) was introduced to ‘….provide a fair and objective framework on which to base review and development for all staff’ (DH, 2004). Comprising six core dimensions (relevant to all parts of the NHS) and twenty four specific dimensions which are job-specific, the KSF is a list of competencies against which all NHS employees can have their performance ‘measured’.
Building a competency profile
As Hammond and Collins have observed, a competency profile is ‘….a detailed list of competencies a ‘competent’ practitioner requires’ (1991: 98). On the surface, such a competency-based approach lends itself well to the use of portfolios. A paramedic, or any allied health professional, need simply list their prescribed competencies in line with the KSF and document how they have demonstrated/met each competency in turn.
In terms of documenting a process of learning, particularly where the CPD activities are linked to an annual development plan, the author believes portfolios in paramedic practice have some value. However, when the focus moves to determining whether a paramedic is competent, the use of competencies (and their inclusion in a portfolio) become problematic.
The most widely accepted construct of competencies is based upon behaviour theory, as the emphasis is on producing specifically desired behaviours (Norris, 1991; Gonczi, 1994; Worth-Butler et al. 1994; Manley and Garbett, 2000; Albanese et al, 2008). Behaviourism is primarily associated with the work of Pavlov (classical conditioning) in Russia and Skinner in America (Atherton, 2009). The ethos of Pavlov’s classical conditioning is that a formerly neutral stimulus elicits a response because it has been paired with another stimulus which have elicited that response in the past (Petri, 1996).
However, of greater relevance to this evaluation is the concept of what Skinner referred to as operant conditioning (Skinner, 1953). Skinner argued that it is possible to ‘shape behaviour’ through positively reinforcing desired behaviours (1953: 91).
Applying Skinner’s notion of ‘shaping behaviour’ to a competency-based assessment model, a two year study conducted between 2000 and 2002 in the use of portfolios in nursing, resulted in professional competencies being referred to as ‘round pegs in square holes’ (Scholes et al, 2004: 599). Funded by the former English National Board for Nursing, Midwifery and Health Visiting, the study showed that nurses entering the profession for the first time and using portfolios to document their professional competence found some competencies too abstract and consequently, deconstructed them to fit with their current practice (Scholes et al, 2004).
These findings are consistent with Benner’s belief that performance measurement is only as accurate as the competencies selected to be measured (Benner, 2001). Benner’s work is more aligned to the constructivist theory of learning and self-directed approach to learning and uses the Dreyfus and Dreyfus (1980) model of skill acquisition to explore how nurses progress in the development of professional competence (Benner, 2001).
Moving from an initial position of reliance on abstract principles (competencies?), Benner charts a nurse’s professional development to ultimately drawing on past experiences to influence their professional practice. Emphasising the gap between theory and practice, Benner argues that a nurse needs to develop ‘tools’ in order to learn from experience and develop from being a detached observer to an involved performer (Benner, 2001: 184).
There is again an affinity with the ‘process’ dimension of CPD although Benner does not offer any proposed methodology for documenting this learning process. In relation to the ‘tools’ which may be useful to the health care practitioner to aid in the learning process, the wider literature on self-directed learning and use of portfolios clearly highlights the benefit of engaging in a process referred to as reflexivity (Schön, 1983; Kolb, 1984; Gibbs, 1989; Brew, 1993; Boud et al 1993; Usher, 1993; Brown and Sorrell, 1993; Jasper, 1999; Blaber, 2008; Cork, 2008; Sibson, 2009).
Reflective learning
The notion that ‘….wisdom may come through experience, but it does not come through an accumulation of experience’ (Brew, 1993: 97) is fundamental in reflexivity and supports the ethos that central to learning is personal reflection (McMullan, 2003; Rolfe, 2002).
The idea of a ‘reflective practitioner’ was introduced by Donald Schön in the early 1980s (Schön, 1983) but the strength of the reflective process lies in the individual taking responsibility for their own learning, which is a basic tenet of adult learning principles (Knowles, 1975). Learning from experience is a process in itself, which can be described and evaluated through different models of reflection (Usher, 1993) and these written reflections are an essential component of the portfolio process (Johnson et al. 2010).
The HPC have cited reflective practice as a work-based CPD activity and support its inclusion in paramedic portfolios (HPC, 2006). In keeping with the earlier discussion on portfolio format, no one template or reflective model is promoted by the HPC, simply that paramedics engage in reflexivity and document it as evidence of their CPD.
However, the recurring question arises with respect to the use of portfolios in paramedic practice. Is reflective practice evidence of a process of CPD or evidence of an outcome e.g fitness to practice? Writing for a portfolio is different to writing for an assessment (Blaber, 2008) and in accepting this premis, concerns emerge regarding the validity of personal reflection as evidence of fitness to practice.
Conclusion
If the reliability of an assessment tool relates to the consistency of a given method as opposed to a valid tool measuring what it is intended to measure (Sibson, 2010), the author believes that paramedic portfolios in their current form have an obvious restriction.
With the existing audit process of paramedic portfolios not involving some form of verification of evidence, as opposed to the nursing profession in New Zealand which is required to have portfolio contents verified prior to submission (Dawbin, 2005), the reliability of portfolios as a tool to demonstrate a paramedic’s competence can be questioned.