Email your comments

Prof James Barrett's reply

While we accept that e-portfolios will be with us from now on, we have an opportunity as a specialty to try to adapt them to our advantage. Presently, many of the trainees are of the view that they are not fit for purpose.

In principle, the advantages of e-portfolios are clear - a neat web based record of training that can be easily scrutinised and provides evidence of your progression. However, the general feeling is that the current software is cumbersome and slow, with many trainees and trainers struggling. I can also see the advantage of having a more rigid assessment procedure and hope this will also improve the standard of assessors - an improvement on the sometimes cursory RITA process..

However, the disadvantages of the current system are also clear. Geriatric medicine is a difficult speciality, and is a challenge to do well. It involves complex decision making and often, highly complex medicine. There are no set procedures with easy outcomes to measure and against which the geriatrician can be judged. It is not easy to assess a geriatrician. Some specialities, such as anaesthetics, surgery and procedure based medical specialities (cardiology, gastroenterology), lend themselves very well to a log book approach to assessment. Geriatric medicine, however, does not.

This is something we should be proud of. A difficult speciality needs more than a simplistic tick box assessment method. I, for one, have yet to work out what a DOPS should be, and struggle with the rigid tick boxes of the ‘mini-cex’. There isn’t an official CBD document yet to use, but this tool would seem to be a more sensible way, to assess us. The rigid structure of the supervisor’s report is also not fit for purpose. Indeed, many trainees are reporting that supervisors are refusing to do them and continue to dictate a formal letter style report. I know the move is away from what is seen as ‘old school’ assessment, however I do feel that we should continue to embrace the good bits inherent in the old methods. What can be more important than what your colleague and current consultant thinks about how you work?

My concern is also that rather than making the assessment procedure more robust, the “tick box” route will lead to a lazy approach, and these will be used to assess one remotely. Indeed, the West Midlands deanery suggested that the RITA/ARCP process could be done remotely with only some problem STR’s being seen face to face (I am pleased to say this was wholeheartedly rejected).

The pilots were certainly of dubious validity, with only 1 Spr in the Mersey deanery choosing to continue using it out of 5. The CMT pilot from which this is based also showed poor uptake. It would appear to a group of trainees who already feel disgruntled having been disadvantaged by MTAS, that this is being thrust upon us with not enough thought, and certainly no evidence to suggest that the new approach produces better doctors or assessments.

I suggest that, as a group, we expand an assessment process developed for our strengths as a speciality. Let us be leaders of this, and not have a ‘one size fits all’ approach, hurriedly and messily imposed on us. I feel this should include elements of formal assessment, using CBD and mini-cex assessments, but it should also include more traditional methods.

Thomas Jackson
St4 West Midlands Deanery

Prof James Barrett's reply:

Dear Zoe and colleagues

Thank you for asking me to respond to Dr Jackson’s letter. He clearly was one of the attendees at the lively discussion we had about the e portfolio at the BGS trainees group meeting in Bournemouth.

The curricula
Following the Bristol and other scandals the general public now expect that doctors in training (and thereafter) should be required to show evidence that they have been adequately trained and that they have been assessed in the workplace to have achieved the necessary competencies. The PMETB approved curriculum for Geriatric Medicine training plus the GIM (Acute) and Generic Curricula describe the competencies to be achieved during the training of a geriatrician and the assessments that are required during training. A further revision of the curriculum is currently in progress.

Why an e portfolio is used
The e portfolio is designed to support this process and is based on sound adult learning principles. The linkage to the curriculum competencies is there to help with the setting and review of the objectives included in the personal development plan. It is designed to be easy to access and use to record assessments and reflections and other evidence and to organise that against the curriculum objectives in a way that facilitates the review of progress made in acquiring the curriculum competencies.

Dr Jackson in his letter supports the concept of using an e portfolio but he and most trainees and supervisors have experienced the frustrations of using a web based system in NHS hospitals. The time taken to access the e portfolio and enter assessments, appraisals and other entries is dependent on the hospital IT system rather than the speed of the e portfolio itself. There are however, certain features on the portfolio which are admittedly more time consuming than they need to be.

There have been a number of long standing requests lodged with the NES developers to facilitate the quick review of the assessments entered without clicking on each one individually and to streamline the sign off procedure for competencies in the curriculum record of progress section. This is now top of the physicians portfolio development agenda for the next few months.

These time consuming elements have been supervisors’ main gripe with the e portfolio especially as the time that needs to be dedicated to proper educational supervision of doctors in training has not yet been adequately recognised in consultant job plans. Very few consultants have the 1 hour per week per trainee supervised that has been advocated by the Royal College of Physicians.

Some craft specialties do have log books to record the numbers of procedures performed. These may need to be integrated into the e portfolio for their use but I agree that moves to introduce this approach for Geriatric Medicine trainees should be resisted as it does not add value.

The evidence base to support the use of case based discussions and other new assessments has been gathered over the last 2 years to meet PMETBs forthcoming standard that only validated assessment methods should be used and there is general agreement that case based discussions are a good way to assess the achievement of competence when used as part of a collection of assessment methods performed by multiple experienced and trained assessors.

It has been suggested that e portfolio has not been successfully used with SpRs in Mersey and there was poor uptake in CMT. I lead the pilot work in Mersey and there were some people who did not want to use e portfolio but they were also not keen to do any other form of appraisal or engage in educational supervision. The portfolio is now well established with CMT doctors and we are approaching the end of 2 years use in Mersey with the higher specialty trainees who have only recently started to be able to link their evidence to the curriculum record. It will be a few months before a steady state is achieved with e portfolio use in the higher specialties which should coincide with the introduction of the revised GIM curriculum and the updated specialty curricula once they have been approved by PMETB.

The other big change is around the ARCP process which is prescribed by the Gold Guide to be a trainee not present process. It is a review of evidence. All trainees should still, however, expect a face to face discussion with their training programme director at least once per year and use the opportunity to feedback about the quality of their training.

The targets to be achieved at each stage of training need to be explicit and we should all demand greater clarity from the ARCP decision aids to be used with the revised curricula. The original curricula were written for use with the RITA process before that was changed. The Geriatric Medicine decision aid was praised by PMETB but it and the others now need to be rewritten to fit the ARCP process. In Mersey, John Clague and a few colleagues devised an interim local decision aid for the GIM and Generic curricula which has been shared with the BGS trainees group.

One of the main pieces of evidence the ARCP panel requires is the supervisor report but regrettably, many supervisors do not yet sufficiently understand the curriculum objectives and are not writing their reports with those in mind. The e portfolio supervisor report includes some fields which are automatically filled and is surely preferable to hand-writing long lists of the evidence that is to be presented to the ARCP panel.

There is a section for the supervisor to write a short pen picture of the trainee’s progress towards the end of the supervisor report which fulfils the function of the previous free text report. Programme Directors value

those comments highly within the context of a structured report that fulfils all the ARCP requirements.

The JRCPTB e portfolio is now the only training record for physicians in training and there is a genuine desire for it to be easy to use and to support learning. Further constructive comments are welcome and will be fed into the future development of the portfolio.

James Barrett

P.S. We have posted on the website, for those interested, the MDSM Supervisor Report Help file and the Level 2 GiM (Acute) ARCP Decision Aid for ST3+ acute medical specialties in Mersey.

PPS. At a brainstorming meeting at the Royal College of Physicians (London) meeting on 23 June, plans were made on how to steamline the most time consuming processes within the e-portfolio. A work request is being forwarded to NES to implement the changes but it is not possible to guess at this stage how long that will take to put in place.

JB

BGS Newsletter, July 2009
Issue 22 ISSN 1748-6343 22