Email your comments

The pilot Knowledge-Based Assessment (KBA) examination took place on the 11th May in Scotland and Northern Ireland and on the 18th May in England and Wales.

On behalf of the SAC and the Joint Assessment Board I wish to thank the local organisers and the candidates for enabling the project to be brought to fruition. I am pleased to report that all regions took part, and the paper was taken by 411 trainees (79% of the UK total) and 37 consultants. Trainees from all years were included and LATS were analysed as year 1 registrars. The 3 hour examination consisted of 100 best-of-five questions that had been written and peer-reviewed by the assessment board, using principles developed for the MRCP written examinations. We deliberately set an examination a little on the hard side to enable us to range the standard in future live versions of the paper. The overall mean result was 58.6 % (range 15-90), and a steady rise in the mean by year of training was seen from year 1 to consultant. With a notional pass mark of 50% around 80% of candidates would have passed, including most year 4 and 5 trainees and all but 1 consultant!! The alpha coefficient (a measure of the internal consistency) was 0.67 for the 100 question paper, with a projected alpha of 0.81 for 200 questions (assuming all other variables such as question quality, difficulty and candidate ability remain the same). Alpha coefficients of 0.8 or more are usually required for high stakes examinations. This is an extremely good performance for a new exam, and paves the way for further development of the exam.

A candidate questionnaire completed on the day of the exam showed that 64% thought the level of difficulty was about right (only 0.4% thought it was easy) and 70% thought 3 hours was the right length of time (only 2.5% found it to be too short).

The next phase
The pilot examinations will all have been completed by July 2006 (Geriatric Medicine and 4 other specialties). The exact pattern of future development has not been finally decided, though the most likely outcome is that the KBA will evolve into MRCP part 3 for the individual specialties. If this goes ahead as envisaged, the first live examination, for the specialties involved in the pilot, could be held as early as the end of 2007, though the timing and format will depend on further discussions with PMETB. There will a further update for the newsletter once the position is clear.

Meanwhile, a summary of the KBA pilot exams can be found on the JCHMT website, and candidates can use their examination number to find out their score. www.jchmt.org.uk/assessment/kbapilot_geriat.asp

Steve Allen
Chairman SAC Geriatric Medicine