Surgical Audit and Peer Review Guide 2014
Surgical Audit and Peer Review Guide 2014
Surgical Audit and Peer Review Guide 2014
http://www.surgeons.org/policies-publications/publications/
Mr G Campbell FRACS
Chair, Professional Standards Committee
Dr A J Green FRACS
Chair, Surgical Audit Committee
1
Thomson M, Oxman A, Davis D, Haynes R, Freemantle N, & Harvey E (2000) Audit and
feedback to improve health professional practice and health care outcomes (Cochrane
Review). In: The Cochrane Library, issue 3, part 2. Oxford: Update Software, 1997.
2
RACS (2013) Continuing Professional Development Information Manual
3
Aitken R, Nixon S & Ruckley C 1997 Lothian Surgical Audit: a 15-year experience of
improvement in surgical practice through regional computerised audit The Lancet 350
(9080):pp.800-804
4
The Western Australian Audit of Surgical Mortality – a 30% reduction in deaths over ten
years: Aitken R, Azzam D, Itotoh F, Neo A: Medical Journal of Australia 2013; 199 (8): 539-
542
5
The causes and effects of delay to diagnosis: outcomes from surgical mortality data: North
J, Blackford F, Wall D, Ware, R, Rey-Conde, T, Allen J: British Journal of Surgery 2013;
Volume 100, Issue 3
6
Clinicians Toolkit http://www0.health.nsw.gov.au/pubs/2001/pdf/clintoolkit.pdf
7
Van Rij, A. (1996) Continuing Education & Surgical Performance, The Bulletin 16 (3):pp. 5-8
8
Australian Medical Association Position Statement Clinical Indicators 2012
https://ama.com.au/ition-statement/clinical-indicators-2012
Outcome Indicators
Number and percentage
Individual Fellow rate compared to audit aggregate
OUTCOME
Intraoperative complications
Postoperative complications
Blood transfusion
Return to theatre
In-hospital death
Readmission within 30 days
30-day mortality
Length of Stay:
Mean and Range expressed as ‘days’.
Individual Fellow result compared to audit aggregate.
LENGTH OF STAY
Intubation Mean
Range
Initial post-operative ICU Stay Mean
Range
Post-operative hospital stay Mean
Range
Indicators
Biliary tract indicators
Bile leak requiring intervention
Bile duct injury
Unplanned ERCP post-operative
Colorectal Cancer Indicators
Unplanned reoperation
Mortality
Anastomotic leak
Hernia Indicators
Postoperative Haematoma
Referral to pain clinic
Hernia recurrence
Thyroid indicators
Recurrent Laryngeal Nerve Palsy
Postoperative hypocalcaemia Ca <2.0mmol/l
9
Dindo D, Demartines N, Clavien P. Classification of surgical complications. A new proposal
with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg. 2004, 240:
pp.205-213.
In many hospitals some of the above listed adverse events are already
reported and there should be no need to recollect the data. The challenge for
those responsible for preparing an audit is to ensure that reportable adverse
events such as nosocomial infections, falls, pressure sores and medication
errors are known to the surgical team and where appropriate discussed.
Complications
Complications should be graded according to the Clavien-Dindo
10
classification. The first three editions of this surgical audit guide combined
Grades 3 & 4 below so had only a four level classification system. In recent
years the Clavien-Dindo classification has become widely accepted and has
replaced the previous level system.
Grade 1: Any deviation from the normal postoperative course without the
need for pharmacological treatment, surgical, radiological or
endoscopic interventions. For example, problems that did not
prolong admission and had little impact on the patient’s well-being,
such as a mild wound infection, urinary tract infection, atelectasis,
unexplained pyrexia
Grade 2: Requiring pharmacological treatment, including blood transfusion
or TPN.
Grade 3: Complication which necessitates intervention, surgical,
endoscopic or radiological
10
Dindo D, Demartines N, Clavien P. Classification of surgical complications. A new
proposal with evaluation in a cohort of 6336 patients and results of a survey, p. 18
Grade 5: Death
2.9. The Minimum Data Set, Expanded Data Set and Trainee
Logbook Data Set
To give guidance in collecting the data that are essential for effective
surgical audit, the Surgical Audit Committee developed recommended
data sets which allow some consistency of data and easier comparison
of outcomes. The Minimum Data Set, which is the bare minimum of
information, is particularly suited for large volume low risk procedures.
Peer Review
Peer review is a learning exercise. Whilst rights, responsibilities,
apportionment of blame, punishment, compensation and access to
justice can be valid processes, they should not be confused or interfere
with the processes of education, risk management and quality
assurance. Peer review is not an opportunity to blame or brag.
Peer review involves an evaluation of one’s work by one’s peers. Peers
are other surgeons with comparable training and experience. It can
often also be helpful to include other non-surgical members of the team
in the review group e.g. surgical trainee or senior nursing staff. The
review should be conducted in an atmosphere of confidentiality, of trust
and teamwork, and be seen as an evolving process.
11
Aberg, T Svenmarker, S Hohner, P Hentschel, J (1997) Routine registration of deviations
from the norm in cardiac surgery: a potent clinical research tool and quality assurance
measure. [Journal Article] European Journal of Cardio-Thoracic Surgery. 11(1):10-2, 13-6
Computer systems
A database management system is required. It may be written using
commercially available general-purpose programs or it may be custom built.
It is recommended that, where practicable, particularly for individuals in
private practice, data be used for multiple purposes such as billing, reporting
or clinical records, so that the system provides multiple benefits in addition to
surgical audit.
As manual data recording and entry can be tedious and prone to error, it is
recommended that advantage be taken where possible, of automated or
semi- automated entry, such as bar codes, scanners, down-loading from
other systems or use of look-up tables, etc.
Smartphones, iPads and tablets enable audit data capture on the move.
They offer the opportunity for data entry using touchpad, handwriting or voice
in addition to a computer keyboard, and will be able to share information
wirelessly through clouds, mobile phones and other wireless networks.
Whatever system is used, contemporaneous data backup is important for
when devices and drives fail.
Logbooks
Logbooks used by surgical trainees provide an opportunity to start a data
collection system as part of an ongoing process towards surgical audit.
There are a number of programs available, including Morbidity Audit and
Logbook Tool (MALT), that have been developed by the College. It will
enable all surgeons (as well as specific groups) to easily undertake a
Qualified Privilege
RACS considers that confidentiality is essential for effective surgical audit. All
audit information collected and discussions must remain strictly confidential.
One means of assuring confidentiality for audit is to obtain legal protection
under ‘qualified privilege’ schemes. The surgeon in charge of an audit and
peer review activity should ensure that the activity is registered under
appropriate Federal or State or Territory schemes and that there is
12
New Zealand Privacy Amendment Act
http://privacy.org.nz/assets/Files/Codes-of-Practice-materials/HIPC-1994-incl.-amendments-
revised-commentary.pdf
13
Health Practitioners Competence Assurance Act 2003, Public Act 2003 No 48 Date of
assent,18 September 2003
14
Australian Health Practitioners Regulation Agency (AHPRA) National Law
http://www.ahpra.gov.au/Legislation-and-Publications/Legislation.aspx
8. AUDIT REPORTING
Peer review meetings are limited in time. Discussion needs to focus on the
most important issues. This makes a formal audit report of a service, unit or
surgeon's work a valuable document to refer to the routine and mundane.
A pre-agreed structure and format of reports assists in their generation
regardless as to whether this is done by hand or automated from the
database.
A minimum standard of reporting is expected for any surgical audit. It should
identify all the cases that were done for the area of interest. It should be able
to sub-classify the cases according to preagreed criteria within the minimum,
expanded, trainee or specialty datasets. For example, surgeon D wishes to
audit all cholecystectomies sub-classified by whether they were emergency
or elective admissions. The audit report should identify particular adverse
15
Yap C, Colson M, Watters D (2007) Cumulative sum techniques for surgeons: a brief
review, ANZ J Surg. 2007 Jul;77(7):583-6.
Mandatory Reports
Search/ find/ sort by any of the minimum data fields (total, pre, post op, LOS)
Aggregate reports
Number of admissions
Number of operations, Sub classified by procedure, urgency, magnitude
Ability to list cases with/ without complications
Sub classified by grade 1 – 5
Advisable Reports
Binary recording of key performance indicators
At minimum, identify:
• Unplanned reoperations;
• Unplanned readmissions;
• Unplanned ICU admissions/ readmissions;
• Prolonged LOS (varies according to procedure);
• Unanticipated blood transfusion.
Other procedure specific Indicators
Calculation of key performance indicators from numerator/ denominator
Ability to generate CUSUM
Specialty specific risk adjustment tools are available, for example Fellows
16 17
performing cardiac surgery can use AUSScore and POSSUM can be
used within General Surgery. Ideally, risk adjustment tools should be based
on a small number of variables.
There are two major comorbidity scoring systems, based on papers from
18
Elixhauser and Charlson. Elixhauser is simpler in that it apportions a score
19
of 1 to every comorbidity from a list of 35, whereas Charlson’s group
created an index where some comorbidities were deemed more important
than others and so were allocated a score of 2 rather than 1. Generally, an
Elixhauser comorbidity count of 4 or more represents a significantly comorbid
patient in terms of risk.
16
Reid C, Bilah B, Diem D, Smith JA , Shardey GC. The AusScore – a validated model for
predicting outcome from isolated coronary bypass graft procedures in Australia. Heart,
Lung and Circulation 2007;16:S179.
17
POSSUM is a scoring system that correlates with outcome. It has been used in
General Surgery, particularly colorectal surgery. Portsmouth (P-POSSUM), Colorectal (CR-
POSSUM), vascular (V-POSSUM) and Eldery (E-POSSUM) versions are also available.
POSSUM was developed for surgical audit and has two components: an acute
physiology score and operative severity score. POSSUM scores can only be completed
after discharge and therefore it is an audit tool for risk adjustment but cannot be used for
individual patients during their care. (please see Further References).
18
Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with
administrative data. Med Care 1998;36:8-27
19
Charlson ME, Pompei P, Ales KL, MacKenzie CR A new method of classifying prognostic
comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):
pp.373-83.
Gillies T, Ruckley C, Nixon S. Still missing the boat with fatal pulmonary
embolism. Br J Surg. 1996, 83(10): pp.1394-5.
Goligher JC, Pulvertaft CN, Irvin TT, Johnston D, Walder B, Hall RA,
Willson-Pepper J, Matheson TS. Five- to eight-year results of truncal
vagotomy and pyloroplasty for duodenal ulcer. Br Med J. 1972 Jan
1;1(5791):7-13.
North JB, Blackford FJ, Wall D, Allen J, Faint S, Ware RS, Rey-Conde T..
Analysis of the causes and effects of delay before diagnosis using
surgical mortality data. Br J Surg. 2013 Feb,100 (3), p.419-25. Doi,
10.1002/bjs.8986. Epub 2012 Dec 6.
http://www.ncbi.nlm.nih.gov/pubmed/23225342
It can also be useful to include a free content field to record qualitative information.
Free content fields may also assist imports from other databases.
These may be related to specific procedure or outcome in a specialty
• Agreed clinical indicators (e.g. bile duct injury in laparoscopic
cholecystectomy)
• Late outcomes (e.g. I, 2, and 5 year cancer outcomes)
• Patient satisfaction.
These fields should be considered very carefully as data overload is a danger to
successful audit.
20
Goligher JC, Pulvertaft CN, Irvin TT, Johnston D, Walder B, Hall RA, Willson-Pepper J,
Matheson TS. Five- to eight-year results of truncal vagotomy and pyloroplasty for duodenal
ulcer. Br Med J. 1972 Jan 1;1(5791):7-13.
The College’s Morbidity Audit and Logbook Tool (MALT) has been designed with
the primary aim of enriching the education and training experiences for a trainee
by:
• Promoting communication between the trainee, supervisor and training board
through the ability to generate summary and individual reports
• Enabling the system to be portable, web-based which includes mobile and
handheld devices that are web enabled
• Ensuring validity/consistency in data collection
• Increasing data collection opportunities for specialty areas
• Progression towards the establishment of an effective audit process for
Fellows.
The Australian and New Zealand Society of Cardiac and Thoracic Surgeons’
Database Program.
(formerly The Victoria Cardiac Surgery Database Audit)
Contact: Mr Gil Shardey, FRACS
Email: [email protected]
Web: http://www.ascts.org
For further information regarding the audit approval application process, please
contact the Department of Professional Standards:
Phone: +61 3 9429 1200
Fax: +61 3 9276 7432
Email: [email protected]