To monitor the safety and quality of e-health implementations as they roll out nationally, the CRE is developing, and will in the early stages operate, a national e-health critical incident system. Analysis of incident reports can generate critical alerts for government, vendors, clinicians and the community, as well as contribute to the development of an international classification of information technology (IT) related incidents, and theoretical and empirical models of IT failure.
Working with the Australian Patient Safety Foundation (APSF) we collate and analyse incidents associated with IT using data captured from the Australian Incident Monitoring system (AIMS).
The systematic analysis of incidents is well-established in medical practice. Incidents can trigger root-cause analyses in health services, or provide early warnings of unexpected drug reactions or infectious outbreaks. Our research extends these methods to incidents associated with e‑health (i.e. patient harm due to an IT problem or difficulty in using software), and we are pioneering this approach internationally. The aims are to :
- Develop a robust classification system to characterise the identified incidents
- Use this information to tract the evolving causes of IT related harm in Australia
- Promulgate the classification internationally.
IT incident detection and classification
To detect IT incidents in general practice we have developed and trialed a new incident-monitoring system called TechWatch. Incidents can be reported to TechWatch either online or over the phone to trained operators. Since 2009 we have analysed 1,385 IT incidents in Australia, the United States and the United Kingdom. The methods our research has generated have become the de facto international standard to detect and classify IT incidents.
Our classification had been used to examine 4,883 incidents, including by governments in the US and UK. In 2012 the Pennsylvania Patient Safety Authority used our classification system to examine one of the largest repositories of incidents in the US, and issued a Patient Safety Advisory with specific recommendations for the procurement, implementation and use of IT systems. At the same time the ECRI Institute, a US federal patient safety organisation, used our classification to undertake an in-depth analysis of incidents nationally (called a Deep Dive™). In the UK our classification system was used by the National Health Service in Wales and we collaborated with the Health and Social Care Information Centre in England to examine incidents from one of the largest civilian IT programs ever undertaken worldwide. Most recently our classification was implemented into the provincial incident monitoring system in British Columbia, Canada.
We welcome enquiries about our classification and are happy to assist individuals and organisations who wish to use the schema to analyse IT incidents.
Automated identification of incident reports
Ten percent of admissions to Australian acute-care hospitals are associated with harm to patients or adverse events. The reporting of critical incidents by health professionals is now well established and the rate of reporting continues to increase worldwide. Current methods, which rely on retrospective manual review of incident reports, do not permit timely detection of safety problems and can no longer keep up with this growing volume of data. In New South Wales alone, more than 137,000 patient-safety incidents were reported in 2011.
We are evaluating text classification methods to capture incident reports automatically by type and risk rating. The goal is to track ten types of patient-safety problems nationally working in collaboration with St Vincent’s Hospital, Sydney and the NSW Clinical Excellence Commission. Working with the Australian Commission on Safety and Quality in Health Care we have shown that text classifiers based on well-evaluated machine-learning techniques such as Naïve Bayes and Support Vector Machines can be effective in automatically identifying incidents in two priority areas – clinical handover and patient identification. More recently we have shown the feasibility of using machine learning to identify IT incidents.
- Q&A: Is practice software a safety risk?: 23 Oct 2015, Farah Magrabi on Australian Doctor (available to subscribers only)
- Healthcare IT doesn’t guarantee safety: 19 Oct 2015, Farah Magrabi on ABC Radio National, 8mins, 53secs
- Aiming for high-value healthcare: 19 Oct 2015, Farah Magrabi on ABC Radio National, 28mins, 58secs
Associate Professor Farah Magrabi, Centre of Research Excellence in e-health and Team Leader e-health safety, Centre for Health Informatics, interview on the results of her research into the safety of e-health by analysing critical incidents in e-health. Sep 2015
For more information or to join our team