Wrist & Ankle fractures are one of the most commonly seen fractures at NHS-UK.
Some significant issues faced by the NHS, UK here are-
Pressure on A&E: Annually, out of approx. 23 Million NHS A&E visits, approx. 8 Million (35%) are referred by primary care (1), of that 50% needs radiography (3). Out of this, approx. 1.4 million cases are suspected wrist & ankle fracture referred by primary-care to A&E for radiography (2,3).
Diagnostic Error: As per the BMJ report, clinicians are missing reading X-Rays in approx. 35-39% cases (3,4), thereby potentially approx. 400K missed or undetected fracture ( wrist-and-ankle only ) every year leading to patient sufferings and reduced QALY.
Skill Shortage: Vacancy rates are 12.5% for radiologists and 15% for radiographers (5,10) leading to backlogs in radiography.
1. Baker C. Accident and emergency statistics: demand, performance and pressure. Briefing paper. 2017 Feb 21;6964.
2. Jennison T, Brinsden M. Fracture admission trends in England over a ten-year period. The Annals of The Royal College of Surgeons of England. 2019 Mar;101(3):208-14.
3. Susan Liver. Broken bones are being missed in A&E – medical negligence.2019.
4. Emergency Medicine Journal. 2001 Jul 1;18(4):263-9.
A blog by our AI researcher – Sunny shows how AI based solution can address the problem:
AI in fracture detection- A technical perspective to solve the problem
Globally, fractures are a common reason for hospital admission, including admissions due to missed diagnosis or undetected diagnosis. The healthcare systems across the planet are under pressure due to lack of resources (radiologists, devices or technical-infra) which impact the quality and outcomes of care delivery.
In the last few decades, technology has progressively assisted in medical diagnosis and could be the rainmaker in future with the advent of Artificial intelligence assistance in care delivery. The prominent impact of AI could be the prioritisation and segregation between critical and non-critical cases across all healthcare settings and medical management.
Detecting and prioritising a fracture with AI assistance could not only ease the burden of radiologist or health system but also enable clinicians to spend more time on critical cases where a slight delay could be fatal.
However, there are a few technical aspects to be addressed in developing an AI solution.
Architecture to detect bone-fracture
An AI-enabled system that can auto detect/localized and prioritise the region of broken-bone/bone-fracture encountered many challenges. Some key challenges are:
Challenge 1: Using Computer Vision, detection of fracture is a challenging task, considering the various types of fractures those are small in size, wide range of different shaped and hard to detect.
Solution: Problem is more centric towards precision, hence algorithm must be configured to serve two tasks, first is to identify that fracture is present in the given image or not. Second, is to locate the fracture, using bounding box, if fracture is present.
Challenge 2: ‘Region Proposal Network’ is to handle objects of very different scales in real-life scenario.
Solution: This problem can be tackled by training a set of RPN for various scales. Each RPN will take different convolution layer or set of layers as input so the receptive field will of different size. This significantly improves detection of both small and large object.
Challenge 3: Which architecture can serve the best job?
AI is a science where persistence and continuous experimentation gives results.
- Single Shot multi-box Detector (SSD) is fastest with mobileNet base model and support low resolution of image (hence, Input image should be of low resolution).Hence,SSD based algorithms struggles especially in detecting the small objects like bones fractures.
- The most accurate model is Faster R-CNN with its complicated inception Resnet-based architecture and 300 proposals per image.
- Hot spot, where we reach a balance between precision and speed are, Faster R-CNN with Resnet Architecture having 100 proposals. Since it supports higher resolution images to process, it is easy for algorithm to find out abnormality from the image which is considerable factor when we have to deal with small size of fracture.
- Resnet module is also beneficial in case of deep feature extraction where increase in number of layers does not lead to cause of Over-fitting.
Architectural Workflow – fracture detection system
A computer vision algorithm having Faster R-CNN base with Resnet under Tensorflow and Keras Framework is used to classify as well as localized fractured area within the X-ray image. Several base architectures were used (in experimental phase) such as VGG, mobileNet, inception v2 and inception v3 before to finalize the Inception Resnet v2 to serve as base architecture (having highest “mean Average Precision – mAP” rate among all tried algorithms) with Faster R-CNN. This finalized architecture is capable to even detect minor abnormality in the processed image and ensure us that deep neural network can provide an effective way to identify the fracture in X-ray images, precisely.
It is a state of art detection network, mainly consists of three parts:first part take-cares of classification and generates feature map (Inception Resnet v2, convolution deep neural network is used here as based architecture). Second part is a regional proposal network for generation region proposals and third is regressor, for detecting precise location of each object and its classification. In this architecture, finer localization information has been encoded in the channels of convolution feature response and Region Proposal Network (RPN) slide a small window on the feature map to build a small network for classifying fractured pattern/object or non-fractured. Regressing bounding box locations and position of the sliding window provides localization information with reference to the image. Box regression provides finer localization information with reference to this sliding window and at each sliding window position a set of k-object proposals is defined and each proposal has different size and aspect ratio. Such proposals are called anchors which improve handling of objects of different sizes and aspect ratio. Further, RPN keeps anchors (if intersection over union is larger than 0.7 or) that pass the threshold in a way that has the highest intersection over union (IOU) with the ground truth box.
About Loss Function: Faster R-CNN is optimized for multi-task loss function that combines the losses of bounding box regression and classification. Especially Regression Loss whereOutput of regression compute the predicted bounding box and regression loss signify the offset from the true bounding box.In training of RPN Box regression with reference to anchors, propagation and stochastic gradient descent (SGD) play important role where new layers are initialized through zero-mean Gaussian distributions and rest other layers is initialized using pre-trained network of classification.
L({Pi }, {ti }) = Pi, Pit) + λ
PitLreg(ti, tit)
Symbol “i” represents index of anchors in mini batch and Pi stands for predicted probabilities of anchor being an object. Pit is 1 if anchor is positive and 0 in case of negative anchor. Here, ti is a vector which represents the four parameterized coordinates of the predicted bounding box. Also, tit denotes the ground truth vector of the coordinates associated with positive anchors. Symbol “L” denotes classification log loss over two classes such as object vs. no object.
What lies ahead By 2030, AI will change the way health systems across the planet operate and deliver care to patients, covering all specialities. Technology will assist health systems to focus on “predictive, preventive and curative” care with better outcomes.
