Editor's Pick: Augmented Reality Can Improve Accuracy in Identifying Botulinum Toxin Injection Sites - European Medical Journal

Continue

Editor’s Pick: Augmented Reality Can Improve Accuracy in Identifying Botulinum Toxin Injection Sites

8 Mins
Innovations
Download PDF
EMJ Innovations 6.1

Facial injection is a highly pressurised and challenging procedure for developing clinicians, with trainees traditionally studying medical anatomy diagrams and models. This paper by van Rhee et al., selected as Editor’s Pick for this issue, provides a forward-thinking look at the opportunities for augmented reality and personalisation in this field. The study investigates use of a smartphone app, which delivers overlaying filters, individualised to the face of the user, on muscles and botulinum toxin injection sites. The paper is a valuable contribution to those included in this journal, and is expected to provoke conversations amongst the EMJ readership.

Authors:
Chevonne van Rhee,1,2 John Jacob,2,3 *Kinan Muhammed2

1. University Hospitals Birmingham NHS Foundation Trust, Mindelsohn Way, UK
2. Nuffield Department of Clinical Neurosciences, University of Oxford, UK
3. Milton Keynes University Hospital, Standing Way, UK
*Correspondence to [email protected]

Disclosure:

The authors have declared no conflicts of interest.

Received:
15.10.21
Accepted:
16.11.21
Citation:
EMJ Innov. ;6[1]:25-32. DOI/10.33590/emjinnov/21-00224. https://doi.org/10.33590/emjinnov/21-00224.
Keywords:
Augmented reality (AR), botulinum toxin, innovation, neurology.

Each article is made available under the terms of the Creative Commons Attribution-Non Commercial 4.0 License.

Abstract

Facial botulinum toxin injection is a skill developed with experience. Inaccurate injections of the toxin can cause local complications as well as patient distress. Trainees typically learn to perform facial injections following detailed study of medical anatomy diagrams. However, anatomy diagram depictions of a ‘standard’ face may not be generalisable to the varied facial anatomy of real patients. Augmented reality (AR) technology may provide a more individualised approach. In this study, an AR smartphone app, designed for the development of recreational social media filters, was repurposed to create a face filter that overlaid facial muscles and corresponding botulinum toxin injection sites onto the face of any subject detected by the supporting device’s camera. The primary outcome was to determine if accuracy in injection site identification was superior using the AR app versus a standard facial anatomy diagram. Ten participants who were naïve to administering facial injections used both the AR app and anatomy diagram to mark 10 injection sites on the face of a test subject using a makeup pen. The distance between these sites and the ‘gold standard’ injection sites as determined by an expert botulinum toxin practitioner was calculated. Participants were more accurate with the AR app than with the diagram, with average distance from expert-identified location 4.60 mm versus 6.75 mm, respectively (p<0.01). Further research is needed in optimising this technology prior to trialling its use in patients; however, AR has tremendous potential to become a useful adjunct for procedures requiring anatomical knowledge of facial muscles.

INTRODUCTION

Botulinum toxin is a neurotoxin produced by the Gram-positive anaerobic bacterium Clostridium botulinum. It acts by cleaving soluble N-ethylmaleimide-sensitive factor attachment protein receptor complex proteins in the presynaptic motor neuron, inhibiting the release of acetylcholine at the neuromuscular junction, and resulting in local chemical denervation.1,2 Botulinum toxin injection is used to treat many neuromuscular conditions affecting the face, including strabismus, blepharospasm, hemifacial spasm, and oromandibular dystonia.3,4 It is also used for cosmetic purposes in the elimination of vertical glabellar eyebrow furrows, horizontal forehead lines, and lateral canthal wrinkles.5 Identifying the appropriate site for facial botulinum toxin injection is a skill developed with practise, detailed study of anatomy diagrams, and experience. Inaccurate injections of the toxin can cause complications, including blepharoptosis, brow ptosis, and facial asymmetry.5 While these adverse effects are largely transient, they may be caused by an injection site inaccuracy of only a few millimetres6 and result in distress for patients. This highlights the need for increased precision and replicability during injection. Beginners typically learn to identify injection sites through the study of medical diagrams of facial anatomy. These depictions of a ‘standardised’ face are not necessarily generalisable to the varied shapes and contours of the faces of patients.

A more individualised approach is needed, and emerging augmented reality (AR) software may provide a solution. AR is a variation of virtual reality, where a user’s surroundings are supplemented, or augmented, by additional digital information. One of the most widespread applications of AR technology in today’s society is the omnipresent ‘face filter’. More than 4 billion people are active on social media platforms that support facial recognition software and face filters for recreational purposes.7 This widely used technology has the potential to also lend itself to personalised medicine and act as an adjunct in facial botulinum toxin therapy.

Face filters make use of face meshes, or virtual representations of the face. When a subject’s face is detected by a smartphone camera, features called nodal points such as the eye position, eye separation distance, nose position, and mouth position are extracted. These nodal points are then linked through vertices to create a virtual face mesh, applicable to any face recognised by the camera. When a face tracker and the face mesh are combined, they produce a surface that can detect facial movements. A face filter overlays an effect or image onto the face mesh, producing a form of AR where the virtual image is superimposed onto the subject’s own face in real time.8

AR apps using facial recognition and face filters could enable mapping of the optimal location for injection onto any face recognisable by the supporting device’s camera. This technology could improve the speed and accuracy when identifying sites for facial botulinum toxin injection, while accounting for anatomical variation across subjects. This may not only function as a useful learning tool for trainees in facial injection but reduce complication rates of the procedure in clinical practice. A recent study demonstrated that a dedicated AR guide for botulinum toxin injection, developed using prior radiological imaging of patients, was sufficiently accurate for use in clinical practice. Accuracy was up to 3 mm for all facial regions,suggesting that use of AR technology is becoming more relevant and feasible in this domain.9

In this study, an AR smartphone app designed for the development of social media face filters was used to create a filter that overlayed facial muscles onto the face of a subject detected by a smartphone camera. Accuracy in identifying facial injection sites using this AR smartphone app was compared to that using a standard medical anatomy diagram in participants naïve to administering facial injections. The aim was to establish if injection site identification accuracy was superior using the AR smartphone app versus a standard facial anatomy diagram. Secondary outcomes were time taken to identify injection sites, confidence using either tool, perceived usefulness of the tools, and perceived accuracy with the tools.

METHODS

A consultant neurologist and qualified botulinum toxin practitioner with extensive experience in administering botulinum toxin injection for facial dystonia identified and marked 10 points appropriate for botulinum toxin injection on the face of a test subject (Figure 1A). The equivalent 10 points were also marked on a facial anatomy diagram (Figure 1B). These were taken to be the ‘correct’ sites for botulinum toxin injection as per the expert opinion.

Figure 1: Injection site locations on the test subject, anatomy diagram, and the augmented reality image.
A) Sites for injection on the face of test subject as marked by the expert botulinum toxin practitioner. B) Anatomy diagram provided to participants with injection sites marked by expert botulinum toxin practitioner. C) Sites for injection on the face of test subject as viewed using the augmented reality app.

The AR face filter was created using Spark AR studio (Facebook, Menlo Park, California, USA).10 The basis for the filter was the same anatomy diagram on which the 10 injection sites were demarcated by the expert. When accessed through the Spark AR Player smartphone app, this face filter allowed the facial muscles displayed in the anatomy diagram and the expert’s 10 injection sites to be mapped onto any face recognised by the app in real time (Figure 1C).

Ten participants who were naïve to administering facial injections used both the AR app and a medical anatomy diagram to mark the 10 injection sites on the face of a test subject using a makeup pen. No real botulinum toxin injection was administered. The AR app was tested either on an Android smartphone (Google, Mountain View, California, USA) or an Apple iPad (Apple, Cupertino, California, USA). Half the participants performed the task with the diagram first and the other half with the app first in a counterbalanced fashion. Accuracy of markings and time taken to complete the task were measured. The same test subject was used for all participants.

Participants completed a short survey to compare confidence using either tool and perceived usefulness and accuracy of both tools following the task. They were asked to rate their confidence using the AR app and the anatomy diagram on a scale of 0–10, rate the perceived usefulness of both tools individually and combined on a scale of 0–10, and their perceived accuracy with both tools on a scale of 0–10. They were asked which tool they preferred and whether they would consider using the app for real administration of botulinum toxin injections. Participants were invited to leave comments on their experience of using the app. The subjective opinion of users was obtained to assess the likelihood of users adopting this technology for its intended purpose.

To calculate accuracy, photographs were taken of participants’ markings on the face of the test subject. These images were compared with the images of the expert botulinum toxin practitioner’s markings using graphics software Inkscape (Software Freedom Conservancy Inc., New York, USA). Accuracy values for all 10 sites were obtained by measuring the distance between participant injection site mark and botulinum toxin practitioner’s injection site mark to the nearest 0.5 mm. Injection sites 1–4 were classed as having medial locations, and sites 5–10 were classed as lateral locations (Figure 1A). The corresponding muscle groups for each injection site are denoted in Table 1. Participants had a mean age of 24.2 years. All participants were right-handed. Nine participants were male, and one was female.

Table 1: Injection site anatomical labels.
N/A: not applicable.

Statistical analysis of accuracy and time data was performed using paired t-tests, with significance levels set at p<0.05 and p<0.01, respectively. A sensitivity analysis was performed to account for up to 1 mm error in distance measurements during image analysis by adding 1 mm to measured distance between participant mark using the AR face filter and expert botulinum practitioner mark. Survey data was analysed using Friedman’s test and if significance found, post-hoc analysis with Wilcoxon signed-rank tests and Bonferroni correction was applied.

RESULTS

Accuracy

Participants were more accurate with the smartphone app than with the diagram, with average distance from expert-identified location 4.6 mm versus 6.8 mm, respectively, and this difference was statistically significant (p<0.01 [hl]Figure 2A[/hl]). When data was analysed by distribution of the injection site locations, participants were more accurate with the app versus the anatomy diagram when identifying injection locations on the lateral aspect of the face (5.4 mm versus 8.4 mm, respectively; p<0.01 [hl]Figure 2B[/hl]). Accuracy was also greater with the app than with the anatomy diagram when identifying injection sites on the medial aspect of the face, although this difference did not meet statistical significance (3.6 mm versus 4.6 mm; p=0.07 [hl]Figure 2C[/hl]).

Figure 2: Assessment of mean accuracy using the app versus the anatomy diagram. Error bars denote 95% confidence interval (*p<0.05, **p<0.01).
A) Accuracy across all injection sites using the app versus the anatomy diagram (p<0.01). B) Accuracy for lateral injection sites using the app versus the anatomy diagram (p<0.01). C) Accuracy for medial injection sites using the app versus the anatomy diagram. No significant difference was demonstrated. D) Accuracy using the app versus the anatomy diagram. Sensitivity analysis performed to account for up to 1 mm error during image analysis. Error bars denote 95% confidence intervals (p<0.05).

Among individual points, participants were only significantly more accurate with the app than the diagram in identifying sites 2 (3.4 mm versus 5.4 mm; p<0.05), 7 (6.6 mm versus 13.6 mm; p<0.05), and 8 (4.3 mm versus 8.7 mm; p<0.05). Differences in accuracy with use of the app versus the diagram did not meet statistical significance for all other points.

After sensitivity analysis accounting for 1 mm error during image analysis, accuracy remained significantly higher with use of the app versus the diagram (5.6 mm versus 6.8 mm respectively; p<0.05 [hl]Figure 2D[/hl]).

Time

Time taken to mark all 10 injection sites on the face of the test subject was available for eight participants. Participants took significantly longer to complete the task when using the app versus the anatomy diagram (79.8 versus 57.8 seconds; p<0.01).

Survey

Confidence

There was a statistically significant difference in participants’ confidence in identifying injection sites among participants before the task, using the app, and using the anatomy diagram, χ2(2)=8.970; p=<0.05. Post-hoc analysis with Wilcoxon signed-rank tests was conducted and Bonferroni correction applied. Median (interquartile range [IQR]) for confidence levels in identifying injection sites before the task, using the app, and using the anatomy diagram were 0.00 (0.00–2.25), 5.50 (3.00–8.00), and 6.00 (2.75–6.25), respectively, and significance level was set at p<0.017. Participants demonstrated a statistically significant increase in confidence levels when using the app compared with their confidence levels prior to the task (Z=-2.395; p<0.05) and a statistically significant increase in confidence levels when using the diagram compared with their confidence levels prior to the task (Z=-2.530; p<0.05). However, there was no significant difference between participants’ confidence levels using the app versus using the anatomy diagram (Z=-0.852; p=0.394).

Usefulness and perceived accuracy

There was no statistically significant difference in participants perceived usefulness of the app alone, anatomy diagram alone, nor app and anatomy diagram combined, as tools for identifying botulinum toxin injection sites χ2(2)= 1.067; p=0.587. Median (IQR) for participants’ perceived usefulness in identifying injection sites using the app, using the anatomy diagram, and both adjuncts together were 6.00 (4.75–7.25), 6.00 (4.75–6.25), and 6.00 (5.00–8.25), respectively.

There was no statistically significant difference in participants perceived accuracy using the app versus the anatomy diagram χ2(2)= 0.000; p=1.000. Median (IQR) for participants’ perceived accuracy using the app and using the anatomy diagram were 5.50 (3.00–8.00) and 5.00 (3.75–6.25), respectively.

Participant comments

Of the participants, 70% reported they were willing to use the app to guide them if they were to perform real botulinum toxin injections. Despite this, a number of participants stated they preferred using the diagram as an aid.

Participants reported two significant issues that caused movement of the face filter when using the app: difficulty in holding the supporting device still; and adjustment of the image frame when a participant’s hand moved into view of the camera during marking of an injection site. These frame shifts were particularly apparent when marking injection sites on the side opposite to a participant’s dominant hand and contributed to time delays in marking injection sites when using the app.

DISCUSSION

In this study, an AR face filter of facial muscles was developed by repurposing readily available software that is used recreationally to make social media face filters. This AR face filter allowed users to identify sites for facial botulinum toxin injection significantly more accurately than through traditional means of using a reference anatomy diagram. This improvement in accuracy is important in facial injection of botulinum toxin as it may minimise risk of side effects for patients. The results also demonstrate the potential of this readily available technology both as a learning tool and as a clinical aide for practitioners with limited experience.

Despite improvements in accuracy, participants were slower and no more confident in their performance when using the AR app compared with the anatomy diagram. Participants attributed this to difficulties in keeping the supporting device still, resulting in small but notable movements of the face filter. In the future, these concerns could be addressed by using hands-free devices or stands to keep the device camera stable. Specially designed head-mounted displays, or ‘smart glasses’, have already been trialled with success in neurosurgery11 and maxillofacial surgery,12 and the app-supporting headset Microsoft HoloLens (Microsoft Corporation, Redmond, Washington, USA) is being applied in multiple healthcare settings, including facilitating virtual ward rounds13 as well as AR-assisted surgery.14.15 A similar approach may therefore show benefit in botulinum toxin facial injections.

Over the previous decade, research into the use of AR within medicine has become increasingly prevalent. AR has predominantly been used to facilitate imaging-guided surgery by enabling pre-operative CT or MRI images to be overlain onto the surgical field and guide an operation in the context of the patient’s real anatomy.16 This technology has been trialled in areas of the body that are generally non-deformable as manipulation of these tissues during surgery is minimal, requiring less processing power to track anatomy and maintain accurate image overlay. Consequently, AR research has focused on maxillofacial surgery,12,17,18 neurosurgery,11,19-22 orthopaedic surgery,15,23,24 and hepatobiliary and pancreatic surgery.25-29

In contrast, this preliminary study suggests that AR has the potential to be harnessed in clinical domains without the need for prior imaging of the test subject. Here, facial recognition software designed for social media purposes was used to detect facial landmarks and allowed the filter of facial muscles to be overlayed, such that the resultant AR image forms an accurate representation of the test subject’s underlying musculature. This technology can be relied upon further as facial recognition systems are becoming increasingly more accurate. In 2020, the best face identification algorithm had an error rate of 0.08%, compared to 4.10% for the leading algorithm in 2014.30 In a recent study, a dedicated AR guide for botulinum toxin injection was developed by combining facial recognition software with a standard oral maxillofacial model based on CT or MRI images of patients.9 With this guide, a mean accuracy of 0.40±0.25 mm was demonstrated with a range of 0–3 mm, a standard the authors deemed sufficient for use in clinical practice.9

AR filters must clearly achieve a higher standard of accuracy for use in clinical practice than for recreational use. With an average accuracy of 4.6 mm, the AR app used in this study does not meet the 3.0 mm error margin proposed as the limit suitable for clinical practice.9 However, this preliminary study is the first to suggest that readily available software, designed for recreational social media purposes, can be harnessed to improve the accuracy of facial botulinum toxin injection when compared to the use of a standard anatomy diagram.

The study’s findings are limited by its small sample size. While the results provide an encouraging basis for future research into use of AR for improvement of facial injection accuracy, the study is not adequately powered to draw definitive conclusions. Further research would involve using a larger cohort of participants as well as test subjects of different ages and genders to examine the reliability of the AR app. The ‘gold standard’ reference injection sites could also be refined by averaging opinions from multiple experienced botulinum practitioners, instead of the single expert consulted in this study.

CONCLUSION

As face filters and facial recognition technology are refined for entertainment and recreational purposes on social media, it is only a matter of time before this AR technology is routinely used in medical practice. In this preliminary study, participants were more accurate using an AR face filter app, developed using popular social media software, than they were with a traditional anatomy diagram. While participants did not perceive the app to be any better than the diagram, the improved accuracy using the app demonstrates a clear benefit. It is evident that this technology opens a promising avenue for not only training purposes, but with refinement and further advances, it has the potential to improve the accuracy of facial injections and reduce rates of complication in clinical practice. Further research is needed in optimising this technology prior to trialling its use in patients; however, AR seems to be a viable and useful adjunct for procedures requiring anatomy knowledge of the facial muscles.

References
Kao I et al. Botulinum toxin: mechanism of presynaptic blockade. Science. 1976;193(4259):1256-8. Blasi J et al. Botulinum neurotoxin A selectively cleaves the synaptic protein SNAP-25. Nature. 1993;365(6442):160-3. Münchau A, Bhatia KP. Uses of botulinum toxin injection in medicine today. BMJ. 2000;320(7228):161-5. Nigam PK, Nigam A. Botulinum toxin. Indian J Dermatol. 2010;55(1):8-14. Small R. Botulinum toxin injection for facial wrinkles. Am Fam Physician. 2014;90(3):168-75. Raffaela P et al., “Botulinum toxin adverse events,” Serdev N (ed.), Botulinum Toxin (2018), 1st edition, London: IntechOpen, pp.33-6. Kemp S. Digital 2021 April global statshot report. 2021. Available at: https://datareportal.com/reports/digital-2021-april-global-statshot. Last accessed: 18 November 2021. Shaburova E; United States Patent Application Publication. Method for real time video processing for changing proportions of an object in the video. 2015. Available at: https://patentimages.storage.googleapis.com/6d/f0/58/aaca29a68d77b5/US20150222821A1.pdf. Last accessed: 18 November 2021. Kim HJ et al. Augmented reality for botulinum toxin injection. Concurr Comput. 2020;DOI:10.1002/cpe.5526. Spark AR Studio. Available at: https://sparkar.facebook.com/ar-studio/. Last accessed: 18 November 2021. Tabrizi LB, Mahvash M. Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg. 2015;123(1):206-11. Badiali G et al. Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Craniomaxillofac Surg. 2014;42(8):1970-6. Microsoft. Imperial College Healthcare NHS Trust uses mixed reality in the fight against COVID-19. 2020. Available at: https://customers.microsoft.com/en-us/story/825289-imperial-nhs. Last accessed: 18 November 2021. Microsoft. A hospital revolution: 3D assistance in operations. 2019. Available at: https://customers.microsoft.com/en-us/story/770890-apoqlar-azure-hololens-cognitive-services-health-en. Last accessed: 18 November 2021. Dennler C et al. Augmented reality in the operating room: a clinical feasibility study. BMC Musculoskelet Disord. 2021;22(1):451. Fida B et al. Augmented reality in open surgery. Updates Surg. 2018;70(3):389-400. Qu M et al. Precise positioning of an intraoral distractor using augmented reality in patients with hemifacial microsomia. J Craniomaxillofac Surg. 2015;43(1):106-12. Cercenelli L et al. The wearable VOSTARS system for augmented reality-guided surgery: preclinical phantom evaluation for high-precision maxillofacial tasks. J Clin Med. 2020;9(11):3562. Cabrilo I et al. Augmented reality-assisted skull base surgery. Neurochirurgie. 2014;60(6):304-6. Watanabe E et al. The trans-visible navigator: a see-through neuronavigation system using augmented reality. World Neurosurg. 2016;87:399-405. Kersten-Oertel M et al. Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. Int J Comput Assist Radiol Surg. 2015;10(11):1823-36. Haemmerli J et al. Evaluation of the precision of operative augmented reality compared to standard neuronavigation using a 3D-printed skull. Neurosurg Focus. 2021;50(1):E17. Keating TC, Jacobs JJ. Augmented reality in orthopedic practice and education. Orthop Clin North Am. 2021;52(1):15-26. Molina CA et al. Augmented reality-assisted pedicle screw insertion: a cadaveric proof-of-concept study. J Neurosurg Spine. 2019;33(1):139-46. Erratum in: J Neurosurg Spine. 2020;120(1):120. Pessaux P et al. Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch Surg. 2015;400(3):381-5. Golse N et al. Augmented reality during open liver surgery using a markerless non-rigid registration system. J Gastrointest Surg. 2021;25(3):662-71. Okamoto T et al. Navigation surgery using an augmented reality for pancreatectomy. Dig Surg. 2015;32(2):117-23. Luo H et al. Augmented reality navigation for liver resection with a stereoscopic laparoscope. Comput Methods Programs Biomed. 2020;187:105099. Tang R et al. Augmented reality-assisted pancreaticoduodenectomy with superior mesenteric vein resection and reconstruction. Gastroenterol Res Pract. 2021;DOI:10.1155/2021/9621323. Grother P et al.; National Institute of Standards and Technology (NIST) US Department of Commerce. Face recognition vendor test (FRVT): part 2: identification. 2019. Available at: https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8271.pdf. Last accessed: 18 November 2021.

Rate this content's potential impact on patient outcomes

Average rating / 5. Vote count:

No votes so far! Be the first to rate this content.

Thank you!

Please share some more information on the rating you have given