|
 |
|
TECHNICAL NOTE |
|
|
|
| Year : 2021 | Volume
: 46
| Issue : 4 | Page : 341-346 |
| |
Automation of DVH constraint checks and physics quality control review improves patient safety in radiotherapy
Isak Wahlstedt1, Nikolaj Jensen2
1 Department of Health Technology, Technical University of Denmark, Kongens Lyngby; Department of Clinical Oncology, Rigshospitalet, Copenhagen; Department of Oncology, Herlev Hospital, Herlev, Denmark
2 Department of Clinical Oncology, Rigshospitalet, Copenhagen, Denmark
| Date of Submission | 01-Feb-2021 |
| Date of Decision | 25-Jul-2021 |
| Date of Acceptance | 31-Jul-2021 |
| Date of Web Publication | 31-Dec-2021 |
Correspondence Address:
Ms. Isak Wahlstedt
Department of Oncology, Section for Radiotherapy – 3993, Rigshospitalet, Blegdamsvej 9, 2100 Copenhagen
Denmark
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/jmp.JMP_23_21
 Abstract | | |
This study investigates whether patient safety can be enhanced by the implementation of an automated electronic checklist (PlanCheck) for physics quality control review (QCR) of radiotherapy photon plans. PlanCheck evaluates both technical aspects and DVH constraints. Three hundred and thirty-one consecutively approved radiotherapy plans previously reviewed with manual QCR were retrospectively checked with PlanCheck. Four hundred and thirty-three (3.4%) of the 12783 automated technical checks executed in the 331 plans yielded an error. All errors were scored using the severity rating from the American Association of Physicists in Medicine TG-100 report. Nineteen of these errors (4%) either could have affected or affected target dose (severity 5+) implicating a maximum dose difference to the target or a critical organ at risk of 0.5% to 10% and 3 errors could have resulted in stereotactic brain treatments being delivered to the wrong location (severity 10). Forty-seven breast cancer plans were retrospectively subjected to automated DVH check, 10 undocumented dose constraint violations were found. PlanCheck has been shown to reduce errors in manually reviewed radiotherapy plans and thus to enhance patient safety.
Keywords: Eclipse Scripting Application Programming Interface script, patient safety, quality control, treatment planning
How to cite this article:
Wahlstedt I, Jensen N. Automation of DVH constraint checks and physics quality control review improves patient safety in radiotherapy. J Med Phys 2021;46:341-6 |
How to cite this URL:
Wahlstedt I, Jensen N. Automation of DVH constraint checks and physics quality control review improves patient safety in radiotherapy. J Med Phys [serial online] 2021 [cited 2022 Jan 19];46:341-6. Available from: https://www.jmp.org.in/text.asp?2021/46/4/341/334388 |
| Introduction | |  |
The physics quality control review (QCR) of radiotherapy treatment plans has been proven the most effective check for preventing accidents in radiation oncology.[1] However, the increased complexity of treatment plans, time pressure, and shortage of qualified medical physicists (QMPs) have made QCR more challenging. As an aid in the QCR process, checklists are now recommended by the American Society for Radiation Oncology[2] and in 2015 American Association of Physicists in Medicine (AAPM) published guidelines for the implementation and maintenance of checklists.[3] However, QCR checklists constantly need to be adapted to new treatment techniques and updated with new departmental guidelines and new national laws and regulations. The difficulty of ensuring that all QMPs use the same physical paper checklists has been obvious at our institution and has been a threat to patient safety. As an alternative to paper checklists, electronic checklists have proven successful both with regard to standardization, reduction of plan rejection rates, reducing patient delays, reducing QCR time, and for enhancing patient safety.[4],[5],[6],[7],[8] However, the implementation of an electronic checklist in a radiotherapy clinic can be both time-consuming and require extensive programming skills. PlanCheck is a semi-automated electronic checklist containing 39 automated checks of technical radiotherapy plan aspects. As opposed to previously published electronic checklists,[4],[5],[6],[7],[8],[9],[10],[11] PlanCheck not only checks technical plan aspects but also contains automated dose-volume histograms (DVHs) constraints checks for all local dose constraints for all plans treated with photons at our institution. In this study, we assess the impact on patient safety of both the technical checks and the DVH constraint checks. PlanCheck is created with minimum effort using the Eclipse Scripting Application Programming Interface (ESAPI)[12] (Varian Medical Systems, Inc, Palo Alto, CA). Developing a checklist as a script greatly simplifies and shortens the implementation of an automated checklist compared to stand-alone programs,[8],[10],[11] thus making plan QCR automation more readily available for clinics with sparse programming experience and time.
| Materials and Methods | | |
The PlanCheck script is a C# in-house-developed electronic checklist created using ESAPI, partly based on a previous scripting project,[13] and interacting with a local database created in MySQL[14] (Oracle Corporation Redwood City, CA) containing all structure names and DVH constraints.
Our institution has made use of a physical manual checklist for QCR of treatment plans for many years. The purpose of PlanCheck was to automate as many of the checks in this physical checklist as technically possible. PlanCheck currently automates 39 checks [Appendix Table 1][Additional file 1] and checks 856 dose constraints for all 224 diagnoses treated with photons at our institution. PlanCheck is executed on a plan-by-plan basis and generates a report showing both the values expected by the script (fetched from the database) and the values extracted from the plan and the DVHs through ESAPI. Checks producing no errors are showed in green and checks that are out of tolerance are showed in red. Furthermore, the checks are written in a traffic light system, where the color of the traffic light indicates the importance of the check, red traffic light indicating the highest level of importance, yellow being the intermediate importance level, and green being the lowest level of importance. In order to standardize the physics QCR procedure, the checks that are not automated are printed in the report in the form of a manual checklist.
PlanCheck is dynamic in the sense that only the checks that are relevant for the plan in question will be activated [see what checks are activated for what plans in [Appendix Table 1]]. Adherence to all checks activated for a specific plan ensures a complete physics QCR of that plan at our institution and is followed by a checkpoint to be signed off electronically. In order to save time in the clinical workflow, PlanCheck is not only executed by the QMP as part of the physics QCR but the automated part of the script is also executed by the treatment planner at the planning stage before reaching plan approval and physics QCR, thus avoiding unnecessary plan iterations due to errors caught late in the planning process.
This study investigates whether automated physics QCR with PlanCheck reduces the number of errors in plans previously clinically approved with manual QCR, i.e., using a paper checklist. To assess the impact on patient safety of the 39 technical checks, a retrospective study was conducted. Thus, 331 consecutively approved plans, approved for treatment with manual QCR between July 1 and August 31, 2017 (before the implementation of PlanCheck), were subjected to automated QCR with the script. All errors were automatically saved in a database and the error categories (a combination of the type of check and the type of the plan) where given a severity score using the recommendations from the AAPM TG-100 report.[15] Errors with a possible severity score of 5 or higher (5+) where reviewed and the errors where scored and evaluated individually. The dosimetric impact of errors with severity of 5+ were assessed by subtracting the treatment approved dose distribution from the intended dose distribution in the Eclipse treatment planning system (TPS) (Varian Medical Systems, Palo Alto, CA). The largest dose difference (DD) in the target or a critical organ at risk (OAR) thus found was recorded.
The assessment of the impact of the automated DVH checks on patient safety is made difficult since these results are not saved in the database. However, our local dose constraints for breast cancer patients were revised in December 2020 while PlanCheck was not updated until January 2021. Thus, in December 2020, 47 consecutive breast cancer radiotherapy plans were approved using manual DVH checks only. This gave us the possibility to in February 2021 retrospectively investigate whether PlanCheck could catch DVH violations that were overlooked in the manual QCR in these breast cancer plans. The DVH constraint violations thus found were recorded and assessed dosimetrically. Dose constraint violations detected by PlanCheck but documented in the patient journal were excluded from the analysis.
| Results | | |
A total of 12783 automated technical plan checks were executed in 331 consecutively approved plans, resulting in 433 potential errors detected (3.4% of the checks resulted in an error). The distribution of the detected errors between the checks is shown in [Figure 1] [Descriptions of checks in [Appendix Table 1]]. | Figure 1: Histogram showing the per-check distribution of the 433 errors caught by the automated checks in PlanCheck sorted in pareto order. The dashed line showing the cumulative percentage of each error to the total amount of detected errors
Click here to view |
In [Table 1], the severity distribution of the detected errors, according to the AAPM TG-100 report is shown. Eighty-four percent of the errors (362) had no impact while 11% (48 errors) were assessed to potentially cause inconvenience, either to the staff or to the patient (severity 1–3). Four plans had errors that could have led to suboptimal dose deliveries (severity 4). Of the 14 plans scored with severity 5, six plans had a wrong dose normalization method (DD 0.5%–2%), two breast cancer plans did not include the couch in the dose calculation (DD 0.5% and 2%), 5 plans had an incorrect mean dose to the target compared to the prescription dose (DD 2%–2.5%), and one plan had an incorrect dose resolution (DD 5%). Two plans with errors in the dose resolution were assessed to have a severity score of 6 (DD 8% and 10%). Three plans had errors that could have led to stereotactic brain treatments being delivered to a very wrong location (severity 10). | Table 1: Overview of the distribution of severities among the technical plan errors detected by the automated checks
Click here to view |
In the assessment of the automated DVH checks on the 47 breast cancer plans subjected to physics QCR with PlanCheck, 10 errors were found. Three of these plans delivered 5-14 Gy too high max dose to the spinal cord while four plans delivered 0.4-0.9 Gy% too high mean dose to the heart. Two plans showed 0.3 and 1.4 Gy increased D35% to the ipsilateral lung, a single plan was found to have excessive dose outside the CTV by 0.2 Gy.
| Discussion | | |
In agreement with previously published work,[7],[16] our results show that the automated electronic checklist can detect errors overlooked in manual QCR.
In addition to the obvious enhancement in patient safety gained by reducing errors of geographical or dosimetric impact (severity 5+), reducing also clerical errors saves time in the clinic and has a positive impact on patient safety since corrective measures by dosimetrists and physicists are often made under time pressure. Furthermore, our previous study[16] showed that PlanCheck reduces the mean time spent per plan QCR from 16:20 minutes ±8:50 to 12:00 minutes ±9:20 (P = 0.009).
PlanCheck is continuously updated to ensure that the checks included are catching the errors seen in incidents or in clinical practice and to ensure that the script is keeping up with the technical advances in the clinic. Furthermore, the amount of checks being automated is continuously increasing, thus easing the time pressure on the physicist performing the QCR. Some parts of the check, for example, the shape and position of the gross tumor volume and clinical target volume or the dose distribution outside of the target and OARs, will be manual for still quite some time. However, due to the new technical possibilities, the size of the planning target volume is something that will be checked automatically in a future version of the software. Occasionally, we have seen that errors that could have been caught by PlanCheck have remained uncorrected in the approved treatment plan. To deal with this issue, we have recently implemented check points to be signed off whenever PlanCheck should be run, either at the planning stage or during QCR.
Since all errors were detected retrospectively some patients in this study were treated with defective treatment plans. The 19 errors with a severity of 5 + were reviewed manually, and no error was assessed to have impacted the clinical outcome to the patient, however all 19 errors were reported in accordance with national guidelines. Specifically, the three plans with severity 10 were found to not have resulted in mistreatment, although they could have resulted in a complete geographic miss. This study shows that PlanCheck contributes to improving patient safety. PlanCheck is currently used routinely for all diagnoses treated with photons at our institution. Since all structure names and dose constraints are held in a database, PlanCheck is both easy to maintain and easily implementable in other institutions using the Eclipse TPS. Versions of PlanCheck are currently used at Rigshospitalet (Copenhagen, Denmark) and at Zealand University Hospital (Næstved, Denmark) and another version is being implemented at Herlev Hospital (Herlev, Denmark). To aid in the implementation of PlanCheck in other institutions we have recently made PlanCheck publicly available on GitHub along with an SQL file containing the full constraint database.[17] As one of the authors performed the implementation at Zealand University Hospital, we have no data of how long a full implementation of the script would take for institutions unfamiliar with the script. However, the documentation on GitHub not only contains information about the content of the files and the functionality of the methods but also on what methods need editing upon implementation of PlanCheck in a new institution.
| Conclusion | | |
PlanCheck reduces the number of undetected technical errors and DVH constraint violations in treatment plans compared with manual QCR at our institution and thus enhances patient safety. Furthermore, PlanCheck is both easy to maintain and easily implementable in other institutions that are using the Eclipse TPS. It has proven its ability to catch rare errors with high potential severity for the patients, i.e., errors easily missed in a manual QCR.
Acknowledgments
The authors of this article would like to acknowledge Professor Ivan Richter Vogelius for proofreading the article and medical physicists Sidsel Damkjær and Kristian Boye for helping in identifying the checks that were included in PlanCheck. Isak Wahlstedt's PhD studies are partly funded by ViewRay, Inc.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Ford EC, Terezakis S, Souranis A, Harris K, Gay H, Mutic S. Quality control quantification (QCQ): A tool to measure the value of quality control checks in radiation oncology. Int J Radiat Oncol Biol Phys 2012;84:e263-9.  |
| 2. | Hendee WR, Herman MG. Improving patient safety in radiation oncology. Med Phys 2011;38:78-82. |
| 3. | Fong de Los Santos LE, Evans S, Ford EC, Gaiser JE, Hayden SE, Huffman KE, et al. Medical Physics Practice Guideline 4.a: Development, implementation, use and maintenance of safety checklists. J Appl Clin Med Phys 2015;16:5431. |
| 4. | Covington EL, Chen X, Younge KC, Lee C, Matuszak MM, Kessler ML, et al. Improving treatment plan evaluation with automation. J Appl Clin Med Phys 2016;17:16-31. |
| 5. | Dewhurst JM, Lowe M, Hardy MJ, Boylan CJ, Whitehurst P, Rowbottom CG. AutoLock: A semiautomated system for radiotherapy treatment plan quality control. J Appl Clin Med Phys 2015;16:5396. |
| 6. | Breen SL, Zhang B. Audit of an automated checklist for quality control of radiotherapy treatment plans. Radiother Oncol 2010;97:579-84. |
| 7. | Liu S, Bush KK, Bertini J, Fu Y, Lewis JM, Pham DJ, et al. Optimizing efficiency and safety in external beam radiotherapy using automated plan check (APC) tool and six sigma methodology. J Appl Clin Med Phys 2019;20:56-64. |
| 8. | Holdsworth C, Kukluk J, Molodowitch C, Czerminska M, Hancox C, Cormack RA, et al. Computerized system for safety verification of external beam radiation therapy planning. Int J Radiat Oncol Biol Phys 2017;98:691-8. |
| 9. | Yang D, Wu Y, Brame RS, Yaddanapudi S, Rangaraj D, Li HH, et al. Technical note: Electronic chart checks in a paperless radiation therapy clinic. Med Phys 2012;39:4726-32. |
| 10. | Yang D, Moore KL. Automated radiotherapy treatment plan integrity verification. Med Phys 2012;39:1542-51. |
| 11. | Siochi RA, Pennington EC, Waldron TJ, Bayouth JE. Radiation therapy plan checks in a paperless clinic. J Appl Clin Med Phys 2009;10:43-62. |
| 12. | Eclipse Scripting API Reference Guide. Amsterdam: Elsevier Inc.; 2011. |
| 13. | |
| 14. | |
| 15. | Huq MS, Fraass BA, Dunscombe PB, Gibbons JP Jr., Ibbott GS, Mundt AJ, et al. The report of Task Group 100 of the AAPM: Application of risk analysis methods to radiation therapy quality management. Med Phys 2016;43:4209. |
| 16. | Jensen N, Boye K, Damkjær S, Wahlstedt I. Impact of Automation in External Beam Radiation Therapy Treatment Plan Quality Control on Error Rates and Productivity. Int J Radiat Oncol Biol Phys. 2018;102:S149-50. Available from: https://www.redjournal.org/article/S0360-3016(18)31363-4/pdf. [Last accessed on 2020 Oct 28, 2021 Jan 14]. |
| 17. | |
[Figure 1]
[Table 1]
|
