|Year : 2021 | Volume
| Issue : 3 | Page : 221-227
Assessment of uncertainty depending on various conditions in modulation transfer function calculation using the edge method
School of Radiological Sciences, Faculty of Health Science, Gunma Paz University, Takasaki, Gunma, Japan
|Date of Submission||09-Mar-2021|
|Date of Decision||14-Jun-2021|
|Date of Acceptance||06-Jul-2021|
|Date of Web Publication||8-Sep-2021|
Mr. Sho Maruyama
3-3-4 Tonyamachi, Takasaki, Gunma 370-0006
Source of Support: None, Conflict of Interest: None
| Abstract|| |
In medical X-ray imaging, to perform optimal operations, it is essential for the user to understand whether a required image quality level which depends on a diagnostic task can be achieved with the imaging system used. This study focuses on the effects of noise on the modulation transfer function (MTF) using the edge method, the most widely used to evaluate the task dependence property. The purpose is to verify the uncertainty of the MTF value at each spatial frequency and examine the conditions under which the accuracy is ensured. By using a Monte Carlo simulation, edge images with various contrast-to-noise ratio (CNR) are acquired. MTFs are then calculated with different edge spread function (ESF) lengths. The uncertainties for each spatial frequency are estimated based on the independent MTF calculations obtained from the five edge data. The uncertainty of the MTF is inversely proportional to the CNR. In the frequency range up to the Nyquist frequency, the uncertainty in five calculations is <0.01 when the CNR is more than 60. In addition, it is observed that the uncertainty increases as the ESF length increases. This relationship depends on the frequency range, but it is proportional to the 0.3–0.5 power of the ESF length. The results in which the uncertainty is most likely to be large in the MTF calculation are clearly shown. Therefore, it is expected to provide an important barometer and useful insights for a proper image quality measurement.
Keywords: Contrast-to-noise ratio, modulation transfer function, uncertainty
|How to cite this article:|
Maruyama S. Assessment of uncertainty depending on various conditions in modulation transfer function calculation using the edge method. J Med Phys 2021;46:221-7
|How to cite this URL:|
Maruyama S. Assessment of uncertainty depending on various conditions in modulation transfer function calculation using the edge method. J Med Phys [serial online] 2021 [cited 2022 May 27];46:221-7. Available from: https://www.jmp.org.in/text.asp?2021/46/3/221/325673
| Introduction|| |
In medical X-ray imaging, it is required to minimize the exposure dose and acquire useful images for diagnosis. For that purpose, it is necessary to quantitatively grasp the features of an imaging system by correctly measuring the physical image quality characteristics. In addition, understanding whether a required image quality level which depends on a diagnostic task can be achieved with the system is an important requirement for users.
Evaluating the resolution properties, which explain the signal transfer characteristics of an imaging system, plays a significant role in characterizing the system. The modulation transfer function (MTF) is an established evaluation method that describes the impulse response of a system in the spatial frequency domain.,, There are various methods for measuring the MTF,,,,,, but the International Electrotechnical Commission (IEC) recommends the edge method with a precisely polished tungsten (W) plate. Although the edge method can easily acquire image data because it is tolerant of alignment error, image noise is amplified by the differential processing performed to convert the edge spread function (ESF) to the line spread function (LSF), affecting the accuracy of the calculated MTF. Several methods have been proposed to reduce this effect.,,,,,, Furthermore, some studies are being conducted for developing new measurement methods to obtain accurate MTFs for imaging systems.,,
Measuring MTF requires that the linearity and the shift-invariant of the output signal with respect to the input signal are established., In recent years, however, image processing technology with nonlinear characteristics has been introduced,, and images that have undergone complex processing are used in clinical practice. To evaluate an image with nonlinear properties, a task-based MTF (MTFtask) was developed, and the resolution characteristics of computed tomography images reconstructed by the iterative reconstruction (IR) method were evaluated. Richard et al. investigated the resolution characteristics of an image by applying the IR method using the and clarified the contrast and dose dependency of the signal transfer in the IR method. Urikura et al. proposed an approach using a bar pattern to measure accurate in moderate contrast materials and reported the required contrast-to-noise ratio (CNR) levels. Takada et al. examined the effect of the edge shape when measuring the MTFtask and showed there was no difference in the results obtained by straight and circular edges (i.e. no shape dependency). These studies show that an MTF measurement with low contrast signals is required to evaluate the task-dependent performance of the system, not only the high-contrast signals used in traditional MTF measurement intend to evaluate the performance of the system itself. Such comprehensive image quality evaluations in linear imaging systems have been also performed in various ways to optimize imaging conditions.,
Because the MTF represents the spatial frequency characteristics of the signal transfer, it should not include the noise effects in the measurement and the results should not fluctuate with noise levels. However, it is difficult to completely eliminate the noise, so this effect must be taken into consideration. The effect of noise on the accuracy of the MTF has been theoretically verified by Cunningham et al. and Antonio., In complicated evaluations with a nonlinear property or a task-dependency, it may be necessary not only to measure the characteristics of low-contrast signals but also to perform measurements under conditions that include the effects of noise. It is important to fully understand the effects of signal strength and noise on MTF measurement as shown previously. The required CNR level reported by Urikura et al. is a recommended value in the measurement using the bar pattern and has not been clarified concretely in measurements using the edge method. Therefore, there remains ample room to verify the effects of noise on MTF acquisition.
This study focuses on the effects of noise on the MTF calculations using the edge method, the most widely used and applied method, to evaluate the . The purpose was to verify the uncertainty of the MTF value at each spatial frequency and examine the conditions under which the accuracy of the MTF is ensured. In particular, we investigated the effect of the CNR of an edge image and the ESF length for the MTF calculation. The verification was conducted using a Monte Carlo simulation, and the discussion is developed based on the results obtained by the actual MTF calculation using the simulation image. To clarify the factors of the uncertainty and simplify the interpretation of the results, only the quantum noise was considered and the effects of additional component (e.g. electrical noise) and multiplicative component (e.g. structural noise) were not included.
| Subjects and Methods|| |
Image formation by simulation
To evaluate the effect on the measured MTF value properly, it is necessary to clarify the factors that may affect it; it is important to eliminate uncontrollable factors other than a subject contrast and the quantum noise. In addition, it would be reasonable to understand the degree of quantum noise that depends on the number of photons contributing to image formation when verifying the effect of image noise. Therefore, we considered that various image quality factors could be controlled using a Monte Carlo simulation for an accurate estimation. In this study, the Electron Gamma Shower ver. 5 (EGS5) was applied for image formation.
During the formation of the edge images, the blurring component of the detector was only the sampling aperture. The stochastic blurring component, was not included, X-ray photons were incident to the detector perpendicularly, and the conversion to the light and the diffusion of the light in the sensor were prevented. Because the scattered radiation from the edge is also an error factor in the MTF measurement, a 1-mm-thick W plate was used to ignore the influence of the scattered radiation, with the photon energy set to 50 keV. Following the IEC standard, a 100 mm × 100 mm W edge was placed at an angle of 2.5° to the pixel array to obtain the presampled MTF. The detector in the simulation setting operated as a photon counter that counted all incident photons with a pixel size of 0.2 mm × 0.2 mm (pixel fill factor = 1), and the pixel value of the output image was the number of incident photon in each pixel. That is, it has a linear input-output characteristic.
In this simulation, a general-purpose personal computer with an Intel Core i7-8700HQ 3.20 GHz CPU and 32 GB RAM was used, and the calculation time required was 2 h per 1.0 × 108 histories. This computer can perform up to eight independent simulations simultaneously, but it took approximately 2 weeks to acquire all image data.
Modulation transfer function measurement and conditions of the verification
Using the central part of the simulated edge image, the LSF was obtained by differentiating the ESF after the binning process was performed to obtain the finely sampled ESF (bin width 0.02 mm). The MTF was calculated by performing a Fourier transform of the LSF. During the calculation process, additional MTF conditioning techniques,,,,,, such as windowing, filtering, or extrapolation of the LSF tails were not used to purely evaluate the effects of the quantum noise.
To calculate the uncertainty of the MTF value as a function of the CNR, edge images were acquired by changing the number of photons incident on the detector. The number of photons per unit area was changed from 500 to 500,000 mm − 2. The range of the CNR obtained under the conditions of this verification was between 4 and 140. We considered that the range used for the MTF measurements under various conditions was sufficiently covered. [Table 1] lists the conditions used in this verification (the CNR and the number of incident photons per unit area). The CNR is defined as the following formula:
|Table 1: The conditions of edge image formations for modulation transfer function calculations|
Click here to view
where SBG and Sedge are the mean pixel values of the background area and the edge area, respectively, and σBG is the standard deviation of the background area. These values were obtained by measuring in the region of interest, which was set 10 × pixels as shown in [Figure 1]. Five independent simulations were performed under one condition to obtain the edge images for the analysis.
|Figure 1: Region of interest settings in an edge image for the calculation of the contrast-to-noise ratio. The gray part of the image shows the area with higher incident photons, and the black part shows lower. The contrast-to-noise ratio of this image is 19.8, and this edge image displayed at Window level is the mean pixel value and Window width is 256|
Click here to view
The length of the ESF should be as long as possible to obtain an accurate measurement of the low-frequency response of the MTF. However, if it is too long, the effects of noise included in the ESF become significant., The length of the ESF was set to 20.48 mm for the standard in the analysis. In addition to this standard length, 2.56, 5.12, 10.24, 40.96, and 81.92 mm were also examined to verify the effect of the ESF length. Because the edge image does not contain the glare or scattered radiation that affects the MTF value in the low spatial frequency region, effects such as low-frequency drop, estimated using the long ESF, can be ignored.
The uncertainty of the measured modulation transfer function
The uncertainties of the MTF values were evaluated according to international standards. The standard deviations of the MTF values for each spatial frequency were calculated based on the results of the independent MTF measurements obtained from the five edge data. Because an aperture size with a pixel fill factor of 1.0 is the only factor affecting the MTF, the true value of the MTF can be described by the sinc function, which depends on the aperture size. Therefore, the calculated MTFs were compared with the sinc function, and the relative error with respect to the true value was obtained.
The CNR for calculating the accurate MTF was estimated using the uncertainty described above. Under the conditions of each CNR, the average value of the uncertainty of the MTF value up to the Nyquist frequency and the sampling frequency was calculated and the result was verified.
In task-based evaluations, the image quality or the signal strength may not be suitable for an evaluation, in terms of low-contrast and high-noise. As the CNR was considered to be an easily understandable indicator of this image quality situation, we have applied the CNR as a parameter in this verification.
| Results|| |
[Figure 2] shows the comparisons between the true value and the average MTF obtained by five measurements under each condition of various CNR values. [Figure 3] illustrates the relative error with respect to the true value of the average MTF and the degree of error that would occur in a single measurement. The uncertainty of the MTF value at each frequency decreased as the CNR increases. [Figure 4] shows the average value of uncertainty with respect to the MTF value as a function of the CNR; the values up to the Nyquist frequency and the sampling frequency are shown. This graph shows that the uncertainty is inversely proportional to the CNR, independent of the frequency range in which the value is evaluated. In the frequency range up to the Nyquist frequency, the uncertainty in five measurements was <0.01 when the CNR is more than 60.
|Figure 2: Modulation transfer function calculation results at different contrast-to-noise ratio conditions. The blue line shows the mean value of calculated modulation transfer function and error bars correspond to 1 standard deviation. The red line is the theoretical modulation transfer function (i.e. sinc function). edge spread function length for the calculations were 20.48 mm|
Click here to view
|Figure 3: Relative error rate of the average modulation transfer function value to the theoretical true value in the different contrast-to-noise ratio conditions. Error bars indicate the degree of error that could occur in a single measurement. Edge spread function length for the calculations were 20.48 mm|
Click here to view
|Figure 4: The average value of uncertainty shown in Figure 1, as a function of the contrast-to-noise ratio. The circle mark shows the average value from zero to the Nyquist frequency (0–2.5 cycles/mm) and the rectangle mark is the average value up to the sampling frequency (0–5 cycles/mm), respectively|
Click here to view
[Figure 5] shows the comparisons of the true value and the average MTF obtained by five measurements at each ESF length under the condition that the CNR is approximately 30. [Figure 6] illustrates the relative error with respect to the true value of the average MTF and the degree of error that would occur in a single measurement. It was observed that the uncertainty of the MTF value at each frequency increased as the ESF length increased. [Figure 7] shows the average value of uncertainty for the MTF value of each frequency as a function of the ESF analysis length. This relationship depended on the frequency range in which the value was evaluated, but the uncertainty was proportional to the 0.3–0.5 power of the ESF length.
|Figure 5: The comparisons of the true value and the calculated modulation transfer function in different edge spread function length conditions. The blue line shows calculated modulation transfer function and error bars correspond to 1 standard deviation. The red line is the theoretical modulation transfer function (i.e. sinc function). The contrast-to-noise ratio of the image for the modulation transfer function measurement was approximately 30|
Click here to view
|Figure 6: Relative error rate of the average modulation transfer function value to the theoretical true value in the different edge spread function length conditions. Error bars indicate the degree of error that could occur in a single calculation. The contrast-to-noise ratio of the image for the modulation transfer function measurement was approximately 30|
Click here to view
|Figure 7: The average value of uncertainty shown in Figure 4, as a function of the edge spread function length. The circle mark shows the average value from zero to the Nyquist frequency (0–2.5 cycles/mm) and the rectangle mark is the average value up to the sampling frequency (0–5 cycles/mm), respectively|
Click here to view
| Discussion|| |
According to the theoretical verification of Cunningham et al., the SNR of the MTF obtained by the edge method was proportional to the square root of the number of photons and inversely proportional to the square root of the ESF length. In addition, Carton et al. reported that the longer the ESF, the larger the MTF error due to the effects of noise. Antonio showed that the variation in the obtained MTF was affected by noise and increased as the spatial frequency increased. The uncertainty of the MTF value at each spatial frequency shown by this study decreased in proportion to the increase in the CNR and increases as the length of the ESF increases. This tendency is in good agreement with previous studies.
For an ESF of 20.48 mm, we found that the CNR must be set to approximately 60 or more to suppress the uncertainty to 0.01 or less. However, the uncertainty of the MTF values is greatly influenced by the analysis range of the ESF as shown in [Figure 5]. Therefore, it should be noted that this is not a recommended CNR value generalized for all measurements. Because the edge image created by this simulation does not include a stochastic blurring component, it is as sharp as a direct-type flat panel detector (FPD) or a photon counting system, so the high spatial frequency noise strongly influences the calculated MTF. Therefore, in the actual measurement in the system with various unsharpness components, because high spatial frequency noise is blurred and suppressed, it is estimated that the uncertainty of the MTF will be smaller if the CNR is set to the same level as in this study. These results show the result in the condition where the uncertainty in MTF measurements is most likely to be large, and it provides an important index from the viewpoint on the suppression of the uncertainty of the MTF value.
MTF measurement using edges is an essential and versatile technique that can be applied to task-based evaluation because it is easy to reproduce various imaging conditions such as shape, contrast, and the influence of scattered radiation. However, it has the disadvantage of being sensitive to noise. It is necessary to reduce the noise as much as possible to address this problem. When calculating the MTF, there is a possibility to mask the original image quality characteristics by adding an additional process for noise reduction. Therefore, it is necessary to apply complex works such as ESF averaging, image summation, and MTF averaging to accurately evaluate the MTF characteristics depending on the imaging task., The accuracy of the MTF can easily decrease under a low contrast and noise-rich condition (i.e. low CNR image) meaning it is important to fully understand the reliability of the MTF results measured. In addition, during the evaluation of various image quality factors in general X-ray imaging, it is necessary to measure the effects of geometrical unsharpness or scattered radiation to reproduce complicated clinical conditions. To properly estimate the effect of the spatial spread of scattered radiation, the ESF must be set as long as possible to improve the accuracy in the low spatial frequency region. Furthermore, the measurement condition will be such that the accuracy of the MTF tends to deteriorate because the contrast is degraded by the scattered radiation. Although the effect of scattered radiation is not verified in this study, it may be possible to roughly estimate the uncertainty contained in the MTF obtained under various conditions by referring to the results of this study.
This verification was performed using a Monte Carlo simulation, and image quality factors included in the actual measurement were not taken into consideration to clarify the factors of uncertainty and the interpretation of the results. When there is a lot of electrical noise not filtered by the system MTF, such as a direct-type FPD, it is expected that additional noise affects the MTF as high spatial frequency noise. As shown in the results, it can be a factor to overestimate the MTF in the high spatial frequency region. Under conditions that contain some structural noise, the effect of the analysis length will be significant. Setting a long ESF will improve the accuracy of the low spatial frequency region but will increase the sensitivity on the effect of various noises, affecting the uncertainty of the MTF. After fully recognizing the factors hidden in the analysis target, the conditions for the MTF analysis must be set. It is necessary to determine whether the calculated MTF properly represents the characteristics of the image.
Although the evaluation including the effect of scattered radiation was not performed in this study, it is an important research target in the comprehensive evaluation, so the verification based on the results shown in this study is planned to proceed. In addition, it would be of particular interest to perform task-based evaluations under various imaging conditions such as low-CNR using multiple actual image systems having different physical characteristics to verify the occurrence of uncertainties in detail.
| Conclusions|| |
In this study, we verified the effect of noise in MTF calculations using the edge method and the uncertainty of the MTF value at each spatial frequency using a simulation image. The results showed that uncertainty of MTF decreased as the CNR increased, and increased proportionally with an increase in the 0.3–0.5 power of the ESF length. With an ESF of 20.48 mm, the uncertainty of five calculations was lower than 0.01, when the CNR was 60 or more in the spatial frequency range up to the Nyquist frequency. Because a stochastic blurring was not considered in the simulation, the results were strongly influenced by high spatial frequency noise. The results in which the uncertainty is most likely to be large in the MTF measurement are clearly shown in this study. Therefore, it is expected to provide an important barometer and useful insights for a proper image quality measurement.
The verification based on the results of this simulation study will be continued using multiple actual image systems having different physical characteristics and also performed the evaluation associated with the effect of scattered radiation.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Vañó E, Miller DL, Martin CJ, Rehani MM, Kang K, Rosenstein M, et al
. ICRP publication 135: Diagnostic reference levels in medical imaging. Ann ICRP 2017;46:1-144.
Samei E, Dobbins JT 3rd
, Lo JY, Tornai MP. A framework for optimising the radiographic technique in digital X-ray imaging. Radiat Prot Dosim 2005;114:220-9.
Beutel HK, Kundel HL, Metter RL, editors. Handbook of Medical Imaging. Vol. 1. Bellingham, WA: SPIE; 2000.
Dainty JC, Shaw R. Image Science. New York: Academic Press; 1974.
Giger ML, Doi K. Investigation of basic imaging properties in digital radiography. 1. Modulation transfer function. Med Phys 1984;11:287-95.
IEC Publication. Medical Electrical Equipment – Characteristics of Digital X-ray Imaging Devices – Part 1-1: Determination of the Detective Quantum Efficiency – Detectors Used in Radiographic Imaging, IEC 62220-1. Geneva, Switzerland: IEC Publication; 2015.
ICRU. Modulation Transfer Function of Screen-Film Systems, ICRU Report 41. Bethesda, USA: ICRU; 1986.
Cunningham IA, Reid BK. Signal and noise in modulation transfer function determinations using the slit, wire, and edge techniques. Med Phys 1992;19:1037-44.
Fujita H, Tsai DY, Itoh T, Doi K, Morishita J, Ueda K, et al
. A simple method for determining the modulation transfer function in digital radiography. IEEE Trans Med Imaging 1992;11:34-9.
Samei E, Flynn MJ, Reimann DA. A method for measuring the presampled MTF of digital radiographic systems using an edge test device. Med Phys 1998;25:102-13.
Buhr E, Günther-Kohfahl S, Neitzel U. Accuracy of a simple method for deriving the presampled modulation transfer function of a digital radiographic system from an edge image. Med Phys 2003;30:2323-31.
Samei E, Ranger NT, Dobbins JT 3rd
, Chen Y. Intercomparison of methods for image quality characterization. 1. Modulation transfer function. Med Phys 2006;33:1454-65.
González-López A. Effect of noise on MTF calculations using different phantoms. Med Phys 2018;45:1889-98.
Boone JM, Seibert JA. An analytical edge spread function model for computer ﬁtting and subsequent calculation of the LSF and MTF. Med Phys 1994;21:1541-5.
Carton AK, Vandenbroucke D, Struye L, Maidment AD, Kao YH, Albert M, et al
. Validation of MTF measurement for digital mammography quality control. Med Phys 2005;32:1684-95.
Friedman SN, Cunningham IA. Normalization of the modulation transfer function: the open-field approach. Med Phys 2008;35:4443-9.
Higashide R, Ichikawa K, Kunitomo H, Ohashi K. Application of a variable filter for presampled modulation transfer function analysis with the edge method. Radiol Phys Technol 2015;8:320-30.
Kuhls-Gilcrist A, Jain A, Bednarek DR, Hoffmann KR, Rudin S. Accurate MTF measurement in digital radiography using noise response. Med Phys 2010;37:724-35.
González-López A, Campos-Morcillo PA, Lago-Martín JD. Technical Note: An oversampling procedure to calculate the MTF of an imaging system from a bar-pattern image. Med Phys 2016;43:5653.
González-López A, Campos-Morcillo PA, Vera-Sánchez JA, Ruiz-Morales C. Performance of a new star-bar phantom designed for MTF calculations in x-ray imaging systems. Med Phys 2020;47:4949-55.
Kawamura T, Naito S, Okano K, Yamada M. Improvement in image quality and workflow of x-ray examinations using a new image processing method, “Virtual Grid Technology”. Fujifilm Res Dev 2015;60:21-7.
Ito R, Takagi T, Yoshida K, Ishizaka A. Development of the intelligent grid scattered X-ray correction processing system for digital radiography. Konicaminolta Technol Rep 2016;13:52-6.
Richard S, Husarik DB, Yadava G, Murphy SN, Samei E. Toward task-based assessment of CT performance: System and object MTF across different reconstruction algorithms. Med Phys 2012;39:4115-22.
Urikura A, Ichikawa K, Hara T, Nishimaru E, Nakaya Y. Spatial resolution measurement for iterative reconstruction by use of image-averaging techniques in computed tomography. Radiol Phys Technol 2014;7:358-66.
Takata T, Ichikawa K, Mitsui W, Hayashi H, Minehiro K, Sakuta K, et al
. Object shape dependency of in-plane resolution for iterative reconstruction of computed tomography. Phys Med 2017;33:146-51.
Kyprianou IS, Rudin S, Bednarek DR, Hoffmann KR. Generalizing the MTF and DQE to include x-ray scatter and focal spot unsharpness: Application to a new microangiographic system. Med Phys 2005;32:613-26.
Samei E, Ranger NT, Mackenzie A, Honey ID, Dobbins JT 3rd
, Ravin CE. Effective DQE (eDQE) and speed of digital radiographic systems: An experimental methodology. Med Phys 2009;36:3806-17.
Hirayama H, Namito Y, Bielajew AF, Wilderman SJ, Nelson WR. The EGS5 Code System. SLAC Report Number: SLACR-730. Stanford: SLAC; 2007.
Maruyama S. Visualization of blurring process due to analog components in a digital radiography system using a simple method. Phys Eng Sci Med 2020;43:1461-8.
Maruyama S, Shimosegawa M. The effects of scattered radiation from a semitransparent edge on MTF measurement: Verification of several factors by Monte Carlo simulation. Phys Eng Sci Med 2020;43:547-56.
Joint Committee for Guides in Metrology (JCGM). Evaluation of measurement data-Guide to the expression of uncertainty in measurement. JCGM 100:2008, ISO, Geneva, Switzerland; 2008.
Boone JM, Seibert JA. Monte Carlo simulation of the scattered radiation distribution in diagnostic radiology. Med Phys 1988;15:713-20.
Rivetti S, Lanconelli N, Campanini R, Bertolini M, Borasi G, Nitrosi A, et al
. Comparison of different commercial FFDM units by means of physical characterization and contrast-detail analysis. Med Phys 2006;33:4198-209.
Moninn P. A Comprehensive model for x-ray projection imaging system efficiency and image quality characterization in the presence of scattered radiation. Phys Med Biol 2017;62:5691-722.
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6], [Figure 7]