Name: medical-grade sterilizable target for fluid-immersed fetoscope optical distortion calibration
Uploaded: 2017-02-23T13:00:00.000+00:00
Duration: 7 min 3 s
Description: Watch this Scientific Journal Video about Medical-grade Sterilizable Target for Fluid-immersed Fetoscope Optical Distortion Calibration at JoVE.com

This work was supported through an Innovative Engineering for Health award by the Wellcome Trust [WT101957], the Engineering and Physical Sciences Research Council (EPSRC) [NS/A000027/1], and a National Institute for Health Research Biomedical Research Centre UCLH/UCL High Impact Initiative. Jan Deprest is being funded by the Fonds voor Wetenschappelijk Onderzoek Vlaanderen (FWO; JD as clinical researcher 1.8.012.07). Danail Stoyanov receives funding from the EPSRC (EP/N013220/1, EP/N022750/1), the EU-FP7 project CASCADE (FP7-ICT-2913-601021), and the EU-Horizon2020 project EndoVESPA (H2020-ICT- 2015-688592). Sebastien Ourselin receives funding from the EPSRC (EP/H046410/1, EP/J020990/1, EP/K005278) and the MRC (MR/J01107X/1). Marcel Tella is supported by the EPSRC-funded UCL Centre for Doctoral Training in Medical Imaging (EP/L016478/1).

1. Target Fabrication
<ol>
	<li>Sandblasting
	<ol>
		<li>Prepare a 316 stainless steel sheet with a 1.2-mm thickness. Using a pencil or a nail, draw a 40 mm x 40 mm square onto the sheet with the aid of a ruler.</li>
		<li>Cut the drawn square using a manual metal cutter. CAUTION! Watch the fingers.</li>
		<li>Use a file to round the corners and sides of the sample. CAUTION! They are very sharp; be careful.</li>
		<li>Prepare a straight wooden or metal block slightly larger in size than the stainless steel sheet. Place the cut sheet on it; do this in order to avoid bending the sample during sandblasting.</li>
		<li>Place the assembly in the internal blast chamber. Remember to use a dust collector and to tightly seal the internal blast chamber; otherwise, the sand will spread all over during the process. Wear safety goggles to protect the eyes.</li>
		<li>Position a blast gun perpendicular to and at least 4-5 cm away from the metal surface. Apply the foot control for sandblasting. Put the sample on the piece of wood (1-2 cm thick) using a vice, as the high-pressure sand flow can deform the sample. During sand-blasting, hold the sample tightly by the edge of the piece of wood or by using another vice.</li>
		<li>Repeat the sandblasting on the other side if it is desirable to have a calibration pattern engraved on both sides.</li>
	</ol>
	</li>
	<li>Laser patterning
	<ol>
		<li>Design a pattern of asymmetric circles, as shown in Figure 1.</li>
		<li>Prepare a drawing exchange format (DXF) file of the design either using CAD software or another suitable programming language. 
		NOTE: For convenience, a Python application that can generate DXF files for the design mentioned in this paper is provided as part of the compact GUI application16.</li>
		<li>Import the DXF files into the laser cutting software.</li>
		<li>Set up the following parameters for background etching. Laser Power: 40%, Scan Speed: 80 cm/s, Frequency: 4,000 Hz, Number of Passes: 1.</li>
		<li>Set up the following parameters for etching the pattern. Laser Power: 40%, Scan Speed: 2.1 cm/s, Frequency: 4,000 Hz, Number of Passes: 1.</li>
		<li>Put the sample on the working platform and align the cutting pattern using the software.</li>
		<li>After the laser performs the cut, clean the sample by dipping it in alcohol. Do not use any wipes, as they usually leave undesirable residue.</li>
	</ol>
	</li>
	<li>Sterilization
	<ol>
		<li>Wrap the sterilized sample in a sterilization package and insert it in the sterilization unit (autoclave).</li>
		<li>Add water (not distilled water) to the autoclave and follow the user&#39;s guide/manufacturer&#39;s recommendations to sterilize the target.</li>
	</ol>
	</li>
</ol>
2. Peri-operative Calibration
<ol>
	<li>Calibration software
	<ol>
		<li>Install the &#34;endocal&#34; endoscope calibration software package provided on GitHub16 (follow the instructions in the README file therein). 
		NOTE: This software wraps the OpenCV camera calibration module17 in an easy-to-use convenience application. The provided application runs in two modes: online and offline. The online mode acquires the video stream directly from compatible frame-grabber hardware. The offline mode allows for loading endoscope images either from a video file or a folder with a number of video frames saved as image files. See README for supported hardware and detailed instructions on how to use these two modes.</li>
	</ol>
	</li>
	<li>Endoscopic video acquisition 
	NOTE: The following instructions are for online calibration (as described above), but they are also applicable to offline calibration.
	<ol>
		<li>Place the calibration target in a sterile fluid container, such as a gallipot.</li>
		<li>Fill the container with the target fluid or a similar sterile substance. 
		NOTE: For instance, in fetoscopic procedures, the target fluid is amniotic fluid. Since the optical properties of amniotic fluid are similar to saline water18,19, sterile saline water can be used for calibrating the fetoscope.</li>
		<li>Adjust the zoom and sharpness of the endoscope as desired.</li>
		<li>Immerse the endoscope in the fluid and hold it at a distance from the calibration target similar to the distance from the anatomy that the endoscope will later be used at.</li>
		<li>Launch the calibration application and start the camera acquisition.</li>
		<li>Move the tip of the endoscope slightly for different views while keeping the whole calibration pattern in view of the camera. For optimal performance, keep the elliptical legend around the calibration pattern within the circular view of the endoscope. 
		NOTE: Video frames that are usable for calibration are indicated by a virtual pattern overlay, as seen in Figure 3.</li>
		<li>Acquire at least the minimum number of endoscopic camera views required for calibration (as indicated in the endocal window). 
		NOTE: The current version of endocal requires at least 10 endoscopic camera views for calibration, a heuristically selected number of views where the calibration error appears to be minimal and follow a stable pattern20.</li>
		<li>Press the calibration key, as indicated on the endocal window, to start the calibration process using the images acquired so far.</li>
	</ol>
	</li>
	<li>Saving and using the calibration parameters
	<ol>
		<li>Press the indicated calibration key to save the resulting calibration parameters in a YAML (&#34;YAML Ain&#39;t Markup Language&#34;) file21.</li>
		<li>Group the calibration parameters into the camera matrix and distortion coefficients, as explained in the OpenCV camera calibration module17. 
		NOTE: After performing the calibration, the calibration application automatically displays the distortion-corrected image to the right of the original endoscope image.</li>
		<li>Use the distortion-corrected video feed during a fetoscopic procedure for pure visualization or for real-time placental mosaicing11.</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Zhang, Z., Matsushita, Y., Ma, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Camera+calibration+with+lens+distortion+from+low-rank+textures.">Camera calibration with lens distortion from low-rank textures.</a> , 2321-2328 (2011).</li><li>Devernay, F., Faugeras, O. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automatic+calibration+and+removal+of+distortion+from+scenes+of+structured+environments.">Automatic calibration and removal of distortion from scenes of structured environments.</a> , 62-72 (1995).</li><li>Duane, C. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Close-range+camera+calibration.">Close-range camera calibration.</a> Photogramm. Eng. 37 (8), 855-866 (1971).</li><li>Mallon, J., Whelan, P. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Which+pattern?+biasing+aspects+of+planar+calibration+patterns+and+detection+methods.">Which pattern? biasing aspects of planar calibration patterns and detection methods.</a> Pattern recognition letters. 28 (8), 921-930 (2007).</li><li>Balletti, C., Guerra, F., Tsioukas, V., Vernier, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Calibration+of+Action+Cameras+for+Photogrammetric+Purposes.">Calibration of Action Cameras for Photogrammetric Purposes.</a> Sensors. 14 (9), 17471-17490 (2014).</li><li>Heikkila, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Geometric+camera+calibration+using+circular+control+points.">Geometric camera calibration using circular control points.</a> IEEE Transactions on pattern analysis and machine intelligence. 22 (10), 1066-1077 (2000).</li><li>Deprest, J. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fetal+surgery+is+a+clinical+reality.">Fetal surgery is a clinical reality.</a> Seminars in fetal and neonatal medicine. 15 (1), 58-67 (2009).</li><li>Watanabe, M., Flake, A. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fetal+surgery:+Progress+and+perspectives.">Fetal surgery: Progress and perspectives.</a> Advances in pediatrics. 57 (1), 353-372 (2010).</li><li>Lewi, L., Deprest, J., Hecher, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+vascular+anastomoses+in+monochorionic+twin+pregnancies+and+their+clinical+consequences.">The vascular anastomoses in monochorionic twin pregnancies and their clinical consequences.</a> American journal of obstetrics and gynecology. 208 (1), 19-30 (2013).</li><li>Yamashita, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Miniature+bending+manipulator+for+fetoscopic+intrauterine+laser+therapy+to+treat+twin-to-twin+transfusion+syndrome.">Miniature bending manipulator for fetoscopic intrauterine laser therapy to treat twin-to-twin transfusion syndrome.</a> Surgical Endoscopy. 22 (2), 430-435 (2008).</li><li>Daga, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-time+mosaicing+of+fetoscopic+videos+using+SIFT.">Real-time mosaicing of fetoscopic videos using SIFT.</a> Proc. SPIE 9786, Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling. 97861R. , (2016).</li><li>Yang, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Image+mapping+of+untracked+free-hand+endoscopic+views+to+an+ultrasound+image-constructed+3D+placenta+model.">Image mapping of untracked free-hand endoscopic views to an ultrasound image-constructed 3D placenta model.</a> The International Journal of Medical Robotics and Computer Assisted Surgery. 11 (2), 223-234 (2015).</li><li>Liao, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fast+image+mapping+of+endoscopic+image+mosaics+with+three-dimensional+ultrasound+image+for+intrauterine+fetal+surgery.">Fast image mapping of endoscopic image mosaics with three-dimensional ultrasound image for intrauterine fetal surgery.</a> Minimally invasive therapy &amp; allied technologies. 18 (6), 332-340 (2009).</li><li>Chadebecq, F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Practical+Dry+Calibration+With+Medium+Adaptation+For+Fluid-Immersed+Endoscopy.">Practical Dry Calibration With Medium Adaptation For Fluid-Immersed Endoscopy.</a> , (2015).</li><li>Wengert, C., Reeff, M., Cattin, P. C., Sz&#233;kely, G., Handels, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Bildverarbeitung f&#252;r die Medizin 2006: Algorithmen Systeme Anwendungen. Proceedings des Workshops vom 19. - 21. M&#228;rz 2006 in Hamburg. , 419-423 (2006).</li><li>. Compact GUI application for optical distortion calibration of endoscopes Available from: <a target="_blank" href="https://github.com/gift-surg/endocal">https://github.com/gift-surg/endocal</a> (2016)</li><li>Reeff, M., Gerhard, F., Cattin, P. C., Sz&#233;kely, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Mosaicing of endoscopic placenta images. , (2011).</li><li>Steigman, S. A., Kunisaki, S. M., Wilkins-Haug, L., Takoudes, T. C., Fauza, D. O. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Optical+properties+of+human+amniotic+fluid:+implications+for+videofetoscopic+surgery.">Optical properties of human amniotic fluid: implications for videofetoscopic surgery.</a> Fetal diagnosis and therapy. 27 (2), 87-90 (2009).</li><li>Zhang, Z. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+flexible+new+technique+for+camera+calibration.">A flexible new technique for camera calibration.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 22 (11), 1330-1334 (2000).</li><li>. Open Source Computer Vision Library Available from: <a target="_blank" href="https://github.com/opencv/opencv">https://github.com/opencv/opencv</a> (2016)</li></ol>

The authors have nothing to disclose.

Sandblasting is an important step in the fabrication process because the raw metal surface prominently reflects the endoscope light, making it impossible for the circles to be detected. It is difficult to distinguish the circles even with the naked eye (see Figure 5). Note that the surface of the target shown was already etched with a laser. However, this does not diminish light reflection.
<img alt="Figure 5" src="/files/ftp_upload/55298/55298fig5.jpg" /> 
Figure 5: Calibration target with no sandblasting applied. As seen from the endoscope view on the left, the glare from the endoscope light on the material surface makes it difficult even for the naked eye to distinguish the circles (there is a circle just to the southeast of the large reflection). Note that the surface of this target (i.e., the &#34;background&#34;) was already etched, but this is not helpful in the absence of sandblasting. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/55298fig5large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Prior to pattern etching, it is also important to etch the surface of the whole sample. This is necessary because the sandblasted surface has many specular reflections (see Figure 6), which interfere with blob detection.
<img alt="Figure 6" src="/files/ftp_upload/55298/55298fig6.jpg" /> 
Figure 6: Sandblasted surface with no etching. Although not as prominent as the raw metal surface, the relatively small specular reflections (some of which are highlighted with yellow arrows) are still sufficient to prevent blob detection from succeeding, so no calibration can be performed with this target. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/55298fig6large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Applying the laser at different speeds gives different background colors. The background color plays a significant role in the contrast between the circles and the background. Hence, it is vital to determine the optimal background color. For this purpose, a plate with circles etched against a set of different backgrounds was created (see Figure 7). The backgrounds were tested using the feature detection module of OpenCV23, which is used in the OpenCV camera calibration module17. In this work, the target was made of stainless steel, as it is the most common and reliable material used in clinics for medical devices. This material is freely available, not expensive, robust, and easy to sterilize. Other materials could potentially be used for the calibration target, such as aluminum or iodized metals, but this is the scope of future work.
<img alt="Figure 7" src="/files/ftp_upload/55298/55298fig7.jpg" /> 
Figure 7: Stainless steel plate featuring a palette of different background colors etched with the laser. Practical experiments were conducted in conjunction with the OpenCV feature detection module to determine which background color gives the optimal result in terms of blob-to-background contrast23. The endoscope view on the left shows the plate. The moderate background colors (i.e., those other that the darkest and lightest ones) in this palette yield better blob detection. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/55298fig7large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
One of the advantages of this work is that performing a calibration using the fabricated target takes 2-3 min. Most of the effort goes to manually stabilizing the endoscope to obtain decent views of the calibration pattern. Using a custom-built endoscope holder could eliminate the need for manual stabilization, which in turn could significantly shorten calibration time.
<img alt="Video 1" src="/files/ftp_upload/55298/55298video1.jpg" /> 
Video 1: Video showing how optical distortion calibration can be performed using the developed calibration target together with the endocal software. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/fetoscope-calibration-480.mov" target="_blank">Please click here to view this video. (Right-click to download.)</a>
An advantage of our work compared to the work of Wengert et al.15 is that the OpenCV camera calibration module17 can be used as is for calibration, without requiring any modification or custom parameterization. Because OpenCV is a well-established and well-maintained software package and is very popular in the computer vision community, using it eliminates the need for writing and maintaining custom software. For the reader&#39;s convenience, a compact GUI application is provided16, which the reader can easily install and use to test new calibration targets. One disadvantage of our method compared to Wengert et al.15 is that their method is more robust to occlusions of the pattern, as it does not require the detection of all blobs.
Initially, a calibration target with a checkerboard pattern was fabricated for this work. However, this type of calibration target proved to be unsuitable in experiments due to the difficulty of detecting the corners of the checkerboard squares. Corner detection relies on histogram-based image binarization (see the OpenCV source code24). This implies the need for a clear color contrast between the dark and light squares, which could not be guaranteed with our checkerboard pattern, partially due to specular reflections, like the ones shown in Figure 6. Such specular reflections are present even after background etching; however, the detection of the circles seems to be less sensitive to this shortcoming.
In the current setup, only perpendicular views of the calibration target allow for successful blob detection. This is due to the specular reflections from the target surface hampering blob detection at oblique angles. We are working to further improve the target so as to allow for the acquisition of views at a wider range of angles, which could potentially improve the quality of performed calibrations20.
In the real-time placental mosaicing pipeline that was previously proposed11, the computation of the transformation that maps image pairs relies on the successful detection and grouping of features. Optical distortions, on the other hand, cause a group of features with a rigid geometry to appear different across images. As a consequence, this difference leads to inaccuracies in the computed transformations, which cause drifts in the resulting image mosaics. Because the most prominent optical distortions are present towards the edges, endoscopic images are currently cropped to their innermost regions. A good correction for optical distortions would potentially allow for the incorporation of a larger part of each image into the mosaicing process. The advantage of this method is two-fold. First, it would increase the number of detected features in each image, potentially improving the computation of the image transformations. Second, it would allow for the whole target anatomical surface to be reconstructed in a shorter time.

Camera calibration is a well-known problem in the computer vision field that has been intensively studied over the years1,2,3. A key step of camera calibration procedures is to estimate the parameters of a distortion model, as well as the intrinsic camera parameters, by extracting a grid of points with a known geometry from camera images with sub-pixel accuracy. Calibration targets with a checkerboard pattern featuring black and white squares are commonly used for this purpose. Circular blobs offer an alternative pattern4,5,6.
In recent years, there has been a growing interest in the development of surgical navigation technology for fetal surgery procedures, such as the treatment of twin-to-twin transfusion syndrome (TTTS) on fetuses7,8,9,10. As the field of view of the fetoscope (i.e., an endoscope used in fetal surgical procedures) is very limited, methods for mapping the placental vasculature without the use of external trackers have been proposed to aid TTTS surgery11,12,13. Optical distortions within fetoscopic images have adverse effects on these computational mosaicing methods that rely on visual information extraction11. Thus, there is an unmet need for a cost-efficient and fast tool for peri-operatively calibrating fetoscopes so that optical distortion compensation can be done in real-time during the intervention.
Due to the fact that the fetoscope is immersed in amniotic fluid during the intervention, the refraction index difference between air and amniotic fluid renders classical in-air camera calibration methods unsuitable for fetal surgery procedures. Estimating fluid-immersed camera parameters from in-air camera parameters is a difficult task and requires at least one image of the fluid-immersed calibration target14. Furthermore, peri-operative, fluid-immersed fetoscopic camera calibration is currently impractical due to sterilization requirements and restrictions on the materials allowed in the operating theater. Due to these reasons, calibrating endoscopes for optical distortions is typically not part of the current clinical workflow. The work in this manuscript is an attempt to close this camera calibration gap by designing and producing a sterilizable and practical optical distortion calibration target featuring a pattern of asymmetric circles. Previously, Wengert et al. fabricated a custom calibration device featuring an oxidized aluminum plate as the calibration target. Their method, however, works only in conjunction with the custom calibration algorithm they developed15.

<table><tbody><tr><td>1.2 mm Metal sheet 316 Grade, 40 mm by 40 mm</td><td></td><td></td><td></td></tr><tr><td>Water container at least 50 mm by 50 mm by 30 mm</td><td></td><td></td><td></td></tr><tr><td>A sterilization package</td><td></td><td></td><td></td></tr><tr><td>Saline water</td><td></td><td></td><td></td></tr><tr><td>Manual metal cutter</td><td></td><td></td><td></td></tr><tr><td>A file to round up the corners</td><td></td><td></td><td></td></tr><tr><td>A wooden or metal block 50 mm by 50 mm at least 10 mm thick</td><td></td><td></td><td></td></tr><tr><td>A vice (desirable but not required)</td><td></td><td></td><td></td></tr><tr><td>Sand Blasting machine</td><td></td><td></td><td></td></tr><tr><td>GUI application to create .dxf file with the pattern</td><td></td><td></td><td>https://github.com/gift-surg/endocal</td></tr><tr><td>PC</td><td></td><td></td><td></td></tr><tr><td>Laser Cutter</td><td></td><td></td><td></td></tr><tr><td>Autoclave</td><td></td><td></td><td></td></tr><tr><td>An endoscope calibration software</td><td></td><td></td><td>from GitHub: https://github.com/gift-surg/endocal</td></tr><tr><td>Endoscope</td><td></td><td></td><td></td></tr><tr><td>OpenCV camera calibration module</td><td></td><td></td><td>http://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html</td></tr><tr><td>Safety goggles</td><td></td><td></td><td></td></tr><tr><td>A lab coat</td><td></td><td></td><td></td></tr><tr><td>A ruler and a marker</td><td></td><td></td><td></td></tr><tr><td>Alcohol (preferably ethanol) for dust removal and cleaning</td><td></td><td></td><td></td></tr></tbody></table>

medical-grade sterilizable target for fluid-immersed fetoscope optical distortion calibration

We have developed a calibration target for use with fluid-immersed endoscopes within the context of the GIFT-Surg (Guided Instrumentation for Fetal Therapy and Surgery) project. One of the aims of this project is to engineer novel, real-time image processing methods for intra-operative use in the treatment of congenital birth defects, such as spina bifida and the twin-to-twin transfusion syndrome. The developed target allows for the sterility-preserving optical distortion calibration of endoscopes within a few minutes. Good optical distortion calibration and compensation are important for mitigating undesirable effects like radial distortions, which not only hamper accurate imaging using existing endoscopic technology during fetal surgery, but also make acquired images less suitable for potentially very useful image computing applications, like real-time mosaicing. In this paper proposes a novel fabrication method to create an affordable, sterilizable calibration target suitable for use in a clinical setup. This method involves etching a calibration pattern by laser cutting a sandblasted stainless steel sheet. This target was validated using the camera calibration module provided by OpenCV, a state-of-the-art software library popular in the computer vision community.

We created a sterilizable calibration target by etching a pattern of asymmetric circle on a sandblasted stainless steel metal sheet, whose design is shown in Figure 1. An exemplar setup showing this calibration target in action together with a fetoscope is shown in Figure 2. To feed this design into the laser etching software, a custom application was implemented in the Python programming language16. Creating the design pattern involves iteratively etching parallel lines on a metal sheet. For the pattern to have a consistent color in the end, the distance between these lines should be less than the width of the laser beam (see the inset of Figure 1)&#8212;this value is 45 &#181;m for the Violino (Laservall) laser cutter.
<img alt="Figure 1" src="/files/ftp_upload/55298/55298fig1.jpg" /> 
Figure 1: Design of the engraved pattern featuring a 3-by-11 grid of asymmetric circles. Inset: zoomed-in view of the grid of asymmetric circles. The distance between the lines is 45 &#181;m (equal to the laser beam width), and each circle has a diameter of 1 mm. Other sizes could be used for the grid as well, but this was found to be optimal with respect to the fetoscope field of view. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/55298fig1large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" src="/files/ftp_upload/55298/55298fig2.jpg" /> 
Figure 2: Exemplar setup with the calibration target in use. The tip of the water-immersed fetoscope is directed at the calibration target on the right. On the left is a British penny to provide scale information. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/55298fig2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
The fabricated calibration target allows for the detection of the circular pattern in the endoscopic video stream with OpenCV17, whose locations are then sorted into the pre-defined asymmetric circular grid (see Figure 3). Using this information in conjunction with the already-known grid geometry, internal camera parameters can be estimated. These include the camera matrix and the distortion coefficients. The camera matrix consists of the focal lengths and the optical centers along the x- and y-axis of the 2D image plane. The distortion coefficients are based on the Brown-Conrady model3. Note that for this work, only the radial distortion parameters were estimated. For a brief discussion of the theory, with practical examples, see the webpage of the OpenCV camera calibration module17 and the MATLAB camera calibration toolbox22. More details about the camera calibration procedure are available in Zhang&#39;s work20. The endocal software repository features a sample dataset of 10 endoscopic views of the fabricated calibration target16. Using this dataset, a calibration with an average re-projection error of 0.28 pixels (min: 0.16, max: 0.45) was obtained. This is comparable to the 0.25 pixels reported by Wengert et al. using their custom calibration algorithm15. The same research group, however, reported a re-projection error of 0.6 pixels in a more recent paper when using the method in15 for calibrating an endoscopic camera used for placental mosaicing18.
<img alt="Figure 3" src="/files/ftp_upload/55298/55298fig3.jpg" /> 
Figure 3: Real-time detection of the calibration pattern. A screenshot from the calibration application16 featuring the detected calibration pattern overlaid onto the live video stream using the virtual reality visualization from OpenCV17. Note that each detected column of the calibration pattern is emphasized by a different color. The detected circles, in conjunction with the known geometry, are used for computing the camera parameters. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/55298fig3large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
The estimated camera parameters are used for optical distortion correction. Figure 4 shows a rectangular chessboard pattern, as viewed using a fetoscope, where optical distortions make the lines appear as curves. Note that the lines appear normal in the distortion-corrected image.
<img alt="Figure 4" src="/files/ftp_upload/55298/55298fig4.jpg" /> 
Figure 4: Optical distortion correction. A screenshot from the calibration application16 featuring the live video image from a fetoscope recording from the checkerboard pattern (left) with the distortion-corrected image (right). Three exemplary lines are drawn in both images, each from one corner to another, where the trajectory is linear. Due to the optical distortions, these lines appear as curves in the original fetoscope images. <a href="https://cloudflare-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/55298/55298fig4large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>

Watch this Scientific Journal Video about Medical-grade Sterilizable Target for Fluid-immersed Fetoscope Optical Distortion Calibration at JoVE.com

Medical-grade Sterilizable Target for Fluid-immersed Fetoscope Optical Distortion Calibration

This article describes the design and development of a sterilizable custom camera optical distortion calibration target for the peri-operative, fluid-immersed calibration of endoscopes during endoscopic interventions.

We have developed a calibration target for use with fluid-immersed endoscopes within the context of the GIFT-Surg (Guided Instrumentation for Fetal ...

medical-grade-sterilizable-target-for-fluid-immersed-fetoscope

Nanyang Technological University

Research

JoVE Journal

Bioengineering

7.6K Views.  University College London. This article describes the design and development of a sterilizable custom camera optical distortion calibration target for the peri-operative, fluid-immersed calibration of endoscopes during endoscopic interventions.

Video: Medical-grade Sterilizable Target for Fluid-immersed Fetoscope Optical Distortion Calibration

Multiplexing Focused Ultrasound Stimulation with Fluorescence Microscopy

By focusing low-intensity ultrasound pulses that penetrate soft tissues, LIPUS represents a promising biomedical technology to remotely and safely manipulate neural firing, hormonal secretion and genetically-reprogrammed cells. However, the translation of this technology for medical applications is currently hampered by a lack of biophysical mechanisms by which targeted tissues sense and respond to LIPUS. A suitable approach to identify these mechanisms would be to use optical biosensors in combination with LIPUS to determine underlying signaling pathways. However, implementing LIPUS to a fluorescence microscope may introduce undesired mechanical artefacts due to the presence of physical interfaces that reflect, absorb and refract acoustic waves. This article presents a step-by-step procedure to incorporate LIPUS to commercially-available upright epi-fluorescence microscopes while minimizing the influence of physical interfaces along the acoustic path. A simple procedure is described to operate a single-element ultrasound transducer and to bring the focal zone of the transducer into the objective focal point. The use of LIPUS is illustrated with an example of LIPUS-induced calcium transients in cultured human glioblastoma cells measured using calcium imaging.

Low-Intensity Pulsed Ultrasound Stimulation (LIPUS) is a modality for non-invasive mechanical stimulation of endogenous or engineered cells with high spatial and temporal resolution. This article describes how to implement LIPUS to an epi-fluorescence microscope and how to minimize acoustic impedance mismatch along the ultrasound path to prevent unwanted mechanical artefacts.

By focusing low-intensity ultrasound pulses that penetrate soft tissues, LIPUS represents a promising biomedical technology to remotely and safely ...

Engineering

Microtensiometer for Confocal Microscopy Visualization of Dynamic Interfaces

Adsorption of surface-active molecules to fluid-fluid interfaces is ubiquitous in nature. Characterizing these interfaces requires measuring surfactant adsorption rates, evaluating equilibrium surface tensions as a function of bulk surfactant concentration, and relating how surface tension changes with changes in the interfacial area following equilibration. Simultaneous visualization of the interface using fluorescence imaging with a high-speed confocal microscope allows the direct evaluation of structure-function relationships. In the capillary pressure microtensiometer (CPM), a hemispherical air bubble is pinned at the end of the capillary in a 1 mL volume liquid reservoir. The capillary pressure across the bubble interface is controlled via a commercial microfluidic flow controller that allows for model-based pressure, bubble curvature, or bubble area control based on the Laplace equation. Compared to previous techniques such as the Langmuir trough and pendant drop, the measurement and control precision and response time are greatly enhanced; capillary pressure variations can be applied and controlled in milliseconds. The dynamic response of the bubble interface is visualized via a second optical lens as the bubble expands and contracts. The bubble contour is fit to a circular profile to determine the bubble curvature radius, R, as well as any deviations from circularity that would invalidate the results. The Laplace equation is used to determine the dynamic surface tension of the interface. Following equilibration, small pressure oscillations can be imposed by the computer-controlled microfluidic pump to oscillate the bubble radius (frequencies of 0.001-100 cycles/min) to determine the dilatational modulus The overall dimensions of the system are sufficiently small that the microtensiometer fits under the lens of a high-speed confocal microscope allowing fluorescently tagged chemical species to be quantitatively tracked with submicron lateral resolution.

This manuscript describes the design and operation of a microtensiometer/confocal microscope to do simultaneous measurements of interfacial tension and surface dilatational rheology while visualizing the interfacial morphology. This provides the real-time construction of structure-property relationships of interfaces important in technology and physiology.

Adsorption of surface-active molecules to fluid-fluid interfaces is ubiquitous in nature. Characterizing these interfaces requires measuring surfactant ...

Ferric Chloride-induced Murine Thrombosis Models

Arterial thrombosis (blood clot) is a common complication of many systemic diseases associated with chronic inflammation, including atherosclerosis, diabetes, obesity, cancer and chronic autoimmune rheumatologic disorders. Thrombi are the cause of most heart attacks, strokes and extremity loss, making thrombosis an extremely important public health problem. Since these thrombi stem from inappropriate platelet activation and subsequent coagulation, targeting these systems therapeutically has important clinical significance for developing safer treatments. Due to the complexities of the hemostatic system, in vitro experiments cannot replicate the blood-to-vessel wall interactions; therefore, in vivo studies are critical to understand pathological mechanisms of thrombus formation. To this end, various thrombosis models have been developed in mice. Among them, ferric chloride (FeCl3) induced vascular injury is a widely used model of occlusive thrombosis that reports platelet activation and aggregation in the context of an aseptic closed vascular system. This model is based on redox-induced endothelial cell injury, which is simple and sensitive to both anticoagulant and anti-platelets drugs. The time required for the development of a thrombus that occludes blood flow gives a quantitative measure of vascular injury, platelet activation and aggregation that is relevant to thrombotic diseases. We have significantly refined this FeCl3-induced vascular thrombosis model, which makes the data highly reproducible with minimal variation. Here we describe the model and present representative data from several experimental set-ups that demonstrate the utility of this model in thrombosis research.

We report a refined procedure of the ferric chloride (FeCl3)-induced thrombosis models on carotid and mesenteric artery as well as vein, characterized efficiently using intravital microscopy to monitor time to occlusive thrombi formation.

Arterial thrombosis (blood clot) is a common complication of many systemic diseases associated with chronic inflammation, including atherosclerosis, ...

Medicine

Spatio-Temporal In Vivo Imaging of Ocular Drug Delivery Systems using Fiberoptic Confocal Laser Microendoscopy

Subconjunctival injection is an attractive route to administer ocular drugs due to easy trans-scleral access that bypasses anterior ocular barriers, such as the cornea and conjunctiva. While therapeutic effects and pharmacokinetics of the drugs upon subconjunctival injection have been described in some studies, very few assess the ocular distribution of drugs or drug delivery systems (DDS). The latter is critical for the optimization of intraocular DDS design and drug bioavailability to achieve the desired ocular localization and duration of action (e.g., acute versus. prolonged). This study establishes the use of fiberoptic confocal laser microendoscopy (CLM) to qualitatively study the ocular distribution of fluorescent liposomes in real-time in live mice after sub-conjunctival injection. Being designed for in vivo visual inspection of tissues at the microscopic level, this is also the first full description of the CLM imaging method to study spatio-temporal distribution of injectables in the eye after subconjunctival injection.

Spatio-Temporal In Vivo Imaging of Ocular Drug Delivery Systems using Fiberoptic Confocal Laser Microendoscopy

We present a protocol for the use of fiberoptic confocal laser microendoscopy (CLM) to non-invasively study the spatio-temporal distribution of liposomes in the eye after subconjunctival injection.

Subconjunctival injection is an attractive route to administer ocular drugs due to easy trans-scleral access that bypasses anterior ocular barriers, such ...

Model Surgical Training: Skills Acquisition in Fetoscopic Laser Photocoagulation of Monochorionic Diamniotic Twin Placenta Using Realistic Simulators

Fetoscopic laser coagulation of arterio-venous anastomoses (AVA) in a monochorionic placenta is the standard of care for twin-twin transfusion syndrome (TTTS), but is technically challenging and can lead to significant complications. Acquiring and maintaining the necessary surgical skills require consistent practice, a critical caseload, and time. Training on realistic surgical simulators can potentially shorten this steep learning curve and enables several proceduralists to acquire procedure-specific skills simultaneously. Here we describe realistic simulators designed to allow the user familiarity with the equipment and specific steps required in the surgical treatment of TTTS, including fetoscopic handling, approaches to anterior and posterior placenta, recognition of anastomoses, and efficient coagulation of vessels. We describe the skills that are especially important in conducting placental laser coagulation that the surgeon can practice on the model and apply in a clinical case. These models can be adapted easily depending on the availability of materials and require standard fetoscopy equipment. Such training systems are complementary to traditional surgical apprenticeships and can be useful aids for fetal medicine units that provide this clinical service.

Practicing the specific skills required for fetoscopic laser coagulation of monochorionic placental anastomoses on realistic models can aid less experienced surgeons in overcoming the steep learning curve associated with this procedure that is now regarded as the standard of care for twin-twin transfusion syndrome.

Fetoscopic laser coagulation of arterio-venous anastomoses (AVA) in a monochorionic placenta is the standard of care for twin-twin transfusion syndrome ...

A Fluorescent Intravital Imaging Approach to Study Load-Induced Calcium Signaling Dynamics in Mouse Osteocytes

Bone tissue is exquisitely sensitive to differences in mechanical load magnitude. Osteocytes, dendritic cells that form a syncytium throughout the bone, are responsible for the mechanosensory function of bone tissue. Studies employing histology, mathematical modeling, cell culture, and ex vivo bone organ cultures have greatly advanced the understanding of osteocyte mechanobiology. However, the fundamental question of how osteocytes respond to and encode mechanical information at the molecular level in vivo is not well understood. Intracellular calcium concentration fluctuations in osteocytes offer a useful target for learning more about acute bone mechanotransduction mechanisms. Here, we report a method for studying osteocyte mechanobiology in vivo, combining a mouse strain with a fluorescently genetically encoded calcium indicator expressed in osteocytes with an in vivo loading and imaging system to directly detect osteocyte calcium levels during loading. This is achieved with a three-point bending device that can deliver well-defined mechanical loads to the third metatarsal of living mice while simultaneously monitoring fluorescently indicated calcium responses of osteocytes using two-photon microscopy. This technique allows for direct in vivo observation of osteocyte calcium signaling events in response to whole bone loading and is useful in the endeavor to reveal mechanisms in osteocyte mechanobiology.

The current article describes performing an intravital imaging approach to observe mechanically induced calcium signaling of embedded osteocytes in vivo in real-time in response to tissue-level mechanical loading of the mouse third metatarsal.

Bone tissue is exquisitely sensitive to differences in mechanical load magnitude. Osteocytes, dendritic cells that form a syncytium throughout the bone, ...

Adaptable Angled Stereotactic Approach for Versatile Neuroscience Techniques

Stereotactic surgery is an essential tool in the modern neuroscience lab. However, the ability to precisely and accurately target difficult-to-reach brain regions still presents a challenge, particularly when targeting brain structures along the midline. These challenges include avoiding of the superior sagittal sinus and third ventricle and the ability to consistently target selective and discrete brain nuclei. In addition, more advanced neuroscience techniques (e.g., optogenetics, fiber photometry, and two-photon imaging) rely on targeted implantation of significant hardware to the brain, and spatial limitations are a common hindrance. Presented here is a modifiable protocol for stereotactic targeting of rodent brain structures using an angled coronal approach. It can be adapted to 1) mouse or rat models, 2) various neuroscience techniques, and 3) multiple brain regions. As a representative example, it includes the calculation of stereotactic coordinates for targeting of the mouse hypothalamic ventromedial nucleus (VMN) for an optogenetic inhibition experiment. This procedure begins with the bilateral microinjection of an adeno-associated virus (AAV) encoding a light-sensitive chloride channel (SwiChR++) to a Cre-dependent mouse model, followed by the angled bilateral implantation of fiberoptic cannulae. Using this approach, findings show that activation of a subset of VMN neurons is required for intact glucose counterregulatory responses to insulin-induced hypoglycemia.

Described here is a stereotactic procedure that can target challenging and difficult-to-reach brain regions (due to spatial limitations) using an angled coronal approach. This protocol is adaptable to both mouse and rat models and can be applied to diverse neuroscientific applications, including cannula implantation and microinjections of viral constructs.

Stereotactic surgery is an essential tool in the modern neuroscience lab. However, the ability to precisely and accurately target difficult-to-reach brain ...

Neuroscience

Biomechanical Characterization of Porcine Ocular Lens with Automated Compression

Automated Compression Testing of the Ocular Lens

The biomechanical properties of the ocular lens are essential to its function as a variable power optical element. These properties change dramatically with age in the human lens, resulting in a loss of near vision called presbyopia. However, the mechanisms of these changes remain unknown. Lens compression offers a relatively simple method for assessing the lens' biomechanical stiffness in a qualitative sense and, when coupled with appropriate analytical techniques, can help quantify biomechanical properties. A variety of lens compression tests have been performed to date, including both manual and automated, but these methods inconsistently apply key aspects of biomechanical testing such as preconditioning, loading rates, and time between measurements. This paper describes a fully automated lens compression test wherein a motorized stage is synchronized with a camera to capture the force, displacement, and shape of the lens throughout a preprogrammed loading protocol. A characteristic elastic modulus may then be calculated from these data. While demonstrated here using porcine lenses, the approach is appropriate for the compression of lenses of any species.

Author Spotlight: Advancing Understanding of Age-Related Lens Stiffness Changes 

We present an automated method for characterizing the effective elastic modulus of an ocular lens using a compression test.

The biomechanical properties of the ocular lens are essential to its function as a variable power optical element. These properties change dramatically ...

In vivo Calcium Imaging in Mouse Inferior Olive

Inferior olive (IO), a nucleus in the ventral medulla, is the only source of climbing fibers that form one of the two input pathways entering the cerebellum. IO has long been proposed to be crucial for motor control and its activity is currently considered to be at the center of many hypotheses of both motor and cognitive functions of the cerebellum. While its physiology and function have been relatively well studied on single-cell level in vitro, presently there are no reports on the organization of the IO network activity in living animals. This is largely due to the extremely challenging anatomical location of the IO, making it difficult to subject to conventional fluorescent imaging methods, where an optic path must be created through the entire brain located dorsally to the region of interest.
Here we describe an alternative method for obtaining state-of-the-art -level calcium imaging data from the IO network. The method takes advantage of the extreme ventral location of the IO and involves a surgical procedure for inserting a gradient-refractive index (GRIN) lens through the neck viscera to come into contact with the ventral surface of the calcium sensor GCaMP6s-expressing IO in anesthetized mice. A representative calcium imaging recording is shown to demonstrate the feasibility to record IO neuron activity after the surgery. While this is a non-survival surgery and the recordings must be conducted under anesthesia, it avoids damage to life-critical brainstem nuclei and allows conducting large variety of experiments investigating spatiotemporal activity patterns and input integration in the IO. This procedure with modifications could be used for recordings in other, adjacent regions of the ventral brainstem.

In vivo Calcium Imaging in Mouse Inferior Olive

We present a protocol to expose the brainstem of adult mouse from the ventral side. By using a gradient-refractive index lens with a miniature microscope, calcium imaging can be used to examine the activity of inferior olive neural somata in vivo.

Inferior olive (IO), a nucleus in the ventral medulla, is the only source of climbing fibers that form one of the two input pathways entering the ...

FRET Imaging in Three-dimensional Hydrogels

Imaging of F&#246;rster resonance energy transfer (FRET)&#160;is a powerful tool for examining cell biology in real-time. Studies utilizing FRET commonly employ two-dimensional (2D) culture, which does not mimic the three-dimensional (3D) cellular microenvironment. A method to perform quenched emission FRET imaging using conventional widefield epifluorescence microscopy of cells within a 3D hydrogel environment is presented. Here an analysis method for ratiometric FRET probes that yields linear ratios over the probe activation range is described. Measurement of intracellular cyclic adenosine monophosphate (cAMP) levels is demonstrated in chondrocytes under forskolin stimulation using a probe for EPAC1 activation (ICUE1) and the ability to detect differences in cAMP signaling dependent on hydrogel material type, herein a photocrosslinking hydrogel (PC-gel, polyethylene glycol dimethacrylate) and a thermoresponsive hydrogel (TR-gel). Compared with 2D FRET methods, this method requires little additional work. Laboratories already utilizing FRET imaging in 2D can easily adopt this method to perform cellular studies in a 3D microenvironment. It can further be applied to high throughput drug screening in engineered 3D microtissues. Additionally, it is compatible with other forms of FRET imaging, such as anisotropy measurement and fluorescence lifetime imaging (FLIM), and with advanced microscopy platforms using confocal, pulsed, or modulated illumination.

F&#246;rster resonance energy transfer (FRET) imaging is a powerful tool for real-time cell biology studies. Here a method for FRET imaging cells in physiologic three-dimensional (3D) hydrogel microenvironments using conventional epifluorescence microscopy is presented. An analysis for ratiometric FRET probes that yields linear ratios over the activation range is described.

Imaging of F&#246;rster resonance energy transfer (FRET)&#160;is a powerful tool for examining cell biology in real-time. Studies utilizing FRET commonly ...

Medical-grade Sterilizable Target for Fluid-immersed Fetoscope Optical Distortion Calibration

Summary

Explore More Videos

Chapters in this video

Multiplexing Focused Ultrasound Stimulation with Fluorescence Microscopy

Microtensiometer for Confocal Microscopy Visualization of Dynamic Interfaces

Ferric Chloride-induced Murine Thrombosis Models

Spatio-Temporal In Vivo Imaging of Ocular Drug Delivery Systems using Fiberoptic Confocal Laser Microendoscopy

Model Surgical Training: Skills Acquisition in Fetoscopic Laser Photocoagulation of Monochorionic Diamniotic Twin Placenta Using Realistic Simulators

A Fluorescent Intravital Imaging Approach to Study Load-Induced Calcium Signaling Dynamics in Mouse Osteocytes

Adaptable Angled Stereotactic Approach for Versatile Neuroscience Techniques

Automated Compression Testing of the Ocular Lens

In vivo Calcium Imaging in Mouse Inferior Olive

FRET Imaging in Three-dimensional Hydrogels