Deep Learning On Medical 3D Imaging

Deep Learning On Medical 3D Imaging

World of 3D Imaging AI

Medical 3D Imaging overview

For a better comprehension of Medical 3D Imaging ecosystem, we need to get a handle of following aspects:

  • Medical 3D Imaging DEVICES
    • Medical 3D Imaging (hardware players) VENDORS
    • Medical 3D Imaging SOFTWARES
  • Medical Imaging Stoarge & Exchnage protocol / Standard (DICOM)
  • Medical 3D Imaging DATASET formats
  • Medical 3D Imaging DATASET Handling/ PRE-PROCESSING PYTHON Libraries

Mediacal Images comes from various Medical 3D Imaging Devices; & are generally, stored, exchanged / transmissted as DICOM images.

Medical 3D Imaging DEVICES

  • MRI Scan (magnetic resonance imaging)
  • CT Scan (computed tomography)
  • Xray Scans
  • PET Scan (positron emission topography)
  • PET/CT or PET/MRI Scans
  • InfraRed Images
  • HD Video Camera Recordeers for Medical Pre/Intra/Post 'Operative Procedures'

Medical 3D Imaging (hardware players) VENDORS

  • Philips Healthcare
  • Siemens Healthineers AG
  • GE Healthcare Inc.
  • Hitachi Medical Corporation
  • Carestream Health Inc.

Medical 3D Imaging SOFTWARES

  • 3D Rendering Software
  • 3D Scanning Software
  • 3D Modelling Software

3D Medical Imaging (Vantage View)

medical imaging

Medical Imaging Stoarge & Exchnage protocol | DICOM (So, What & Why about DICOM?)

DICOM Images:Digital Imaging and Communications in Medicine: The standard for (& most commonly used for)

  • communication and management of medical imaging information and related data.
  • storing and transmitting medical images enabling the integration of medical imaging devices such as scanners, servers, workstations, printers, network hardware, and picture archiving and communication systems (PACS) from multiple manufacturers.
  • widely adopted by hospitals and is making inroads into smaller applications like dentists' and doctors' offices.
  • File Format — All patient medical images are saved in the DICOM file format. This format has PHI (protected health information) about the patient such as — name, sex, age in addition to other image related data such as equipment used to capture the image and some context to the medical treatment. Medical Imaging Equipments create DICOM files.

  • DICOM Viewer: Doctors use DICOM Viewers, computer software applications that can display DICOM images, read and to diagnose the findings in the images. A useful DICOM viewer for Windows users - Mango

  • Communications Protocol — The DICOM communication protocol is used to search for imaging studies in the archive and restore imaging studies to the workstation in order to display it. All medical imaging applications that are connected to the hospital network use the DICOM protocol to exchange information, mainly DICOM images but also patient and procedure information. There are also more advanced network commands that are used to control and follow the treatment, schedule procedures, report statuses and share the workload between doctors and imaging devices.

Medical 3D Imaging DATASET formats

Image file format is often a confusing aspect for someone wishing to process medical images.

A medical image data set consists typically of

  • one or more images representing the projection of an anatomical volume onto an image plane (projection or planar imaging),
  • a series of images representing thin slices through a volume (tomographic or multislice two-dimensional imaging),
  • a set of data from
    • a volume (volume or three-dimensional imaging), or
    • multiple acquisition of the same tomographic or
    • volume image over time to produce a dynamic series of acquisitions (four-dimensional imaging).

The file format describes how the image data are organized inside the image file and how the pixel data should be interpreted by a software for the correct loading and visualization.

  • Data file extensions for MRI/CT/X-Ray scan
    1. .dcm - DICOM
    2. .nii (Neuroimaging Informatics Technology Initiative (Nifti))
    3. .mnc (Minc)
    4. .img and .hdr (Analyze)


Reference: Data file extensions

Medical 3D Imaging DATASET PRE-PROCESSING/ Handling PYTHON Libraries

Python library that handle MRI/CT Scan/X-Ray imaging data

  • pydicom - A very popular python package used for analyzing DICOM images.
  • MedPy - Another populat python library

Python libraries to handle Infrared images

White paper for reference.

   pip install pydicom
   pip install pydicom medpy nibabel
   pip install pyradi spectral rampy

Medical Imaging manipulation & vectorization

MRI/CT-Scan/X-rays/ Infrared Scanning - An Overview

MRI Background

Magnetic Resonance Imaging (MRI) is the most common diagnostic tool brain tumors due primarily to it's noninvasive nature and ability to image diverse tissue types and physiological processes. MRI uses a magnetic gradient and radio frequency pulses to take repetitive axial slices of the brain and construct a 3-dimensional representation. Each brain scan 155 slices, with each pixel representing a 1mm3 voxel.

Basic MRI Workflow 3D rendering produced by T2 MRI scan

CT-Scan Background

A computerized tomography scan (CT or CAT scan) uses computers and rotating X-ray machines to create cross-sectional images of the body. These images provide more detailed information than normal X-ray images. They can show the soft tissues, blood vessels, and bones in various parts of the body. (Read more...)

Basic MRI Workflow

MRI/CT-Scan/X-rays pre-processing (code)

One of the challenges in working with MRI/CT-Scan/X-rays data is dealing with the artifacts produced either by inhomogeneity in the magnetic field or small movements made by the patient during scan time. Oftentimes a bias will be present across the resulting scans, which can effect the segmentation results particularly in the setting of computer-based models.

Bias correction before and after

Infrared Background

In infrared photography, the film or image sensor used is sensitive to infrared light. The part of the spectrum used is referred to as near-infrared to distinguish it from far-infrared, which is the domain of thermal imaging. Wavelengths used for photography range from about 700 nm to about 900 nm. Film is usually sensitive to visible light too, so an infrared-passing filter is used; this lets infrared (IR) light pass through to the camera, but blocks all or most of the visible light spectrum

InfraRed image

Medical Imaging 'Annotated Dataset' Sources:

One of the prominent Data Source with proper labeling - Brain Tumor MRI Dataset: 2015 MICCAI BraTS Challenge Data Science Bowl

CNN models work for Image featurization

picture alt

Convolutional Neural Network is a neural network consists of convolutional layer, pooling layer (max pooling or average pooling but max pooling is preferable), fully connected layer and at last a softmax pooling

Did you find this article valuable?

Support Harsh Vardhan by becoming a sponsor. Any amount is appreciated!