Working with a sequence of images

In this notebook, we will look at how to work with a sequence of images, and possibly change some parameters on the fly. As in all examples, we will start with setting the Keras backend.

   

‼️ Important: This notebook is optimized for GPU/TPU. Code execution on a CPU may be very slow.

If you are running in Colab, please enable a hardware accelerator via:

Runtime → Change runtime type → Hardware accelerator → GPU/TPU 🚀.

%%capture
%pip install zea

import os

os.environ["KERAS_BACKEND"] = "jax"

import keras

import zea
from zea import init_device, load_file
from zea.visualize import set_mpl_style
from zea.internal.notebooks import animate_images

zea: Using backend 'jax'

n_frames = 15
n_tx = 11
n_tx_total = 127

We will work with the GPU if available, and initialize using init_device to pick the best available device. Also, (optionally), we will set the matplotlib style for plotting.

init_device(verbose=False)
set_mpl_style()

Let’s initialize a default B-mode ultrasound image formation pipeline.

pipeline = zea.Pipeline.from_default(enable_pfield=False, with_batch_dim=False)

We will load a sequence of acquired RF data frames (carotid scan) and reconstruct a B-mode image from each frame. We will then animate the sequence of images. But first let’s load the data and parameters.

# this can take a while to download
file_path = "hf://zeahub/zea-carotid-2023/2_cross_bifur_right_0000.hdf5"  # ~25GB
# so let's use the smaller one by default:
file_path = "hf://zeahub/zea-carotid-2023/2_cross_bifur_right_0000_small.hdf5"  # ~2.5GB

frames = list(range(n_frames))  # use first 15 frames for demonstration
data, scan, probe = load_file(file_path, "raw_data", indices=frames)

scan.set_transmits(n_tx)  # reduce number of transmits for faster processing
scan.zlims = (0, 0.04)  # reduce z-limits a bit for better visualizations
scan.xlims = probe.xlims
scan.n_ch = data.shape[-1]  # rf data

config = zea.Config(dynamic_range=(-40, 0))

zea: DEBUG Skipping invalid parameter 'n_frames'.

Reconstructing a sequence of B-mode images

images = []
n_frames = data.shape[0]
progbar = keras.utils.Progbar(n_frames, stateful_metrics=["frame"])

params = pipeline.prepare_parameters(probe, scan, config)

for frame_no in range(n_frames):
    output = pipeline(data=data[frame_no, scan.selected_transmits], **params)
    image = output.pop("data")
    params = output
    images.append(image)
    progbar.update(frame_no + 1)

animate_images(images, "./bmode_sequence.gif", scan, interval=100, cmap="gray")

15/15 ━━━━━━━━━━━━━━━━━━━━ 3s 124ms/step

Change transmits on the fly

We now used 11 transmits throughout for every frame. We can also sweep through the the transmits for each frame to see how it affects the image quality.

images = []
n_frames = len(frames)
progbar = keras.utils.Progbar(n_frames, stateful_metrics=["frame"])

for idx, frame_no in enumerate(frames):
    tx_idx = int(round(idx * (n_tx_total - 1) / (n_frames - 1)))
    scan.set_transmits([tx_idx])
    raw_data_frame = data[frame_no, tx_idx][None, ...]
    with zea.log.set_level("WARNING"):  # to surpress info messages
        params = pipeline.prepare_parameters(probe, scan)
        output = pipeline(data=raw_data_frame, **params)
    images.append(output["data"])
    progbar.update(idx + 1)

animate_images(images, "./tx_sweep.gif", scan, interval=100, cmap="gray")

15/15 ━━━━━━━━━━━━━━━━━━━━ 1s 26ms/step