{ "cells": [ { "cell_type": "markdown", "id": "862b383e", "metadata": {}, "source": [ "# LPIPS: perceptual similarity for ultrasound images\n", "\n", "Assessment of ultrasound image quality is one of the more notorious challenges. zea offers a variety of commonly used metrics in `zea.metrics`. In this notebook, we demonstrate how to use the [LPIPS](https://github.com/richzhang/PerceptualSimilarity) model to compute perceptual similarity between ultrasound images using the `zea` toolbox. Besides common distortion metrics, a perceptual metric can be a good complementary metric to assess image quality. For more info, see an interesting paper called: [The Perception-Distortion Tradeoff](https://arxiv.org/abs/1711.06077). First we'll set an appropriate backend for Keras." ] }, { "cell_type": "markdown", "id": "c4a7176a", "metadata": {}, "source": [ "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tue-bmd/zea/blob/main/docs/source/notebooks/metrics/lpips_example.ipynb)\n", " \n", "[![View on GitHub](https://img.shields.io/badge/GitHub-View%20Source-blue?logo=github)](https://github.com/tue-bmd/zea/blob/main/docs/source/notebooks/metrics/lpips_example.ipynb)\n", " \n", "[![Hugging Face model](https://img.shields.io/badge/Hugging%20Face-Model-yellow?logo=huggingface)](https://huggingface.co/zeahub/lpips)" ] }, { "cell_type": "markdown", "id": "70003754", "metadata": {}, "source": [ "‼️ **Important:** This notebook is optimized for **GPU/TPU**. Code execution on a **CPU** may be very slow.\n", "\n", "If you are running in Colab, please enable a hardware accelerator via:\n", "\n", "**Runtime → Change runtime type → Hardware accelerator → GPU/TPU** 🚀." ] }, { "cell_type": "code", "execution_count": 1, "id": "b8d5a8b4", "metadata": {}, "outputs": [], "source": [ "%%capture\n", "%pip install zea" ] }, { "cell_type": "code", "execution_count": 2, "id": "b4d5f7dd", "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "os.environ[\"KERAS_BACKEND\"] = \"jax\"\n", "os.environ[\"TF_CPP_MIN_LOG_LEVEL\"] = \"3\"" ] }, { "cell_type": "code", "execution_count": 3, "id": "b45af7c7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m\u001b[38;5;36mzea\u001b[0m\u001b[0m: Using backend 'jax'\n" ] } ], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from keras import ops\n", "\n", "from zea import init_device\n", "from zea.backend.tensorflow.dataloader import make_dataloader\n", "from zea.models.lpips import LPIPS\n", "from zea.visualize import plot_image_grid, set_mpl_style" ] }, { "cell_type": "markdown", "id": "1231f240", "metadata": {}, "source": [ "We will work with the GPU if available, and initialize using `init_device` to pick the best available device. Also, (optionally), we will set the matplotlib style for plotting." ] }, { "cell_type": "code", "execution_count": 4, "id": "1a5db076", "metadata": {}, "outputs": [], "source": [ "init_device(verbose=False)\n", "set_mpl_style()" ] }, { "cell_type": "markdown", "id": "ba75fecd", "metadata": {}, "source": [ "## Load data\n", "We load a small batch from the CAMUS validation dataset hosted on Hugging Face Hub." ] }, { "cell_type": "code", "execution_count": 5, "id": "c784bb91", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "6cfd915761fc4f55bea8f3cc82ccf982", "version_major": 2, "version_minor": 0 }, "text/plain": [ "val/patient0401/patient0401_2CH_half_seq(…): 0%| | 0.00/44.5M [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "batch_np = ops.convert_to_numpy(batch)\n", "batch_np = ((batch_np + 1) / 2 * 255).astype(np.uint8)\n", "\n", "fig, _ = plot_image_grid(batch_np, remove_axis=False)\n", "axes = fig.axes[:n_imgs]\n", "\n", "# Red border for reference image\n", "for spine in axes[0].spines.values():\n", " spine.set_edgecolor(\"red\")\n", " spine.set_linewidth(2)\n", " spine.set_linestyle(\"--\")\n", "\n", "# Green border for most similar image\n", "most_similar_idx = lpips_scores.argmin()\n", "for spine in axes[most_similar_idx + 1].spines.values():\n", " spine.set_edgecolor(\"green\")\n", " spine.set_linewidth(2)\n", " spine.set_linestyle(\"--\")\n", "\n", "# LPIPS scores as text\n", "for ax, lpips_score in zip(axes[1:], lpips_scores):\n", " ax.text(\n", " 0.95,\n", " 0.95,\n", " f\"LPIPS: {float(lpips_score):.4f}\",\n", " ha=\"right\",\n", " va=\"top\",\n", " transform=ax.transAxes,\n", " fontsize=8,\n", " color=\"yellow\",\n", " )\n", "plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.11" } }, "nbformat": 4, "nbformat_minor": 5 }