Detect and visualize differences between two images with OpenCV Python

Method #1: Structural Similarity Index (SSIM)

To visualize differences between two images, we can take a quantitative approach to determine the exact discrepancies between images using the Structural Similarity Index (SSIM) which was introduced in Image Quality Assessment: From Error Visibility to Structural Similarity. This method is already implemented in the scikit-image library for image processing. You can install scikit-image with pip install scikit-image.

Using the skimage.metrics.structural_similarity function from scikit-image, it returns a score and a difference image, diff. The score represents the structural similarity index between the two input images and can fall between the range [-1,1] with values closer to one representing higher similarity. But since you’re only interested in where the two images differ, the diff image is what we’ll focus on. Specifically, the diff image contains the actual image differences with darker regions having more disparity. Larger areas of disparity are highlighted in black while smaller differences are in gray.

All differences -> Significant region differences

The gray noisy areas are probably due to .jpg lossy compression. We would obtain a cleaner result if we used a lossless compression image format. The SSIM score after comparing the two images show that they are very similar.

Image Similarity: 91.9887%

Now we filter through the diff image since we only want to find the large differences between the images. We iterate through each contour, filter using a minimum threshold area to remove the gray noise, and highlight the differences with a bounding box. Here’s the result.

To visualize the exact differences, we fill the contours onto a mask and on the original image.

from skimage.metrics import structural_similarity
import cv2
import numpy as np

# Load images
before = cv2.imread('left.jpg')
after = cv2.imread('right.jpg')

# Convert images to grayscale
before_gray = cv2.cvtColor(before, cv2.COLOR_BGR2GRAY)
after_gray = cv2.cvtColor(after, cv2.COLOR_BGR2GRAY)

# Compute SSIM between the two images
(score, diff) = structural_similarity(before_gray, after_gray, full=True)
print("Image Similarity: {:.4f}%".format(score * 100))

# The diff image contains the actual image differences between the two images
# and is represented as a floating point data type in the range [0,1] 
# so we must convert the array to 8-bit unsigned integers in the range
# [0,255] before we can use it with OpenCV
diff = (diff * 255).astype("uint8")
diff_box = cv2.merge([diff, diff, diff])

# Threshold the difference image, followed by finding contours to
# obtain the regions of the two input images that differ
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]

mask = np.zeros(before.shape, dtype="uint8")
filled_after = after.copy()

for c in contours:
    area = cv2.contourArea(c)
    if area > 40:
        x,y,w,h = cv2.boundingRect(c)
        cv2.rectangle(before, (x, y), (x + w, y + h), (36,255,12), 2)
        cv2.rectangle(after, (x, y), (x + w, y + h), (36,255,12), 2)
        cv2.rectangle(diff_box, (x, y), (x + w, y + h), (36,255,12), 2)
        cv2.drawContours(mask, [c], 0, (255,255,255), -1)
        cv2.drawContours(filled_after, [c], 0, (0,255,0), -1)

cv2.imshow('before', before)
cv2.imshow('after', after)
cv2.imshow('diff', diff)
cv2.imshow('diff_box', diff_box)
cv2.imshow('mask', mask)
cv2.imshow('filled after', filled_after)
cv2.waitKey()

Limitations: Although this method works very well, there are some important limitations. The two input images must have the same size/dimensions and also suffers from a few problems including scaling, translations, rotations, and distortions. SSIM also does not perform very well on blurry or noisy images. For images that do not have the same dimensions, we must switch from identifying pixel-similarity to object-similarity using deep-learning feature models instead of comparing individual pixel values. See checking images for similarity with OpenCV using Dense Vector Representations for scale-invariant and transformation indifferent images.

Note: scikit-image version used is 0.18.1.

Method #2: cv2.absdiff

For completeness, OpenCV provides a very simple built-in method using cv2.absdiff but the results are not as good as SSIM and also does not calculate a similarity score between the two images. This method only generates a difference image.

The results are very washed and still suffers from the same limitations. Although this method is much simpler, the recommendation is to use SSIM.

import cv2

# Load images as grayscale
image1 = cv2.imread("left.jpg", 0)
image2 = cv2.imread("right.jpg", 0)

# Calculate the per-element absolute difference between 
# two arrays or between an array and a scalar
diff = 255 - cv2.absdiff(image1, image2)

cv2.imshow('diff', diff)
cv2.waitKey()