Counting Bricks

Hello all !!

Long back when I was in my initial stage of learning computer vision, my friend asked me what are you always doing with your laptop? I answered Computer Vision. He then asked me what’s the big deal about it, so I explained. He then challenged me counting number of bricks in an image.

bricks

Original Image to be processed

Challenge accepted !!

It’s a difficult task to count number of bricks in an image manually. But very easy with computer vision. I then opened my favorite IDE to code it and solve the challenge. Here we will solve this challenge in two steps which involves general image processing techniques like blurring,thresholding,etc which gives us number of bricks. But the result found to be less accurate and then we make our script accurate by utilizing Connected component analysis.

Let’s dive in to the code.

import numpy as np
import cv2
from skimage.filters import threshold_adaptive
from skimage import measure

We first start by importing necessary libraries. You should be familiar with numpy and cv2 which are almost handy for everyone. I used skimage for adaptive thresholding even though we have in cv2 as skimage implementation is more pythonic. And skimage.measure is used for connected component analysis.

image = cv2.imread('bricks.jpg')
print "Height: {},Width:{}".format(*image.shape[:2])

We then read the image from disk and display the (Height,Width) of the image to the console.

gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
blurred = cv2.bilateralFilter(gray,21,41,41)
edged = cv2.Canny(blurred,50,200)

Then we convert the image to grayscale as it is very suitable for further processing.

We used bilateralFilter for blurring the image rather than vanilla blurring or Gaussian blurring since we are interested in counting number of bricks so we need to preserve the edges and therefore bilateralFilter is a good choice as it blurs the image while preserving the edges.

bilateralFilter(Image,d,sigmaColor,sigmaSpace)

d – diameter of neighborhood of pixels we need to examine
sigmaColor – color standard deviation, larger the value more colors will be considered in computing the blur.
sigmaSpace – space standard deviation, larger the value pixels farther out from the center pixel diameter will influence blur calculation.

From the above two images we can observe that the bilateral filtered image reduces considerable noise while preserving the edges as they are. We then apply Canny edge detector to the blurred image with T_{lower}=50 and T_{upper}=200 for hysteresis thresholding.

Canny edged applied to blurred image

Canny edged applied to blurred image

thresh = threshold_adaptive(blurred,21,offset=15).astype("uint8")*255

mask = cv2.bitwise_not(edged)
thresh = cv2.bitwise_and(thresh,thresh,mask=mask)

We then continue to adaptive thresholding which involves considering some neighborhood of pixels rather than global quantification which results in better results.

Adaptive thresholded

Adaptive thresholding applied to blurred image

We further apply not to the edged image as we are interested in counting bricks where in the edged image they appear to be black so the better approach is to invert them.

bitwise_not applied to edged image

bitwise_not applied to edged image

The better performance can be acquired by applying bitwise_and for the image obtained above and the thresholded image.

bitwise_and applied to thresholded and *edged image

bitwise_and applied to thresholded and *edged image

We can observe a considerable difference between the image above and the thresholded image and that is why it is sometimes better to process the image in this way.

kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(5,5))
thresh = cv2.morphologyEx(thresh,cv2.MORPH_OPEN,kernel)

thresh_copy = thresh.copy()
(cnts,_) = cv2.findContours(thresh_copy,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)

output = image.copy()

for c in cnts:
    cv2.drawContours(output,,-1,(255,255,255),2)

print "# of bricks:",len(cnts)
cv2.imshow("Bricks",output)
cv2.waitKey(0)

On observing the image above we can say that some bricks are connected lightly and therefore we need to open their connections, in order to achieve that we use some morphological operations. It involves creating of a structuring element and I choose rectangular kernel of size 5x5 which then used for morphological opening operation.

Morphology open applied to previous image

Morphology open applied to previous image

Now that we have our bricks got separated and now we can proceed to find contours which gives us the number of bricks in the image. And also draw the contours to visualize the bricks in the image.

bricks marked

Bricks marked

Let’s run our script and examine the results.

$ python bricks.py
Height: 394,Width:525
# of bricks: 323

We can see that there are 323 bricks in the image. But wait ! On verifying the number of bricks that are actually present in the image 323 tends to be more and it is not correct. The problem we have with this is as you can see the bricks that some small bricks which got separated while in morphological operations and produces some more chunks.

This problem can be solved by applying Connected component analysis to our previous image.

labels = measure.label(thresh, neighbors=8, background=0)
mask = np.zeros(thresh.shape, dtype="uint8")

Connected component analysis works by labeling the pixels. To be more clear it can be performed either on binary image or a thresholded image. It works by examining neighborhood or 4 or 8 and then labels the current pixel. The value for the label can range from 0. While 0 represents background and the label other than 0 represents a foreground pixel.

It processes through all the pixels and label them accordingly. To visualize the each of the bricks we create a mask of same dimensions as thresholded image.

for (i, label) in enumerate(np.unique(labels)):
    if label == 0:
        continue
 
    labelMask = np.zeros(thresh.shape, dtype="uint8")
    labelMask[labels == label] = 255
    numPixels = cv2.countNonZero(labelMask)
 	
    mask = cv2.add(mask, labelMask)
    cv2.imshow("Mask", mask)
    cv2.waitKey(0)

print "# of bricks (accurate):",len(np.unique(labels))-1

We then loop over through all the unique labels and visualize each of the label associated the image pixels. We create labelMask which is used to visualize the current label.

Finally we print the number of bricks that are actually present in the image to the console by counting the number of unique labels found - 1 as 0 is labeled as background.

$ python bricks.py
Height: 394,Width:525
# of bricks: 323
# of bricks (accurate): 316

And the labels at each iteration is visualized as follows.

connected component analysis

Connected Component Analysis

That’s it!! We found the number of bricks in the image to be 316 which is correct. We have seen that applying connected component analysis sometimes make our computer vision tasks even easier with more accuracy. The task shown is fine tuned and the parameters for thresholding and edge detection may vary for image to image. Finally I have found the number of bricks in a go and I won the challenge.

  • Kapil Varshney

    Awesome! Interesting challenge to solve for someone starting out with CV. Thanks Saideep.