Version 1 - Segmenting cells

Version 1 - Segmenting cells

This post will look at the segmentation of a cell that has been selected by the user. The code for the following results can be seen here. As this is a group project my task is to implement this aspect of the project. Other members of my group are currently working on creating an efficient filter to accurately remove noise and creating a threshold that will identify the cell from non cells accurately. For this reason I have implemented a vary basic filter that will do the bare minimum so that the user selected segmentation code can be written.

1. Removing noise

The first step is to use a threshold and noise removal. I wont be going into much detail about thresholding and noise removal as a better version is being worked on by other members of the group. The noise removal on this blog is used to allow the segmentation aspect of the project possible. The threshold uses Otsu binarization with the threshold value set to 200 and is applied to the grey-scale cervical image.  The threshold mask can be seen in Figure 1. [1]

Figure 1 - Threshold created of cervical image 

This threshold mask is then used to create a smoother mask with more defined edges using the morphology functions. Figure 2 shows the mask for noise removal.

Figure 2 - Mask after morphology

2. Watershed Algorithm 

The Watershed algorithm has 3 main key components (labels or markers), identifying the foreground, background and unknown areas. These markers are important as it allows the watershed algorithm to achieve its goal while not being over-segmented. The background area is done so by using dilating the mask created after morphology. This is done using the following function.

# Background area

background = cv2.dilate(opening,kernel,iterations=2) [1]

This dilate function takes in the mask and expands the edges of the region of interests. This will result in the following mask output in Figure 3. 

Figure 3 - Background mask 

Finding the foreground does the opposite and shrinks the borders of the mask. This is done using the following functions. The result of which can be seen in Figure 5. 

# Foreground area

dist_transform = cv2.distanceTransform(opening,cv2.DIST_L2,5)

ret, foreground =       cv2.threshold(dist_transform,0.05*dist_transform.max(),255,0) [1]

Figure 5 - Foreground mask

The unknown region is that of which could have a chance of being in either the foreground or the background. It is found by subtracting the two from each other. The areas of which white pixels are in both images are left behind and everything else turns black. The result can be seen in Figure 6.

# Unknown region

foreground = np.uint8(foreground )

unknown = cv2.subtract(background,foreground ) [1]

Figure 6 - Unknown Mask

The next step is to take the foreground mask and use the function cv2.connectedComponents function which computes the connected components labeled image of the Boolean image [2]. This is used to create the marker label using the foreground mask. One is then added to ensure that the background is not 0, but 1. The unknown region is then set to 0 as it could be either in the foreground or the background. The watershed algorithm is then applied to the image which sets the borders of the cells to -1 and everything else from the original image stays at it was. The code for the watershed algorithm can be found below and the resulting image in Figure 7.

# Marker labell
ret, markers = cv2.connectedComponents(foreground)

# Add one to all labels so that sure background is not 0, but 1
markers = markers+1

# Now, mark the region of unknown with zero
markers[unknown==255] = 0 [1]

#Apply watershed algorithm 
markers = cv2.watershed(img,markers)

#Where value = -1 set value to blue
img[markers == -1] = [255,0,0]

Figure 7 - Image after watershed with defined edges

The one problem with this image is the enclaves within the cells (like the way the Vatican is an enclave of Italy) . We don't want this as it will create holes in the final image of the cells. This can be fixed with better filtering to be done by other members of the group. The reason why the edges are blue and not white or black will be shown in the next part.

3. User select

The next step is to use the image created by the watershed algorithm to generate a mask that will allow the user to select the cell of interest. This is done by first splitting the the image from Figure 7 into its individual components B G and R. then using a binary threshold a mask can be created turning everything that's blue to white and everything else will turn black. The code is below with the result in Figure 8. Note that it looks in the image that there are gaps in some sections of the white line. This is not actually the case and it only looks that way when I uploaded it to the blog. They are actually continuous (Take my word for it). 

#Split image colors 

blue, green, red = cv2.split(img)

#Create binary mask of edges 

ret,B = cv2.threshold(blue,254,255,cv2.THRESH_BINARY)

Figure 8. Binary Mask of edges

This is the mask that will be used in conjunction with the original image to allow the user to isolate cells or clumps of cells. The next step is to hand over control to the user. The first step is to display the image and allow the users mouse to have access to the image. In the mouse function it references a function called SEGMENT. This is the function that will be discussed shortly with all the necessary bits and bobs to isolate a particular cell. 

cv2.setMouseCallback('Cervical',SEGMENT)

cv2.namedWindow('Cervical',600)

cv2.imshow('Cervical',img)

For good measure the window size is lowered also as the image is quite large and using the imshow function on its own would have parts of the image cut off the screen. Figure 9 shows what the user will be greeted with when the program runs. No surprises here it is the original image cervical.jpg.

Figure 9 - cervical.jpg will greet the user

The next step is to set up a while loop that will wait for a user input. This is done with the following code

while(1):

    

    if cv2.waitKey(20) & 0xFF == 27:

        break

cv2.destroyAllWindows()

The loop allows for the function SEGMENT to be called. It will also close the window when the escape key is pressed. The Segment function can be seen below. What it does is it waits for the user to left click on a point of the screen. That point that is selected is then stored as an x , y coordinate value. That value if in a cell then takes the mask generated earlier and uses the x,y coordinates to fill from the x,y point out to the boundary around that point creating a new mask with the cell of interest completely white (Figure 10) . This is like using the bucket in Microsoft paint . This Mask is then applied to the image from figure 7 leaving only the cell and the blue boundaries (Figure 11). Then by changing all solid blue and black pixels to white leaves the cell isolated from everything else on a white background (Figure 12). Note: there are some messing around with the borders of the images in this function as this is required to enable  cv2.floodfill.

def SEGMENT(event,x,y,flags,param): 

if event == cv2.EVENT_LBUTTONDOWN: 

h, w = B.shape[:2]

mask = np.zeros((h+2, w+2), np.uint8)

                cv2.floodFill(B, mask, (x,y), 255);

constant = cv2.copyMakeBorder(img,2,2,2,2,                                        cv2.BORDER_CONSTANT,(0,0,0))

img1_bg = cv2.bitwise_and(img,img,mask = B)

img1_bg[np.where((img1_bg == [255,0,0]).all(axis                                                  = 2))] = [255,255,255]

img1_bg[np.where((img1_bg == [0,0,0]).all(axis =                                                    2))] = [255,255,255]

cv2.imshow('Cervical',img1_bg)

Figure 10 - cell mask with cell boundries

Figure 11 - cell after masking 

Figure 12 - Isolated cell

Conclusion and improvements for next version

The segmentation works only on ideal cells. some cells have holes in them due to poor noise removal which needs to be enhanced. If the background is selected the background will show and all the cells are left white. This is also not ideal. The finished design will also crop the image removing the majority of white space. also not to segment out the area where the blue line lies on as it is also part of the cell.

References

[1] Docs.opencv.org. (2018). OpenCV: Image Segmentation with Watershed Algorithm. [online] Available at: https://docs.opencv.org/3.1.0/d3/db4/tutorial_py_watershed.html [Accessed 1 Mar. 2018].

[2] Docs.opencv.org. (2018). OpenCV: Structural Analysis and Shape Descriptors. [online] Available at: https://docs.opencv.org/3.1.0/d3/dc0/group__imgproc__shape.html#gac2718a64ade63475425558aa669a943a [Accessed 1 Mar. 2018].