First, check out /samples/c/squares.c in your OpenCV distribution. This example provides a square detector, and it should be a pretty good start on how to detect corner-like features. Then, take a look at OpenCV’s feature-oriented functions like cvCornerHarris() and cvGoodFeaturesToTrack().
The above methods can return many corner-like features – most will not be the “true corners” you are looking for. In my application, I had to detect squares that had been rotated or skewed (due to perspective). My detection pipeline consisted of:
- Convert from RGB to grayscale (cvCvtColor)
- Smooth (cvSmooth)
- Threshold (cvThreshold)
- Detect edges (cvCanny)
- Find contours (cvFindContours)
- Approximate contours with linear features (cvApproxPoly)
- Find “rectangles” which were structures that: had polygonalized contours possessing 4 points, were of sufficient area, had adjacent edges were ~90 degrees, had distance between “opposite” vertices was of sufficient size, etc.
Step 7 was necessary because a slightly noisy image can yield many structures that appear rectangular after polygonalization. In my application, I also had to deal with square-like structures that appeared within, or overlapped the desired square. I found the contour’s area property and center of gravity to be helpful in discerning the proper rectangle.