VEHICLE DETECTION AND CLASSIFICATION FROM HIGH RESOLUTION SATELLITE IMAGES

In the past decades satellite imagery has been used successfully for weather forecasting, geographical and geological applications. Low resolution satellite images are sufficient for these sorts of applications. But the technological developments in the field of satellite imaging provide high resolution sensors which expands its field of application. Thus the High Resolution Satellite Imagery (HRSI) proved to be a suitable alternative to aerial photogrammetric data to provide a new data source for object detection. Since the traffic rates in developing countries are enormously increasing, vehicle detection from satellite data will be a better choice for automating such systems. In this work, a novel technique for vehicle detection from the images obtained from high resolution sensors is proposed. Though we are using high resolution images, vehicles are seen only as tiny spots, difficult to distinguish from the background. But we are able to obtain a detection rate not less than 0.9. Thereafter we classify the detected vehicles into cars and trucks and find the count of them. * Corresponding author.


INTRODUCTION
Efforts to extract information from imagery have been in place ever since the first photographic images were acquired.Low resolution satellite images such as that obtained from LANDSAT, MODIS and AVHRR sensors provide only a vague idea about the scenes and will not provide any information about the objects found in the scene.Images obtained from these types of sensors can be used only for weather forecasting or meteorological applications.For feature extraction and object detection problems high resolution images is a basic requirement.High resolution satellites like CARTOSAT, IKONOS, QuickBird, SPOT and WorldView provide detailed information about the objects as that of aerial images.The panchromatic band of QuickBird images reaches up to 60cm resolution which is as good as aerial images.Recently launched (August 13, 2014) WorldView-3 provides commercially available panchromatic imagery of 0.31 m resolution.
Though the availability of high resolution satellite images accelerate the process of object detection and automate such applications, vehicle detection from satellite images are still a challenging task.This is because, even in high spatial resolution imagery, vehicles are seen as minute spots which are unidentifiable from the foreground regions to detect.Classifying the detected vehicles is more serious as it is sometimes unable to distinguish large and small vehicles using naked eye itself even from high resolution images.Seeing the reference images in our work, this problem is clearly identified.Earlier many researches have been performed on vehicle detection in aerial imagery (Hinz, S., 2005;Schlosser, C., Reitberger, J. & Hinz, S., 2003;Zhao, T., & Nevatia, R.., 2001) and later on the work is extended for satellite imagery.But the methods used for aerial imagery can't be directly applied for satellite images since vehicles are more vivid in aerial images.Fig. 1 shows an aerial image of 0.15m resolution.Figures 2, 3 and 4 are satellite images of resolutions 0.8m, 0.61m and 0.46m respectively obtained from different sensors.It is clear that even by using high resolution satellite images, vehicles are difficult to distinguish from the scene.
The most distinguished works in this field include morphological transformations for the classification of pixels into vehicle and non-vehicle targets.The work done by Jin, X. and Davis, C.H. ( 2007) uses a morphological shared-weight neural network (MSNN) to learn an implicit vehicle model and classify pixels into vehicles and non-vehicles.A vehicle image base library was built by collecting a number of cars manually from test images.The process is time consuming because of neural networks and there is an extra burden of creating an image library.Zheng, H., and Li, L. (2007) suggested another morphology based algorithm using 0.6 meter resolution QuickBird panchromatic images to detect vehicles.Zheng, H., Pan, L. and Li, L. (2006) used similar approach as that of Jin, X. and Davis, C.H., but with increased accuracy.Recent works in this area includes adaptive boosting classification technique (Leitloff, J., Hinz, S. & Stilla, U., 2010) for the updation of weights and an area correlation method (Liu, W., Yamazaki, F. & Vu, T. T., 2011) to detect vehicles from satellite images in which very good accuracy is achieved but the classification stage is not included.The most recent work in this field is by Zheng,Z. et al. (2013) where the quality percentage reaches 92%, but with aerial images of very high resolution having 0.15m.The method uses top-hat and bot-hat transformations successively with Otsu's thresholding for better accuracy.
The specialty of our work is, vehicles are detected even from satellite images of 2.5m resolution with acceptable accuracy.Both bright and dark vehicles are detected using the proposed method and then it is classified as cars and trucks.Results presented in the work reveals that the detection percentage reaches 90% irrespective of the poor quality of vehicles in satellite images.

Pre-processing the Input Image
Satellite images are obtained as panchromatic (greyscale), natural colour (RGB) and multispectral bands.No preprocessing stage is needed for panchromatic images.For applying the vehicle detection algorithm for satellite natural colour images, it is first converted to greyscale.Multispectral images have 4 bands where the fourth band is Near Infra-Red (NIR).This band is not used in our algorithm and can be discarded and finally converted to greyscale image.

Region of Interest (ROI) Extraction
Most of the road segments may not be straight in the concerned satellite image.Therefore, before going directly to the region of interest segmentation we have to rotate the image in such a way that the road segment should be 0 0 (fig.18) with respect to the horizontal plane.After that selection of desired region of interest is done.The procedure is as follows: (i)Rotating the image

Automatic Road Detection using FIS
After evaluating a number of satellite images and their mean and standard deviation, 11 rules are formulated for decision making in order to develop a fuzzy inference system (FIS): a.If (mean is low) and (stddev is low) and (hough is not line) then (output is not road) b.If (mean is low) and (stddev is low) and (hough is line) then (output is road unlikely) c.If (mean is low) and (stddev is high) and (hough is not line) then (output is not road) d.If (mean is low) and (stddev is high) and (hough is line) then (output is not road) e.If (mean is average) and (stddev is low) and (hough is not line) then (output is not road) f.If (mean is average) and (stddev is low) and (hough is line) then (output is road) g.If (mean is average) and (stddev is high) and (hough is not line) then (output is not road) h.If (mean is average) and (stddev is high) and (hough is line) then (output is road unlikely) i.If (mean is high) and (stddev is low) and (hough is not line) then (output is not road) j.If (mean is high) and (stddev is low) and (hough is line) then (output is road unlikely) k.If (mean is high) and (stddev is high) and (hough is line) then (output is road unlikely) The linguistic variables generated using MATLAB are shown below:

Multiple Thresholding for finding Bright Vehicles
Most of the cases, the intensity values of bright vehicles are greater than the intensities of the background.Using this concept we can use a fixed threshold and pixels higher than this particular threshold corresponds to bright vehicles.But some objects or regions on roads, such as lane markers and road dividers may have similar intensity values as that of bright vehicles.Also, each bright vehicle may not have same range of intensity because of the images taken at different times due to sun elevation and azimuth angles and sensor angles.So it is better to use more than a single threshold.But, as the number of threshold values increases we get as many binary images, increasing the time taken for vehicle detection process.Therefore, to identify only the vehicles and to avoid the detection of irrelevant objects, three different thresholds T 1 , T 2 , and T 3 are used in this work.The third threshold T 3 is fixed as the combination of first two thresholds T 1 and T 2 for getting more accurate results.
In order to find the three threshold values, consider the two dimensional matrix of image intensities M 1 .A one dimensional matrix M 2 is constructed using the maximum intensity pixel from each row of M 1 .The threshold T 1 is mean of this 1D matrix M 2 .T 2 is the minimum value in the matrix M 2 .For increasing the percentage of accuracy a third threshold T 3 is calculated which is the mean of first two thresholds T 1 and T 2 .The procedure is briefed in figure 14.The concept is based on the fact that on a highway, bright vehicles have maximum intensity levels than any other objects.Therefore we are considering the maximum intensity in each row of M 1 to calculate M 2 and thereby the three thresholds T 1 , T 2 and T 3 .Thresholds T 1 , T 2 , and T 3 are used to convert the test image to three different binary images Image-1, Image-2 and Image-3.Fig. 15 shows the three thresholded images for calculated threshold values of 200, 149 and 175 for T 1 , T 2 and T 3 respectively.15, it is understood that many irrelevant objects are included in the thresholded images.Also some vehicles are common in the resultant images.In order to remove the irrelevant objects and to extract the common objects, the logical AND operation is performed among the binary images as given in eqn.( 1), ( 2

Otsu's Thresholding for finding Dark Vehicles
For the detection of dark vehicles, the Otsu's threshold (Otsu, N., 1979) is used.Before applying the Otsu's threshold, a sliding neighborhood operation is applied to the test image (Aurdal, L, Eikvil, L., Koren, H., Hanssen, J.U., Johansen, K. & Holden, M., 2007).In this method a 3-by-3 neighborhood of each and every pixel is selected.The nearby pixel is replaced by the minimum intensity value of this 3x3 window.The result is a darker pixel compared to the earlier one.This operation is followed by Otsu's thresholding to get the resultant dark vehicle detected image.Otsu's thresholding method involves iterating through all the possible threshold values and calculating a measure of spread for the pixel levels each side of the threshold, i.e. the pixels that either falls in foreground or background.The aim is to find the threshold value where the sum of foreground and background spreads is at its minimum.This can be achieved by finding a threshold with the maximum between class variance and minimum within class variance.For this, the method check all pixel values in the image using equations 5 and 6 to find out which one is best to classify foreground and background regions, so that foreground regions are clearly distinguished from the scene depending on the quality of the image.The within class variance is simply the sum of the two variances multiplied by their associated weights.Between class variance is the difference between the total variance (sum of background and foreground variances) of the image and within class variance.

Vehicle Classification & Count
Before moving directly to the vehicle classification stage, a morphological dilation operation is performed as some vehicles may get splitted into parts after segmentation operation.Dilation will combine these parts into a single vehicle, which increase the detection percentage of the vehicle detection algorithm.The structuring element used for the process is defined as eqn.( 8): Vehicles are commonly rectangular in shape; we can't see an irregular shaped or circular shaped vehicle.Therefore the above structuring element is sufficient for our detection algorithm.
From the detected vehicles, some of the parameters which are able to classify them as cars and trucks are calculated.In our work, width, height, and area of the detected vehicles are considered for the classification stage.For that connected component labeling is performed on the dilated image.First, taking into account of all the connected components in the reference image, the average of each of these parameters is computed.Then, the three parameters for each and every detected vehicle are compared with the average values.If width, height, and area of the vehicle is greater than the average values it is considered as a truck, or else it is a car.The algorithm is given below: 1) Dilate Vehicle Detected Image using structuring element SE. 2) Perform Connected Component Labeling using 4connected neighborhood.3) Compute area, major axis (width) and minor axis (height) of labeled regions.4) Obtain mean of these parameters.5) Check whether area greater than mean area and Major axis length greater than mean major axis length and Minor axis length greater than mean minor axis length.

EXPERIMENTAL RESULTS
The results obtained for other two panchromatic IKONOS (1m) and SPOT-5 (2.5m) images of highways in San Jose, CA and Barcelona, Spain are given below.The ROI for IKONOS image is with an angle of rotation of +13 0 and for SPOT-5 image is with an angle of rotation of -10 0 .The experimental method is also verified with very high resolution aerial and satellite natural colour (RGB) images given in section I.
The aerial image is taken as a whole as ROI since only road segment is shown in the image.The angle of rotation of the image is taken as 0 0 .Part of the road segment is taken as ROI for other three satellite images with angle of rotations -36 0 , +30 0 and +44 0 respectively.To measure the performance of the algorithm, the results obtained are compared with the manual count of the vehicles; this is, by visually inspecting the region under study.The inferred results are given in table 1.For all the reference images, it is seen that, though the cars and trucks are not vivid even in the actual image, the results show that the detection rate is more than 90%.It is noted that for the very high resolution aerial image, since the vehicles are clearly seen, even by taking the entire image as ROI, the method is able to detect and classify vehicles with a detection rate of 0.94.

CONCLUSION
In this paper, a multistep algorithm is designed for detecting vehicles from satellite images of different resolutions.The method also classifies and counts the number cars and trucks in the image.The proposed method is able to detect exact number of cars and trucks even from satellite images lower than 1m resolution in which vehicles are identified as some noisy white spots.But roads having high density traffic, there may be chances for increased error percentage since vehicles are very much closer in those cases.Also in this work vehicles are classified only as cars and trucks.More number of classes can be included like cars, small trucks, large trucks and buses.

Figure 5 :
Figure 5: Overall Flow of the System (a)Enter the angle of rotation.ifwe are rotating it in the clockwise direction, the angle of rotation is negative and for anti-clockwise direction it will be positive.(b) Convert the value from cell string to string.(c) Convert the value from string to numeric.(d)Display the rotated image.(ii) Selecting the co-ordinates of the ROI (a) Select the (x,y) coordinates of the upper left corner point of the ROI.(b)Select the (x,y) coordinates of the lower right corner point of the ROI.(c)Merge the array x and y. (d)Convert the values from numeric to string.(e)Display the selected ROI co-ordinates.(iii) Region of interest Extraction (a)Subtract x co-ordinates to find width (b)Subtract y co-ordinates to find height (c)Crop the image using the obtained width and height of the image.(d)Display the segmented region which is the ROI.The figure shown below (fig.6) is a SPOT-5 panchromatic image of 2.5m resolution which is a highway in Oklahoma City.

Figure 7 :
Figure 7: Selecting the Coordinates of the Image

Figure 16 :
Figure 16: New Segmented Images mean and variance of the image are represented by w, µ and σ 2 .The foreground and background regions are represented as f and b.The detected dark vehicles are shown in fig.18.

Figure 18 :
Figure 18: Dark Vehicle Detected Image

•
Yes -Increment truck count by 1 • No -Increment car count by 1 6) Conversion from numeric to string for car and truck count.7) Display the count in the message box.The following figures (fig.20) show the car and truck counts for the vehicle detected image.

Figure 20 :
Figure 20: Car & Truck Counts of the ROI in the Reference Image

Figure 32 :
Figure 32: Car & Truck Counts of the ROI

Table 1 .
Performance Evaluation