EFFICIENT LANE DETECTION BASED ON ARTIFICIAL NEURAL NETWORKS

Lane detection is a problem that has attracted in the last years the attention of the computer vision community. Most of approaches used until now to face this problem combine conventional image processing, image analysis and pattern classification techniques. In this paper, we propose a methodology based on so-called Ellipsoidal Neural Networks with Dendritic Processing (ENNDPs) as a new approach to provide a solution to this important problem. The functioning and performance of the proposed methodology is validated with a real video taken by a camera mounted on a car circulating on urban highway of Mexico City.


INTRODUCTION
Autonomous Car Technology (ACT) is recognized nowadays as one of the most important advances in urban mobilization.It is under development since 15 years.Between 2004 and 2005 several vehicles competed to traverse the Mojave Desert in California covering more than 200 km in an autonomous way, following a map of GPS coordinates.The vehicles were built by universities such as Stanford and Carnegie Mellon, as well as by industries like Ensco and Velodyne Acoustics.The event was organized by DARPA-USA.In 2007, another competition was organized.It represented a major challenge: Vehicles were asked to avoid hitting other vehicles, cycles, pedestrian or traffic lights.They should also have the capacity to make "U" turns, when needed, and to transit under different weather conditions.In 2005 and 2007 the two winners were Stanley (Thrun, 2006) and Boss (Urmson, 2008), respectively.
In (Assidiq 2008), for example, the authors solve lane detection by first acquiring a front view by a camera mounted on the vehicle.They then detect lanes by using a pair of hyperbolas which are fitting to the edges of the lane by means of so-called Hough transform.They show that their proposal is robust to light changes and shadows by testing it with both painted and unpainted road as well as curved and straight road in different weather conditions.
In (Fernando 2014), the authors face lane detection by integrating two visual cues: 1) stripe-like features found on lane lines extracted using a two-dimensional symmetric Gabor filter, and 2) texture characteristic determined using the entropy measure of a predefined neighbourhood around a lane boundary line.They integrate both visual cues using a rule-based classifier which incorporates a modified sequential covering algorithm to improve robustness.They separate lane boundary lines from other similar features by using a road mask that uses road chromaticity values estimated from CIE L*a*b* colour transformation.They validate their algorithm on a set of images of suburban streets.
In (Saha 2012), the authors face the lane detection problem by first converting a RGB road scene image into a grey image.They then employ a flood-fill algorithm to label the connected components on the grey image.After that, they take the largest connected component as the road region.They finally subtract the marks or road lane and road boundary from the connected components.Thy show the performance of their algorithm on both straight and slightly curved road scene images under different day light conditions and the presence of shadows on the roads.
In (Van Winden 2016), the authors describe a method for automatically deriving road attributes by analysing and mining movement trajectories (e.g.GPS tracks).They investigate how to automatically extract eight road attributes: directionality, speed limit, number of lanes, access, average speed, congestion, importance, and geometric offset.For this, they propose using a decision tree.They use their proposal to automatically update the OpenStreetMap (OSM) dataset of the Netherlands, increasing its level of completeness.
In short, in (Wang 2006), the authors face real-time lane detection by combining a hyperbola-pair lane boundary model and an improved RANSAC paradigm.They introduce a fuzzy measurement into the RANSAC paradigm to improve accuracy and robustness when fitting boundaries points into the model.They test their proposal in many different conditions, including various conditions of illumination, weather and road.
It is worth mentioning that most lane detection methods are based on the Hough transform (Leng, 2010), (Li, 2014), (Tan, 2014), (Gaikwad, 2015), a well-known technique used to detect lines and curves (Duda, 1972).This kind of line detectors strongly depend on the pre-processing stage; they are typically combined with a Canny or Sobel edge detector or with an adaptive threshold.The main drawback is in the complexity of the methodology and the required time.Our proposed does not use these filters and is so easy to implement.
Most recent approaches make use of so-called deep networks (Kim, 2014), (Huval, 2015), (Gurghian, 2016).These techniques are so accurate and represent the state-of-the-art in image classification.They have three main disadvantages: need 1) thousands of examples for training, 2) specialized hardware to be implemented and 3) are time consuming.The besought method does not require a special hardware neither is time consuming.
To end up with this discussion, in (Kaur 2015), the authors provide a comprehensive review of the literature in lane detection techniques with the aim to present to the reader the limitations of the existing lane detection methods.
Also it is expected that in 2035, the more developed societies will adopt ACT as a normal transportation way.While that moment arrives, several intelligent vehicular assistants are under development.One of these systems concerns automatic lane detection.Systems like these alert the driver when his car starts to move out of the lane; either because the driver is distracted or tired.A system like this is particularly useful in highways where monotony of conduction can augment distraction.
Artificial neural networks, on the other hand, have been successfully used in many tasks including signal analysis, noise cancellation, model identification, process control, object detection, and pattern recognition, in general.
Many artificial neural network models have been reported in literature, since the very simple Threshold Logic Unit (TLU), introduced by McCulloch-Pitts (MuCulloch 1943) at the beginning of the 40´s, passing by the well-known Perceptron, presented to the world by Rosenblatt in the 50´s (Rosenblatt 1958) and (Rosenblatt 1962) until the so called Morphological Neural Models with and without Dendritic Processing introduced by Ritter et al. in (Ritter 1996), (Sussner 1998), (Ritter 1999), (Ritter 2003), to mention a few.
Interest in Morphological Neural Networks (MNNs) has increased in the last five years (Sossa, 2013), (Sossa, 2014), (Arce, 2016) and (Zamora, 2017).This kind of ANNs make use of min and max operations to segment the input space into hyperboxes.In particular, MMNs with dendritic processing have shown the capacity to efficiently solve low-dimensional classification problems where other reported methods fail (Zamora, 2016), (Hernandez, 2017).
In this paper, we propose a new method for lane detection based on the functioning of so-called dendritic ellipsoidal neurons (DEN) (Arce, 2017).Instead of hyper-boxes, these neural networks use hyper-ellipsoids.
The rest of the paper is organized as follows.In Section 2 we briefly present the architecture of an ellipsoidal neuron, while in Section 3 we describe, step by step, how this kind of neuron can be trained.Section 4, in the other hand, is focused to introduce the method for detecting lanes in an image.In Section 5 we present some results, while in Section 6 we summarize the conclusions and directions for further research.

ARQUITECTURE OF A DEN
Before presenting the DEN, we describe the conventional mathematical structure for a Dendrite Morphological Neuron (DMN).The dendrite output for a DMN is given by: where   is one of the  components of an input vector of  = ( 1 ,  2 , … ,   )  .  0 and   1 are the synaptic weights (  0 inhibition,   1 activation) of terminal fibres of the ℎ neuron synapsing the ℎ dendrite for the ℎ class.More details can be found in (Ritter 2003) and (Ritter 2006).
The neural network model used in this paper also uses dendritic processing (Arce 2017), the authors call it as Dendrite Ellipsoidal Neuron (DEN).Instead of using hyper-boxes to cluster data, as normally used by standard DMNs, DENs use hyper-ellipsoids.Next we describe each of the parts of a DEN.This change is key for our lane detector because the hyper-boxes cannot be rotated.The hyper-boxes in the DMNs are always align to the coordinate axes which is a great limitation in the moment to segment the lane.In contrast, the hyper-ellipsoids have the flexibility to be rotated.This allows to need less number of dendrites to segment the same number of pixels for the lanes.Of course, everything has a cost.The DEN requires ( + 1) number of learning parameters per dendrite, while the DMNs requires 2 parameters per dendrite, where  is the dimension of patterns.Since the lane detection is a classification problem with 2 dimensions (for black and white images), this is not important for the lane detector.
The architecture of a DEN is illustrated in Figure 1.As can be seen, it is composed of three layers as follows:

Input Layer
This layer receives at its input -dimensional vectors of the form  = ( 1 ,  2 , … ,   )  .This layer does not perform any processing, it only sends the information provided by the input patterns to the next layer, the so-called dendritic layer.

Dendritic Layer
This layer performs so-called dendritic computation over input patterns   ,  = 1, … , , by applying the following equation: In this case, each variable is defined as follows: is the vector produced by all the dendrites; it represents the Mahalanobis distance between input pattern   and all the generated ellipsoids.Each dendrite performs the operation given by Equation (2) on each input pattern.The number of dendrites as well as the number of centroids and covariance matrices are obtained during the training step.

Output Layer
This layer is composed of only one neuron.Once the dendritic step is performed, the output neuron choses the dendrite that has generated the smallest distance between the input pattern and all generated ellipses; it then assigns the input pattern to the corresponding class.The following equation shows how this calculation is made: is a scalar value that indicates the nearest distance between the input pattern and all the placed ellipsoids.
Given an input pattern:   , if   = 0, means that   is exactly at the centroid of the hyper-ellipse.If   < 1,   is inside the hyperellipse, and if   > 1,   is out of the hyper-ellipse.
Figure 1 shows the architecture of an ellipsoidal neuron with dendritic processing.Figure 2 shows an example (in 2D) of a hyper-ellipse generated by its two dendritic parameters (  and Figure 1.Architecture of an ellipsoidal neuron with dendritic processing, composed of three layers.Taken from (Arce 2017).

TRAINING OF A DEN
One of the most important problems in training a neural network with dendritic processing is the selection of the number of dendrites.The method described in this note automatically determines the number of necessary dendrites to solve a given problem.In this section we explain the operation of the proposed algorithm.Before beginning, let us have the following definitions: Definition 1.For  1 ,  2 , … ,   ,  = 1,2, … ,  a finite set of patterns;   = ( 1 ,  2 , … ,   )  ∈ ℝ  being one of these patterns.Each pattern   belongs to only one of  classes:   ,  = 1,2, … , , with  > 1 a finite number.Definition 2. A hyper-ellipsoid   is an -dimensional ellipsoid that contains a finite set of patterns:  ∈ ℝ  .This   is defined by its dendritic parameters: (  and ∑ ) −1  .
Equation 2 defines the Mahalanobis distance between pattern   and hyper-ellipsoid   .The goal of the proposed training algorithm is to create and position a set of hyper-ellipsoids    ∈ ℝ × , with  ∈ {1,2, … , } that allow stablishing a decision surface among classes with the smallest number of dendrites.

1)
Define a global error (  ) and local error (  ).Both errors are referred to the allowed number of wrong classified patterns.  is the total allowed error while   makes reference to the allowed errors (positive failures and negative failures) for each class.

2)
For each class   , generate  clusters using so-called -means++ algorithm (Arthur, 2007) To better understand the functioning of a DEN, the following straightforward example is provided.As can be seen from Figure 3, it is two class problem: The blue dots belonging to the first class  1 and the green stars to the second class  2 .
We would like to classify pattern  ̃1 = [−1.0000− 0.5000] which is supposed to belong to class  1 and pattern  ̃2 = [0.00000.0000] from class  2 .Table 1 presents the hyperparameters of the two generated ellipsoids by Algorithm 1.

LANE DETECTION USING A DEN
In this section we describe the methodology for lane detection based on ellipsoidal neurons with dendritic processing.The experiment was performed by using a video taken at the "Viaducto Bicentenario" of the State of Mexico.The video consists in 3237 frames with a resolution of 580 × 620 pixels per frame.We used low resolution images due to we are focused to implement our proposal in a near future in an embedded device.
The methodology was applied to each frame and uses the following four steps:

Change of Colour Space
The first step consists in changing the colour space from RGB to HLS.The idea is to work in the S cannel (Saturation) that provides the degree at which a primary colour is dissolved (yellow, orange, or pure red) pure with light (Gonzalez, 1987).
Figure 4 shows an original image while Figure 5 depicts the S cannel of the HLS colour space.In Figure 5 we can observe the slight improvement in colour on the image lanes.

Selection of the Interest Area
The following step consists in selecting an interest area in the image.The idea is to isolate from the S-image an area containing, with a high probability, the lanes we would like to further detect.Figure 6 depicts an example of interest area.
Figure 6.Selection of the area of interest.
The interest area is defined to have a trapezoidal form, taking into account the knowledge that at the distance lanes tend to join.The lower corners of the trapezoid are chosen as the image corners to the left and to the right of the image.The upper corners of the trapezoid are marked by using the perspective information of the camera.
Figure 7. Thresholding of the interest area.

Interest Area Thresholding
Once the interest area has been selected, the proposed methodology proceeds to threshold it.Two thresholds were used: 50 and 120, respectively.Figure 7 presents the thresholded interest area.As can be appreciated this area contains interest pixels (in red) and non-interesting pixels (in black).
Figure 8. Establishment of the ellipses on the binary image by means of an ENNDP.

Application of the ENNDPs to The Binary image
The following step consists in stablishing ellipses over the thresholded image by means of the trained ellipsoidal neuron.To carry out this, during training the red pixels were assigned to a first class (class "0") and the black pixels to the second class (class "1").Instead of assigning an optimal number of ellipses as it will done when applying Algorithm 1, a constant number of 16 ellipses per frame were marked.This allows reducing computation times.Figure 8 shows the ellipses generated by the proposal using the ellipsoidal neuron.

Ellipse Filtering
The last step of the methodology for automatic lane detection consisted in filtering ellipses by their centroids, created in advance by means of RANSAC method.This random algorithm is normally used, as it is known, to eliminate atypical values not coinciding with the model (Fischler, 1981).Figure 9 shows the filtered ellipses using RANSAC.

EXPERIMENTAL RESULTS
As already mentioned in Section 4, the described methodology was applied on a video composed of 3237 frames with a resolution of 580 × 620 pixels per frame, taken with camera mounted on a car circulating on "Viaducto Bicentenario", State of Mexico.Figure 9 shows nine of the results obtained by the application of the described methodology.It is worth mentioning to say that in 74 frames the algorithm fails to correctly superimpose the ellipses.The calculated accuracy of the proposed methodology over this set of images was thus of 0.977.
The main drawback of the proposed algorithm is the required time by the filtering stage; it consumes 40% of the total processing time.A possible solution could be using other filtering techniques and an implementation of the algorithm in a Graphic Processing Unit (GPU).

CONCLUSIONS AND FUTURE WORK
We have presented a methodology based on the pattern recognition capacities of the so-called dendrite ellipsoidal neuron.We have shown how the proposed methodology can be successfully applied to automatically detect lanes in urban highways.
Future work includes the implementation of the besought algorithm on a portable Graphic Processing Unit (GPU) and the execution of this on a scaled exploration car.Another future work will be the use of other filtering techniques focused in the time reduction in this stage.
Finally, a formal evaluation and a deep analysis of the proposed method are necessary to validate its whole capacities compared to other methods and algorithms reported in literature.

Figure 2 .
Figure 2. Example of a hyper-ellipsoid generated by the two parameters (  and ∑ ) −1  of the dendrite.Taken from (Arce 2017).
,  ∈ ℤ + .a. Obtain the dendritic parameters (  and ∑ ) −1  of each cluster .Inverse of the covariance matrix can be obtained by applying the pseudo-inverse Moore-Penrose method.b.Compute   counting the number of wrong classified patterns.c.If   <   continue with the following class and go back to step 2. Else,  =  + 1. Algorithm 1 depicts the training for an ellipsoidal neuron.Note that it uses counter  to avoid falling in an infinite cycle when no better classification improvement is obtained.BEGIN Define   y   FOR each class   ,  = 1: WHILE   >   : Transform the  clusters into hyper-ellipsoids Compute   IF   == 0: STOP IF   ≥ |  [k − 1] −   [k]|:  =  + 1 IF  == 3 STOP  =  + 1 END Algorithm 1. Training algorithm for a DEN.

Figure 3 .
Figure 3. DEN trained to solve a straightforward example.Blue ellipsoid encloses patterns belonging to first class; green ellipsoid does the same but patterns from second class.

Figure 4 .
Figure 4. Original imagen taken at the "Viaducto Bicentenario" of the State of Mexico.

Figure 5 .
Figure 5. Saturation channel S of colour space HLS of the original image.

Figure 9 .
Figure 9. Nine of the images with the ellipses superimposed by the algorithm over the lanes.