DESIGNING DAILY PATROL ROUTES FOR POLICING BASED ON ANT COLONY ALGORITHM

In this paper, we address the problem of planning police patrol routes to regularly cover street segments of high crime density (hotspots) with limited police forces. A good patrolling strategy is required to minimise the average time lag between two consecutive visits to hotspots, as well as coordinating multiple patrollers and imparting unpredictability in patrol routes. Previous studies have designed different police patrol strategies for routing police patrol, but these strategies have difficulty in generalising to real patrolling and meeting various requirements. In this research we develop a new police patrolling strategy based on Bayesian method and ant colony algorithm. In this strategy, virtual marker (pheromone) is laid to mark the visiting history of each crime hotspot, and patrollers continuously decide which hotspot to patrol next based on pheromone level and other variables. Simulation results using real data testifies the effective, scalable, unpredictable and extensible nature of this strategy. * Corresponding author


INTRODUCTION
Police patrolling is one of the most important methods for crime prevention and emergency response in urban areas.Due to its importance, many strategies for police patrolling have been developed and used in practice, such as random preventive patrolling (Weisburd and Eck 2004), high-visibility patrolling (Braga 2001), and hotspot policing (Braga 2001;Koper 1995).Hotspot patrolling, which "focuses on small geographic places or areas where crime is concentrated" (Koper 2014), is considered one of the most important innovations in policing in recent years, and is currently used by many police departments (Weisburd et al. 2003).
In policing operations, a high-level strategy of hotspot patrolling needs to be turned into detailed patrol routes.So far there have been different approaches in designing police patrol routes.Reis et al. (2006) designed the patrol routes based on Genetic Algorithms, and tested this algorithm using a simulation of a constant number of criminals and police officers as agents moving around an open area.This solution is designed and tested in a simplified scenario and requires substantial improvements before it can be applied to police operational planning.Chawathe et al. (2007) modelled the patrolled road network as an edge-weighted graph, and organized hot-spot police patrol routes based on the importance of segments and topology of the road network.The resulting patrol patterns of this approach are deterministic, depending entirely on crime rate distribution and the topology of the road network, and might be predicted by offenders.Chen et al. (Chen and Yum 2010) developed an algorithm for patrol route planning based on a cross entropy method.This method was developed for single patrol unit planning, and faces challenges when extended to multiple-unit patrolling.Tsai et al. (Tsai et al. 2010) derived a strategy for police resource allocation based on modelling the interactions between police and terrorists as an attacker-defender Stackelberg game.However, this method assumes that a player always predicts his opponent's behaviour and chooses the best response, and may have difficulty in generalizing to large numbers of agents and multiple crime types.
More generally, police patrol can be broadly classified as multiagent patrolling, which is a problem that has been well developed in robotics and related domains.Researchers have developed distinct methods for multi-agent patrolling problem based on a variety of concepts, including a probabilistic ants algorithm (Fu and Ang 2009) and Bayesian strategy (Portugal and Rocha 2013), etc.However, these methods are not directly applicable to police patrol, because of the complexity of implementation (Almeida et al. 2004) and oversimplification of the patrolling environment (static environment, uniform distance, etc.) (Fu and Ang 2009).Moreover, one fundamental assumption underlying most previous methods is that patrolling targets are treated as points or nodes so that there is no time cost in traversing targets (Portugal and Rocha 2013).However, police patrol takes place on street segments with physical length, and traversing segments indeed costs some time.Consequently, a detailed police patrol route design on street segments is urgently needed for efficient policing as daily practice.
In this work, we present a police patrol routing strategy for patrolling road network based on Bayesian strategy and the ant colony algorithm.In Section 2, we formulate the police patrolling problem on road network.In Section 3, we describe the Bayesian Ant Patrolling Strategy (BAPS).In Section 4, we present the result of simulation tests of BAPS and compare it with the benchmark strategy.In Section 5, we introduce the extension of BAPS with varied pheromone decay rate.In Section 6, we present our discussion and discuss topics for future study.

PROBLEM FORMULATION
In this work, the problem of effectively patrolling a road network on foot is studied.The environment is the road network, with certain road segments being identified as hotspots through crime mapping and crime prediction (Ratcliffe 2010) based on historical data.Patrollers move on the road network, and communicate frequently with the control centre.Every time a patroller finishes patrolling a hotspot, he or she sends a message to the control centre, and the control centre calculates and sends back the position of next hotspot.Patrollers have a priori map of the environment and always travel to the next hotspot via shortest path.This procedure is a simplification of real police patrol.A more complex model may involve emergency response and cooperating with mobile patrol, which might be addressed in future study.
Informally, a good strategy is the one that minimizes the time lag between two patrols to the same hotspot and for all hotspots.Several criteria have been suggested to quantify the effectiveness of patrolling strategies, including the idleness of targets, the frequency of visits, or the distance travelled by agents (Iocchi et al. 2011).In this study the idleness of targets is used as the main metric, as it records the duration since last visit, and is intuitive to understand and analyse.To explain this metric, some important variables use in this work will be defined.Thus the average idleness of a hotspot i h at time t is defined as: ii hh

I t t t C
(3) where The global average idleness of all hotspots, represented as One assumption from other works (Chevaleyre 2004;Machado et al. 2002) is used in the beginning of patrolling, namely that for any hotspot

It
, as if every hotspot had just been visited when patrol started.Consequently there is a transitory phase when the global average idleness tends to be low.For this reason, the final () It value is evaluated after convergence in the stable phase, as will be seen below.
Considering a patrol path as an array of hotspots, the police patrolling problem may be described as the optimisation problem of finding a set of paths p which visit all hotspots, using a team of R patrollers, with the overall goal of As has been mentioned before, in this work the patrolling route of each patroller is computed online by the control centre according to the state of the system.Patrollers decide the path from current position to the next hotspot, although they always use the shortest paths on road network.

BAYESIAN ANT PATROLLING STRATEGY
This section describes the Bayesian Ant Patrolling Strategy (BAPS).This strategy is inspired by the probabilistic ant algorithm (Fu and Ang 2009) and state exchange Bayesian strategy (Portugal and Rocha 2013).This Bayesian-based model represents the possibility of moving from the current position to any hotspot, based on previous visits of the hotspot, travelling distance, coordination between patrollers, and other factors.For n hotspots, the model is applied independently n times and the final decision of which hotspot to visit next is made via comparison of moving possibility among n hotspots.There are two relevant components of BAPS, namely the pheromone formalization (deposit and decay) and the Bayesian decision model.In the following sections, these two components will be described in sequence.

Pheromone Deposit and Decay
The use of pheromone comes from the ant colony algorithm (Dorigo et al. 1991).The ant colony algorithm is a type of algorithm for solving various optimisation problems by simulating ants' behaviour in seeking food (Dorigo et al. 1991).The ant colony algorithm has been used in multi-agent patrolling algorithm (Fu and Ang 2009;Doi 2013) .
Note that due to the exponential decay process, the decay rate should be chosen to be large enough to avoid extremely low levels of pheromone resulting from decay over a long time.
The combination of pheromone deposits and pheromone decay over time will build a kind of continuous potential field (Parunak et al. 2001) across all hotspots, which will push patrollers towards hotspots with lower pheromone levels.

Bayesian decision model
After patrolling a hotspot, a patroller is faced with a decision stage where it must decide the next hotspot it should patrol, among all n-1 other hotspots.
The probability of moving to a hotspot i h is calculated using the following formula, applying Bayes rule: In the first part of Equation ( 9),

(move( ))
i Ph represents prior knowledge in the problem.For example some special hotspots may require higher visiting frequency than others, which would be represented in this part.In this work, the prior is not used and defined as uniform.
In the second part of Equation ( 9), i h G is the gain of patrolling a hotspot i h at time t, defined as: where R = the number of patrollers and R>1.
Thus, the probability of moving to hotspot i h is given as: where c is constant across all i h .
The next hotspot to patrol is the one with the highest probability: arg max (move( ) | , ) If more than one hotspot has the highest probability, the next hotspot is selected from these candidates with equal probability.

SIMULATION EXPERIMENT
To assess the performance of BAPS and compare it with other strategies, simulation trials using an agent-based simulation were conducted.
Agent-based modelling (ABM) is a simulation technique that seeks to capture how individual behavioural unitsagentsinteract with their environment and with each other, allowing higher-order behaviours and structures to emerge from these interactions (Epstein and Axtell 1996).The model framework used in this work is built in Java, using the MASON simulation toolkit (available at http://cs.gmu.edu/~eclab/projects/mason/).The simulation trial proceeds at a physical scale of 1m 2 resolution, and is updated on a temporal scale of five seconds per step.
To represent the context in which police patrol is conducted, the simulation combines information about the real-world road network and the real crime records.The crime density of one segment is defined as the ratio of the number of crime incidents on the segment to its length.Among all segments, the 311 with the highest crime density and covering 5% of total length are selected as crime hotspots.
Figure 1 shows the road network map of Camden Borough, with hotspot segments highlighted.

Figure 1. Map of Camden Borough
The benchmark test is also conducted using the same environment.Previous research has shown that optimal patrolling can be obtained if all agents follow the same TSP or Hamilton cycle, equally distributed in time and space (Smith and Rus 2010;Pasqualetti et al. 2012).However, this is based on topological representation of the environment in which patrolling targets are treated as points.In this work an adjusted algorithm is adopted to fit the problem that patrolling targets are segments with physical length.The problem to solve, known as the Rural Postman Problem (Christofides et al. 1981), is to find a shortest circuit that traverses a subset of required segments (hotspots) at least once of a connected undirected graph.One well-known algorithm for this problem, the Christofides Algorithm (Christofides et al. 1981), is used in this work as the benchmark strategy.Note that this is a heuristic solution and has been proved that in the case where the underlying network satisfies the triangular inequality property, the performance of this algorithm has a bound of 3/2, which means the performance is bound accordingly: (Christofides Solution)/(Optimal Solution) 3/2(Pearn and Wu 1995).In the following discussion, the patrolling strategy based on Christofides algorithm is referred to as the Christofides Cycle Patrolling Strategy (CCPS).
Both BAPS and CCPS were tested in the above environment with different sizes of patroller groups (18, 30, 48, 60, 72, and 90).Each simulation went on for 11 patrol cycles.The global average idleness is considered to have converged when its value after any patrol cycle converges with no more than 1% difference to that of the previous cycle.
In the BAPS simulation, the distances from current position to hotspots were normalized to the range of [1, 32], a range which was determined experimentally.Without loss of generalization, the amount of pheromone deposit at each visit is set as 1 unit.
As has been mentioned above, the decay rate should be large enough to avoid extremely low pheromone levels.The threshold of pheromone level is set as 0.001 unit, and according to some pilot experiments, the largest value of idleness is smaller than 100000.Thus the decay rate is selected as 0.99993, based on Equation ( 8).Accordingly, M, representing gain saturation, is set as 1000.L, controlling the probability for zero gain, is set as 0.001, as suggested by Portugal & Rocha(Portugal and Rocha 2013).From Table 2, for different patroller sizes, BAPS has lower global average idleness and better performance than CCPS.When team size is relatively small, the difference is noticeable.
The difference becomes smaller as team size rises to 72 or larger.This is possibly due to the increasing possibility of visiting the same hotspots as the number of patrollers increases.Another important aspect of a good patrolling strategy is unpredictability (Yin et al. 2012;Sherman et al. 2014), because predictable patterns in patrolling routes are likely to be exploited by criminals or offenders.In this work, we use the standard deviation of idleness of each hotspot to measure the unpredictability of patrolling routes.Table 2 shows that for over 78% of all hotspots, the average idleness from BAPS is smaller than CCPS, and for 100% of all hotspots, the standard deviation of idleness from BAPS is larger than CCPS.The low idleness deviation of CCPS may be explained by the even distribution of patrollers on the cycle and the same patrolling cycle used by all patrollers.However, low standard deviation indicates observable patterns which can be adopted by smart criminals.By contrast, the high deviation of idleness in BAPS, or the high randomness of patrol routes, would create a perceived "omnipresence" of the police that deters crime in crime hotspots (Sherman and Eck 2002).

EXTENSION OF BAPS -WEIGHTED BAYESIAN ANT PATROLLING STRATEGY
Another advantage of BAPS is its great extensibility.For instance, different decay rates can be adopted to differentiate hotspots.With the other factors fixed, lower decay rates lead to higher gain of patrolling and consequently higher visiting frequency.By setting different decay rates, patrollers are able to pay more attention to higher crime rate hotspots as well as keeping an eye on all hotspots.This variation is called Weighted Bayesian Ant Patrolling Strategy (WBAPS).To test this new functionality, another simulation is conducted, in which hotspots are classified, and a new metric of patrolling performance is defined.The 311 hotspots are divided into five classes (1-5), with Class 1 representing group of lowest crime density and Class 5 highest crime density, and with approximate number of hotspots in each class.Figure 3 shows a hotspots map of different classes.5 shows.The decay rate for Class-1 is the same as the BAPS simulation test, while the other decay rates decrease in order from Class 2 to Class 5.  From Table 6, the new strategy, WBAPS, decreased the weighted global average idleness by about 1.8%, in the cost of 2.4% rise in global average idleness.Specifically, the average idleness of Class-4 and Class-5 hotspots reduced significantly by 4.7% and 9.4% when the WBAPS is used.This is due to the adoption of stratified decay rates.However, the adjustment of decay rate is still an open question for future research, which is related with the spatial distribution and crime rate distribution of hotspots.

CONCLUSIONS
In this work, the Bayesian Ant Patrolling Strategy was introduced to solve the problem of police patrol routing on road networks.The motivation of this strategy is to minimise the global average idleness of all hotspots while avoiding overly predictable patrol patterns.We have shown its effectiveness by agent-based simulation using empirical GIS data and real crime incident data and comparing its performance with the typical cycling algorithm.Moreover, due to its online and probabilistic nature, the strategy can reduce the predictability in the patrol routes.
Future work will be built upon this strategy, aiming at including the relevant dynamics of police activity.In particular, factoring the influence of frequent emergency calls on patrol activity can present a more realistic patrolling scenario, and adjusting the existing strategy to this dynamic environment will vastly improve its usefulness.Research into coordinating patrollers of different types will provide insight into a practical patrolling strategy.Another interesting direction would be customising the patrolling strategy for alleviating specific crime type rather than general crime.
as the metric of patrolling performance and relative change(Bennett and Briggs 2005) taking CCPS idleness as reference value is defined as:

Figure 3 .
Figure 3.Comparison of BAPS and CCPS result.(a) Average idleness of each hotspot.(b) Standard deviation of idleness on each hotspot

Figure 4 .
Figure 4. Hotspot map of different classes Meanwhile the metric called weighted global average idleness is defined, and the weight of hotspots is equal to its class order.The weighted global average idleness is defined as: (Fu and Ang 2009) the probabilistic ants algorithm(Fu and Ang 2009), the introduction of pheromone decay would indicate the frequency and time of recent visits to a location.For this reason we use pheromone trails to mark visits to hotspots in this study.
i h , which is defined in Equation (1).The normalisation is done to avoid local optima where patrollers repeatedly visit vertices that are very close to each other.The normalisation is conducted on the set of distances from the current position to all other hotspots.
(Portugal and Rocha 2013) the state exchange Bayesian strategy(Portugal and Rocha 2013), i h G is a continuous random variable with a probability density function f(g)., and i h S is defined as a discrete variable that represents the number of patrollers intending to visit i h and tracks the intentions of the teammates.Similar to state exchange Bayesian strategy (Portugal and Rocha 2013), the distribution of () i h P S s is defined as: The study area is Camden Borough in Inner London.Roads are drawn from the Ordnance Survey MasterMap Integrated Transport Network Road (ITN) dataset, and are partitioned into individual segments.Locations of crime incidents are drawn from the records of the Call Aided Despatch (CAD) system of the Met Police, and are preprocessed before use.The locations of six police stations, which factor into activities of police officers, are taken from the data provided by the Met Police.

Table 2
presents the simulation result, where

Table 2 .
Global average idleness from simulation with BAPS HIt value in seconds)

Table 5 .
Decay rate for different hotspot classesThis simulation used the same setting as the above simulation except the different decay rate for different hotspot classes, and the result is compared with ordinary BAPS, as showed in Table6.

Table 6 .
Simulation result of BAPS and WBAPS (all values in seconds)