Agent-based Model of Maritime Drug Smuggling. David Tomá²

Czech Technical University in Prague Faculty of Electrical Engineering Department of Cybernetics Bachelor Thesis Agent-based Model of Maritime Drug Smuggling David Tomá² Supervisor: Ing. Ond ej Van k May 23, 2013

ii Aknowledgements I'd like to thank my supervisor, Ing. Ond ej Van k, for his numerous advices during making practical part and writing of this thesis.

Abstract Drug smuggling is a worldwide problem with many negative eects on the society. It is important to understand and predict smuggler's behavior in order to best position patrolling assets and disrupt his supply chain. The goal of this thesis is formalize smuggler's behavior and compare it with real-world data. Smuggler's route planner makes the main component of this thesis, it's formalized as bi-objective route optimizer, weighting route length and risk of interception along the route. Agent-based simulation of the problem is formed. Results show that smugglers can be divided into classes according to their willingness to risk. Keywords: Agent-based model, simulation, bi-objective optimization, drug smuggling, maritime security Abstrakt Pa²ování drog je celosv tový problém s mnoha negativními vlivy na spole nost. Pro co nejefektivn j²í rozloºení hlídek a pro naru²ení pa²eráckého et zce je d leºité porozum t a p edvídat pa²erákovo chování. Cílem práce je formalizovat chování pa²erák a porovnat ho s reálnými daty. Jedním z hlavních prvk je plánova tras, který je denován jako biobjektivní optimalizátor trasy, m ící vzdálenost trasy a riziko chycení v pr b hu plavby. Je vytvo ena agentní simulace tohoto problému. Z výsledk lze pozorovat rozd lení pa²erák do skupin podle jejich ochoty riskovat. Klí ová slova: Agentní model, simulace, bi-objektivní optimalizace, pa²ování drog, námo ní bezpe nost iv

Contents 1 Introduction 1 1.1 Structure of the Thesis............................... 2 2 Problem Description 3 3 Related Work 5 3.1 Agent-based Simulation.............................. 5 3.1.1 AgentC................................... 5 3.2 Motion Planning.................................. 6 3.2.1 Multicriteria Optimization......................... 6 3.2.1.1 Pareto Dominance........................ 6 3.2.1.2 Pareto Optimality........................ 6 3.2.1.3 Scalarization........................... 7 3.2.2 A* Algorithm................................ 7 4 Agent-based Model 10 4.1 Data......................................... 10 4.1.1 Geographical Data............................. 10 4.1.2 Behavioral Data.............................. 14 4.2 Environment Model................................ 14 4.3 Behavior Model................................... 15 4.3.1 Smuggler's Behavior Diagram....................... 15 4.3.2 Patrol's Behavior Diagram......................... 15 4.3.3 Fishermen's Behavior Diagram...................... 16 4.4 Smuggler's Route Planner............................. 17 4.4.1 Distance Model............................... 17 4.4.2 Risk Model................................. 18 4.4.3 Planning as Multi-objective Optimization Problem........... 19 5 Implementation 20 5.1 Environment.................................... 20 5.2 Planner....................................... 20 5.3 Agents........................................ 21 5.3.1 Behavior Structure............................. 21 5.3.2 Smugglers.................................. 21 5.3.3 Patrols.................................... 21 vii

CONTENTS viii 5.3.4 Fishermen.................................. 22 5.4 Conguration / Scenario Set up.......................... 23 5.4.1 Conguration................................ 23 5.4.2 Scenario................................... 23 5.5 Simulation Execution................................ 24 5.6 Reporting...................................... 24 6 Evaluation 26 6.1 Planner Evaluation................................. 26 6.2 Simulation Evaluation............................... 26 6.2.1 Success Rate Dependency on Number of Patrols and Smugglers.... 29 6.2.2 Success Rate Dependency on Number of Patrols and Visibility.... 29 6.2.3 Radar Noise................................. 29 7 Conclusions 32 Bibliography 33 A CD Content 35

Chapter 1 Introduction Contemporary maritime drug smuggling is a pressing concern, requiring continuous eort from international forces. The distribution chain is rich, robust with respect to deployed anti-drug-tracking countermeasures and the smugglers are able to adapt to novel strategies deployed by the international forces. It is important to understand the behavior of the drug smuggler in order to design such strategies that minimize the probability of successful drugshipment delivering. Thesis is focused on smuggler's behavior modeling on the open sea. Agent-based approach is used to capture main decision-making process of drug smuggler when transporting a cargo. We focused on activity in Central and South America, especially smuggling from Colombia and Ecuador to Costa Rica and Mexico. Modeling of illegal sea borne activities is used in another contexts, however, mostly to predict illegal activity in following days of weeks to best position disruptive resources. Researchers in the Naval Research Laboratory (NRL) in Monterey are especially interested in this research. Their work has been so far focused on maritime piracy, e.g., Slootmaker [14] and Hansen et al. [6] describe piracy activity models which employ meteorological forecasts, intelligence reports and historical pirate incidents to predict areas conductive to pirate activity around the Horn of Africa. So far, they have been considering only equation-based models without intelligent reasoning. This research aims to ll this gap. Smuggler's activity is decomposed into a set of principal activities; we design behavior in each of the activity. The main contribution of this thesis is smuggler's route planner, which plans a transit route taking into account both the length of the route as well as potential risk of being caught. We use bi-objective formulation to express this mixed criterion and utilize A* algorithm to compute optimal routes. The results show that for each of the route in the realworld data, there exists a corresponding coecient weighting risk versus the distance taken by the smuggler to transport a shipment. This shows that dierent smugglers are willing to transit the area under dierent risk or that reasoning of every smuggler is dierent, however, we can group smuggler into dierent classes. A part of the work was successully presented on Internation Conference POSTER 2013 [16]. 1

CHAPTER 1. INTRODUCTION 2 1.1 Structure of the Thesis 1. Study the problem of contemporary maritime drug tracking related to drug trade in Middle America Chapter 2 2. Collect data relevant to the operational behavior of drug smugglers and extract main factors having impact on their behavior Chapter 2, gure 2.1 3. Formalize operational decision-making of a drug smuggler on the open sea as a state automata with parameters reecting main factors extracted in the previous point Section 4.3 4. Design a movement planner, capturing movement patterns of the drug smuggler in given area Section 4.4 5. Enrich the model for other actors - patrolling vessels, shing vessels etc. Section 4.3 6. Implement the environment model and drug smuggler behavior model in the multiagent simulation AgentC Chapter 5 7. Compare the outputs of the simulation (simulating the modeled drug smuggler) with the real-world data. Chapter 6, gure 6.4

Chapter 2 Problem Description Drug smuggling is a worldwide problem with many negative eects on the society. For a specic group of people it represents very high (and often only one possible) stable income and it is improbable the problem will disappear in following decades. It is possible to tackle the problem on many levels, in this paper, we focus on disruption of the drug supply chain lead on the open sea. Specically, we aim to model the behavior of a drug smuggler which is transporting a shipment from one harbor to another over sea. This model should aid the decision makers in the area to best position their patrolling assets. As a test case, we have selected frequent drug shipping route leading from South American statesespecially from Ecuador and Colombiato Mexico and Costa Rica. The situation in Central America region in 2007 is depicted on Figure 2.1. The model presented in this paper captures behavior in the southern part of the region, i.e. the East Pacic Ocean 1. The problem of catching smugglers consists in size of open sea so using of high-quality is necessary. All types of ships are used for smuggling, from small high-speed cigarette boats to large commercial cargo carriers, [5]. More and more smugglers use direct route to Mexico and the US west coast which are more expensive from them. It also requires largers (and slower) vessels and better electronic equipment. Recently smugglers improve their go-fast boats to be able to carry more drugs and to be less visible on patrol's radars. Aune [1] depicts smuggler's strategies and gives us a description of the situation. We can observe that the smugglers do not choose the shortest path, but rather sail further from the coast. This is caused by presence of international forces trying to suppress the activity in the region. The smuggler thus weigh distance of the route taken and increased risk of being caught when sailing closer to the shore. Smugglers also use commercial carriers in their legitimate trade; drugs are hidden among the cargo. Sailing conditions for every ship are inuenced by weather which decreases their speed and make visible worse. Rain, fog, darkness and others are perfect for smuggling. Police ships also have a problem with shermen who burden their radars with unnecessary and unwanted entries. The aim of this thesis is thus capturing the behavior of the smuggler in form of an agent optimizing it's utility function in terms of distance sailed and risk taken. To solve this 1 Picture shows 155 samples in South America in 2007, http://www.cipcol.org/?p=590. 3

CHAPTER 2. PROBLEM DESCRIPTION 4 Figure 2.1: Figure discribing maritime smuggling activity in Caribbean Sea, Gulf of Mexico and East Pacic Ocean, specically coast of Mexico, Guatemala, El Salvador, Nicaragua, Ecuador and Colombia problem we dene the environment as a set of obstacles representing borders between sea and land, set of harbors as origin and target smuggler's destinations and some space for sailing ships which restricts their movement on open sea.

Chapter 3 Related Work 3.1 Agent-based Simulation Klügl [9] describes basic denition of agent-based simulation as follows: Multi-Agent Systems provide a description framework that is appropriate for many real world systems consisting of a set of interacting autonomously deciding actors. Human and animal societies form prominent and intuitive examples for real-world multi-agent systems. Multi-agent system has four important parts which describe it perfectly. The parts are: Agents Multi-agent System Interations Environment The agents interact in environment and form multi-agent systems [9]. Agent's denition is dened in [4]: An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to eect what it senses in the future. We can consider environment as all parts which are in interactions with agents. 3.1.1 AgentC Project AgentC is a agent-based simulation framework used in maritime domain developed in ATG 1. It's focused on analysis of maritime domain data, modeling and predicting vessels behavior, planning of optimal routes of long range transport vessels which goes to sailing risk reducing. It oers method for creating agents including behavior model and agent's interactions with environment, which can include weather model. Simulation run can be displayed in Google Earth 2 which allows seeing actual situation with results. 1 Agent Technology Center in Faculty of Electrical Engineering, Czech Technical University in Prague. http://agents.fel.cvut.cz/projects/agentc/ 2 http://www.google.com/earth/index.html 5

CHAPTER 3. RELATED WORK 6 Figure 3.1: Figure from [9] shows relationship between real and simulates world. 3.2 Motion Planning 3.2.1 Multicriteria Optimization Talbi [15] denes multicriteria optimization problem as: Let be x a vector of decision variables from a set of feasible solutions S, then function F(x) = f(x),f2(x)...) is the vector of objectives to be optimized. { min F (x) = (f1 (x), f MOP = 2 (x),..., f n (x)) s.c. x S The search space S represents the decision space or the parameter space of Multicriteria optimization problem. The space in which the objective vector belongs to is called the objective space. The vector F can be dened as a cost function from the decision space in the objective space that evaluates the quality of th solution (or tness), [15]. 3.2.1.1 Pareto Dominance Let be v = (v 1,..., v n ) a vector and u = (u 1,..., u n ) an objective vector. We say u dominate v if no element from v is smaller than corresponding element from u and at least one element from u is strictly smaller. [2] 3.2.1.2 Pareto Optimality We have Pareto optimal solution x S, where every F (x) does not dominates F (x ); x S. Graphically the optimality can be represented as shown in Figure 3.2, e.g. there is no complement in the box restricted by values x and y for every parete optimal element [15].

CHAPTER 3. RELATED WORK 7 Figure 3.2: Figure from [15] shows graphical representation of pareto optimality. 3.2.1.3 Scalarization Transformation of a MOP problem into a monoobjective one. Aggregation Method is one of the most used for generating Pareto optimal solutions [3]. Talbi [15] explains it as follows: It consists in using an aggregation function to transform a MOP into a moobjective problem by combining the various objective functions f i into a single objective function f generally in a linear way: f(x) = n λ i f i (x), x S (3.1) i=1 lambda i are weights in interval [0...1] with sum n i=1 λ i = 1. If lambda i > 0 for every i, than the solution is Pareto optimal. 3.2.2 A* Algorithm A* is computer algorithm used to nding a path in a graph with given start and end points [13]. It uses a best-rst search to nd optimal path from point A to B; optimal means the shortest, cheapest, etc., according to edge weight represenation. Function f(x) is a function which evaluates individual nodes to determine an order, in which these nodes will be visited. Function f(x) is composed of these 2 fuctions [10, 11]: cost function g(x)distance between start and given point

CHAPTER 3. RELATED WORK 8 heuristic function h(x)the best predicted distance to end, it must admissible (distance to target must not be overestimated), e.g. straight-line distance to target. and is given as: Pseudocode below shows how A* works 3. f(x) = g(x) + h(x) (3.2) init Priority Queue; init closed list; put start point to the queue; while open list is not empty do pop (q) o the queue; generate q's successors and set their parents to q; foreach successor do if successor == goal then return; end successor.c = q.c + distance between successor and q; successor.h = distance from goal to successor; successor.f = successor.g + successor.h; if queue contains node with the same position and with lower f than successor then break; end if closed list contains node with the same position and with lower f than successor then break; end put the node to the queue; end push q on the closed list; end Priority queue of open nodes is initialized. Lower value of f(x) means higher priority for given node x. In each step we take a node from queue and compute values f for its neighbor nodes, which are lately added to the queue. Algorithm ends if the rst node in queue is the end one, or when queue is empty. A* is complete algorithm; it will always nd a solution if there is someone. It's also optimal if: closed list (list of visited nodes) is not used 3 http://web.mit.edu/eranki/www/tutorials/search/

CHAPTER 3. RELATED WORK 9 closed list is used and heuristic function h(x) is consistent; for any pair of nearby nodes a, b: where d(a,b) is distance between a and b. h(x) h(y) + d(a, b) (3.3)

Chapter 4 Agent-based Model To properly model the behavior of the smuggler and evaluate it inside a simulation framework, we use agent-based approach, decomposing the scenario into an agent model and an environment model. 4.1 Data We split data also on the agent and the environment ones. To create agent's ship, we need to know its speed and radar visibility, envirnoment is made from data which dene its shape (polygons) and harbors as smuggler's destinations. 4.1.1 Geographical Data Environment is composed of obstacles (or borders), navigable waters and harbors. We use Google Earth to create these parts as a set of les with ".kml" extension 1. Borders are comprised of a set of polygons dening a land. If we set these polygons too accurate, too more edges will be added to visibility graph, and it slows down both creating graph and algorithm for planning routes. So it is necessary to choose compromise between precision and speed. Figure 4.1 represents polygons in their positions. We dene harbors as a GPS points lying on the borders of above mentioned polygons. Either harbor can be origin or destination. We are inspired by the data from Figure 2.1 because exact locations are not known. Figure 4.2 shows estimated harbors used in simulation. We use followings harbors as the origin ones (from the South), in Ecuador: Data de Posorja 2 43'14" J, 80 18'20" Z San Mateo 0 57'26" J, 80 48'28" Z and in Colombia: Tumaco 1 49'43" S, 78 47'39" Z 1 https://developers.google.com/kml/ 10

CHAPTER 4. AGENT-BASED MODEL Figure 4.1: Borders (yellow) as a polygons set in Google Earth. Isla del Gallo 1 59'19" S, 78 40'16" Z El Aticito 3 20'49" S, 77 33'53" Z Punta Bonita 3 35'10" S, 77 20'43" Z Punta Soldado 3 45'50" S, 77 11'5" Z Virudó 5 23'38" S, 77 25'36" Z Nuquí 5 44'17" S, 77 28'2" Z Destination harbors in Central America are only 3, (from the East) in Costa Rica: Tambor 9 39'38" S, 85 0'41" Z and in Mexico: Salina Cruz 16 3'43", 95 6'9" Z Galera 15 53'5" S, 97 41'37" Z 11

CHAPTER 4. AGENT-BASED MODEL 12 Figure 4.2: Origin and destination harbors for smuggler's routes Navigable waters together with borders form bounded part of the Earth. We restrict navigable waters because of optimization. This work is focused on Carribean, making graph in the South America is pointless. Figure 4.3 Environemnt is enriched with Weather model describing weather situation in given location using Esri grid. In this simulation there are wave height used. It in uence following quantities with some weight: Speed Current ship's speed in knots is decreased by half the waves' heights (in meters) in actual ship's position. Visibility One-meter wave reduces patrol's visibility by 5 percent. Esri grid is special le format used in Geographic information systems developed by Esri company. It's a tool for describing features of certain geographic space. Grid is localized by x and y coordinations determining left bottom corner of this grid. As other usual grids it has xed number of rows, columns and size of each square cell. For forming non-square shapes NODATA value is established. It tells us there is no data to use on certain grid position. Esri le example below de nes data from Figure 4.4: ncols 6 nrows 6

CHAPTER 4. AGENT-BASED MODEL 13 Figure 4.3: Navigable waters composed of visibility graph enriched with risk model heat map. The darker the heat map, the riskier the area. xllcenter -101.0 yllcenter -6.5 cellsize 0.25 NODATA_value 999 6 6 4 4 2 2 6 4 4 2 2 2 4 4 4 2 2 2 4 4 2 2 0 0 4 4 2 2 0 999 4 2 2 0 999 999 ncols and nrows are number of columns and rows, xllcenter and yllcenter denotes x and y coordinates of left bottom table cell, cellsize means cell's dimension in degrees and N ODAT A_value is above mentioned feature. Values below these properties are table data.

CHAPTER 4. AGENT-BASED MODEL 14 Figure 4.4: Esri grid visualization example. Left image shows graphic projection and the right one shows the values. Because we do not have any weather data from this location, we use (and customize) data from Indian Ocean oered by NRL 2. 4.1.2 Behavioral Data Patrol's agents are equipped with Rigid Hull Inatable Boat (RHIB) 3 with average speed equal to 10 knots for longer endurance. Smuggler sails on ships called cigarette boat (Gofast boat) 4 with speed of 9 knots because of endurance too (shortime max speed is about 30 knots in local conditions). US Navy also uses helicopters. In this simulation helicopters are not implemented, so the advantage of using them is substitute with increasing patrol's ship speed on 10 knots. 4.2 Environment Model Environment model denes the space where agents can move. The navigable waters are dened by a set of obstacles represented by polygons; the agent can move anywhere except inside of the obstacles. Additionally, we dene a set of origin and destination harbors on the borders of the obstacles. The smuggler then moves between any origin harbor and any destination harbor. As mentioned above weather model in esri format is used. Weather is given in strictly given square areas with their latitude and longitude position as described in Figure 4.4. Each cell has its value of wave height. Between neighbor cells there is no interpolation, because the values are quite low (from 0 to 6) and accurate (on the ten-thousandth), so no interpolation 2 Naval Research Laboratory in Monterey 3 US Navy ship http://www.navy.mil/navydata/fact_display.asp?cid=4200&tid=2200&ct=4 4 High speed boat used for smuggling http://www.london-research-and-development.com/gofast.pdf

CHAPTER 4. AGENT-BASED MODEL 15 is needed. Agents nd out a weather information in each step while they are sailing. They get wave height in their current position and adjust the speed according to it. 4.3 Behavior Model Section is about modeling behavior of all types of agents. We will describe all the states and their reactions on events. 4.3.1 Smuggler's Behavior Diagram We decompose smuggler agent's behavior into following principal activities, captured in Figure 4.5: Waitthis is the initial activity of the agent. In this state, the smuggler chooses its destination harbor, having the origin harbor assigned during the initialization phase. Plan a routethis activity captures the decision-making process of the agent, i.e. route planning. This activity encapsulates route planner, main component of the model and main contribution of this work. Sail to destinationagent is following the route computed by the planner. average speed is given from conguration. His Reach destinationagent has reached the destination and he re-plans to sail home, now using only a simple planner not taking into account the risk component as he does not transport any shipment. Caughtthe agent was caught during the sailing phase by international forces. This is the terminal state of the agent and the agent is removed from the simulation. Sail homewhen sailing home, the agent does not consider the risk component anymore as moves legitimately over the ocean. After the home location has been reached, the cycle is repeated by switching to the Wait state. 4.3.2 Patrol's Behavior Diagram Diagram Figure 4.6 determines patrol's behavior activities: WaitThis activity is the initial one (as in smuggler's behavior). Patrol gets its starting (random) position in given area. There can be more patrol's in the same position. Static patrolpatrol holds its position and looks for smuggler ships to chase them. Patrol predicts smuggler's position to optimize chasing strategy. Duration of this state is set to 8 hours if no smuggler appears.

CHAPTER 4. AGENT-BASED MODEL 16 Figure 4.5: Figure describing smuggler's behavior. Star (*) denotes initial state and plus sign (+) is denotes terminal state. Change patrol positionagent takes the straight (shortest) path to his next static position while he is still looking for smugglers. He also predicts position to optimize. ChaseIn patrol's visibility range smuggler agent appears. Patrol tries to predict his location to catch him as soon as possible. Chase timeout is dened to avoid following uncatchable smugglers. Smuggler is caught when patrol gets close to his ship, exactly on distance of 100 meters. Patrol is predicting when it is at minimal distance of 1 nautical mile from target. Else it sails straight on him. Smuggler caughtpatrol is now searching smuggler's ship. 4.3.3 Fishermen's Behavior Diagram Fishing agent behaves according this simple diagram: WaitAgent is initializing his origin harbor and he is ready for next action. Sail to destinationrandom destination is choosen and agent is moving there. He goes along the shortest route, there no more complicated planner to create indirect route. FishingBoat anchored for 24 hours.

CHAPTER 4. AGENT-BASED MODEL 17 Figure 4.6: Figure describing patrol's behavior. Star (*) denotes initial state. 4.4 Smuggler's Route Planner The planner plans a route in navigable areas dened by polygonal obstacles. In the preplanning phase, a visibility graph is constructed. Visibility graph transforms the continuous space with potentially innite number of voyages into a computationally tractable form without the loss of expressiveness. Vertexes in the visibility graph represent known navigationaly signicant points and edges represents elementary shortest paths between these points. [7] The visibility graph 4.3 captures the distance-minimization component of agent's planning process. To take into account the risk component, we dene an area, where the smuggler can be caught. The area is dened by a polygon, which is discretized into a grid, driven by a discretization step parameter, determining granularity of the grid. The boundary cells of the grid are connected to the visibility graph. 4.4.1 Distance Model To measure the length of an edge in the visibility graph, we use Haversine formula [12], giving great-circle distance between any two points on sphere 5 For an angle θ, the Haversine formula is computed as: H (θ) = sin 2 ( θ 2 ) = 1 cos(θ) 2 (4.1) 5 We use spherical approximation of the Earth. For a more precise distance calculation, Vincenty's formulae can be used.

CHAPTER 4. AGENT-BASED MODEL 18 Figure 4.7: Figure describing sherman's behavior. Star (*) denotes initial state. For two points p 1 (φ 1, λ 1 ) and p 2 (φ 2, λ 2 ) dened by their latitudes φ and longitudes λ, the Haversine formula takes the following form: H(p 1, p 2 ) = H(φ 1 φ 2 ) + cos(φ 1 )cos(φ 2 )H(λ 2 λ 1 ) (4.2) The distance dist(p 1, p 2 ) between two points p 1 and p 2 is then computed as: ( ) dist(p 1, p 2 ) = 2r arcsin H(p1, p 2 ) (4.3) Finally, we dene length of a route x in the graph with edges (p i, p j ) as: 4.4.2 Risk Model dist(x) = (p i,p j ) x dist(p i, p j ) (4.4) To capture the risk, we propose the following risk model, which is a function of distance from the nearest coast line: r(p) = 1 dist(p, b) + 1 (4.5) where r is risk value at point p and dist(p, b) is distance from point p to the nearest obstacle point b. To be able to assign a risk to an edge, we dene risk taken between any two points: r(p 1, p 2 ) = r(p 1) + r(p 2 ) dist(p 1, p 2 ) (4.6) 2 i.e., risk value for an edge on graph given by 2 points p 1 and p 2 is the arithmetic mean of risks at those points, multiplied by length of the edge (p 1, p 2 ) (so that the risk is proportional to the time spend by traversing the edge).

CHAPTER 4. AGENT-BASED MODEL 19 Finally, we dene the risk along a route x in the graph with edges (p i, p j ) as: r(x) = (p i,p j ) x r(p i, p j ) (4.7) 4.4.3 Planning as Multi-objective Optimization Problem We use formalization of multi-objective optimization to describe the problem of nding an optimal smuggler's route, as the smuggler weights two criteriathe length of the route taken and the risk taken along the route. The multi-objective optimization problem is dened over a set of functions. It is possible to use linear scalarization to express the criterion in the optimization problem as a single function: min x X k w i f i (x) (4.8) i=1 where we assign a weight w i to each function f i (x). In our case we solve bi-objective optimazition problem (having distance and risk); our linear scalarization is thus following: x = arg min dist(x) + α r(x). (4.9) x X The sum of route length dist(x) and risk r(x) are weighted by a parameter α capturing the risk aversion (i.e. the willingness to risk) of the smuggler (if α is small, shorter routes with higher risk are preferred). X is set of all possible paths and x is the optimal path minimizing the criterion function. We use A* algorithm [13] to nd the optimal route x. A* algorithm is dened by a cost function g and a heuristics function h. A* uses evaluation function f(p i ) in form of: f(p i ) = g(p i ) + h(p i ) (4.10) where g(p i ) is cost function and h(p i ) is heuristics. The cost function is given as: g(p i ) = dist(home, p i ) + r(p i ) α (4.11) where dist(p i ) is distance from home harbor to p i, r(p i ) is risk along the edge from p i 's parent and α is risk weight coecient. The heuristic function h(p i ) is dened as the direct distance from p i to the destination harbor: h(p i ) = dist(p i, destination) (4.12)

Chapter 5 Implementation Both environment and agent model are implemented within the AgentC framework [7, 8], which oers features to model smuggler's behavior, allows simulating various scenarios and displaying the results in Google Earth. 5.1 Environment Environment consists of followings parts: Geographical data provider or GISDataProvider class; object with all geographical data for planning, behavior etc. It stores all sailing zones and harbor lists and provides methods for getting random point in a range or returning a harbor from list. The rst method is used by patrols, smugglers and shermen use the second one. Vessel data provider This class encapsulates all the informations and services around ship traces. It's mostly used to get all vessels in given distance and to get an information about certain vessel. Agent container basic container containing all the agents, manages simulation steps and eventreporting. For all its saved agents it starts method simulatestep(long stepduration), which execute agent's current state for given time quantum stepduration. 5.2 Planner As mentioned before planner makes routes on visibility graph. To avoid permanent graph making before each simulation, there is a cache used to save and restore it. New cache le is created when graph is changed. There can be many saved graph with dierent properites. Cache saves all important data which can be used later, e.g. harbors location, borders, etc. 20

CHAPTER 5. IMPLEMENTATION 21 5.3 Agents This section describes agent's behavior implementation structure and important part of the behavior states. 5.3.1 Behavior Structure The substancial part of agent's behavior are classes FSMRealisticSmuggler, FSMCaribbean- Army and FSMFishermen which extend abstract class AbstractFiniteStateMachine. These classes are responsible for states management, behavior diagrams are made here. States are successors of abstract class AbstractState containing method simulatetimestep (long deltat), which evaluates given behavior in time quantum deltat and returns the remaining time after all state steps. If this time is greater then 0, actual state is done. 0 means given task is not complete yet and it continues. 5.3.2 Smugglers On the beggining origin and destination harbors are generated for each smuggler. Next step assigns FSM structure appropriate with behavior diagram. There are 2 main states in smuggler's behavior. The rst one is StateSailToDestination which is responsible for sailing loaded and the second one, StateSailHome, oers plan for sailing home, to origin harbor, unloaded. StateSailToDestination generates smuggler's risk weight, number from 0 to 1, which decides the trace character and move with ship on the given created trace by PlanMover oering method move(long deltat, double speed). This method moves the ship with average speed speed for time interval deltat. StateSailHome is similar to StateSailToDestination except for generating risk. When smuggler's ship is unloaded, it takes the shortest route home. 5.3.3 Patrols In addition to FSM structure patrol agents get only their rst position on the open sea when they are initialized. Class StateSailToDestinationArmy is the rst action state in patrol's behavior diagram. It generates destination position in given maximum distance and moves the ship there. While sailing it looks for smuggler ship in its radar range. When smuggler ship appears current state set target to this ship, reports event and ends. If more than one ship are in range, method ndsmuggler chooses the best choice for chasing according to the closest distance from the ships in given time step. It predicts smugglers locations by their speed and heading. It assumes ship straight trace, beacuse it's impossible to predict the location with direction changes. If there no smuggler ship found, state waits for reaching destination. StateStaticPatrolArmy is state when ship is anchored for given hours and checks its surroundings. The same method for looking for smugglers is used, as in previous state to optimize chasing strategy.

CHAPTER 5. IMPLEMENTATION 22 Figure 5.1: Figure indicates predicting proces, where blue mark is a patrol ship and the red one is a smuggler. The circle around is patrol's radar range, green dots show predict positions in dierent time intervals. Patrol chooses the one with red line as target destination. StateChaseArmy - patrol pursues set target with predicting its next position to catch him as soon as possible. Predicting is little bit dierent from the one above in states State- SailToDestinationArmy and StateStaticPatrolArmy. On Figure 5.1 the predicting process is sketched. Patrol (blue) estimates a set of uniformly distributed points (lying in a line) in maximum distance equal to distance between patrol and smuggler (red). Then the point closest to smuggler, where patrol ship will be earlier than the smuggler one, is choosen and set as destination point in current step. 5.3.4 Fishermen Fishermen are based on only two states. Their behavior aren't needed to be complicated because they don't react on any events. The rst step is assigning home harbor to each ship. Main behavior is the same as the patrol's in states StateSailToDestinationArmy and StateStaticPatrolArmy, e.g. random point in open sea is generated and set as destination point to shing. Specically StateSailToDestinationFish and inbuilt state called StateWait which is simple implemented state for waiting for given time.

CHAPTER 5. IMPLEMENTATION 23 5.4 Conguration / Scenario Set up Conguration and scenarios inuence whole simulation. Scenario le have turned particular modules on or o before simulation starts, congurations store constants for using them in code. 5.4.1 Conguration Their extension is in form ".conf" and inside they have its specic tree structure which divide data to sections and subsections to make it transparent. All type of data types can be used. Behind double slash there can be commentary used. Following code shows short conguration example: module.smugglers = false agent { smugglers { origins_harbours_file = "data/agents/smuggler/harbours.kml"; number = 10; avg_speed = 8.0 // Smuggler average travel speed in knots graph { grid_size = 50 // Planning area grid size in km graph_construction = true; // make kml file } } } To get values from conguration, CongReader is used. It's embedded part of AgentC framework. It oers functions for getting dierent data types, which have common parameter part where the wanted item is written by dot notation. From example above, getting value number looks like: int number = AisConfig.getConfig().getInt("agent.smugglers.number"); It follows the conguration by exception paradigm [7]. We dene default conguration data in.conf les and scenario 5.4.2 settings which can update some values from default conguration. Figure 5.2 shows the principal of this paradigm. 5.4.2 Scenario Application starts with scenario chooser which turn set modules on or o and determines future application character. Extension ".groovy" is typical of scenario le and it denotes groovy language 1. Groovy is na object-oriented programming language for Java, and it oers also scripting syntax for this platform. 1 http://groovy.codehaus.org/

CHAPTER 5. IMPLEMENTATION 24 Figure 5.2: Figure shows principal of conguration by exception. conguration are changed by scenario. Default values set in module.smugglers = true module.carmy = true module.fishermen = true agent.transport.long_range.number = 1; Code example above shows groovy le structure (dot notation, same as conguration) and updating default values from conguration. 5.5 Simulation Execution The simulation is step-based, i.e., it runs in steps dening temporal discretization of the model. In each step, the agent is given a time quantum to execute its actual state and perform an action. Simulation duration is set in conguration and it gives a total number of steps to execute in simulation. A step represents a time interval. Duration of this interval is the same for all steps and it is given in seconds. 5.6 Reporting During simulation all agents (except shermen) react on events which can change their current state or do some action, for example when smuggler is caught, he receives an event message changing his state. Agent includes event queue based on waiting for incoming events and follow-up processing of them. After sending event a note appears in visualization timeline so we can observe the simulation progress. Next part of reporting is saving simulation logs used for evaluation. Patrols and smugglers prints their results, each agent to his own le. Pros of saving to many les is that progress of temporary result can be watched, slower simulation speed is cons.

CHAPTER 5. IMPLEMENTATION 25 Smugglers update their les when they change the state to sail home. Their results are total number of journeys started with goods (i.e. number of state "StateSailToDestination" initialization) and number of successfully reached destinations. Patrols save their stats from radar, i.e. number of seen smugglers and shermen. Values are saved before start of states of static patroling, sailing and chasing.

Chapter 6 Evaluation 6.1 Planner Evaluation First, we evaluate the properties of the planner, i.e., we examine the trade-o between risk and distance taken. Second, we integrate the agent model into the simulation and to nd set of α coecients representing main behavior patterns of real-world smugglers. Figure 6.1 captures the trade-o between the risk and distance taken in form of a Pareto frontier for a number of routes between dierent origin-destination harbor pairs 1. For each pair, the Pareto frontier was constructed by varying the α parameter from 0 to 10000. Note that when increasing the value beyond 10000, the route was not dierent anymore due to boundaries of the risk graph. Having the Pareto frontier, we can ask, what α value corresponds to the real-world smuggler behavior. We executed the simulation with various α parameters to capture the behavior of the real-world smuggler, as depicted on Figure 2.1. Figure 6.4 shows result of the simulated behavior matched against the data overlay. Routes on this gure are representatives of the 3 classes of smugglers. Resulting risk weight coecient α was found to be (respectively from the North): 0, 1000 and 100000. This matching quality of the planned route was evaluated by visual comparison only. 6.2 Simulation Evaluation Second part of evaluations is based on running simulations with dierent parameters and nding result dependency on them. A nal result with certain parameter is measured as an arithmetic mean of results from 10 simulations. Because there is too many parameters which can be variable and make big dimension of results (unreal to simulate all of them), we xed some parameter and measure only in two dimensions. Simulation duration is 100,000 steps of 500 seconds, it's about 579 days. 1 For brevity, the harbors are denoted by their country only. 26

CHAPTER 6. EVALUATION 27 Figure 6.1: Pareto frontiers for a few dierent routes: Route 1) from CO to MEX. Route 2) from EC to MEX. Route 3) from EC to MEX. Route 4) from EC to MEX. Route 5) from CO to MEX. Route 6) from CO to MEX. Figure 6.2: Routes from Pareto frontiers 6.1 with risk weight α = 0 (direct routes). Route colors are identical with the ones in Pareto frontiers (best viewed in color).

CHAPTER 6. EVALUATION 28 Figure 6.3: Figure with generated routes with dierent risk weight α ; from the North 0 α 1000000

CHAPTER 6. EVALUATION 29 Figure 6.4: Result of the simulation with representatives of each class of smugglers which are displayed on the background. Generated routes almost exactly imitate the ones from data. 6.2.1 Success Rate Dependency on Number of Patrols and Smugglers The crucial observation is focused on success rate of smuggler's drugs delivery depending on number of patrols and smuggler ships. We xed visility and number of shermen. Visibility is equal to 50 nautical mile, shermen are 15. Graph on Figure 6.5 shows how success rate changes only when number of patrols is dierent. Number of smugglers does not aect total result because the more smugglers are there, the more crowded the space is. That goes to bigger patrol's eciency. On the other side, the more patrols we have, the more expensive the operation is. E.g. if we want to catch 60% of smugglers, we need 10 vessels to patrol the area. 6.2.2 Success Rate Dependency on Number of Patrols and Visibility The view of the situation with variable visibility is dierent. Better quality of ship equipment decrease smuggler's success rate and the same higher number of patrols. On Figure 6.6 the biggest fall is evident when visibility raises. It's better to equip vessels with superior equipment than deploy more patrol vessels. 6.2.3 Radar Noise Last evaluation is about comparing seen shing and smuggler ships after whole simulation. Fishing ships on patrol's radar make it unclear, which negatively aects patrol's eciency. It's necessary to lter shing ships and focus only on the smuggler ones.

CHAPTER 6. EVALUATION 30 Figure 6.5: Chart shows smuggler's success rate depending on number of smugglers and patrols; visibility is 50 nautical mile. Figure 6.6: Dependency on visibility and number of patrols with xed number of smuggler equal to 10.

CHAPTER 6. EVALUATION 31 Figure 6.7: Figure with mean number of smugglers and shermen who were seen by a patrol vessel during simulation. Left image shows number of smugglers which is lineary dependent on visibility. The right one represents the same with shermen. Here the total number dependent both on visibility and number of shermen in simulation. In simulation we xed number of patrols and smugglers to 10. Visibility is set from 10 to 50 nautical miles and number of shermen from 5 to 15. Results of these simulations are shown on Figure 6.7, where is evident linear dependence of total number of seen shermen and smugglers on vessel's visibility. Number of shermen has only a little eect on the results.

Chapter 7 Conclusions The goal of this thesis was to specify smuggler's behavior in Central and South America and to try model this behavior in simulation, where envirnoment contains patrolling and shing vessels and weather model. I studied the problem of smuggling in America and create a conceptual model of smuggler's behavior. It was necessary to nd out important data which can have eects on simulation progress, such as what boats smugglers use, how they sail to their destination, etc. From available data I determined smuggler's strategy which was later implemented into AgentC framework as a behavior structure which is designed as diagram. I dened risk weight α, which represents smuggler's willingness to risk. Smuggler's routes planner was the main component, which optimizes routes according to distance and risk. I used A* algorithm to solve this multi-objective optimization problem. Result of this optimization is Pareto Frontier which shows optimal route for all risk weithts. From simulation results I nd out we can divide smugglers into 3 classes according to their routes. The most risky smugglers belong to the rst class, they sail direct to their destination harbor with maximum risk. Smugglers, who are more careful, take longer routes further from coast and try to reduce the risk. The last group (class) represents smugglers taking the longest route at the expense of risk. I tested dependency of number of patrols and smugglers on total smuggler's delivery success. Results say to reduce smuggler's success rate we need to use more patrols or improve their equipment, especially radars. This work has certainly some limits, for example we did not consider smuggler's reactions on patrols, which could have inuence on their planning, so results would be dierent, we can use another attitude of this problem, instead of agent-based, we could involve cooperation of smugglers or we could add police helicopter etc. 32

Bibliography [1] B. R. Aune. Maritime drug tracking: an underrated problem. on-line, 1990. http://www.unodc.org/unodc/en/data-and-analysis/bulletin/bulletin_ 1990-01-01_1_page008.html. [2] Y. Collette and P. Siarry. Multiobjective Optimization: Principles and Case Studies. Decision Engineering. Springer, 2003. [3] G. Eichfelder. Adaptive Scalarization Methods in Multiobjective Optimization. Vector Optimization. Springer-Verlag Berlin Heidelberg, 2008. [4] S. Franklin and A. Graesser. Is it an agent, or just a program?: A taxonomy for autonomous agents. In Proceedings of the Workshop on Intelligent Agents III, Agent Theories, Architectures, and Languages, ECAI '96, London, UK, UK, 1997. Springer- Verlag. [5] M. Gonzales, K. McEnery, T. Sheehan, and S. Mellody. America's Habit: Drug Abuse, Drug Tracking, and Organized Crime : President's Commission on Organized Crime. Diane Publishing Company, 1998. [6] J. Hanson, L. Hsu, J. Dykes, J. Dastugue, R. Allard, C. Barron, M. Abramson, S. Russell, and R. Mittu. Information domination: Dynamically coupling METOC and INTEL for improved guidance for piracy interdiction. In NRL Review, pages 109115. Naval Research Laboratory, 2011. [7] M. Jakob, O. Van k, B. Bo²anský, O. Hrstka, and M. P chou ek. Adversarial Modeling and Reasoning in the Maritime Domain Year 3 Report. Technical report, ATG, CTU, Prague, 2012. [8] M. Jakob, O. Van k, and M. P chou ek. Using Agents to Improve International Maritime Transport Security. IEEE Intelligent Systems, 26(1):9096, january 2011. [9] F. Klügl. Agent-based simulation engineering. online. http://www.oru.se/pagefiles/ 15636/mas_pdfs/kluegl_habilitation.pdf. [10] S. LaValle. Planning Algorithms. Cambridge University Press, 2006. [11] G. Luger. Articial intelligence: structures and strategies for complex problem solving. Pearson education. Addison-Wesley, 2005. 33

BIBLIOGRAPHY 34 [12] C. C. Robusto. The cosine-haversine formula. The American Mathematical Monthly, 64(1):3840, 1957. [13] S. J. Russell, P. Norvig, J. F. Canny, J. M. Malik, and D. D. Edwards. Articial intelligence: a modern approach, volume 2. Prentice hall Englewood Clis, NJ, 1995. [14] L. Slootmaker. Countering piracy with the next-generation piracy performance surface model (master thesis). Technical report, NPS, Monterey California, 2011. [15] E.-G. Talbi. Metaheuristics : from design to implementation. J. Wiley & Sons, Hoboken (N.J.), 2009. [16] D. Tomá² and O. Van k. Agent-based Model of Maritime Drug Smuggling. International Conference POSTER, 2013.

Appendix A CD Content CD attached with the thesis contains: Runnable simulation \Application Source code \Source code Thesis in PDF \Thesis\tomasda2_BP.pdf 35