Deep learning for pedestrian collective behavior analysis in smart cities: A model of group trajectory outlier detection

This paper introduces a new model to identify collective abnormal human behaviors from large pedestrian data in smart cities. To accurately solve the problem, several algorithms have been proposed in this paper. These can be split into two categories. First, algorithms based on data mining and knowledge discovery, which study the different correlation among human behavioral data, and identify the collective abnormal human behavior from knowledge extracted. Secondly, algorithms exploring convolution deep neural networks, which learn different features of historical data to determine the collective abnormal human behaviors. Experiments on an actual human behaviors database have been carried out to demonstrate the usefulness of the proposed algorithms. The results show that the deep learning solution outperforms both data mining as well as the state-of-the-art solutions in terms of runtime and accuracy performance. In particular, for large datasets, the accuracy of the deep learning solution reaches 88%, however other solutions do not exceed 81%. Additionally, the runtime of the deep learning solution is below 50 seconds, whereas other solutions need more than 80 seconds for analyzing the same database.


Introduction
Our cities and environments are revolutionized by the current information and communication (ICT) technologies including wireless sensing and the Internet of Things (IoT) that enable interconnecting smart objects via machine-to-machine communication protocols as well as through the Internet, advanced machine learning (ML) and knowledge mining approaches, pervasive and mobile computing, and high performance computing (HPC). All of these technologies make our cities and premises smarter, with augmented capacities through cyberphysical systems in which machines and humans interact and act on the environment [1][2][3]. One of the fundamental services needed in many smart city applications is that of automatic human behavior analysis. Human behavior in cities is a complex process on both spatial and temporal dimensions, which is governed by semantic actions, and latent factors [4][5][6]. Human behavior depends on many factors such as

Motivation
This research is inline with the ongoing efforts to use cutting edge ICTs for providing advanced smart city applications, and its aim is to improve modeling human behavior for different smart city applications. The existing smart city systems uses human behavior to enhance smart agents. A human behavior knowledge base can be collected from different sources using sensorial data, and Internet of Things (IoT) technologies, social media tools such as Twitter, Facebook, Flickr, Instagram, etc, which enable different representations, including texts, images, videos, time series, and trajectories.
Our goal in this paper is to design a new framework of modeling human behavior, which integrates data mining and deep learning models to understand and analyze human behavior better. The strategy used in this work is based on collective abnormal human behaviors. The smart agents might also interact with the current human behavior knowledge base for decision making. For instance, if a transportation agent detects a grouping of anomalous trajectory of people in some given circumstance, (e.g., low number of Bikes for a given day), then the agent could interact with the human behavior knowledge base to make an informative decision on whether the observation is a real outlier or just related to a particular behavior of people in a certain circumstance. In the healthcare domain for example, there is a high correlation between human activities such as biking in a city and smart healthcare activities. Understanding collective human health behavior helps the smart healthcare agents to improve and predict healthcare activities.

Contributions
This paper studies collective abnormal human behaviors in a smart city environment and proposes a new model for collective abnormal human behavioral data. Our main contributions can be summarized as follows: • We formally define the collective abnormal human behaviors problem by considering the individual abnormal human behavioral data as potential candidates. We introduce a new concept of density of a collection of abnormal human behaviors. A collection of abnormal human behavioral data is thus defined as a set of individual abnormal human behaviors that are highly correlated, i.e., with a high number of shared locations. • We propose solutions based on data mining, and deep learning to accurately identify the collective abnormal human behaviors data. Data mining based solutions explore the nearest neighbors, the density computation, and the feature selection in deriving anomalies from human behavioral data. In addition, two improvements based on ensemble learning, and GPU computing are shown to boost the performance of these solutions in terms of both runtime, and accuracy of the returned anomalies. The deep learning based solution applies a convolution neural network to learn and extract relevant features, and then retrieve anomalies from human behavior data. • We evaluate the performance of the proposed algorithms using different real human-behavioral databases. To the best of our knowledge, this is the first work that explores collective abnormal human behavior, and there is no relevant candidate in the literature to compare with. We therefore compare the proposed solutions with general anomaly detection solutions (see Section 2 for an overview). The results of our experiments demonstrate that our algorithms outperform the baseline algorithms. Moreover, they also show the effectiveness of the deep learning solution over the data mining based solutions.

Related work
Human behavior has recently studied in some smart city applications, notably smart buildings [15,16]. Several solutions have been proposed ranging from technical solutions based on pervasive computing for optimal monitoring [17][18][19], energy consumption analysis [20], and the use of ML for human for occupancy estimation [21], and activity recognition [22,23]. A detailed review on all these solutions is available in [8]. We are interested in this paper in another category of smart city applications that received relatively less attention amongst the research community; pedestrian behavior analysis. We are particularly interested in analysis from the perspective of collective anomaly detection.
Solutions to collective anomaly detection employ statistical models to derive the group of outliers [24]. Chalapath et al. [25] considered the use of a deep generative model and test it on various image applications. The outlierness for each group in the input data is estimated by group reference function using a standard back-propagation algorithm. Tang et al. [26] defined contextual outlier detection as small group of points that share similarity on some attributes but largely deviates on other attributes. Contextual outlier detection has also been explored in other works such as [27,28]. Li et al. [27] assigned feature weights for each group outlier, that were used to compute chain rule entropy and determine correlation between different feature groups. The use of HPC to deal with contextual outlier detection in high and sparse dimensional space was dealt with by Zhao et al. [28].
Other approaches use clustering strategies on individual outliers to group them into similar clusters, while each cluster is considered as a group of outliers [26,29]. Soleimani et al. [29] proposed a supervised learning approach that groups anomalous patterns when memberships are previously unknown. The salient features are extracted from an appropriate training set with discrete data inputs. This approach applies to topic document modeling and is able to discover irregular topic mixtures from a collection of documents. Sun et al. [30] proposed an approach to detect abnormal group-based joint medical frauds. The abnormal group problem has been converted to the maximal clique enumeration problem by considering the set of patients as a set of vertices, where each edge indicates that the two connected patients are similar. The similarity between patients can be obtained by computing their identical joint behavior. Given that the maximal clique enumeration is an -hard problem, the authors investigated methods to reduce the size of the graph.
We conclude from this literature review that existing group anomaly algorithms proposed in different contexts do not apply to human behavioral data. They focus on finding a group of outliers from the set of candidate groups, and not from the individual outliers. Candidate groups are not always known a priori in human behaviors such as pedestrian trajectories [31][32][33][34][35][36]. Moreover, existing solutions for trajectory outlier detection [37][38][39] are only limited to identifying single outliers. In this paper, we propose the first algorithm to detect collective abnormal human behaviors from individual outliers directly. We then demonstrate the application of our solutions on analyzing human behaviors.

Collective abnormal human behavior
In order to introduce the collective abnormal human behavior problem, we need a few preliminary definitions. We represent human behavioral data as a trajectory which is a sequence of location points in space. Each single spatial location point is a tuple of two values-the latitude and the longitude of this location. As common in the literature [40], the location points which are similar enough are aggregated into regions. Let us denote as a location region in space.  | | }, by the set of all regions. We define the dissimilarity between any two human behaviors data as the distance between them.

Definition 3.3 (Human Behaviors Dissimilarity).
We define the distance between two human behaviors data ( , ) by the number of all regions minus the number of shared regions between the two human behaviors data , and , as We define the collective abnormal human behaviors candidates, i.e., the set of potential human behavioral data belong to the collective abnormal human behaviors. This human behavioral data is retrieved from the individual abnormal human behavioral data.

Definition 3.4 (Abnormal Human Behaviors Candidate).
We define an abnormal human behaviors candidate  by the set of individual abnormal human behaviors retrieved from the set of individual abnormal human behaviors , i.e., The density of collective human behavioral data is an important concept in our analysis. Intuitively, it is defined as the ratio between the number of human behavioral data in the collection and the number of shared regions among the human behavioral data of such a collection.

Definition 3.5 (Density of Collective Human Behaviors Data).
We define the density of the candidate collection of human behaviors data  as To normalize the density function, we divide the result by the density of the collection having maximum density value, this ensures that we can obtain values ranging from 0 to 1. We call this function NormalizedDensity. Now, we are ready to formally define the collective abnormal human behaviors problem. Definition 3.6 (Collective Abnormal of Human Behaviors Problem). The collective abnormal human behavior Problem (CaHB) aims to discover from the set of all mapped human behavioral data, the set of all collections of abnormal human behavioral data, where each collective abnormal human behavior, , is determined as follows, Note that is the density threshold varied from [0 … 1].
A trivial approach for solving the collective abnormal human behaviors problem considers all possible combinations between the mapped human behavioral data, and evaluates each subset separately using Definition 3.5. This method requires high computational cost as well as memory resources, to evaluate the candidate sets and save the potential abnormal human behaviors. The theoretical complexity of this approach is (2 | | ). To address this issue, we propose in the next section, an alternative framework to improve the collective abnormal human behaviors process.

CaHB solutions
Let us begin by describing the key element of our data mining and deep learning based solutions to the CaHB problem (see Fig. 1 for an overview). Generally speaking, our approach builds upon data mining and deep learning. In particular, we use data mining techniques represented by clustering, feature selection, and neighborhood computation. We can boost the performance of such an approach using ensemble learning as well as GPU computing. The deep learning solution is represented by the convolution neural network, where an object detection is adopted to study the collective abnormal of human behaviors data. As shown in Fig. 1, our approach can be divided into distinct parts.
1. Mapping (Pre-processing): Typically, human behavioral data in most applications consists of noisy GPS data points where errors can exceed several meters. This can negatively influence the final output of many algorithms. Hence, first we should use a map-matching step to project GPS data points of each of the human behaviors onto a road network. In this paper, we use the probabilistic model based on a Hidden Markov Model [41]. In particular, we represent each road segment as a hidden state in the Markov chain-it has an emission probability which is the likelihood of observing the GPS point conditional to the candidate road segment being a true match. A higher probability to a road segment is assigned if the observed human behaviors points are close to it. The maximum likelihood path over the Markov chain that has highest probability is then determined, and the corresponding road segment is associated to the observed human behaviors point. In this way, we create the mapped human behaviors database, where each observed human behavior is assigned to the associated road segment. 2. Processing: After constructing the mapped trajectory database, a processing step is performed to find the collective abnormal human behaviors. In this context, we propose two ways to find the collective abnormal human behaviors, where we investigate data mining and deep learning solutions. Furthermore, we present various ways for improving the performance of our techniques by incorporating ensemble learning and high performance computing.

Data mining-based solutions
In the remainder of this section, we describe the details of the data mining based algorithms (Sections 5.1-5.3), ensemble learning (Section 5.4), and the high performance computing framework (Section 5.5).

The DBSCAN-CaHB algorithm
To present the adaptation of the DBSCAN algorithm, we need the following definitions: Definition 5.1 (Human Behavior Data Neighborhoods). We define the neighborhoods of human behavioral data ,  , for a given threshold by Definition 5.2 (Core Human Behavior Data). A human behavioral data is defined as a core human behavioral data if there is at least a minimum number of human behavioral data MinPts such that | | ≤ . In general, solutions to sequence and human behavioral data clustering [42][43][44] are able to derive clusters with different densities. However, these algorithms do not explore the micro clusters property for anomaly detection. This section presents our approach for identifying collective abnormal human behaviors, DBSCAN-CaHB, that A. Belhadi et al. uses the DBSCAN algorithm to search for clusters by checking theneighborhood of each human behavioral data (See Definition 5.1). The core trajectories are determined using Definition 5.2. DBSCAN-CaHB then iteratively collects density-reachable human behavioral data from these core human behaviors directly, which may involve merging a few density-reachable clusters. The process terminates when no new human behavioral data can be added to any cluster. Initially, the set of human behaviors are grouped (as in DBSCAN). This generates several clusters with different sizes. Each micro cluster (See Definition 5.3) is considered as group candidates. For each group, the density of each group is determined using Definition 3.5, if the density exceeds threshold, then the group is selected as abnormal.

The NN-CaHB algorithm
Let us begin the presentation of our adaptation of the NN Algorithm, with the following definition: Definition 5.4 ( NN). We define NN of a human behaviors data , denoted by NN( ), as where ( ) = ( , ) is the -distance human behaviors defined by the set of human behaviors ′ ∈ , such that ( , ) ≥ ( , ′ ).
The following proposition holds: Proposition 5.1. Let us consider two human behavioral data ′ and ′′ . Let  * ( ) be a collective abnormal of human behaviors at the iteration t such that: Then, the following holds: Proof. We have that: From (7) and (8) we have: It follows from the above proposition that if a human behavioral data belongs to the k-nearest neighbors of at least one human behavioral data in the current collective abnormal human behaviors, and is not in the collective abnormal human behaviors of the next iteration, then, any human behavioral data that belongs to the k-nearest neighbors of will not be in the collective abnormal human behaviors of the next iteration. Consequently, it is judicious to prune the search into k-nearest neighbors of the individual human behavioral data. In particular, it considers that as input the set of the first individual human behavioral data  + = { + 1 , + 2 ⋯ + }, ranked according to the NN value, i.e., ∀ ≥ , NN( + ) ≥ NN( + ). The process aims to enumerate the sets of collective abnormal human behaviors,  * , by exploring a search tree of  + . It starts by adding the individual abnormal human behaviors ranked first, + 1 , to the collective abnormal human behaviors, denoted by  * 1 . It then generates all potential candidates from + 1 . A human behavioral data is a potential candidate from + 1 , if and only if, ∈  + ∨ ∈ NN( + 1 ). The density of  * 1 is updated by adding the potential candidates to  * 1 , one by one. Only the potential candidates respecting the density threshold are saved, and the remaining ones are removed. Once the potential candidate is added to  * 1 , it is removed from  + . If  * 1 contains less than two elements, it is removed from  * . The same process is recursively applied to all potential candidates added to  * 1 , and the overall process is repeated for all abnormal human behaviors in  + .

FS-CaHB
We begin by defining the transformation of the CaHB problem to the feature selection problem.

Definition 5.5 (Transformation to FS Problem). Consider CaHB problem
We transform it to the feature selection problem, represented by the set of all features and the subset of selected features * , as follows: =  +  , and * =  * . We evaluate * as follows: where Quality( * ) is computed as per Definition 3.5.
We consider each individual abnormal human behaviors as one feature, and our aim is to select the most relevant features from the set of all features. This set then becomes the collective abnormal human behaviors (see Definition 5.5). The evaluation of the selected set of features (human behavioral data) is computed using the group A. Belhadi et al. density measure, see Eq. (9). The process starts by applying the feature selection algorithm on the set of individual abnormal human behaviors, The output of this step is a ranking of individual anomalies in the descending order in terms of score feature relevance. A ranking vector is created, where a search enumeration tree is generated in the breadthfirst-search (BFS) manner. If the quality of the current group candidate does not reach the criteria from Definition 3.5, a backtracking procedure is launched by taking the next human behavioral data in the ranking vector. While exploring the enumeration tree of individual anomalies, the aim is to maximize the function reported in Eq. (9).

Ensemble learning
Each of the previously proposed techniques -clustering (DBSCAN-CaHB), neighborhood computation (KNN-CaHB) and feature selection (FS-CaHB) -returns potential collective abnormal human behaviors. Out of these collections, some collections are usually good while others may not be useful. In order to improve the accuracy of the detected collections, we propose the use of ensemble learning [45]. In particular, we first launch our algorithms, i.e., DBSCAN-CaHB, kNN-CaHB, and FS-CaHB, in this context called learners. The three outputs -sets , , and -are then merged to derive the final collections. Hence, the main challenge of this approach is to find an efficient merging strategy. We propose the following. For each collection, we determine the number of the occurrences of the three learners, where we look for the collections that are highly frequent. For instance, if there are two collections: the first collection { 1 , 2 , 4 } appears two times -ones in the output of the algorithm, and another time in the output of the algorithm -and the second collection { 1 , 2 , 3 } appears only one time in the output of , then the first collection is better ranked than the second one.

High performance computing
The map is first partitioned on grids { 1 , 2 ... } using the map partitioning step. The set of designed grids are then sent to the GPU. Each block of threads is mapped onto one grid, where the CaHB solutions are applied on each block in parallel. If we consider the size of the shared memory of each block is , the first human behavioral data of the grid are allocated to the shared memory of the block, and the remaining human behavioral data of the grid is allocated to the global memory of the GPU host. It defines a local table, , to store the collections of abnormal human behaviors data of the grid . The local table of each grid is sent to CPU for further processing. In this context, CPU host performs a merging step to find the global collections, where the union of all sets of collections in the local tables is computed. From a theoretical standpoint, high performance computing improves the CaHB solutions by exploiting the massively threaded computing of GPUs while mining the grids of human behavioral data. It also minimizes the CPU/GPU communication, by defining only two points of CPU/GPU communication. The first one takes place when the grids are loaded into the GPU host, and the second one when the local tables are returned to the CPU. It also provides an efficient memory management by using different levels of memories including global and shared memories. However, it may suffer from synchronization between the GPU blocks. This takes place when the GPU blocks process grids with different number of human behaviors data. This issue degrades the performance of the GPU-based implementation of the CaHB solutions. In real scenarios, different number of human behavioral data per grid may be obtained, this depends to the way of the human behavioral data are placed into the map, as the size of the grids are different, as the synchronization cost of the GPU-based implementation will be high. All these statements will be clearly explained in the performance evaluation section.

Deep learning-based solutions
This section presents the proposed CNN-CaHB framework, which use a convolutional neural network (CNN) for identifying collective abnormal human behaviors. CNN-CaHB consists of two stages: 1. Data Collection: The aim of this step is to collect human behavioral data and build the images database. A visual strategy is used, in which each image contains a set of human behavior trajectories. The process starts by recording video frames from cameras. The frames are then transformed to images, and different distortion techniques such as mapping, re-sizing are used to correct the images. The whole trajectory images are stored into the database for training. 2. Training: The convolution neural network is applied to design a training model, which it is considered as a powerful vision machine to learn from the different features of the image trajectory data. The collective abnormal human behaviors is transformed to the object detection problem by setting the input of the object detection model to the trajectories image database, and the output of the object detection model to the collective abnormal human behaviors in each image. We use the regional convolution neural network [46,47]. For every image in the input, regions of interests are determined and passed to the hidden layer where the Relu activation function is performed and a similar process to the convolution neural network is followed.
In fast RCNN, the model performs better and quicker as the regions of interest are found using a selective search method, and all the regions of interests of an image are found at once. This is different from CNN that finds ROI (regions of interest) and applies Relu on each ROI separately which is slower. The process is repeated for a given number of epochs, or until the training stop providing improvement for a given number of iterations. The weight initialization is done using the pre-trained ImageNet model. 1 The trained model is stored on the central workstation. 3. Inference: The aim here is to derive the collective abnormal human behaviors of the input image using the trained model of the previous step. Therefore, a propagation of the different weights of the trained model is performed to detect the objects of the image. The detected objects are considered as the collective abnormal human behaviors. In this step, different kinds of inference are generated. We send the trained model to the computers and infer the model for each new trajectories image data. We can also use smartphones which support Android, and GPU computing to infer the collective abnormal of human behaviors in real time processing. In this context, several technologies could be integrated such as TensorflowLite. 2 In practice, the training images have a very high resolution in today's IoT world, from 5000 pixels to 100,000 pixels. As a result, millions to billions of region proposals have been generated. This makes the whole system very hungry in time-processing and memory. In some cases, the system will be bluntly blocked after several days and weeks of processing. To deal with this problem, we propose a strategy to prune and filter the number of bounding boxes. Two collections in the same frame should not be close to each other, and thus two bounding boxes in the same image should not be close to each other. We then compute the similarity between each new bounding box generated and the bounding boxes that have already been generated. The similarity between two bounding boxes is determined by the number of pixels that separate these bounding boxes. We only keep the bounding boxes, which gives high diversity of the image. That is, the minimal set of bounding boxes which cover the maximum number of pixels in the image.

Performance evaluation
The CaHB framework and its different components are evaluated in this section. The parameters of both data mining and deep learning based CaHB solutions are first tuned, and then the scalability performance of the best implementation is investigated and compared with the state-of-the-art solutions.

Experimental setup
The implementation of the different components of the CaHB framework. 3 has been integrated on the SPMF data mining library [48] The experimental evaluation has been performed on a computer with 64−bit core 7 processor running Windows 10 and 16 GB of RAM, coupled with a GPU device. The GPU device is an NVIDIA Tesla C2075 with 448 CUDA cores (14 multiprocessors with 32 cores each) and a clock speed of 1.15 GHz. It has 2.8 GB of global memory, 49.15 kB of shared memory, and a warp size of 32. Both the CPU and GPU are used in single precision. We used HUMBI dataset, which is a large corpus of high fidelity models of behavioral signals in 3 from a diverse population measured by a massive multi-camera system [49]. The dataset contains human behavior from 164 subjects across gender, ethnicity, age, and physical condition at a public venue. High models of five elementary parts are designed including: gaze, face, hands, body, and cloth. As a byproduct, the 3D model provides geometrically consistent image annotation via 2 projection, e.g., body part segmentation. In general, a common problem of anomaly detection techniques lies in the evaluation step, typically in emerging applications such as CaHB where the ground truth is typically unknown. To facilitate a quantitative evaluation, for CaHB detection techniques, we inject synthetic collective abnormal of human behaviors data. The evaluation is performed using runtime and ROCAUC, which are the common measures for the evaluation of anomaly detection methods.

Parameters setting
The first part of the experiment analysis focuses on tuning the parameters of different proposed CaHB solutions. Several tests have been performed by varying the user threshold from 1 to 10 for DBSCAN-CaHB, the number of neighborhood from 1 to 10 for kNN-CaHB, the tree depth from 1 to 10 for FS-CaHB, the number of epochs from 100 to 1000 for CNN-CaHB. For all the number of human behaviors data used as input, the accuracy of DBSCAN-CaHB and kNN-CaHB increases with the increase of the corresponding parameter values up to reaching the optimal point, and then starts decreasing. In case of FS-CaHB, and CNN-CaHB, the accuracy increases with values of the parameter and stabilizes when reaching the optimal point. The best parameter values obtained in this step are used in the remaining of the experiments. The user threshold is set to 4 for DBSCAN-CaHB, is set to 6 for kNN-CaHB, tree depth is set to 6 for FS-CaHB, and epochs is set 700 for CNN-CaHB.

Comparison with state-of-the-art solutions
The aim of this experiment is to compare the proposed solutions with state-of-the art algorithms in terms of accuracy and processing time. Since this is the first work that explores collective abnormal of human behaviors (to the best of our knowledge), we compare the proposed solutions with general collective abnormal of human behaviors based solutions (see Section 2). For this purpose, we adapted two algorithms, i.e., Das et al. [50], and Sun et al. [30] and used them as base-line. Figs. 2 and 3 present runtime, and accuracy respectively, of the proposed solutions in comparison with baseline algorithms. Fig. 2 presents the runtime of the proposed solutions against the baseline algorithms. Each plot correspond to a specified percentage of regions. Overall, we consider 1000 regions, the first plot represents results of exploring 100 regions, the second plot represents results exploring 200 regions, and so on until the last plot where all the regions are explored. At each pass of the experiment, the percentage of human behaviors data, represented by -axis have been varied from 10% to 100%. The -axis represents the runtime in seconds for identifying the collective abnormal of human behaviors data. The results show that the runtime of our GPU-based, as well as, the convolution neural network solutions outperform the baseline algorithms. However solution based on neighborhood computation required huge computational resources, but still outperform the baseline algorithms. For instance, for mining 1000 regions, and the whole human behaviors data, the CNN-CaHB, and the GPU-CaHB need 42 s, and 81 s, respectively, however, other solutions exceeds 110 s for mining the same number of regions, and the same number of human behaviors data. This is explained by the fact that the convolution neural network only needs time to train the data and find the weighted of the model used in the inference step. Further, the GPU-based approach benefits from the massively threaded and the efficient mapping between the human behaviors data and the GPU blocks. On the other hand, solutions based on neighborhood computation and density estimation have high cost for determining the similarity between each new human behaviors data and the whole human behaviors database. Fig. 3 presents the accuracy represented by the AUC value of the proposed solutions against the baseline algorithms. Each plot correspond to a specified percentage of regions, in overall, we consider 1000 regions, the first plot represents results of exploring 100 regions, the second plot represents results exploring 200 regions, and so on until the last plot where all the regions are explored. At each pass of the experiment, the percentage of human behaviors data, represented by -axis have been varied from 10% to 100%. The -axis represents the AUC value for identifying the collective abnormal of human behaviors data. The results show that the AUC value of our convolution neural based approach outperform the baseline algorithms. However solution based on neighborhood computation, ensemble learning, and GPU computing are less accurate, and converges to the baseline algorithms. For instance, for mining 1000 regions, and the whole human behaviors data, the accuracy of CNN-CaHB is 85%, however, other solutions does not reach 75%, except Sun et al. algorithm, where its accuracy is 81%. This comes from the fact that the convolution neural network used more advanced techniques in identifying the collective abnormal of human behaviors data such as the convolution, the pooling operators, intensive number of layers, computing global image features, while the other solutions use very basic concepts in finding anomalies.

Future work
As far as future perspectives are concerned, we plan to explore other representation of human behavioral data such as time series, events log, and tweets. These representations will help identifying collective abnormalities. This research area needs to investigate other data mining algorithms and other deep learning architectures for maximizing its potential. For instance, the recurrent neural network is more suitable to deal with time series and textual data. Hybrid architectures based on both convolution and recurrent neural networks are needed to study the correlation of human behavioral data gathered from heterogeneous sources. Solutions to human behavioral analysis are very time consuming in particular when dealing with large scale data. Therefore, technologies from different domains could be adapted such as: 1. HPC aims at using parallel frameworks to speed up the sequential solutions. Some of the most well-known architectures apply the multi-core CPU or GPU and perform on MapReduce or Spark platforms A. Belhadi et al.  2. Computational intelligence is a collection of intelligent methods aiming at optimizing complex problems with strategies like meta-heuristics 3. Database systems provide techniques to efficiently store, update, and search human behavior data, such as query optimization and index optimization.
Adapting, combining, and optimizing technologies in human behavioral analysis may also provide many open research questions and future directions. Furthermore, in this research, two measures (runtime, ROCAUC) have been considered in the evaluation step. Evaluating the proposed solutions on other measures such as F-measure, F1-score, and computing the number of grouped human behavioral data is also in our future agenda.

Conclusion
A new model for identifying abnormal human behavior from trajectory data has been introduced in this paper, where data mining and deep learning algorithms have been proposed to tackle the highly important problem. Data mining techniques explore the correlation, the neighborhood computation, and the density estimation among the input. We proposed adaptations of DBSCAN, kNN, feature selection, and ensemble learning, by considering group and collective behavior of people to identify abnormalities. We also boosted the runtime performance of these solutions by using HPC approaches and benefiting from the massively threaded of the GPU architecture. We also proposed a deep learning approach based on the convolution neural network. The DL approach aims to learn from both training as well as historical trajectory data, and be able to identify the abnormal human behavior. The work has been validated through intensive experimentation analysis of both the data mining and deep learning solutions. The results confirm that the deep learning approach outperforms the data mining approaches, as well as the state-of-the-art solutions in terms of runtime and accuracy.