SMART OPTIMIZER SELECTION TECHNIQUE: A COMPARATIVE STUDY OF MODIFIED DENSNET201 WITH OTHER DEEP LEARNING MODELS

. The rapid growth and development of AI-based applications introduce a wide range of deep and transfer learning model architectures. Selecting an optimal optimizer is still challenging to improve any classification type's performance efficiency and accuracy. This paper proposes an intelligent optimizer selection technique using a new search algorithm to overcome this difficulty. A dataset used in this work was collected and customized for controlling and monitoring roads, especially when emergency vehicles are approaching. In this regard, several deep and transfer learning models have been compared for accurate detection and classification. Furthermore, DenseNet201 layers are frizzed to choose the perfect optimizer. The main goal is to improve the performance accuracy of emergency car classification by performing the test of various optimization methods, including (Adam, Adamax, Nadam, and RMSprob). The evaluation metrics utilized for the model’s comparison with other deep learning techniques are based on classification accuracy, precision, recall, and F1-Score. Test results show that the proposed selection-based optimizer increased classification accuracy and reached 98.84%.


Introduction
Due to the rapid population growth in the world, the number of cars and usage of vehicles have exponentially risen [4], which has led to increasingly congested roads, heightened air pollution, and a rise in accidents [10].In such a scenario, effective traffic monitoring of the streets is required, a severe challenge in many cities worldwide.Also, urban traffic congestion can potentially result in various impacts on the environment, public health, and the economic situation.Consequently, an accurate road transportation system is required based on Traffic Signal Timing (TST), considered an optimized and fastest technique to reduce the congestion at the road's intersections and improve the urban traffic flow networks [19].The main challenge in intelligent transportation focuses on controlling traffic signals, impacting the provision of transportation services within urban transportation systems [18].Besides, intelligent transportation systems can be utilized to overcome the related issues to traffic signals.
In particular, the problems that constantly affect our society, namely (human safety, waiting time minimization, effective cost reduction, and decrease of carbon dioxide emission (CO2) to the atmosphere.
Moreover, launching signal timing control ensures individuals' safe and efficient movement through an intersection.Achieving this goal necessitates a well-structured strategy for determining the right-of-way allocation.To make the optimization task attainable, it is necessary to introduce specific assumptions.However, a notable challenge arises from frequent discrepancies and occasionally substantial divergence between these assumptions and real-world conditions.In the meantime, numerous factors influence drivers within real-world traffic scenarios, including interactions influenced by driver preferences with vulnerable road users (such as pedestrians, cyclists, etc.), weather conditions, and road quality [7].Emergency vehicles play a critical role in various life-threatening situations.Over 20% of patients in ambulances are at risk due to traffic congestion.
However, the mortality rate tends to increase in severely ill patients.Countries with high population densities often face significant traffic congestion during peak hours.Emergency vehicles like police, ambulances, and firefighters frequently become trapped in traffic, leading to life-threatening circumstances.Therefore, prioritizing these emergency vehicles and ensuring their seamless flow becomes essential.This goal can be achieved by suggesting an automated traffic system to identify these vehicles even when roads are congested [21].
Additionally, accurate vehicle detection systems are essential for distinguishing between emergency and regular vehicles.Deep learning and transfer learning approaches employing Convolutional Neural Networks (CNNs) have been harnessed in computer vision, producing results that match human capabilities and even surpass those of human experts.The Deep convolutional neural network is one of the best techniques for detecting and classifying objects in image and video [17].The general concept uses knowledge learned from the pre-trained neural network model applied to different but related problems.Various pre-trained models including AlexNet, Visual Geometry Group (VGG)-19, ResNet-101, inception-v3, and DenseNet201 [14] are applied to the customized dataset of non-emergency and emergency cars.
Most recent advances and developments for deep learning applications in many fields have been presented, such as computer vision, medical image, natural language processing, speech processing, and traffic congestion [5].A modified version of MobileNet based on the increase in frame rate is proposed by Ahmed et al. to achieve higher F-values.It could be applied to diverse applications supporting real-time traffic data analysis [2].Another deep learning model based on ResNet-50 is presented for vehicle localization and classification using real data from traffic surveillance cameras [12].Automatic damage vehicle classification has become an interesting area for researchers.An image-based inspection model using an adapted version IAPGOŚ 4/2023 p-ISSN 2083-0157, e-ISSN 2391-6761 of pre-trained convolutional neural networks namely VGG-19 and VGG-16 is proposed to reduce the damage to car's cost [6].
In [1] a new deep-learning model for Bangladesh vehicle classification is proposed using a Multi-class Vehicle Image Network (MVINet) based on DenseNet201 with four additional layers.The proposed approach utilized CNNs to extract features from the image vehicles dataset.
This paper uses different optimizers to improve the efficiency and accuracy of the proposed deep and transfer models.The optimization process undertakes multiple paths until reaching the convergence.Various optimization mechanisms have arisen to solve the problems during the learning process.The following optimizers are taken into consideration including the Stochastic Gradient Descent (SGD), Stochastic Optimization Descent with Momentum, Rung Kutta, Adaptive Learning Rate, Root Mean Square Propagation, Adaptive Moment Estimation, Deep Ensembles, Feedback Alignment, Direct Feedback Alignment, Adfactor, AMSGrad, and Gravity.Experimental results illustrated that best optimizers like SGD and Adam have optimal training, testing, and validation accuracy [8].The general block diagram of the proposed system is depicted in Fig. 1.
This study applied various deep learning techniques to classify vehicles while passing in a traffic intersection.The paper's organization is as follows: Section I provides a general introduction and its associated issues.Section II entails the background of traffic management.Section II presents relevant prior research and surveys previous studies.The methodology is explained in Section III.In Section IV, results are shown and discussed.The concluding remarks are provided in the final section.

Motivation and contribution
In the Kurdistan Region of Iraq (KRG), considered a developing region in Iraq, the main challenges facing people are accidents at the intersection of traffic signaling and control systems.Many factors are involved in such an unreliable environment, such as (planning urban roads, lack of driver awareness, and lack of traffic signs).Furthermore, emergency vehicles require an efficient traffic system to reduce road congestion.To build this system, which is the main aim of this research, a customized dataset is created for ambulance, police, and firefighter cars. Recently

Related works
Researchers have done much work to detect and classify vehicles in traffic intersections and use the acquired data to control traffic signal systems.Deep learning techniques are mainly used to improve those system accuracies for emergency vehicles regarding detection and classification, while few studies have utilized transfer learning-based models.They depend on their proposed techniques' image or video preprocessing and tuning hyper-parameters.Kamaran H. and Aree A. [16] extensively surveyed computer vision-based traffic control and monitoring.The aim was to comprehensively review recent research using computer vision and deep learning algorithms.They also compared the models based on the algorithms, dataset, and performance accuracy used, respectively.In addition, Tomar et al. [22] presented a state-of-the-art review of smart vehicles, including challenges and emerging trends.They proposed solutions for the transportation system, especially controlling and managing traffic at the road intersections.On the one hand, they included synchronizing signals across a specific route and improving the real-time traffic flow at junctions and intersections while cars and emergency vehicles pass.On the other hand, achieved results indicated a remarkable reduction in congestion by applying signal synchronization to the busy roads in the smart cities.In [15], Dallas et al. offered a comprehensive overview aimed at improving the traffic signal system's safety and accuracy through various performance evaluation metrics.Their survey also had a potential benefit to traffic agencies, researchers, and commercial offices.The simulated results performed by several methodologies showed that when sensor data and probe vehicles are used, the performance accuracy of the traffic signal system is better.
To incorporate the machine and deep learning models in traffic flow prediction, Noor et al. [20] introduced a critical review to present the techniques and gaps in the Intelligent Transportation System (ITS) performance evaluation.Based on their findings, the most frequent ML techniques for predicting traffic flow are CNN-based and Long-Short Memory (LSTM).
Joo has proposed an improved reinforcement learning model for traffic signaling management systems, H. et al. [11].They tackled the traffic congestion problem using an effective Traffic Signal Control (TSC) algorithm, which adaptively increases the number of normal and emergency cars passing through an intersection and balances the signals between roads using Q-learning (QL).Based on their simulated results, the proposed model performs better than other research using (QL).Ke, X. et al. [13] designed and developed a model for detecting road congestion in ITS based on CNN and multidimensional visual features.Firstly, the model detects the density estimation of foreground objects utilizing a gray-level co-occurrence matrix; then, the Lucas-Kanade optical flow algorithm with a pyramid implementation is performed to measure the speed of objects in movement.Secondly, a Gaussian mixture method is used for the background estimation, and CNN accurately separates the final foreground from the foreground candidates.Finally, the attained results through objective and subjective metrics are evaluated to exceed the state-of-the-art research work in road-traffic congestion detection methods.Currently, roads and highways in urban networks are supplied with many surveillance cameras that can be used for real-time vehicle detection and identification by efficiently estimating traffic flow.The paper presented a comparative study of object detectors, visually described features, and classification models to implement traffic estimations.Notably, three object detectors are used to categorize vehicles.Additionally, four traditional machine learning models are sequentially employed to extract visual features for classification.These traditional ML approaches are compared to the deep learning-based models.The research findings indicate that when techniques are accordingly implemented and tested, the deep learning method performs most accurately, especially for multi-class classification [9].
Ashir M. introduced a real-time detection method for various emergency vehicles based on a computer vision approach under massive traffic conditions.This allows priority control to emergency vehicles by the traffic controller agencies, which significantly saves people's lives and properties, prevents crimes, and forcefully minimizes the time needed by an emergency vehicle to arrive at its destination.A modified version of the YOLOv5 algorithm for object detection is proposed and applied to four types of emergency vehicles.YOLO is considered one of the most well-known object detection schemes due to its efficiency and accuracy.The connected layers of the YOLO algorithm are modified to have new learning weights while freezing convolutional layers.The test results of the proposed model have indicated promising results for emergency vehicle detection and classification [3].
The abovementioned survey states that conventional deep learning techniques cannot accurately obtain an optimal result.Hence, a transfer learning-based model, DenseNet-201, is developed to fill this gap with some improvements.It is dynamically adapted to select one of the best optimizers used in this research work.

Proposed methods
In this study, four optimizers are tested with different deep and modified transfer learning techniques using a customized dataset created by [17].As shown in figure 1  Use the proposed search mechanism to find out the best optimizer.The condition is based on returning the maximum value of the accuracy and precision metrics. Performance metrics' evaluation: precision, recall, f1-score, and average accuracy. Vehicles classification: The multi-classification method identifies emergency and non-emergency vehicles.

Customized dataset
Because no publicly available dataset contains emergency cars (police, ambulance, and firefighters) and non-emergency cars, a customized dataset has been produced.In this work, the used dataset was collected from the (Kaggle, Fatkun Batch, and Rania traffic directorate in Kurdistan reign-Iraq).In table 1, the dataset for both balanced and unbalanced datasets is displayed.

Modified DenseNet201 model
In this study, a modified version of "Densely Connected Convolutional Neural Network" (DenseNet201) is developed to improve and overcome the state-of-art, which is based on traditional CNN for feature extraction and classification.Concerning neural networks, freezing layers can be considered a mechanism in which the control of learning weight parameters is updated.When the layers are frozen, the weights cannot be further modified during potential processing.This strategy is to reduce the computational complexity for training, whereas it does not have a significant effect on detection accuracy.Therefore, freezing the model's layers is a method to speed up the neural network training by freezing hidden layers in progressive ways.In this regard, many freezing layers are performed on DenseNet201 layers in the interest of its impact on performance accuracy.Figure 2 demonstrates the modified architecture of DenseNet201 by freezing some layers.

Proposed Search Algorithm
All models trained with the input size of (64*64, 128*128, 224*224) are simulated for different epoch numbers.For each model, the performance metrics are calculated based on accuracy, precision, recall, and f1-score, respectively.The conducted results are then sorted to find out the maximum values of accuracy and precision.Our search mechanism relies on the condition where the selection of the best optimizer is performed when both accuracy and precision measures have maximum values, as depicted in Fig. 3.

Performance evaluation metrics
The proposed models are trained with the input mentioned above data for a range of iterations.Performance metrics such as accuracy, precision, recall, and f1-scores are used for estimating the model's performance accuracy.The following equations specify these metrics accordingly [23].

Results and discussions
As explained in previous sections, the main goal of this work is to find an optimal optimizer that can effectively comply with proposed deep and transfer learning techniques.Accordingly, the pre-trained models are compiled using the customized dataset for different epoch numbers (i.e., iterations).Other hyperparameters are fixed for all methods, such as (filter size=7*7, dropout_rate = 0.5, batch_size = 16, and learning-rate = 0.0001).The conducted tests determine the performance metrics (Average accuracy, precision, recall, and F1-Score) over 15-30 iterations.Fig. 4 and 5 show the accuracy and precision results for the different input image sizes when the best optimizer has been chosen.In this case, a modified DenseNet201 with 120 layers freezing is used.
Experimental results in Figures 4 and 5 reveal that the image size 224*224 gives better accuracy and precision values.Therefore, for further tests, we merely used this size with all techniques and optimizers in this research.
Table 2 and 3 present the accuracy and precision metrics results to select the best optimizer using our new search strategy.
The highlighted rows in the above tables indicate that the more accurate model is the DenseNet201 with 120 layers freezing and the best optimizer is RMSprop for both accuracy and precision tests when the input image size is 224*224 and the epoch No. is 30.
In addition, the confusion matrix of the proposed emergency vehicles classification transfer-based model is plotted, which is displayed in Fig. 6.The matrix diagonal represents the proposed model performance accuracy for different types of emergency cars.

Conclusions
This paper uses a new search mechanism to develop an enhanced deep transfer learning model based on freezing layers.It is designed for emergency and non-emergency vehicle detection and classification.Various optimizers are used for training different models to find out the optimal accuracy of detection.In the initial phase, various image preprocessing and augmentation methods are employed on the input image dataset to achieve dataset balance.Afterward, the basic architecture of the DensNet201 model is modified by freezing the layers.Finally, the trained models are simulated with various optimizers to select the optimal one.The higher accuracy attained by the proposed method is 98.84% when the RMSProp is used.To reduce congestion and waiting time, the future work plan will test the proposed model in a real-time traffic signaling system when different emergency vehicles enter the lane intersections.
, the proposed methodology steps are as follows:  Dataset collection: Emergency and non-emergency vehicles are gathered from a publicly available dataset and local traffic offices in KRG-Iraq.The vehicle class types are imbalanced. Image labeling: It is based on emergency (Ambulance, Police, and Firefighter) and non-emergency cars. Image quality enchantment: perform preprocessing algorithms, including image resizing, sharpening, smoothing, and contrast enhancement. Data augmentation: to balance the dataset, various image transformations are applied to overcome the overfitting issue. Dataset splitting: Datasets are portioning to cross-validation of 80% (training data) and 20% (test and validation). Train the proposed deep and transfer models with tuning hyper-parameters of each model.

Fig. 2 .
Fig. 2. Modified DenseNet201 architecture true positive, true negative, false positive, and false negative are expressed by TP, TN, FP, and FN respectively.
The model's (DenseNet201 with 120 layers freezing) loss and accuracy for the training and test validation are depicted in Fig.7and Fig.8respectively.

Fig. 6 .Fig. 7 .Fig. 8 .
Fig.6.Confusion matrix for the optimal model and optimizer few researchers use the DenseNet201 transfer model to address the abovementioned issues.The crucial contributions are summarized as follows:  The proposed models use different image sizes (64*64, 128*128, and 224*224) as an input for the model's training.Various transformations also augment the images to make the emergency vehicles dataset balanced.
, researchers have mainly used deep learning techniques for emergency vehicle detection and classification based on CNN and RNN architectures.Nevertheless,

Table 1 .
Balanced and Unbalanced Datasets

Table 2 .
Accuracy tests