Real-Time Moving Object Detection for Day-Night Surveillance using AI

Project Title: Real-Time Moving Object Detection for Day-Night Surveillance using AI

A Final year project on “Real-Time Moving Object Detection for Day-Night Surveillance using AI” was submitted by Gaurav Barve / Aditya Chache / Vineet Das (from Mumbai University / Datta Meghe College Of Engineering)to extrudesign.com.

Abstract

In this project, we have created a Web App that can be operated in various modes i.e. Object detection of various media like images, videos, and real-time video streams from the user’s device. The app would take a media format and give the list of all detected objects within it as well as an annotation on the original media depicting the detected objects. The project implements image processing provisions so that users can modify their images as per need this would be done with the help of an open CV. The app would be compatible across all platforms and on different operating systems this would help in reaching a far greater audience.

Chapter: 1 – Introduction

Real-time object detection and tracking are important and challenging tasks in many computer vision applications such as video surveillance and vehicle navigation as well as online proctoring nowadays. Our Project belongs to the same domain and in this project, we have to build a one-stop solution for all image processing as well as object detection requirements.

1.1) Motivation

The reason for choosing this project is because we want to implement a similar model which can perform all regular operations as well as to detect objects in daytime as well as night-time and which when deployed will be useful for real-time surveillance. Most of the other models don’t work that well on night-time images because they haven’t been trained on dark images. So we want to bridge that gap by training a model on dark images from scratch. To build an OS-independent and device-independent Web-App that can be used anywhere.

1.2) Aim and Objective

The main purpose of object detection is to identify and locate one or more effective targets from still image or video data. It comprehensively includes a variety of important techniques, such as image processing, pattern recognition, artificial intelligence, and machine learning. On completion and deployment, any user would be able to visit the Web App where he can fulfill all object detection requirements as well as Image Processing be it on Images, Videos, or Real-Time detection.

1.3) Application

Object detection is applied in numerous territories of image processing, including picture retrieval, security, observation, computerized vehicle systems, and machine investigation.

Critical difficulties remain in the field of object detection. The potential outcomes are inestimable with regard to future use cases for object detection. Being a project built keeping the community in mind would benefit in providing a benchmark for future project developments.

Chapter: 2-Literature Review

The list of all the review papers is mentioned in references. Following are the summary of the findings from the papers:

2.1) Existing Systems

The existing system which are still in use are slow in processing as well as the requirement of a near perfect system for them is a big challenge.
Traditionally various CNN models were used to achieve the accurate detection however thesemodels are useful and convenient but they do not provide the most accurate and reliable detection as per today’sstandards.

2.2) Findings

In comparison with Object detection techniques that came before YOLO, like R-CNN, YOLO introduced a single unified architecture for regression go image into bounding boxes andfinding class probabilities for each box. This meant that YOLO performed much faster and also provided more accuracy.
It makes less errors than Fast R- CNN as it can see the bigger context because YOLO, unlike Fast R-CNN, can globally reason the image when making predictions.
Modern object detection is primarily designed for images, and not explicitly for videos. As aresult, videos require to be chunked into individual frames before detection can happen. This creates inefficiencies like delays in detection, overheads in converting the videos into frames,and non- consideration of the frame-level relationships.
Addressing these aspects is a critical area of current and future development.

2.3) Recent Trends

Most detectors require significant amounts of labeled data for training. This is highly inefficient as it increases the development cycle time and costs. Efforts are in progress toresolve the same.
Object detection is resource- intensive, both in terms of human intervention and model processing (compute). As a result, high-speed detection in realtime, particularly for mobile devices, is an important development area. Auto ML in object detection is also an important focus area.
Most detectors struggle with small objects. The inaccuracies in small-object detection are considerably higher than those related to the medium or big sized ones. For instance, remote-sensing of smalltargets in disaster management or military applications is still under-

Chapter: 3-Requirement Analysis

3.1) Software Requirements

Google Colab
Python 3.6 and higher
tensorflow==2.6.0
matplotlib==3.4.3
numpy==1.19.5
streamlit==0.69.62
pandas==1.3.3
OpenCV libraries
Nvidia Cuda & Cudnn

3.1) Hardware Requirements

System	Minimum Requirements
Operating Systems	Windows: 7,8,10 (32/64 bit) macOS: 10.12+Linux: ubuntu any version
CPU	Intel Core i3 5^thgen+ Amd Fx 6350+
GPU	Not required as we will be using cloud GPU
CAMERA	Access to a webcam or any other external camera for real-time detection

Chapter: 4-Project Design

Collecting the Data: The data which is used to train the model is collected in two ways. In the first way we have used the Ex-Dark image dataset which consists of dark images of objects from over 10 classes. In second method we have used web scraping to download images directly from the web.
Annotating the Images: The most important of the project is to annotate the data which is to draw those bounding boxes on the objects that we want to detect. For this task we will use theRoboflow software which is used to annotate the images. Apart from annotating we will also use it to augment the data and basically increase the size of our dataset for better accuracy.
Importing the framework: To use the YOLO model we will import the darknet framework from GitHub. After that is done we will use a .py script to generate the file paths of our images.Then we will make some changes in the .cfg file of our model such as changing the batch size, providing the file paths of our images andlabels, changing the number of epochs, etc.
Training the model: This is also an important part of the workflow. So basically our final goalis to detect objects on unseen data and to achieve this we first need to train the algorithm which weare going to use on our collected and annotated data. The algorithm that we will use for training is the a one-stage object detection model name as YOLOv4.
Testing and Deploying the model: After the training is done we will evaluate the model on our test data and if we’re satisfied with it we will save it and deploy as a web application. For the deployment phase we will use Streamlit which is an open source python framework to create web apps. After the web application is created we will deploy it on Heroku which is a cloud platform for hosting web applications.

Chapter: 5-Design Implementation of Real-Time Moving Object Detection using AI

To understand how we have implemented the project let us go through the Flow Diagram of our Project.

From the above flow diagram we can see the implementation of the project.
Initially we placed all our data in Google Drive and by providing the path into Colab we buildour model after saving the model we stored this model in .cfg file and the weights. The same model would be used for detection in images, videos, real time analysis.
The model would be stored in Streamlit Cache so that there is no processing delay of inputmedia.
For image processing we would use the open cv library provided by python to modify, change,test images.
The drop down menu would provide the user to choose on which media the user wants toaccess the App.

On successful execution we are deploying it with the use of Heroku- cloud platform as aservice (PaaS) supporting several programming languages.
This would enable the user to access the Web App on any platform and any system as it is notlimited to one particular platform or operating system.

Chapter: 6-Technologies Used for Real-Time Moving Object Detection

6.1) Streamlit

Streamlit is an open-source Python library that makes it easy to create and share beautiful, custom web apps for machine learning and data science. In just a few minutes you can build and deploy powerful data apps. Easy and user-friendly interface for users to navigate around

6.2) Google Colab

Colab is a free Jupyter notebook environment that runs entirely in the cloud. Most importantly, it does not require a setup and the notebooks that you create can be simultaneously edited by your team members – just the way you

edit documents in Google Docs. Colab supports many popular machine learning libraries which can be easily loaded in your notebook. Collaboratory, or ‘Colab’ for short, allows you to write and execute Python in your browser, with

Zero configuration required
Free access to GPUs
Easy sharing

6.3) Roboflow

Roboflow is a Computer Vision developer framework for better data collection preprocessing, and model training techniques. Roboflow has public datasets readily available to users and has access for users to upload their own custom data also.

6.4) Yolo

YOLO is an algorithm that uses neural networks to provide real-time object detection. This algorithm is popular because of its speed and accuracy. It has been used in various applications to detect traffic signals, people, parking meters, and animals.

6.1) Darknet Framework

Darknet is a high-performance open-source framework for the implementation of neural networks. Written in C and CUDA, it can be integrated with CPUs and GPUs.

Advanced implementations of deep neural networks can be done using Darknet. These implementations include You Only Look Once (YOLO) for real-time object detection, ImageNet classification, recurrent neural networks (RNNs), and many others.

Chapter: 7-Testing, Results and Future Scope

Training an object detection model on custom images requires a lot of time so to avoid mistakes we will first train it on a subset of the dataset. After training is done on the subset we will test the accuracy of the model on the test images which we had split during preprocessing. If we are satisfied with the accuracy we will then train the model on all the images. The testing results can be visualized with a graph of the loss vs accuracy.

After training on the full dataset if we get a satisfying accuracy we will save the trained weights and the model and then use them to create a web app for object detection and finally deploy the app on the cloud.

The Future Scope of this project is that this technology can be applied in more robust AI fields such as Autonomous Vehicles, Surveillance Systems, etc

The interface for object detection in images-

The following shows the list of detected objects along with their confidence levels.

The output image is as follows:

Chapter: 8-References

Real-time Vehicle monitoring for traffic surveillance and adaptive change detection usingRaspberry Pi camera module 2017 IEEE Region 10 Humanitarian Technology Conference(R10- HTC)
Convolutional Neural Network- Based Human Detection in Nighttime Images Using VisibleLight Camera Sensors. Academic Editor: Vittorio M. N. Passaro Received: 31 March 2017; Accepted: 4 May 2017; Published: 8 May 2017
Making of Night Vision: Object Detection Under Low-Illumination Received May 24, 2020, accepted July 2, 2020, date of publication July 7, 2020, date of current version July 17, 2020. Digital Object Identifier 10.1109/ACCESS.2020.3007610
Learning Enriched Features for Real Image Restoration and Enhancement Inception Institute of Artificial Intelligence, UAE Mohamed bin Zayed University of ArtificialIntelligence, UAE University of California, Merced, USA Google Research
Object Detection for Night Surveillance 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing
Joseph Redmon, Santosh Divval, Ross Girshick, Ali Farhadi. ”You Only Look Once: Unified, Real-Time Object Detection” [J].2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2016:779788.
P. Viola and M. Jones, “Robust real-time face detection,” Int. J. of Comput. Vision, vol. 57, no. 2, pp. 137–154, 2004.
X. Sun, P. Wu, and S. C. Hoi, “Face detection using deep learning: An improved faster rcnn approach,” arXiv:1701.08289, 2017
Anwar, S., Khan, S., Barnes, N.: A deep journey into super-resolution: A survey. arXiv(2019)

Credit: This project on “Real-Time Moving Object Detection for Day-Night Surveillance using AI” was completed by Gaurav Barve, Aditya Chache, and Vineet Das in partial fulfillment of the requirements of the degree of Bachelor of Computer Engineering from Mumbai University / Datta Meghe College Of Engineering.