DESIGN OF AN IOT BASED AUTONOMOUS VEHICLE

USING COMPUTER VISION

01 Introduction
It is a driver less autonomous car. A Line Follower Robot is an Autonomous Robot
which tracks the line and follow them. A line following Robot is a robot (usually a
vehicle) which follows a distinguished colored path (usually a black lined path). It
consists of several electronic components which makes it follow (usually with tires) a
programmed path. The main component of such robot is a microcontroller which is the
brain of the robot. An Obstacle Avoiding Robot is a robot (usually a vehicle) which
follows a straight path but if any obstruction is present/introduced in its path this will
detect through ultrasonic sensor. And Apart from line following capabilities, these
robots should also have the capability to navigate junctions and decide on which
junction to turn and which junction ignore. This would require the robot to have 90
degree turn and also junction counting capabilities. To add on to the complexity of the
problem, sensor positioning also plays a role in optimizing the robots performance for
the tasks mentioned earlier. Line-following robots with pick- and- placement capabilities
are commonly used in manufacturing plants. These move on a specified path to pick the
components from specified locations and place them on desired locations. Basically, a
line-following robot i s a self-operating robot that detects and follows a line drawn on
the floor. The path to be taken is indicated by a white line on a black surface. Also with
the help of camera if the obstacle detect by ultrasonic sensor it will capture the image
and send on mail with record. Also it will update over IOT. This autonomous robot will
detect the traffic signal also if red signal detects it will pass signals to the controller and
automatically robot will stop. Else robot will move based on path. Camera through we
can monitor every thing. All information will be displayed on LCD.
1.1 Motivation

1.2 Problem Definition

In the industry carriers are required to carry products from one manufacturing
plant to another which are usually in different buildings or separate blocks.
Conventionally, c arts or trucks were used with human drivers. Unreliability and
inefficiency in this part of the assembly line formed the weakest link. The
project is to automate this sector, using carts to follow a line instead of laying
railway tracks which are both costly and an inconvenience.
The robot can be further enhanced to let the user decide whether it is a dark line
on a white background or a white line on a dark background. The robot can also
be programmed to decide what kind of line it is, instead of a user interface. Extra
sensors could be attached to allow the robot to detect obstacles, with traffic
signal detection.
2. literature serve:
1. LANE FOLLOWING AND OBSTACLE DETECTION
TECHNIQUES IN AUTONOMOUS DRIVING VEHICLES
Author:- Phanindra Amaradi1, Nishanth Sriramoju2, Li Dang3, Girma S.
Tewolde4, Jaerock Kwon5

This paper presents techniques developed for lane following and obstacle
detection capabilities for autonomous vehicles. The proposed system is based on
a single 120 degrees fish eye camera and a 180 degrees scanning LIDAR sensor.
For the lane detection a modified lane marking technique was used representing
the position of the lines making the lane with respect to the position of the
camera mounted on the vehicle. The paper describes implementation of the lane
detection system using Hough transform to detect the lanes. Based on the
detected lane markings, the position of the relative vehicle is determined as to
whether it is in its lane or if it has drifted outside the lane. The system shows the
offset amount, as a percentage of distance, it has drifted from the centre of the
lane. The SICK LIDAR LMS-291 S-O5 laser range sensor is adopted for the
obstacle detection.

2. STEREOVISION-BASED REAL-TIME OBSTACLE DETECTION

SCHEME FOR UNMANNED GROUND VEHICLE WITH
STEERING WHEEL DRIVE MECHANISM
Author:- Maham Khan1, Saad Hassan2, Syed Irfan Ahmed3, Jamshed Iqbal4,*
This paper highlights the importance of an autonomous navigation scheme
for an Unmanned Ground Vehicle (UGV) operating under a complex
operational scenario that requires obstacle detection. The proposed scheme is
based on the construction and investigation of disparity images and follows
a two-stage perception structure. During detection phase, the relationship
between encountered obstacles and the robot’s path is inferred. Based on the
extracted Region Of Interest (ROI) and statistical information about
projections, the confirmation phase characterizes the contours and obstacles
positions. Computationally intensive stereovision techniques are optimized
for use in real-time applications. The scheme has been tested on a custom-
developed wheeled mobile robot. The experimental findings show the
benefits of our scheme. The robot detects obstacles of any size and shape
within a range of 80–200cm. The results demonstrate that the robot has the
ability to precisely navigate in a wide range of illumination conditions. The
proposed approach does not need any extraction of lane markers since it
fully exploits the information contained in the disparity images. The
anticipated applications of the proposed scheme include autonomous
navigation of vehicles for environment exploration, perimeter surveillance
and automated delivery of goods in a factory.

3. VISUAL SERVOING LINE FOLLOWING ROBOT: CAMERA-

BASED LINE DETECTING AND INTERPRETING
Author:- Mukhamad Aji Putra, Endra Pitowarno, and Anhar Risnumawan

Autonomous self-driving car has gained a lot of interest with the advance of
robotic technology. An autonomous car using only visual information for
autonomous navigation is expected to mimic the way human driving capability.
The car has to navigate robustly and completing the tasks in a specified
environment. These tasks for example detecting road lanes, pedestrians, or road
signs. In this paper, as a step toward the goal of developing an autonomous self-
driving car, we develop a prototype of autonomous line following robot. Using
only visual information, we study a visual servoing technique for which the
robot able to detect road lanes and navigate autonomously. An image processing
technique is employed to obtain effective features for feedback control. The
controller utilizes features extracted from the image plane, without using the
complete geometric representation. Motion is controlled only on x-axis or
heading angle control. The proportional integral and derivative controller is used
to maintain the heading of the robot. Experiments show encouraging result for
the robot to navigate autonomously given the road lanes and maintain heading
angle.

4. LINE FOLLOWING ROBOT BASED ON VISION TECHNIQUES

Author:- Inian Roy. A1, Karthick raja. B2, Chakkaravarthy. G3, Arun Prakash.
C4
A line follower robot is automated device programed to follow a specific path.
They have a vital role in industries and domestic application. Some of the
existing techniques used in controlling line follower robot are by using
microcontroller, guided tape method etc. This paper aims at providing a machine
vision based line follower robot. Machine vision is an image processing
technique where in pre-programing algorithm are used to process the obtained
image. A camera is used to obtain the image of the track and the obtained image
is processed using suitable image processing software and depending on the
results generated the robot is tracked. Thus by using
machine vision technique the line follower robot is guided along their track.

5. A NOVEL DESIGN OF LINE FOLLOWING ROBOT WITH

MULTIFARIOUS FUNCTION ABILITY

Author:- Hasan U. Zaman, Md. Majidul Haque Bhuiyan, Montashir Ahmed,

and S.M Tarek Aziz
This paper describes the design and implementation of a Line following robot
that follows a line which specifically is black line on white surface. This line
following robot has two distinguished modes, such as line following mode and
obstacle detection mode. It is like a robot car that follows a designated path. This
robot can identify obstacles rightward, leftward and in front obstacles while
following its designated path on it pathway. So, altogether, it can be said the
robot has capability to move following a black line while detecting obstacles
three way round. The design includes knowledge of Arduino programming,
electrical circuit integration with the designed coding and some architectural as
well as basic mechanical engineering knowledge for its development using
Arduino Uno. The robot has three sonars connected in three ways round for
better obstacle detection and two infrared sensors connection which undergoes
with a connection of a motor driver IC (L293D). This follower with multiple
modes has given power by a rechargeable lithium battery of 6 volts. All over the
robot costs 4500 BDT (Bangladeshi Taka)/4253.15 INR (Indian Rupees), 64.136
USD (United States Dollars). The robot is able to carry up to 600gms of
component above while detecting line with identifying obstacles without any
 problem of changing its course on its designated pathway

Software Requirements Specification

Introduction
3.1.1 Project Scope

The purposed for proposed rood is that the line flowing robot. .it is
driverless autonomous robot also it will detect traffic signal with obstacle
detection using ultrasonic sensor. Capture the image using camera.
The study is concerned with autonomous line following robot with
computer vision this mainly used for it is driverless autonomous robot
also it will detect traffic signal with obstacle detection using ultrasonic
sensor.

3.1.2 User Classes and Characteristics

1. Admin

2. System

3. Robot

3.1.3 Assumptions and Dependencies

 It is assumed that the micro-controller recommended input voltage
required for on the micro-controller. Hardware will be made available for
the project in some phase of its completion. Until then, test hardware will
be used for providing the demo for the presentations. It is assumed that the
user is familiar with internet and also familiar with handling the hardware.
Since the application is a hardware based application there is a need for
the different hardware specification. It will be assumed that the users will
possess decent internet connectivity.

Functional Requirements

A system failure can occur because of a hardware failure or a severe software issue,
causing over power supply, short-circuit, damaging of component, wrong connection
etc.. or stop functioning altogether. A system failure may or may not result in an error
being displayed on the screen. The computer may shut off without warning and without
any error message. If an error message is displayed, it often is displayed as error.
System Feature 1

3.1.1 without battery system fails.

The without battery we can’t run the project the robot will stop automatically
system failed.

System Feature 2 (and so on)

4. External Interface without power supply.

3.3 External Interface Requirements (If Any)

There will be two user types – the cluster user and the cluster administrator – each of
which will have its own corresponding interface.
Hardware Interfaces

All hardware interfaces of social question and answer system will be those of the Sun
Grid Engine on top of which it will be running. Hence, it will incorporate SGE’s
interfaces for:

 CPU usage

 Memory usage

 Network communication

Software Interfaces

It has already been stated, it runs every system.so it does not any specification software
it just install the python.

Communications Interfaces

Communication architecture defines the frequency and fidelity of information flow

between individuals in your organization. It helps structure how and when you
communicate, both within a team and cross-functionally. The specific tactics are unique
to each organization, but it requires proactive thought and investment. Other
Nonfunctional Requirements

Nonfunctional Requirements
Performance Requirement:
1. The application will be line follow robot, it detect obstacle and send it on mail also
data will updated over IOT .
2. The scheme ensures confidentiality, integrity and freshness of prove.
3. For good performance the resources must be dedicated and must be normalized.

Nonfunctional Requirements
Performance Requirements
Safety Requirements
Security Requirements
Software Quality Attributes
Safety and Security Requirement:
In Safety Requirement it is ensured that hardware is not harmful to any other program or
Software running on particular machine in any manner.

 Performance Requirement:
System can produce results faster on raspberry pi .
It may take more time for peak loads at main system.
The system will be available 100% of the time. Once there is a fatal error, the system
will provide understandable feed back to the user.
 Safety Requirement
Only administrators have access to the database which is send in robot of each
individual user.
All data will be backed-up every day automatically and also the system administrator
can back-up the data as a function for him.
This makes it easier to install and updates new functionality if required.

 Security Requirement
Our System is being developed in python. Python is an interpreted high-level
programming language for general-purpose programming. Created by Guido van
Rossum and first released in 1991, Python has a design philosophy that
emphasizes code readability, notably using significant whitespace.

 Usability
The system is designed keeping in mind the usability issues considering the end-
users who are developers/programmers. It provides detailed help which would lead
to better and faster learning. Navigation of system is easy.
 Availability

The system shall be available during 24 hours of a day.

Multi lingual Support

Front End: python

Backend: Hardware
 Open Issues

Currently we are in only design phase so we have not faced such kind of open issue
which will cause our system.
System Requirements
3.3.1 Database Requirements:

3.3.2 Software Requirements(Platform Choice)

Introduction of Embedded System:

An Embedded System is a combination of computer hardware and
software, and perhaps additional mechanical or other parts, designed to
perform a specific function. A good example is the microwave oven. Almost
every household has one, and tens of millions of them are used everyday, but
very few people realize that a processor and software are involved in the
preparation of their lunch or dinner.
This is in direct contrast to the personal computer in the family room.
It too is comprised of computer hardware and software and mechanical
components (disk drives, for example). However, a personal computer is not
designed to perform a specific function rather; it is able to do many different
things. Many people use the term general-purpose computer to make this
distinction clear. As shipped, a general-purpose computer is a blank slate; the
manufacturer does not know what the customer will do wish it. One customer
may use it for a network file server another may use it exclusively for playing
games, and a third may use it to write the next great American novel.
Frequently, an embedded system is a component within some larger
system. For example, modern cars and trucks contain many embedded
systems. One embedded system controls the anti-lock brakes, other monitors
and controls the vehicle's emissions, and a third displays information on the
dashboard. In some cases, these embedded systems are connected by some
sort of a communication network, but that is certainly not a requirement.
5.3.3 Hardware Requirements
HARDWARE REQUIREMENTS:
 Raspberry Pi.
 System : Pentium IV 2.4 GHz. And above
 Hard Disk : 40 GB.
 Floppy Drive : 1.44 Mb.
 Monitor : 15 VGA Colour.
 Mouse : Logitech.
 Ram : 512 Mb.
 Ultrasonic sensor
 Arduino Uno
 CAMERA
 LED’S
 LCD DISPLAY
 DC MOTOR
 L293D MOTOR DRIVER

SOFTWARE REQUIREMENTS:

 Operating system : Windows XP/7.

 Coding Language : Python
3.6 Analysis Models: SDLC Model to be applied

Methodologies used. SDLC Module

We are using waterfall model for our project.

1. Requirement gathering and analysis:

In this step of waterfall we identify what are various requirements are need for our
project such are software and hardware required, database, and interfaces.

2. System Design:

In this system design phase we design the system which is easily understood for end
user i.e. user friendly.
We design some UML diagrams and data flow diagram to understand the system flow and
system module and sequence of execution.
3. Implementation:

In implementation phase of our project we have implemented various module

required of successfully getting expected outcome at the different module levels.
With inputs from system design, the system is first developed in small programs called
units, which are integrated in the next phase. Each unit is developed and tested for its
functionality which is referred to as Unit Testing.
3. System Implementation Plan

Sr Task Description Duration

No (Days)
1 Domain selection Deciding the domain as 15
network security for the
project
2 Searching key terms Searching key terms 2
related to the domain
3 Deciding topic for Selecting Smart Crawler 10
project as project topic
4 Literature survey Studied Papers related to 20
smart crawler
5 Problem statement Making the problem 10
statement
6 Synopsis formation Forming the synopsis 15
7 Studying existing system Methodologies and 15
drawbacks of existing
system are made
8 Mathematical Modeling For the mathematical 25
model of project
9 Existing System Implementation of 60
existing system
10 Proposed system Implementation of 70
proposed system
11 Testing Going through various 20
types of testing
12 Data Analysis Analysis of existing and 30
proposed system
13 Report generation Latex report preparation 20
04 System Design

4. System Architecture
4. Data Flow Diagrams

DFD 0

DFD 1

DFD 2
4. Entity Relationship Diagram
4. UML Diagrams

Activity diagram
Advantages
• Robot movement is automatic.
• Fit and Forget system.
• Used for long distance applications.
• Defence applications.
• Used in home, industrial automation.
• Cost effective.
• Simplicity of building.

Limitations
Applications
• Guidance system for industrial robots moving on shop floor etc.
• Industrial applications.
• Home applications.

Conclusions & Future Work

Implementation a lane following and obstacle detection system for autonomous vehicle
applications. The tools developed are implemented to processes the video frames from
the camera to capture the image of signal. Video streaming possible through this project.

Appendix A: Problem statement feasibility assessment using, satisfiability analysis and

NP Hard,NP-Complete or P type using modern algebra and relevant mathematical
models.

What is P?
 • P is set of all decision problems which can be solved in polynomial time by a
deterministic.

• Since it can be solved in polynomial time, it can be verified in polynomial time.

• Therefore P is a subset of NP.

P:
The main objective of this paper is to implement an effective lane following and
obstacle detection system. The work which develops image processing based lane
detection system is comprised of the following steps: filtering, edge detection, lane
detection and offset distance calculation based on position of the vehicle.

What is NP?
• "NP" means "we can solve it in polynomial time if we can break the normal rules
of step-by-step computing".

What is NP Hard?
A problem is NP-hard if an algorithm for solving it can be translated into one for solving
any NP-problem (nondeterministic polynomial time) problem. NP-hard therefore means
"at least as hard as any NP-problem," although it might, in fact, be harder.
Np-Hard:
In the industry carriers are required to carry products from one manufacturing plant to
another which are usually in different buildings or separate blocks. Conventionally, c
arts or trucks were used with human drivers. Unreliability and inefficiency in this part of
the assembly line formed the weakest link. The project is to automate this sector, using
carts to follow a line instead of laying railway tracks which are both costly and an
inconvenience.
The robot can be further enhanced to let the user decide whether it is a dark line on a
white background or a white line on a dark background. The robot can also be
programmed to decide what kind of line it is, instead of a user interface. Extra sensors
could be attached to allow the robot to detect obstacles, with traffic signal detection.

What is NP-Complete?
• Since this amazing "N" computer can also do anything a normal computer can,
we know that "P" problems are also in "NP".

• So, the easy problems are in "P" (and "NP"), but the really hard ones are *only*
in "NP", and they are called "NP-complete".

• It is like saying there are things that People can do ("P"), there are things that
Super People can do ("SP"), and there are things *only* Super People can do
("SP-complete").

NP-Complete:
. Basically, a line-following robot i s a self-operating robot that detects and follows a
line drawn on the floor. The path to be taken is indicated by a white line on a black
surface. Also with the help of camera if the obstacle detect by ultrasonic sensor it will
capture the image and send on mail with record. Also it will update over IOT. This
autonomous robot will detect the traffic signal also if red signal detects it will pass
signals to the controller and automatically robot will stop. Else robot will move based on
path. Camera through we can monitor every thing. All information will be displayed on
LCD.

1.1 References
[I] Pakdaman, M.; Sanaatiyan, M.M., "Design and Implementation of Line
Follower Robot,", Second International Conference on Computer and Electrical
Engineering TCCEE '09, vol.2, pp.585-590,Dec.2009.
[2] PriyankPatil, "A VR Line Following Robot," Department of Information
Technology K. 1. Somaiya College of Engineering Mumbai, India. Mar 5, 2010.
[3] Dean A. Pomerleau; Jay Gowdy; Charles E. Thorpe, "Combining Artificial
Neural Networks and Symbolic Processing for Autonomous Robot Guidance,"
Engng Applic. Artif. Intell. Vol. 4, No. 4, pp. 279-285,1991.
[4] Tstv'an Harmati; Krzysztof Skrzypczyk, "Robot team coordination for target
tracking using fuzzy logic controller in game theoretic framework," Robotics and
Autonomous Systems, 57 ,pp. 75-86,2009.
[5] K.D. Do, "Output-feedback formation tracking control of unicycle-type
mobile robots with limited sensing ranges," Robotics and Autonomous
Systems,57, pp. 34--47, 2009.
[6] L1U Shi-Cai ; TAN Da-Long; L1U Guang-Jun, "Formation Control of
Mobile Robots with Active Obstacle Avoidance," Acta Automatica Sinica, Vol.
33, No. 5, 2007.
[7] YANG Tian-Tian; L1U Zhi-Yuan; CHEN Hong; PEl Run, "Formation
Control and Obstacle Avoidance for Multiple Mobile Robots," Acta Automatica
Sinica, Vol. 34, No. 5, 2008.
[8] Fuyi Xu; Hendrik Van Brussel; Marnix Nuttin; Ronny Moreas, "Concepts for
dynamic obstacle avoidance and their extended application in underground
navigation," Robotics and Autonomous Systems, 42 pp. I-IS, 2003.
[9] Guilherme Augusto Silva Pereiraa, Vijay Kumarb, Mario Fernando
Montenegro Campos, "Closed loop motion planning of cooperating mobile
robots using graph connectivity," Robotics and Autonomous Systems, 56 ,pp.
373-384,2008.
[10] Jose Manuel Gascuena, Antonio Fernandez-Caballero, "Agent-oriented
modeling and development of a person-following mobile robot," Expert Systems
with Applications, 38, pp. 4280-4290,2011.
[11] Sandhana, Lakshmi (2002-09-05), "A Theory of Evolution for Robots,"
Wired Magazine .. Retrieved 2007-10-28.