Introduction of Backtracking

The Backtracking is an algorithmic-method to solve a problem with an additional way. It uses a recursive
approach to explain the problems. We can say that the backtracking is needed to find all possible
combination to solve an optimization problem.

Backtracking is a systematic way of trying out different sequences of decisions until we find one that
"works."

In the following Figure:

o Each non-leaf node in a tree is a parent of one or more other nodes (its children)
o Each node in the tree, other than the root, has exactly one parent

Generally, however, we draw our trees downward, with the root at the top.

Hello Java Program for Beginners

A tree is composed of nodes.

Backtracking can understand of as searching a tree for a particular "goal" leaf node.

Backtracking is undoubtedly quite simple - we "explore" each node, as follows:

To "explore" node N:

1. If N is a goal node, return "success"

2. If N is a leaf node, return "failure"

3. For each child C of N,

Explore C

If C was successful, return "success"

4. Return "failure"

Backtracking algorithm determines the solution by systematically searching the solution space for the given
problem. Backtracking is a depth-first search with any bounding function. All solution using backtracking
is needed to satisfy a complex set of constraints. The constraints may be explicit or implicit.

Explicit Constraint is ruled, which restrict each vector element to be chosen from the given set.

Implicit Constraint is ruled, which determine which each of the tuples in the solution space, actually satisfy
the criterion function.

N-Queen Problem.
The N Queen is the problem of placing N chess queens on an N×N chessboard so that no two queens
attack each other. For example, following is a solution for 4 Queen problem.
The expected output is a binary matrix which has 1s for the blocks where queens are placed. For example,
following is the output matrix for the above 4 queen solution.

{ 0, 1, 0, 0}

{ 0, 0, 0, 1}

{ 1, 0, 0, 0}

{ 0, 0, 1, 0}

Backtracking Algorithm: The idea is to place queens one by one in different columns, starting from the
leftmost column. When we place a queen in a column, we check for clashes with already placed queens. In
the current column, if we find a row for which there is no clash, we mark this row and column as part of
the solution. If we do not find such a row due to clashes then we backtrack and return false.
1) Start in the leftmost column

2) If all queens are placed

return true

3) Try all rows in the current column. Do following for every tried row.

a) If the queen can be placed safely in this row then mark this [row,

column] as part of the solution and recursively check if placing

queen here leads to a solution.

b) If placing the queen in [row, column] leads to a solution then return

true.

c) If placing queen doesn't lead to a solution then unmark this [row,

column] (Backtrack) and go to step (a) to try other rows.

3) If all rows have been tried and nothing worked, return false to trigger

backtracking.
8 Queens Problem using Backtracking
You are given an 8x8 chessboard, find a way to place 8 queens such that no queen can attack any other
queen on the chessboard. A queen can only be attacked if it lies on the same row, or same column, or the
same diagonal of any other queen. Print all the possible configurations.

To solve this problem, we will make use of the Backtracking algorithm. The backtracking algorithm, in
general checks all possible configurations and test whether the required result is obtained or not. For the
given problem, we will explore all possible positions the queens can be relatively placed at. The solution
will be correct when the number of placed queens = 8.
The time complexity of this approach is O(N!).

Input Format - the number 8, which does not need to be read, but we will take an input number for the sake
of generalization of the algorithm to an NxN chessboard.
Output Format - all matrices that constitute the possible solutions will contain the numbers 0(for empty
cell) and 1(for a cell where queen is placed). Hence, the output is a set of binary matrices.
Visualisation from a 4x4 chessboard solution :
In this configuration, we place 2 queens in the first iteration and see that checking by placing further queens
is not required as we will not get a solution in this path. Note that in this configuration, all places in the third
rows can be attacked.

As the above combination was not possible, we will go back and go for the next iteration. This means we
will change the position of the second queen.
In this, we found a solution.

Now let's take a look at the backtracking algorithm and see how it works:

The idea is to place the queens one after the other in columns, and check if previously placed queens cannot
attack the current queen we're about to place.

If we find such a row, we return true and put the row and column as part of the solution matrix. If such a
column does not exist, we return false and backtrack*

Pseudocode

START
1. begin from the leftmost column
2. if all the queens are placed,
return true/ print configuration
3. check for all rows in the current column
a) if queen placed safely, mark row and column; and
recursively check if we approach in the current
configuration, do we obtain a solution or not
b) if placing yields a solution, return true
c) if placing does not yield a solution, unmark and
try other rows
4. if all rows tried and solution not obtained, return
false and backtrack
END

Implementation
Implementaion of the above backtracking algorithm :

#include <bits/stdc++.h>

using namespace std;

int board[8][8]; // you can pick any matrix size you want

bool isPossible(int n,int row,int col){ // check whether

 // placing queen possible or not
// Same Column
for(int i=row-1;i>=0;i--){
if(board[i][col] == 1){
return false;
}
}
//Upper Left Diagonal
for(int i=row-1,j=col-1;i>=0 && j>=0 ; i--,j--){
if(board[i][j] ==1){
return false;
}
}
// Upper Right Diagonal
for(int i=row-1,j=col+1;i>=0 && j<n ; i--,j++){
if(board[i][j] == 1){
return false;
}
}
return true;
}
void nQueenHelper(int n,int row){
if(row==n){
// We have reached some solution.
// Print the board matrix
// return
for(int i=0;i<n;i++){
for(int j=0;j<n;j++){
cout << board[i][j] << " ";
}
}
cout<<endl;
return;
}
// Place at all possible positions and move to smaller problem
for(int j=0;j<n;j++){
 if(isPossible(n,row,j)){ // if no attack, proceed
board[row][j] = 1; // mark row, column with 1
nQueenHelper(n,row+1); // call function to continue // further
}
board[row][j] = 0; // unmark to backtrack
}
return;
}
void placeNQueens(int n){
memset(board,0,8*8*sizeof(int)); // allocate 8*8 memory
// and initialize all
// cells with zeroes
nQueenHelper(n,0); // call the backtracking function
// and print solutions
}

int main(){

int n;
cin>>n; // could use a default 8 as well

placeNQueens(n);
return 0;
}
C++
Copy

Output ( for n = 4): 1 indicates placement of queens

0 0 1 0
1 0 0 0
0 0 0 1
0 1 0 0

Explanation of the above code solution:

These are two possible solutions from the entire solution set for the 8 queen problem.

main()
{
call placeNQueens(8),

placeNQueens(){

call nQueenHelper(8,0){ row = 0

if(row==n) // won't execute as 0 != 8

for(int j=0; j<8; j++){

{
if(isPossible==true)
{ board[0][0] = 1 // board[row][0] = 1

call nQueenHelper(8,row+1) // recur for all rows further

print matrix when row = 8 if solution obtained

and (row==n) condition is met

}
board[0][0] = 0 // backtrack and try for
// different configurations
}
 }

}
}
C++
Copy

for example, the following configuration won't be displayed

Time Complexity Analysis

1. the isPossible method takes O(n) time

2. for each invocation of loop in nQueenHelper, it runs for O(n) time

3. the isPossible condition is present in the loop and also calls nQueenHelper which is recursive

adding this up, the recurrence relation is:

T(n) = O(n^2) + n * T(n-1)

solving the above recurrence by iteration or recursion tree,

the time complexity of the nQueen problem is = O(N!)

N-Queens Problem

N - Queens problem is to place n - queens in such a manner on an n x n chessboard that no queens attack
each other by being in the same row, column or diagonal.

It can be seen that for n =1, the problem has a trivial solution, and no solution exists for n =2 and n =3. So
first we will consider the 4 queens problem and then generate it to n - queens problem.

Given a 4 x 4 chessboard and number the rows and column of the chessboard 1 through 4.
Since, we have to place 4 queens such as q1 q2 q3 and q4 on the chessboard, such that no two queens attack
each other. In such a conditional each queen must be placed on a different row, i.e., we put queen "i" on row
"i."

Now, we place queen q1 in the very first acceptable position (1, 1). Next, we put queen q2 so that both these
queens do not attack each other. We find that if we place q2 in column 1 and 2, then the dead end is
encountered. Thus the first acceptable position for q2 in column 3, i.e. (2, 3) but then no position is left for
placing queen 'q3' safely. So we backtrack one step and place the queen 'q2' in (2, 4), the next best possible
solution. Then we obtain the position for placing 'q3' which is (3, 2). But later this position also leads to a
dead end, and no place is found where 'q4' can be placed safely. Then we have to backtrack till 'q1' and place
it to (1, 2) and then all other queens are placed safely by moving q2 to (2, 4), q3 to (3, 1) and q4 to (4, 3).
That is, we get the solution (2, 4, 1, 3). This is one possible solution for the 4-queens problem. For another
possible solution, the whole method is repeated for all partial solutions. The other solutions for 4 - queens
problems is (3, 1, 4, 2) i.e.

The implicit tree for 4 - queen problem for a solution (2, 4, 1, 3) is as follows:
Fig shows the complete state space for 4 - queens problem. But we can use backtracking method to generate
the necessary node and stop if the next node violates the rule, i.e., if two queens are attacking.

4 - Queens solution space with nodes numbered in DFS

It can be seen that all the solutions to the 4 queens problem can be represented as 4 - tuples (x1, x2, x3, x4)
where xi represents the column on which queen "qi" is placed.

One possible solution for 8 queens problem is shown in fig:

1. Thus, the solution for 8 -queen problem for (4, 6, 8, 2, 7, 1, 3, 5).
2. If two queens are placed at position (i, j) and (k, l).
3. Then they are on same diagonal only if (i - j) = k - l or i + j = k + l.
4. The first equation implies that j - l = i - k.
5. The second equation implies that j - l = k - i.
6. Therefore, two queens lie on the duplicate diagonal if and only if |j-l|=|i-k|

Place (k, i) returns a Boolean value that is true if the kth queen can be placed in column i. It tests both
whether i is distinct from all previous costs x1, x2,....xk-1 and whether there is no other queen on the same
diagonal.

Using place, we give a precise solution to then n- queens problem.

1. Place (k, i)
2. {
3. For j ← 1 to k - 1
4. do if (x [j] = i)
5. or (Abs x [j]) - i) = (Abs (j - k))
6. then return false;
7. return true;
8. }

Place (k, i) return true if a queen can be placed in the kth row and ith column otherwise return is false.

x [] is a global array whose final k - 1 values have been set. Abs (r) returns the absolute value of r.

1. N - Queens (k, n)
2. {
3. For i ← 1 to n
4. do if Place (k, i) then
5. {
6. x [k] ← i;
7. if (k ==n) then
8. write (x [1....n));
9. else
10. N - Queens (k + 1, n);
11. }
12. }

Branch and Bound Algorithm

Branch and bound is an algorithm design paradigm which is generally used for solving combinatorial
optimization problems. These problems are typically exponential in terms of time complexity and may require
exploring all possible permutations in worst case. The Branch and Bound Algorithm technique solves these
problems relatively quickly.
Let us consider the 0/1 Knapsack problem to understand Branch and Bound.
There are many algorithms by which the knapsack problem can be solved:

Let’s see the Branch and Bound Approach to solve the 0/1 Knapsack problem: The Backtracking Solution
can be optimized if we know a bound on best possible solution subtree rooted with every node. If the best in
subtree is worse than current best, we can simply ignore this node and its subtrees. So we compute bound (best
solution) for every node and compare the bound with current best solution before exploring the node.
Example bounds used in below diagram are, A down can give $315, B down can $275, C down can $225, D
down can $125 and E down can $30.

Given weights and values of n items, put these items in a knapsack of capacity W to get the maximum total
value in the knapsack. In other words, given two integer arrays, val[0..n-1] and wt[0..n-1] represent values
and weights associated with n items respectively. Also given an integer W which represents knapsack
capacity, find out the items such that sum of the weights of those items of a given subset is smaller than or
equal to W. You cannot break an item, either pick the complete item or don’t pick it (0-1 property).
Examples :

Input : val[] = {60, 100, 120};

wt[] = {10, 20, 30};

W = 50;

Output : 220 //maximum value that can be obtained

30 20 //weights 20 and 30 are included.

Input : val[] = {40, 100, 50, 60};

wt[] = {20, 10, 40, 30};

W = 60;

Output : 200

30 20 10

Approach :
Let val[] = {1, 4, 5, 7}, wt[] = {1, 3, 4, 5}
W = 7.
The 2d knapsack table will look like :

Start backtracking from K[n][W].Here K[n][W] is 9.

Since this value comes from the top (shown by grey arrow), the item in this row is not included. Go
vertically upward in the table without including this in the knapsack. Now, this value K[n-1][W] which is 9
doesn’t come from the top which means the item in this row is included and go vertically up and then left by
the weight of the included item ( shown by black arrow). Continuing this process include weights 3 and 4
with a total value 9 in the knapsack.
NP-Completeness | Set 1 (Introduction)
We have been writing about efficient algorithms to solve complex problems, like shortest path, Euler
graph, minimum spanning tree, etc. Those were all success stories of algorithm designers. In this post,
failure stories of computer science are discussed.
Can all computational problems be solved by a computer? There are computational problems that can
not be solved by algorithms even with unlimited time. For example Turing Halting problem (Given a
program and an input, whether the program will eventually halt when run with that input, or will run
forever). Alan Turing proved that general algorithm to solve the halting problem for all possible program-
input pairs cannot exist. A key part of the proof is, Turing machine was used as a mathematical definition of
a computer and program (Source Halting Problem).
Status of NP Complete problems is another failure story, NP complete problems are problems whose status
is unknown. No polynomial time algorithm has yet been discovered for any NP complete problem, nor has
anybody yet been able to prove that no polynomial-time algorithm exist for any of them. The interesting part
is, if any one of the NP complete problems can be solved in polynomial time, then all of them can be solved.

What are NP, P, NP-complete and NP-Hard problems?

P is set of problems that can be solved by a deterministic Turing machine in Polynomial time.
NP is set of decision problems that can be solved by a Non-deterministic Turing Machine in Polynomial
time. P is subset of NP (any problem that can be solved by deterministic machine in polynomial time can
also be solved by non-deterministic machine in polynomial time).
Informally, NP is set of decision problems which can be solved by a polynomial time via a “Lucky
Algorithm”, a magical algorithm that always makes a right guess among the given set of choices
(Source Ref 1).
NP-complete problems are the hardest problems in NP set. A decision problem L is NP-complete if:
1) L is in NP (Any given solution for NP-complete problems can be verified quickly, but there is no efficient
known solution).
2) Every problem in NP is reducible to L in polynomial time (Reduction is defined below).
A problem is NP-Hard if it follows property 2 mentioned above, doesn’t need to follow property 1.
Therefore, NP-Complete set is also a subset of NP-Hard set.
Decision vs Optimization Problems
NP-completeness applies to the realm of decision problems. It was set up this way because it’s easier to
compare the difficulty of decision problems than that of optimization problems. In reality, though, being
able to solve a decision problem in polynomial time will often permit us to solve the corresponding
optimization problem in polynomial time (using a polynomial number of calls to the decision problem). So,
discussing the difficulty of decision problems is often really equivalent to discussing the difficulty of
optimization problems. (Source Ref 2).
For example, consider the vertex cover problem (Given a graph, find out the minimum sized vertex set that
covers all edges). It is an optimization problem. Corresponding decision problem is, given undirected graph
G and k, is there a vertex cover of size k?
What is Reduction?
Let L1 and L2 be two decision problems. Suppose algorithm A2 solves L2. That is, if y is an input for L2 then
algorithm A2 will answer Yes or No depending upon whether y belongs to L2 or not.
The idea is to find a transformation from L1 to L2 so that the algorithm A2 can be part of an algorithm A1 to
solve L1.

Learning reduction in general is very important. For example, if we have library functions to solve certain
problem and if we can reduce a new problem to one of the solved problems, we save a lot of time. Consider
the example of a problem where we have to find minimum product path in a given directed graph where
product of path is multiplication of weights of edges along the path. If we have code for Dijkstra’s algorithm
to find shortest path, we can take log of all weights and use Dijkstra’s algorithm to find the minimum
product path rather than writing a fresh code for this new problem.
How to prove that a given problem is NP complete?
From the definition of NP-complete, it appears impossible to prove that a problem L is NP-Complete. By
definition, it requires us to that show every problem in NP is polynomial time reducible to L. Fortunately,
there is an alternate way to prove it. The idea is to take a known NP-Complete problem and reduce it to
L. If polynomial time reduction is possible, we can prove that L is NP-Complete by transitivity of reduction
(If a NP-Complete problem is reducible to L in polynomial time, then all problems are reducible to L in
polynomial time).
What was the first problem proved as NP-Complete?
There must be some first NP-Complete problem proved by definition of NP-Complete problems. SAT
(Boolean satisfiability problem) is the first NP-Complete problem proved by Cook (See CLRS book for
proof).
It is always useful to know about NP-Completeness even for engineers. Suppose you are asked to write an
efficient algorithm to solve an extremely important problem for your company. After a lot of thinking, you
can only come up exponential time approach which is impractical. If you don’t know about NP-
Completeness, you can only say that I could not come with an efficient algorithm. If you know about NP-
Completeness and prove that the problem as NP-complete, you can proudly say that the polynomial time
solution is unlikely to exist. If there is a polynomial time solution possible, then that solution solves a big
problem of computer science many scientists have been trying for years.

We will soon be discussing more NP-Complete problems and their proof for NP-Completeness.

P and NP problems and solutions | Algorithms

P Problems

P is the set of all the decision problems solvable by deterministic algorithms in polynomial time.

NP Problems

NP is the set of all the decision problems that are solvable by non - deterministic algorithms in polynomial
time.

Since deterministic algorithms are just the special case of non - deterministic ones, so we can conclude that
P is the subset of NP.

Relation between P and NP

NP Hard Problem

A problem L is the NP hard if and only if satisfiability reduces to L. A problem is NP complete if and only if
L is the NP hard and L belongs to NP.

Only a decision problem can be NP complete. However, an optimization problem may be the NP hard.
Furthermore if L1 is a decision problem and L2 an optimization problem, then it is possible that L1 α L2.
One can trivially show that the knapsack decision problem reduces to knapsack optimization problem. For
the clique problem one can easily show that the clique decision problem reduces to the clique optimization
problem. In fact, one can also show that these optimization problems reduce to their corresponding decision
problems.

NP Completeness Problem

Polynomial time reductions provide a formal means for showing that one problem is at least as hard as
another, within a polynomial time factor. This means, if L1 <= L2, then L1 is not more than a polynomial
factor harder than L2. Which is why the “less than or equal to” notation for reduction is mnemonic. NP
complete are the problems whose status are unknown.

Some of the examples of NP complete problems are:

1. Travelling Salesman Problem:

Given n cities, the distance between them and a number D, does exist a tor programme for a salesman to
visit all the cities so that the total distance travelled is at most D.

2. Zero One Programming Problem:

Given m simultaneous equations,

3. Satisfiability Problem:

Given a formula that involves propositional variables and logical connectives.

A language L is the subset [0, 1]* is NP complete if,

1. L belongs to NP and
2. L' ← L for every L' belongs to NP

All NP complete problems are NP hard, but some NP hard problems are not known to be NP complete.

If NP hard problems can be solved in polynomial time, then all the NP complete problems can be solved in
polynomial time.