Computational Complexity

Definition of Mixed-integer programming:

max {cx+ hy : Ax+Gy ≤b , x ∈ Z +¿ , y ∈ R+¿ }¿ ¿
n P

Here,
p and n are non-negative integers such that p+n ≥ 1
c , h , A , G , b are matrices with integral coefficients (More convenient than rational coefficients)

Matrix Dimension [m is positive integer]

C 1xn
H 1xp
A mxn
G mxp
B mx1

 A problem consists of infinite number of instances

o An instance is specified by assigning numerical values called data to the problem
parameters
 In case of mixed integer programming, the parameters are the value of p,n,m as well as
the dimension of the matrices
 For example:
o In pure integer programming, p=0, and h and G do not exist
o In linear programming, n=0, and c and A do not exist
 An algorithm that can solve all instances of mixed integer programming can solve all
instances of the special cases of pure-integer and linear programming
o Every instance of a linear or pure-integer program is also an instance of a mixed-
integer program
o Mixed integer programming is at least as hard as pure-integer and linear
programming
 If there is a directed path from problem X1 to problem X2, then every instance of X2 is
also an instance of X1.
Set packing:

 Each coefficient of matrix A is 0 or 1

 b is a column of 1s
Node Packing:

 Instance of set packing

 Each row of A has exactly two 1s
Algorithm Efficiency and Problem Complexity:

 Computation time is related to problem size

Size of Problem:
The amount of information required to describe the instance in a standard representation

 The data needed are generally obvious

o For example: An instance of integer programming is specified by integers m and n and
matrices A,b and c
 A robust approach for representing the essence of real computation is to use a binary (0,1)
alphabet for the representation of numerical and logical data
o In this model, a positive integer x (20 ≤ x n ≤2n +1 ¿ , is represented by the vectors
(δ 1 ,δ 2 ,… … .. , δ n), where-

δ i ϵ { 0,1 } ∀ i=1 , … ,n
n
x=∑ δ i 2i
i=0

o An additional digit is required for representation of the sign of x

o Rational numbers are represented by pairs of integers
 Initial data are assumed to be integral or rational
 Subsets of a set can be represented by incidence or characteristic vector
o If Q= {1,2 , … . , n } , the subset Q j is given by the vector (q 1 , q 2 , … … , q n) where-

q i= 1 if q i ∈ Q
{
0 otherwise
 Graphs can be represented by-
o Node-incidence matrix
 A graph with m nodes and n edges can be represented by a m x n matrix
o Adjacency matrix
o A list of its elements
 Size of an instance of optimization problem is traditionally defined by the number of variables
and constraints
o Not always adequate
o Some algorithms’ number of steps depends explicitly on the magnitude of the numerical
data
 An algorithm in integer knapsack problem’s number of computations is
proportional to number of variables times the right hand side of the constraint
 In Ellipsoid method, the number of computation depends on the magnitude of
numerical data
o Size of numbers involved in elementary calculations (e.g: addition and multiplication)
may also be of concern
 It is frequently assumed that these operations are done in constant or unit time
 
Not reasonable for huge integers
 Alphabet used to represent data must contain at least two symbols

Running Time of an Algorithm:

Let us consider an optimization problem X consisting an infinite number of instances ( d 1 , d 2 , … … … . ) ,

where the data for the instance d i is given by a binary string of length l i=l(d i )

Let A be an algorithm that can solve each instance of X in finite time g A (d i). Then the running time
associated with all instances of size k is given by-

f A ( k )=max { g A ( di ) :l ( d i )=k }

 Conservative measure of running time

o Considers the worst-possible outcome for each size
 Since two instance having the same length does not necessarily have the
same running time, some statistic must be used to aggregate the running
times for all instances of the same length
 Advantage-
o Absolute guarantee on running time
o Independent of probability distribution of the instances
o Easiest measure for analysis
 Disadvantage
o Failure to give a true picture when a large percentage of instances of a given size
can be solved rapidly and only a small percentage need more time
 Expected running time may be used
Polynomial Time Algorithm:
A polynomial-time algorithm is an algorithm whose execution time is either given by a
polynomial on the size of the input, or can be bounded by such a polynomial

 Algorithm A is said to be a polynomial time algorithm for problem x if f A ( k )=O ( k p ) for

some constant p.
 P is the class of problems that can be solved in polynomial time
 Problem x ∈ P denotes that there exists a polynomial time algorithm for solving x
Exponential Time:

f is said to be exponential if for some constants c 1 , c2 >0 and d 1 , d 2>1 and a positive integer k’,
we have-

c 1 d k1 ≤ f (k )≤ c 2 d k2

Decision/Feasibility Problem:
A feasibility or decision problem X is a pair (D, F) with F ⊆ D, where
D is the set of instances of X
F is the set of feasible instances

 The elements of D are finite binary strings.

Given an instance d ∈ D, we want to determine whether d ∈ F.

 The answer is either yes or not

o One does not needs to identify the solution
o Just needs to assure that the solution to Yes instance has a polynomial length

For example: Associated with an optimization problem, we define a feasibility problem in which
an instance corresponds to a description of a constraint set. F is the set of instances for which
the constraint set is nonempty.
Let us consider the optimization problem max {cx : x ∈ S }. An instance of this consists of {c, and
a standard representation of S}
Here, D=S and F should be a value greater than the value of cx for the previous instance. We
can consider that value as k.
Its corresponding decision problem can be:
Isthere any x ∈ S with value cx ≥k
Here, an instance consists of {c, a standard representation of S, and an integer k}
NP:

It is the class of decision problems with the property that for any instance for which the answer
is Yes, there is a short (polynomial) proof of Yes.

 If P={ x ∈ R +¿ : Ax≤b }¿ is an unbounded polyhedron and max{cx : x ∈ P } is finite, then any

n

extreme point of P has a polynomial length in the description length of P

Polynomial Reducibility:
If an instance of P∈ NP can be converted to an instance of Q ∈ NP in polynomial time, then P is
said to be polynomially reducible to Q.

 If we have an algorithm for problem Q, it can be used to solve problem P with an

overhead that is polynomial in the size of the instance
 Polynomial reduction is denoted by sign

NP-Complete Problem:
The subset of problems P ∈ NP such that for all Q∈ NP , Q is polynomially reducible to P.
  If we have a NP-Complete problem, we may be able to find others by polynomial reduction
 If x 1 ∈ NPC , and x 1 is polynomially reducible to x 2 ∈ NP , then x 2 ∈ NPC

Certificate of Feasibility:
Information that can be used to check feasibility in polynomial time is called a certificate of feasibility

 Given X =(D , F), for each instance d ∈ F we let Q d denote such a certificate
 Qd must be short

Non-deterministic Algorithm:

 Used to find certificate of feasibility

 Two stages-
o Guessing stage
 Input is d ∈ D
 A binary string Q is guessed and passed to second stage
o Checking stage
 Works with pair (d,Q) and may provide output that d ∈ F
 Work done measured
 Non-deterministic algorithm is feasible when the checking stage is a polynomial function
of the length of the encoding of d for some Qd for which it replies that d ∈ F.
o When d ∈ F, there is a short proof of feasibility

Satisfiability Problem:
Let a finite set N={1,2,……,n}
Also let us assume m pairs of subsets of N be denote by-
C i=¿

An instance is feasible if the following set is non-empty

¿
 First problem in NP shown to be NP−complete
The problem is in NP as one can simply guess a x ¿ ∈ Bn and check easily if it satisfies the constraint

Cook’s Theorem:
The satisfiability problem is NP-complete.
Examples of NP-Complete Problems:

 Satisfiability Problem
  0-1 Integer Program Feasibility Problem
 Node Packing Problem
 Set-partitioning Problem
Node Packing:

A node packing on a graph G = (V, E) is a subset U ⊆ V of the vertices so that no two vertices in U are
adjacent.

{x ∈ Bn : xi + x j ≤ 1 for all(i, j )∈ E }
Here,

n=¿ V ∨¿

x i= 1if node iis∈ packing

{ 0 otherwise
 No edge between nodes in U

Does a collection of sets contain at least   mutually disjoint subsets?

Node Packing Problem:

Given a graph G=(V,E) and integer k, is there a U ⊆ V such that ¿ U∨≥k and U is a node packing

Set Partition Feasibility Problem:

The task of deciding whether a given multiset S of positive integers can be partitioned into two
subsets S1 and S2 such that the sum of the numbers in S1 equals the sum of the numbers in S2.
The feasibility or decision problem is-

Given a 0-1 matrix Am x n, does there exist a x ∈ { 0,1 }n such that Ax=1?

Clique:
A clique in a graph G= (V , E ) is a subset C ⊆ V with the property that every pair of nodes in C is
joined by an edge

 A node packing can contain no more than one node from each clique

NP-hard Problem:
An optimization problem whose decision/feasibility version is NP-complete is known as NP-hard

 A problem is called NP-hard if there is a NP-complete problem that can be polynomially reduced
to it
 An NP-hard problem is at least as difficult as any NP-complete problem
 For example: Finding the maximum cardinality node packing in a graph is NP-hard

 An instance is given by-

o An integer n
o An implicit description of polyhedron P ⊆R n in the family
o An x ¿ ∈ Rn
 A solution is to check whether x ∈ P , and if x ∉ P , then find a valid inequality πx ≤ π 0 for P
¿ ¿

¿
that is violated by x
 Given a family of full-dimensional polyhedral, there is a polynomial time algorithm for the
separation algorithm, if and only if there is a polynomial time algorithm for the linear
programming problem

Ways to Show a problem is in P:

 Finding a polynomial time algorithm

 Polynomial reduction to another problem which we already know is in P
  Checking if the separation problem is solvable

Separation Problem:

A separation problem can be described by-

Is x ∈ conv (S)? If not, find an equally satisfied by all points of S, but cutting off x

 Given a family of polyhedral associated with a class of problems, the family of optimization
problems max { cx : x ∈ conv ( s ) } is polynomially solvable if and only if the family of separation
problems is polynomially solvable
 If separation problem can be solved in polynomial time, it can be used as a subroutine in the
ellipsoid method to solve the optimization problem in polynomial time
o In each iteration of ellipsoid algorithm, we need to check whether the current point is
feasible
o If it is not feasible, then the current ellipsoid is cut to half using a hyperplane that
separates the current solution from the polyhedron
o At each iteration, the separation oracle can be used to generate such a cutting
hyperplane in polynomial time

Ellipsoid Algorithm: