Design and Analysis of Algorithm
Definition of lower-bound arguments
Different algorithms using decision tree
Application Application of P, NP and NP-complete NP -complete problems
Limitations of algorithm Power
* Property of STI
Page Page 1 of 25 25
Design and Analysis of Algorithm
A reasonable evaluation of algorithms as problemsolving tools is inevitable. inevitable.
These algorithms are considered as powerful instruments, particularly when these are processed by modern computers.
Sometimes there are problems that cannot be solved by an algorithm or the problems seem not to have efficient solutions.
Limitations of algorithm Power
* Property of STI
Page Page 2 of 25 25
Design and Analysis of Algorithm
is a comparison-based comparison-based algorithm where the behavior of the algorithm is based only on the comparison between elements [www.cs.toronto.edu].
The lower bound deals with the complexity of a problem instead of the algorithm.
Basically we have to prove that no algorithm , no matter how complex, can do better than our bound
Limitations of algorithm Power
* Property of STI
Page Page 3 of 25 25
Design and Analysis of Algorithm
number of comparisons needed to find the t he largest element in a set of n numbers
number of comparisons needed to sort an array of size n
number of comparisons necessary for searching in a sorted array
number of multiplications needed to multiply two n-by-n matrices
Limitations of algorithm Power
* Property of STI
Page Page 4 of 25 25
Design and Analysis of Algorithm
Establish the asymptotic efficiency class similar to the worst case
Single out where this class stands with respect to the hierarchy of efficiency class
Limitations of algorithm Power
* Property of STI
Page Page 5 of 25 25
Design and Analysis of Algorithm
is based on counting the number of items that must be processed in input and generated as output [LEV07].
Examples:
finding finding max eleme element nt -- n steps steps or n/2 comparisons
polynomial evaluation
sorting
element uniqueness
Hamiltonian circuit existence
Conclusions:
may and may not be useful
be careful in deciding how many elements must be processed
Limitations of algorithm Power
* Property of STI
Page Page 6 of 25 25
Design and Analysis of Algorithm
seek to establish a lower bound based on the amount of information it has to produce [LEV07]. Example:
Deducing a positive integer between 1 and n selected by somebody by asking that person questions with yes/no answers.
Conclusions:
The approach is related to information theory because it has proved to be useful for finding the information-theoretic bounds in different problem types concerning comparisons, such as sorting and searching.
Its core concept can be apprehended more precisely through decision trees.
Limitations of algorithm Power
* Property of STI
Page Page 7 of 25 25
Design and Analysis of Algorithm
Malevolent simply means that it will change a problem to force algorithm down worst path and we can say that it is fair because we must remain consistent with the work that is already done for selection problems.
Examples:
Three-Card Monte
n-Card Monte
Finding Patterns in Bit Strings
Evasive Graph Properties
Conclusions:
The adversary should be thought of as a very powerful, clever being that is trying to make your algorithm run as slowly as possible.
The adversary cannot "read the algorithm's mind", but it can try to be prepared for anything the algorithm does.
Limitations of algorithm Power
* Property of STI
Page Page 8 of 25 25
Design and Analysis of Algorithm
proves a lower bound by transforming a problem that already has an established lower bound into the new problem.
Examples:
Reduction to linear programming
Algorithmic problem solving
Reduction to graph problems
Evasive Graph Properties
Limitations of algorithm Power
* Property of STI
Page Page 9 of 25 25
Design and Analysis of Algorithm
We must prove that an arbitrary instance of problem Q can be transformed in a logically efficient manner to an instance of problem P.
As a result, any algorithm solving P would solve Q as well. Therefore, the lower bound for Q will also be the lower bound for P.
The figure below are the list of problems used for establishing lower bounds by problem reduction:
Limitations of algorithm Power
* Property of STI
Page 10 of 25 25
Design and Analysis of Algorithm
is a convenient model of algorithms involving comparisons in which internal nodes represent comparisons and leaves represent outcomes (or input cases).
The figure below represents the decision tree of an algorithm for finding a minimum value of three numbers:
Limitations of algorithm Power
* Property of STI
Page 11 of 25 25
Design and Analysis of Algorithm
By studying the properties of decision trees for comparison-based comparison-based sorting algorithms, we can derive important lower bounds on time efficiencies of such algorithms. algorithms. Consider the following figure:
Limitations of algorithm Power
* Property of STI
Page 12 of 25 25
Design and Analysis of Algorithm
Inequality implies that the height of a binary decision tree for any comparison-based sorting algorithm and the worst-case number of comparisons made such an algorithm cannot be less than [log 2n!]:
We can also use the decision trees for analyzing analyzing the average-case behavior of a comparison-based comparison-based sorting algorithm.
Limitations of algorithm Power
* Property of STI
Page 13 of 25 25
Design and Analysis of Algorithm
We can compute the average number of comparisons for a particular algorithm as the average depth of its decision tree’s leaves like the average path length from the root to the leaves.
The figure shows an example of a decision tree for the three element in the insertion sort :
Limitations of algorithm Power
* Property of STI
Page 14 of 25 25
Design and Analysis of Algorithm
An algorithm solves a problem in polynomial time if its worst-case time efficiency belongs to O( p(n)) time, where p(n) is a polynomial of problem’s input size n [LEV07].
: problems that can be solved in polynomial time.
problems that cannot be solved in polynomial time.
Limitations of algorithm Power
* Property of STI
Page 15 of 25 25
Design and Analysis of Algorithm
Reasons for drawing the intractability line :
We cannot solve arbitrary instances of intractable problems in a reasonable amount of time unless such instances are very small.
Although there might be a huge difference between the running time in O(p(n)) for polynomials of drastically different degrees, there are very few useful polynomial time algorithm with the degree of polynomial higher than three.
Polynomial functions posses many convenient properties, in particular both the sum and composition of two polynomials are always polynomials, too.
The choice of this class led to a development of an extensive theory called computational complexity.
Limitations of algorithm Power
* Property of STI
Page 16 of 25 25
Design and Analysis of Algorithm
is a class of decision problems that can be solved in polynomial time by deterministic algorithms. This class class of problems is called called polynomial [LEV07].
The restriction of P to decision problems can be justified by the the following: following:
It is sensible to exclude problems not solvable in polynomial time because of their exponentially large output.
Different important problems that are not decision problems that are easier to study.
Limitations of algorithm Power
* Property of STI
Page 17 of 25 25
Design and Analysis of Algorithm
There are many important problems showing the polynomial time algorithm:
Hamilton circuit
Traveling salesman
Knapsack problem
Partition problem
Bin packing
Graph coloring
Integer linear programming
Limitations of algorithm Power
* Property of STI
Page 18 of 25 25
Design and Analysis of Algorithm
the class of decision problems whose proposed solutions can be verified in polynomial time, which is solvable by a nondeterministic polynomial algorithm.
A nondeterministic polynomial algorithm is an abstract two-stage procedure that:
generates a solution of the problem (on some input) by guessing
checks whether this solution is correct in polynomial time
Limitations of algorithm Power
* Property of STI
Page 19 of 25 25
Design and Analysis of Algorithm
the class of decision problems whose proposed solutions can be verified in polynomial time, which is solvable by a nondeterministic polynomial algorithm.
A nondeterministic polynomial algorithm is an abstract two-stage procedure that:
generates a solution of the problem (on some input) by guessing
checks whether this solution is correct in polynomial time
Sample NP or Nondeterministic algorithm problems:
Guess a truth assignment
Substitute the values into CNF formula to see if it evaluates to true
Limitations of algorithm Power
* Property of STI
Page 20 of 25 25
Design and Analysis of Algorithm
is the class of decision problems that can be solved by nondeterministic polynomial algorithms. This class class of problems is called called nondeterministic polynomial [LEV07].
All the problems in P can also be solved in this manner (but no guessing is necessary), so we have: •P
NP
Limitations of algorithm Power
* Property of STI
Page 21 of 25 25
Design and Analysis of Algorithm
NP sample problems also includes:
Hamiltonian circuit existence
Partition problem: Is it possible to partition a set of n integers into two disjoint subsets with the same sum?
Decision versions of TSP, knapsack problem, graph coloring, and many other combinatorial optimization problems.
Limitations of algorithm Power
* Property of STI
Page 22 of 25 25
Design and Analysis of Algorithm
is a problem in NP that is as difficult as any other problem in this class, because by definition, any problem in NP can be reduced to it in polynomial time.
A decision problem D1 is said to be polynomially reducible to a decision problem D2 if there exists a function t that transforms instances of D 1 to instances of D2 such that:
t maps all yes instances of D 1 to yes instances of D2 and all no instances of D 1 to all no instances of D2
t is computable by a polynomial time algorithm.
A decision problem D is NP -complete -complete if it is as hard as any problem in NP , i.e.,
D is in NP
every problem in NP is polynomial-time reducible to D
Limitations of algorithm Power
* Property of STI
Page 23 of 25 25
Design and Analysis of Algorithm
This definition immediately implies that if a problem D is polynomially reducible to some problem D that can be solved in polynomial time, then problem D.
problems NP problems
know now n -complete NP -complete problem candidate for f or NP completeness
Limitations of algorithm Power
* Property of STI
Page 24 of 25 25
Design and Analysis of Algorithm
Other NP -complete -complete problems obtained through polynomial-time reductions from a known NP complete problem, as illustrated in the figure below.
problems NP problems
know know n -complete NP -complete problem candidate for NP completeness
Limitations of algorithm Power
* Property of STI
Page 25 of 25 25
