The Maximum Subsequence Sum Problem Prob lem To emphasize algorithm analysis, we will consider the problem of finding the maximum sum of a contiguous subsequence of integers from a given input data sequence. For example, if the input data sequence contains
CSCI 3320/8325 Data Structures Struct ures
–2 11 –4 13 Module 2 First Algorithms for Analysis
–5 –2
the answer is 20 20 (11 – 4 + 13 = 20). For convenience, we define the maximum subsequence sum as 0 if all the integers are negative.
The Solutions
2
Algorithm 1: Exhaustive Trials
We will consider four different algorithms to solve this problem. Each algorithm will be explained, and the C++ code will be examined. We will also compute the worst case running time for each of the algorithms, and observe how relatively simple algorithms can yield enormous improvement in the running time.
3
Algorithm 1: C++ Code
The most obvious way to solve this problem is to compute the sum of each possible subsequence, and retain the largest sum as the result. Each possible subsequence is identified by the subscripts of the starting and ending elements in the subsequence (assuming the entire sequence has been stored in an array). Thus a pair of nested nested loops will be used to generate all possible pairs of starting and ending subscripts. For each subsequence, we will also need a loop to compute the sum of the elements in the candidate subsequence. Note that we are not concerned with identifying the subsequence itself in this problem, only with obtaining its sum. 4
Algorithm 1: Analysis
int maxSubSum1(c maxSubSum1(const onst vector
vector &a) { int maxsum maxsum = 0; // 1 for (int i=0; i maxsum) // 7 maxSu m = this Sum; // 8 } return maxSum; // 9 } 5
The analysis of this algorithm is quite simple. It is easy to see that the maximum number of iterations of each loop (the worst case) is governed by the size of the sequence, N . There is just one statement (6) repeated inside the three nested loops (2, 3 and 5), and it has a constant, or O (1), running time. Thus, the worst case running time for this algorithm is O (N ?3). While a more precise analysis can be done, it yields an expression that is still dominated by N 3, and thus the worst case running time is not affected. 6
1
Algorithm 2: Eliminating a Loop
Algorithm 2: C++ Code
A simple observation allows us to eliminate one of the nested loops. The sum computed by the loop on lines 5 and 6 includes just one more element that the previous calculation (for the same value of i). Thus we can eliminate the innermost loop and rewrite the code with only two nested loops. The running time of this improved algorithm is clearly O (N 2). 7
Algorithm 3: A Recursive Solution
int maxSubSum2 (const vector & a) { int maxSum = 0; for (int i = 0; i < a.size(); i++) { int thisSum = 0; for (int j = i; j < a.size(); j++) { thisSum += a[j]; if (thisSum > maxSum) maxSu m = this Sum; } } return maxSum; } 8
Algorithm 3: The Three Cases
The idea in this algorithm is to split the problem into two pieces, each approximately the same size, and solve them independently. Recall that the division of the size of a problem (usually by 2) typically results in a logarithmic worst case running time; that is the case with this solution. If we divide the sequence, then compute the maximum subsequence sum of each part, the result is, with one exception, just the maximum of the two resulting sums. The exception occurs when the maximum subsequence crosses the middle, and has elements in each of the two parts.
Consider this input: Left Half 4 -3 5 -2
Right Half -1 2 6 -2
The maximum of the left half is 6 (4 –3 +5), and the maximum of the right half is 8 (2+6). The maximum sum in the left half that includes its rightmost element is 4 (4 –3 +5 –2); the maximum sum in the right half that includes its leftmost element is 7 (-1 +2 +6). The maximum sum that crosses the middle is thus just 11 (4 +7).
9
Algorithm 3: Putting It All Together The solution has a recursive function that receives the entire array along with the subscripts of the left and right “border” elem ents. It checks first for the base case (just one element). If not the base case, the two parts of the sequence are checked recursively to obtain their maximum sums. The maximum sums of the left and right sequences that include the border elements are then added to obtain the maximum sum that crosses the middle. Finally, the maximum of these three sums is returned as the result. 11
10
Algorithm 3: C++ Code (Part 1) int maxSumRec (const vector &a, int l, int r) { if (l == r) // base case: only one element if (a[l] > 0) return a[l]; else return 0; int int int int for
c = (l + r) / 2; // approximate center maxlsum = maxSumRec(a,0,c); // solve left part maxrsum = maxSumRec(a,c+1,a.size()); // right part lbsum=0, maxlbsum=0; // left border sum (int i=c; i>=l; i--) { lbsum += a[i]; if (lbsum > maxlbsum) maxlbsum = lbsum;
}
12
2
Algorithm 3: C++ Code (Part 2)
Algorithm 3: Analysis If we consider only the base case for the recursion (a single element), it is clear that T (1) = O (1), since only one of two return statements is executed. If more than one element is in the subsequence, then we recursively invoke the function with the left and right halves of the original vector, each taking time T (N /2), then compute the sum of (potentially all) the elements in the vector, taking time O (N ). Thus the total worst case running time for the recursive cases is
int rbsum=0, maxrbsum=0; // right border sum for (int i=c+1; i<=right; i++) { rbsum += a[i]; if (rbsum > maxrbsum) maxrbsum = rbsum; } return max3(maxlsum, maxrsum, maxlbsum + maxrbsum); } int maxSubSum3 (const vector &a) { return maxSumRec(a, 0, a.size()- 1); }
T (N ) = 2 T (N /2) + O (N ). 13
Algorithm 3: Analysis Conclusion
14
Algorithm 4: Further Improvements
We won’t bother with formally solving these equations now, but will revisit them later in the course.
Our final algorithm for the maximum subsequence sum problem is not only the simplest, but also the most efficient (in terms of running time growth). Eliminating the need for the “i” loop can be understood if we make the following observations.
At this poi nt, however, w e simply no te that the fina l result is T (N ) = O (N log N ) Again, recall t hat when a n algori thm work s by divi ding the work to be done by a constant factor in each iteration (or invocation), the running time will likely include a factor that is logarithmic in the problem size (N ).
If a[i] is negative, then any sequence that begins with it can be improved by starting with the next element. Any sub sequence t hat begi ns with a negati ve subsequence can be improved by eliminating that negative subsequence.
15
Algorithm 4: A Single Loop
16
Algorithm 4: C++ Code
To eliminate the “i” loop, we just keep track of the subscript of the last element of the subsequence being examined. As soon as the sum of a subsequence becomes negative, we just set the sum back to zero, essentially eliminating it from the current subsequence’s sum.
int maxSubSum4 (const vector &a) { int maxSum=0, thisSum = 0; for (int j=0; j maxSum) maxSum = thisSum; else if (thisSum < 0) // eliminate negative prefix thisSum = 0; } return maxSum; }
17
18
3
Algorithm 4: Analysis
Algorithm 4: An On-line Algorithm
The running time for algorithm 4 should now be easy for you to determine. The body of the for loop has running time O (1), since it contains one assignment statement, with an addition, clearly requiring ti me O (1), and one if statement, with the “then” and “else” parts containing one assignment statement each, with running time O (1).
Since the number of times the body of the for loop is executed is equal to the problem size, we clearly have T (N ) = O (N )
This last algorithm has the property of being an on-line algorithm. This means that the algorithm requires only constant space (since only three integers – maxSum, thisSum, and the current value from the a vector – are needed at any time), and can instantly provide an answer to the problem for the data it has already processed.
19
Binary Search
20
Binary Search: C++ Code
Given an integer X and integers A 0 , A 1 , …, A N-1 , which are presorted and already in memory, find i such that A i = X, or return i = -1 if X is not in the input. The most obvious solution is a linear search, examining A 0, then A 1, and so forth. It should be easy to see that this solution has T (N ) = O (N ). The linear search does not use the fact that the data is presorted. The binary search does better by examining the middle element which is either the desired value X, or identifies which of the remaining parts (left or right of the middle element) should be examined further.
int binarySearc h(const vector &a, const int x) { int low = 0, high = a.size()-1; while (low <= high) { int mid = (low + high) / 2; // middle subscript if (a[mid] < x) low = mid + 1; else if (x < a[mid]) high = mid – 1; else return mid; // found } return –1; // not found }
21
Binary Search: Analysis
22
Euclid’s Algorithm
The work done inside the while loop clearly takes O (1) time, so the total running time depends on the number of iterations of the while loop. Without loss of generality, we can consider the number of elements in A to be equal to 2k . Each iteration reduces the number of elements remaining to be considered by half, so clearly the number of iterations required is k , where 2k-1 ? N ? 2k . Thus T (N ) = O (log N ). 23
Euclid’s algorithm is used to compute the greatest common divisor (gcd) of two integers a and b (that is, the largest integer that divides each of a and b). The algorithm works by repeatedly computing the remainder of a / b, replacing a by b, and b by the remainder, until the remainder is zero. The last nonzero remainder is the answer. For example, computation of the gcd of 137,912 and 151,360 yields the following sequence of remainders: 137,912 13,448 3,432 3,152 280 72 64 8 0 Thus the greatest common denominator is 8. 24
4
Euclid’s Algorithm: C++ Code long gcd(long a, long b) { while (b != 0) { long rem = a % b; a = b; b = rem; } return a; }
// // // //
Euclid’s Algorithm: Analysis Each execution of the body of the loop started with statement 1 (that is, statements 2, 3, and 4) takes constant time, so the running time of the algorithm depends only on the number of iterations of the loop, which depends on the length of the sequence of nonzero remainders.
#1 #2 #3 #3
If we could show that each iteration of the loop decreased the value of the remainder by at least a constant factor, then we could predi ct a logarithmic running time. But this is not the case (refer back to the computation of the gcd of 137,912 and 151,360).
// #4
We can, however, show that after two iterations the remainder is at most half its original value (the proof appears on the next slide). Thus, since 2 log N = O (log N), we have established the logarithmic running time of the algorithm. 25
Remainder Reduction Rate Analysis To show that the remainder decreases by at least half with every pair of iterations, it is sufficient to prove that if a > b, then a mod b < a / 2. (Recall that the remainder of a / b, or a mod b, will always be less than b.) There are two cases to consider: If b ? a / 2, the remainder is clearly less than a / 2. If b > a / 2, then a / b = 0 with a remainder of a – b, which must be less than a / 2.
Thus, even if a < b, at most two iterations of the loop will be required to obtain a remainder that is at most half as large as a.
26
Exponentiation Computation of a b (where a and b are both integers) can be done using the obvious technique involving b –1 multiplications. We can do better than this is we observe that if z = a k , then z ? z = a 2 k . Thus, if b is even, a b = (a 2) b / 2, eliminating almost half of the multiplications required by the obvious technique.
27
Exponentiation: C++ Code long pow(long a, long b) { if (b == 0) return 1; if (b == 1) return a; if (isEven(n)) return pow (a * a, b / 2); else return pow (a * a, b / 2) * a; }
// // // // //
28
Exponentiation: Analysis The base cases for the recursive function (statements 1 and 2) clearly take O (1) time. Each recursive invocation of pow reduces the size of the exponent by a c onstant factor (2), so the running time is logarithmic. If we count the number of multiplications, it’s easy to see that at most two multiplications are involved for each invocation of the function, so the maximum number of multiplications is 2 log2 b.
#1 #2 #3 #4 #5
29
30
5
