. n + 1), where

is all of S minus its first and last elements. We know that S has to start with 0 because every string in the B list starts with the symbol ~.The only string in the A list that starts with the symbol fl. is the first one (which we've numbered 0). So the only way that the strings that are generated from the two lists can match is for the first index to be 0. We know that S must end with n + 1 because every string in the A list except the last ends with ft. But no string in the B list does. But B's n + p• element provides that final ~and it provides nothing else except the final$. The string that S produces is identical. if we remove all instances of rt and $,to the string that the sequence (1,

) would produce given . This is true because we constructed elements 1 through n of < A, B> to be identical to the corresponding elements of < X. Y > except for the insertion of 'I and$. And we guaranteed, again ignoring¢ and$, that 110 = a 1 = x1• So the sequence (1 ,

) generates the same string from both the X andY lists and its first element is l. So it is an M PCP solution for M P = . So we have that MPCP s PCP. Let R(< X. Y> ) be the reduction that we have just described. If there existed a Turing machine M that decided PCP, then M ( R( < X. Y> )) would decide MPCP. But MPC P is not in D, so no decider for it exists. So M does not exist either and PCP is not in D.

APPENDIX

F

The Theory: Complexity I f.1

n this appendix, we will prove some of the claims that were made but not proved in Part V.

Asymptotic Dominance In this section we prove the claims made in Section 27 .5.

f. 1.1 Facts about 0 We wiH prove separately each of the claims made in the theorem. The basis for these proofs is the definition of the relation·CJ: f(n) e CJ(g(n}) iff there exists a positive integer k and a positive constant c such that: 'Vtt

~

k (f(n) s cg(n)).

Let f, ft· /2. g. g,, and K2 be functions from the natural numbers to the positive reals. let a and b be arbitrary real constants, and let c, co. ch ... c,. be any positive real constants.

Fact 1: f(n) e O(f(n)) Let k

= 0 and c =

1. Then 'Vn

2:

k (f(n) s cf(n)).

Fact 2: Addition

+ co) (if we make the assumption, which will always be true for the functions we will be considering, that 1 e O(f(n))). We first note that. for any function g(n). if g(n) e CJ(f(n)) then it must also be tr_ue l~at g~n) e O(j(n) + co) since f(n) < f(n) + c0• Now we show the other dtrectton. Smce 1 e c.?(f(n)), there must exist k 1 and c 1 such that:

1. CJ(f(n)) = O(f(n)

870

Appendix F

The Theory. Complexity

'Vn H g(n) e O(f(n)

~

k 1 (1 s ctf(n)) (co s CoCtf(n)).

(1)

+ eo) then there must exist k 2 and c2 such that: 'Vn

~ k2

(g(n)

S

C2(/(n)

+ CQ)).

Combining that with (1), we get:

Vn

~

max(kh k2) (g(n) s c2(f(n) + c~tf(n))

s (c2 + CoCt)(f(n))). So let k = max(k1, k2) and c Vn

:::r

~

c2 +

eoc1• Then,

k (g(n) s cf(n)).

2. If f 1(n)eO(g1(n)) and fz{n)e0(g2(n)) then f1Cn) + fz(n)eO{gt(n) + ~(n)). If f 1(n) e O(g1(n)) and fz(n) e O(g2(n)), then there must exist ktt Ctt k2 and c2 such that: 'Vn ~ k1 (f,(n) s clg,(n)). Vn ~ k2 (fz(n) s C282(n)). So:

Vn ~ max(k1, k2) (f,(n) + f2(n) s CtKt(n) + c2~ (n) s max(c~o c2)(g1(n) + g2(n))).

So let k = max(k 1, k 2) and c

Vn

~

k (f1(n)

= max(ch c2). Then, + f2(n) s c(g1(n) + g2 (n))).

3. O(f1(n) + /z{n)) = O(max(f1(n),fz(n))). We first ~how that if g(n) e 0(/J(n) + /2(n)) then g(n) eO(max(f1(n). fl(n))). If g(n) e O(f1(n) + fl(n)), then there must exist k 1 and c1 such that:

Vn

~

k 1 (g(n) s c1(/1(n) + fz(n)) s 2c1 • max(f1(n), /z(n))).

So let k = k 1 and c = 2c1• Then, Vn

2:

k (g(n) s c max(f1(n), fz(n))).

Next we show that if g(n) e O(max(f1(n), fl(n))) then g(n) e O(f1(n) If g(n) e O(max(f1(n), fl(n))), then there must exist k 1 and c 1 such that:

Vn :: k 1 (g(n) s c1 max(f1(n). fl(n)) :s Ct (f,(n) + fl(n))). So let k = k1 and c = c1• Then, Vn

2:

k (g{n) s c(f1(n) + fl(n))).

+ fl(n)).

F.1

Asymptotic Dominance

871

Fact 3: Multiplication L O(f(n)) = O(eaf(n)). We first show that, if g(n) e O(f(n)), then g(n) e O(cof(n)). If g(n) e O(f(n)), then there must exist k 1 and Ct such that: ~

'r/n So let k = k 1 and c

kt (g(n) s ctf(n)).

= c11eo. (Thus Ct = ceo.) Then, ~

'r/n

k (g(n) s ccof(n)).

Nextweshowthat,ifg(n) e O(cnf(n)), theng(n) eO(f(n)). Ifg(n) eO{cof(n)), then there must exist k 1 and c1 such that

'r/n > kt (g(n) s Ctcnf(n)).

So let k = k 1 and c = c1co- Then, 'r/n

~

k (g(n) s cf(n)).

2. If ft(n) eO(gt(n)) and f2(n) e0(8l(n)) then / 1(n)f2(n) eO(g1(n)g2(n)).

If ft(n) e O(gt(n)) and f2(n) e O(g2(n)), then there must exist kh c1o k2 and~ such that:

'r/n ~ k1 (ft(n) s Ct 8t(n)), and 'r/n ~ k2 {fl(n) s c2g2(n)). Thus: 'Vn ~ ma.x(ktt (ft(n)fl(n) s c1 c2 gt(n) 12(n)). So let k = max(k., k2) and c = c1 c2• Then,

kv

Fact 4~ Polynomials 1. If a s b then O(n4 ) ~ O(nb). If g(n) e O(n4 ), then there must exist k 1 and c1 such that:

Vn

~

kt (g(n) < c1n4 s c1nb)

(since (a s b)-+ (1f s nh)).

So let k = k 1 and c = c1.1ben:

'r/n > k (g(n) s crf). 2. lf/(n)

= Cfli + CJ-1,.1-t 'Vn

C!!

+ . •. ctn +eo, thenf(n)eO(ni).

1 (cf" + ci-tni-l + ... c 1n +Cos c1nl + c _1nl + ... c1ni +coni 1 S (cj + Cj-l + ... Ct + Co) ni).

872

Appendix F

The Theory: Complexity

So let k = 1 and c

= (ci + cj-L + ... c1

+co). Then:

'Vn ~ k (f(n) s en').

Fact 5: Logarithms

1. For a and b > 1, O(log, n} = O(logbn). Without loss of generality, it suffices to show that O(log, n) ~ 0(1081, n) . H g(n) e O(log, n), then there must exist k 1 and c1 such that: 'Vn

~

k 1 (g(n) s c, log, n).

Note that log, n = Jog, b lo8b n. So let k = k1 and c 'Vn

2:

= c1 Jog, b. Then,

k (g(n) s c logh n).

1. IfO 1 then O(n 11 ) ~O(ff lo& n) ~ O(nb). First we show that O(nll) C O(n11 Ioac n). For any n 2: c, loSe n 11 n lo&c n. If g(n) e O(n"), then there must exist k 1 and c1 such that:

~

1, so n•

~

'tin ~ k 1 (g(n) s c1n").

So let k = max(ka. c) and Co= c1• Then, 'tin ~ k (g(n) s c0 n11 lo&c n).

Next we show that O(n11 lo&c.~ n) ~ O(nh). First, notice that, for p > 0 and n ~ 1, we have, II

lo8rn

=

J

J If

x-l dx

s

L

x-l+pdx =

~(nP -1) < ~nP,

I

In particular, for p = b- a, we have: 1

lo~n < - - n6

b-a

- ".

If g(n) e O(n 11 lo&c n}, then there must exist k 1 and c1 such that: 'Vn

2:

So, for all n

k 1 (g(n) s c1ff lo& n). ~

max(l, k 1), we have that:

g(n) s en" lo&c n = en" lo&c t> log,. n < c lo&c e So let k =max( I. k1) and c0 = b . Then,

-a

'tin 2: k (g(n) s c0 n").

clo&- e

b-a

n11nb- a s

c loo e cc

b-a

,/',

F.1 Asymptotic Domin~nce

873

Fact &: Exponentials (including the fact that exponentials dominate polynomials) L If 1 < a s b then O(a") ~ O(b"). If g(n) e O(a"), then there must exist k 1 and c1 such that:

Vn

2::

k 1 (g(n) s c1a" (since (a s b)-+ (a" s b")).

s c1b")

So let k = k1 and c • c1• Then, Vn

C!::

k (g(n) s cb").

l. If a 2:: 0 and b > 1 then O(n11) ~ O(b"). If a = 0, then we have that 0(1) c O(b"), which is trivially true. We now consider the case in which a > 0. First notice that, if p C!:: 1 and n C!:: p, then,

"

s log,p

+ ~~dx

s log,p

+ n- p

,

p

n

s log,p + -. p

.

.

If, m particular, p

= max(-1og,a b' 1) , then-1p s lo~ n

s log, p

log,b -,p

a

2::

1 and:

+ log, b n

a a log, n s a log, p + log, b • n.

And: n• =

~o~n

s

~·toc.p+,b· ,.

s p". b". If g(n) e O(rt'), then there must exist k1 and c1 such that:

Vn

So, again letting P =

~

k1 (g(n) s c1rf).

max( 10~ b' 1), let k

Vn

C!::

k (g(n) s cb").

• max(k1• p) and c =

c1 ~. Then.

874

Appendix F

The Theory: Complexity

3. lf/(n)

= c;+ 12" + c;ni + c; _1ni - l + , .. c 1tr + Ct),thenf(n)e0(2").

From Fact 4.2. we have that Cjlli + ci _,nJ- I +... c 1n + c11 e O(ni). From Fact 6.2. we have that ni e 0(2"). So, using the transitivity property that we prove in 8, we have that ci'zi + c;_1ni- l + ... c1n + coe0(2"). So there must exist k 1 and k2 such that:

\In ~ k 1 (cjlti + c;- 1ni- t + ... c 1n + c0 c;+ 12" + c;ni + c;- 1n i- l + ... c 1n

~ k 22" + c0 s t·1_ 12'' ~

So let k = k 1 and c

+ k22" (c;+t. + k2)2").

= c1+ 1 + k 2• Then, \In

~

(f1(n) s

k

c2").

Fact 7: Factorial dominates exponentials: If a ce 1 then O(a") c O(n!) First notice that. if a

a"

~

1. then: n

la1- l

f:.~

ksJ

= n a = IT a·

i, l-1

11

11

n _a ~ f~~ n u. kIT .k k!iu t =•a t fu' - I

rr n a •..!=.L .. rr k k

l., a- 1

s

t :.A;,

Iu - I

Ilk

If

k =>

II;

k =- l

~

lu'- la

,

IT-·Ilk k

k=l

k= l

t, l- 1a ~rr - ·n!. k :. J k If g(n) e O(a"), then there must exist k 1 and c 1 such tha t:

Vn

~

kt (g(n)

~

Ctll'').

~

en!).

rai- l a So let k = k 1 and c = c1 -k. Then.

IT

k= l

'Vn

~

k (g(n)

Fact 8: Transitivity: If f(n) E O(f1(n)) and f 1(n) E O(f1 (n)) then f(n) E O(f2 (n)) If f (n) e O(f1(n)) and f 1(n) e O(f2(n)), then there must exist k 1• c., k 2 and c:2 such that: \;In ~

k 1 (/( n) ~ ctfl(n)).

Vn ~ k2 (/,(n) ::; clfi(n)).

F.2 The Linear Speedup Theorem

So let k

875

= max(k 1• k2) and c = Ct~· Then, Vn

2:

k (f(n) s cfl(n)).

F.1.2 Facts about u We will prove separately the two claims made in the theorem. The basis for these proofs is the ddinition of the relation u: f(n) e u(g(n)) iff. for every positive c, there exists a positive integer k such that:

Vn

~

k (f(n) < cg(n)).

Let f and g he functions from the natural numbers to the positive reals. Then, 1. f(tr) fl. cr(f(tr)): Let c = l..Then there exists no k such that Vn

2:

k (f(n) < cf(n)).

Z. tT(/(11)) C 0(/(11)):

H g(n) e t1(f(11)) then, for every positive c 1, there exists a k 1 such that: Vn 2: k (g(11) < cf(n)). To show that g(n) e CJ(f(11)). it sufHces to find a single c and k that satisfy the definition of 0. Let c = 1 and let k be the k 1 that must exist if Ct = 1.

f. 2

The Linear Speedup Theorem In Section 27.5 we introduced the theory of asymptotic dominance so that we could describe the time and space requirements of Turing machines by the rate at which those requirements grow. rather than by some more exact measure of them. One consequence of this approach is that constant factors get ignored. This makes sense for two reasons. The first is that. in most of the prohlcms we want to consider. such factors are dominated by much faster growing ones, so they have little impact on the size of the problems that we can reasonably solve. But a second reason is the one that we will focus on here: All but the most efficient Turing machines can be sped up by any constant factor. The idea behind this claim is simple. At each step of its operation, a Thring machine visits one tape square. If we can compress the contents of the tape so that they fit on fewer squares, we can reduce the number of steps that a Turing machine must execute to process the tape. Compressing a tape is easy. We simply increase the size of the tape alphabet. That enables us to encode a chunk of squares from the original tape into a single square of the new one.

EXAMPLE F.1 Encoding Multiple Tape Squares as a Single Square

a }.

Let. M be a ~uring ,machine whose tape alphabet is {0, 1, We can build a new Tunng mach me M that uses the tape alphabet {A-Z. a-z. 0-9, a-w}. Using the new alphabet, we can encode four squares of M's tape as a single square on the tape

876

.Appendix F

The Theory: Complexity

M' will use. Initially, we'll include in the encoding at leost one blank on either side of the input (plus more on the right for padding if necessary). If M ever moves off its input, new squares can be encoded as necessary. So, for example:

If the tape of M is:

... D':DOOOOOOOOOOOlOOOOlOCD .•• ..,

~

c

Then the tape of M' might be:

8

8

f

.,.-;

1r

If we design the transitions of M' appropriately, it will be ahle to do in one step what it takes M four steps to do.

l11e compression idea that we have just dcscribL:d is the hasis of the Linear Speedup ·n,eorem that we are ilhout to present. Before we go in ttl the details of the theorem and its proof. one caveat is in order. While this theorem is of some theoretical interest. its application to real computers is limited. Real computers hc1ve a fixed si1.e alphabet (generally consisting of two binary symhols). So the sort of compression that we ure using here cannot be applied. However. it is worth noting that other compression algorithms (that explnit patterns in particular kinds of input strings and are thus ahle to reduce the number of bits required) arc routinely used in many kinds of real applications. THEOREM F.1 The Linear Speedup Theorem

-

-

-

Theorem: Let M he a k-tape Turing machine where k > 1and timert•q(M) = f(n). Given any constant c: > 0, there exists another k-tape Turing machine M' such that L(M) = L(M') and timert•q(M ) s 1

rf(n)l -c- + 2n + 2 • f 6c 1.

Notice that cis a factor by which we reduce the tinH: a computation requires.

So. if we want to say that the new program take~ half as long, we set c: to 2. In some statements of the thcorcm~ c i·s a multiplicative factor. Using those versions. we: 'd set c to 1, 2 in that case. The two formulations arc equivalent.

Proof: We prove the claim by describing the operation of M'. M' will begin by making one pass across its input tape to encode it as described in Example F.l. It will store the encoded string on tape 2 and blank out tape l. During the rc:st of its operation. it will use tape 1 as M would have used tape 2 and vice versa. The number of symbols to be colhapscd into one at this step is dl.!termined by c, the speedup that is desired. We'll call this collapsing factor m and set it to r6c 1. Nl.!xt M' simulates thl.! execution of M. The idea hL:re is that. since M' encodes m symbols as one. it can process m symbols as tmc. Unfortunately, on any one tape them symbols that M' needs may he spread across two of the new encoded ta ' squares: They may fall on the current square plus the unc to its right or they m~~

F.2 The linear Speedup Theorem

877

fall on the current square plus the one to its left. So M' must make one move to the left. then one back. and then one more to the right and then back before it can be sure that it has all the information it needs to make the move that simulates m moves of M. We illustrate this with the following example. Let k = 2 and m = 5. Each tape square actually contains a single symbol that encodes five original symbol~ but we've shown them here with the original sequences so that it is possible to sec how the simulation works. Suppose that, at some point in its computation. M has entered some state q and its tapes contain the following fragments: Tapct•

Tapc2:

. .. lo

o 1 o olo 11 o ol111 o 1!1 o o o olo o o 1 o! ...

... jo

o 111!11 o o olo o 111!111 o 1!11 o 11j .. ..

t

t If the next five moves of M move the read/write head on tape 1 to the right five times and they move the read/write head on tape 2 to the left five times. M' will n~;:ed ll'l examine both one encoded square to the left and one encoded square to the right before it wilJ have enough information to simulate all five of those moves. So M' simulates M by doing the following:

I. Move one square to the left on each of the tapes and record in the state the encoded symbol it finds on each of the k tapes.

·:z. Move one square back to the right on each tape and record in the state the encoded symbol it finds on each of the k tapes. 3. Move une mme square to the right on each tape and record in the state the encoded symbol it finds on each of the k tapes. 4. Move one square back to the left on each tape. At this point, the-read/write heads of M' are back where they started and the state of M' includes the following vector of information that captures the current state of M~ (q,

L 1• C 1• Rt. tb

M''s state. The relevant contents of tape 1: Lt .is the encoded square to the left of the one that contains M's simulated read/write head. is the encoded square. that contains M's simulated read/write head. R, is the encoded square to the right of the one that contains M's simulated read/write head. f1 (an integer between l and m) is the position within C 1 of M's simulated read/write head.

c,

878

Appendix F

The Theory: Complexity

Tape 2: similarly Tape k: similarly 5. Using this information. make one move that alters the C squares as necessary and moves each read/write head as required to simulate M. Also update M 's state. 6. Make one more move if necessary. If. on some tape. M 's simulated read/write head moved off the C square. it will be necessary to make this second move in order to alter the contents of either the Lor R square to match what M would have done. But note that. on any given tape. it will only be necessary to work with the current square plus one to the left or the current square plus one to the right. So two moves suffice to make all necessary changes to the tapes. The first phase. encoding the input tape. requires thnt M make one complete pass through the input and then move back to the left. Jt mny have to use up tom padding blanks. So. in the worst case, on an input of length n this phase requires 2(n + m) steps. The second phase. simulating M. requires at most six steps for every m steps that M would have executed. So. if timer<•q(M) = f(n), then. timereq(M')

Since m =

~ r61~;)

l

+ 2(n + m).

r6c 1. we then have, timereq(M') ~

/(n)l + 2n + 2 • r6<:1. 7

r

,APPEND

C E S

G - Q

Applications n appendices G through Q , we describe applications of the techniques that have been covered throughout the book. Most of the discussion is organized around a collection of key application areas that make use of more than one of the ideas that we have discussed. We will consider all in the following Jist, although we will barely scratch the surface of each..

I • •

Programming languages: syntax and compilers Functional programming

·•

Tools for programming and software engineering, including techniques for verifying the correctness of programs and of hardware designs Network protocols, network modellng. and the Semantic Web Computer system security, cryptography, hackers and viruses Computational biology Natural language processing

• • • •

• Artificial intelligence and computational reasoning • Music • Classic game.s and puzzles • Interactive video games Then we will look at three of the specific tools that we have introduced. We will brieOy survey some of their applications that lie outside the particular application areas that we will already have covered. The three tools are •

regular expressions,

• •

finite state machines and transducers, and grammars.

APPE ND

X

G

APPLICATIONS: PROGRAMMING LANGUAGES AND COMPILERS he idt!as that we have discussed throughout this book form the foundation of modern programming. Programming languages are typically described with context-free grammars. Regular expression matchers are built into many modem progrumming environments. Finite state transition diagrams enable visual programming.

T G.1

Defining the Syntax of Programming Languages Most programming languages are mostly contcxt-frce.1llere are some properties, such as type constraints. that cannot usually be describ«!d within the context-free framework. We will consider those briefly in Section 0.2. But context-frl!e grammars provide the basis for defining most of the syntax of most programming languages.

G.1. 1 BNF It became clear l.!arly on in the history of programming language development that designing a language was not enough. It was also necessary to produce an unambiguous language specification. Without such a specification. compiler writers were unsure what to write and users didn't know what code would compile. The inspiration for a solution to this problem came from the idea of a rewri te or production system as described years earlier by Emil Post. (See Section lR2.4.) In 1'>59. John Backus confronted the specification problem as he tried to write a description of the new language ALGOL 5H. Backus later wrote {Backus 19~0) . "As soon as the need for precise description was noh:d. it became obvious that Post's productions were well-suited for ~ha~purposc. t .hastily adaptc~ lhc!m for usc _i~ dcs~rihin~ the syn tax of IAL {Algol :"~H). lllc notatton that he destgned was mcx..hftec.l shghtly tn collaboration with Peter Naur and used in the clcfinition. two years later1of ALGOL 60. Tile ALGOL 6() notation

880

G.1

Defining the Syntax of Programming Languages

881

became known as BNF 0.. for Backus Naur form or Backus Normal form. For the definitive specification of ALGOL 60. using BNF. see [Naur 1%3]. Just as ~he ALGOL 60 language influenced the de!Sign of generations of procedural programmmg languages. BNF has served as the hasis for the description of those new languages, as well as others.

The BNF language that Backus and Naur used exploited these special symbols: • ::= corresponds to - , • I means or. and • < > surround the names of the nonterrninal symbols.

EXAMPLE G.1 Standard BNF Our term/factor grammar for arithmetic expressions would be written as follows in the original BNF language:

::= + I ::- * I ::= id I () While it seems obvious to us now that formal specifications of syntax are important and BNF seems a natural way to provide such specifications, the invention of BNF was an important milestone in the development of computing. John Backus received the 1977 Turing Award for ··profound, influential. and lasting contributions to th e design of practical high-level programming systems. notably through his work on FORTRAN. and for seminal publication of formal procedures for the specification of programming languages." Peter Naur received the 2005 Turing Award .. For fundamental contributions to programming language design and the definition of Algo\60, to compiler design, and to the art and practice of computer programming." Sine~ its introduction in 1960, BNF has become the standard tool for describing the context-free part of the syntax of programming languages, as well as a variety of other formal languages: query languages, markup languages, and so forth. In later years. it has been extended hot h to make better use of the larger character codes that are now in widcspre4td use and to mukc specifications more concise and easier to read. For example. mtx.lcrn vcrsitms of BNF •

often use ...... instead of::=.

•

provide a convenient notation for indicating optional constituents. One approach is to use the subscript l'rt· Another is to declare square brackets to be metacharacters that surround optional constituents. The following rules illustrate three wavs to say the same thing: •

S-Tie s - rurt

s-{Tl •

may .in~ludc many of the features of regular expressions. which are convenient for spec1fymg those parts of a language's syntax that do not require the full power of the context-free formalism. 'These vnrinus dialects arc called Extended BNF or EBNF ~.

882

Appendix G

Applications: Programming Languages and Compilers

EXAMPLE G.2

EBNF

In standard BNF, we could write the following rule that describes the syntax of an identifier that must be composed of an initial letter, followed by zero or more alphanumeric characters: ::= I ::- I ::= I ln EBNF, it can be written as: identifier • letter (letter

I

digit)*

But note, this is a simple example that illustrah!s the point. In any practical system, the parsing of tokens, such as identifiers. is generally handled by a lexical analyzer and not by the context-free parser.

G.1.2 Railroad Diagrams Context-free grammars fill reference books on every programming language that has bl!cn created since BNF was used to define ALGOL 60. Sometimes more modem definitions look superficially different from BNF. since other nota tions have been developed over the years. For example. railroad diagrams ~ (alsn called syntax diagrams or railway tracks) are! graphical rl!nditions of the rules of a context-free grammar. Railroad diagrams have the same expressive power as docs BNF. but they are sometimes easier to read.

EXAMPLE G.3 A Railroad Diagram for a Switch Statement Consider the following BNF specification for a switch statement like the one in Java (where the subscriptQPT indicates an optional cnnstitucnt): ::=SWITCH ( 1 } ) {} ::- opr ::= l : :• CASE : BREAKon ~== DEFAULT: We assume that , which is used in other places in the grammar, is defined elsewhere.

G.2

Are Programming Languages Context-Free?

883

Here's the corresponding railroad diagram (again assuming that is defined elsewhere): $'111itch·stmt:

~

SWITCH- (

.

.

T rnt·C111'fn5=-t .L •:num-typo:

)I

CASE-value-:-SI.mt-list

DEFAULT - .: -lllmt-list

1-

Terminal strings are shown in upper case. Nonterminals are shown in lower case. To generate a switch statement, we follow the lines and arrows, starting from switch-stmt. The word SWITCH appears first, followed by(. Then one of the two alternative paths is chosen. They converge and then the symbols ) and { appear. There must be at least one case alternative, but, when it is complete, the path may return for more. The BREAK command is optional. So is the DEFAULT clause, in bot.h cases because there are detours around them.

G.2 Are Programming Languages Context-Free? So far. we have considered the use of context-free grammars to specify the syntax of individual statements. They are also used to specify the structure of entire programs. However. there are global properties of programs that cannot be described within the context-free framework. Recall that, in Section 13.3, we showed that WcW = { 'WC'I.V: we {a.b }*} is not context-free. The structure of the strings in WcW is very similar to the declaration-use pattern that is common in typed programming languages.

EXAMPLE G.4

Why Java Isn't Context Free

Here's a syntactically legal Java program:

public class example {public static void main () { char todayistuesday; todayistuesday. 'a';}}

884

Appendix G

Applications: Programming Languages and Compilers

EXAMPLE G.4 (Continued)

Here's a string that is not a syntactically legal Java program: public class example {public static void main () { char todayiswednesday; todayistuesday ='a';}} The problem with the second program is that the variable that is used hasn't been declared. Observe the relationship between the strings of this sort that are legal and the language WcW = {'wcw: lV e {a. b }*} by substituting ; for c and the variable name for w.

To prove that Java is not context-free. let: J = {syntactically legal Java programs}

n

string a*b*; a~:b* = • a';}). We've used the shorthand for some particular opening string that will transform the remaining fragment into a legal Java program. So J includes a set of Java programs that declare a single variable whose name is in a *b"' and then do a single opt!ration. namely assigning to that variable the value •a·. By Theorem 13.7. if Java were context free then J would also be context-free since it would he the intersection of a context-free language with a regular language. But we can show that J is not context-free using the Pumping 1neurcm. Let: w

= string a~
t

12131415 161

= 'a •:} }

1

I

If either t' or y contains any portions of regions l . 4. or 7, then set q to 2. The resulting string violates the form constraint of J. If cith~r ., or y uverlaps the boundary between regions 2 and 3 or regions 5 and ·6, then set tf to 2.111c resulting string violates the form constraint of J. It remains to consider the following cases:

•

•

(2. 2). (2. 3). (3, 3 ). (5. 5), (5. 6). (6. 6). (3, 5): Set q to 2.11u: resulting strings will have a declaration for one variable and a use of another. thus violating the type constraints of Java. So s (1. {syntactically legal Java programs}. (2.5).(2. 6).(3.6): Violate the requirement that l·tJ xyl s k.

There is no way to carve w into uvxyz such that all the conditions of the Pumping Theorem are met. So J is not context-free. So Java is not context-free.. Recall that. in Exercise 133. we considered another aspect of type checking: guaranteeing that each invocation of a declared procctlure contains the same number of paramt:ters as the declaration. We saw that a simple language that exhibits a similar

G,3

Designing Programming Languages and Their Grammar$

885

property is not context free. Because of these issues. type checking typically cannot be done with a cc..mtcxt-free parser.

G.3 Designing Programming Languages and Their Grammars So fur. we have discussed the syntax of programming languages as though it were a natural phenomenon over which we have no control. In Appendix L, we'll consider English; it is such a phenomenon and we have no control. But programming languages are artificial things. designed by people to serve a particular purpose. It makes sense to design them so that they have the properties we want. Clearly we want them to be expressive. easy to usc. and hard to make mistakes in (alternatively, easy to check for mistakes). We also want to design the syntax so that: •

The language is not inherently ambiguous. If this is true, then we. will be able to design an unambiguous grammar that will generate exactly one parse tree (and thus one interpretation) for each string in the language.

•

The language can be parsed efficiently (i.e.• deterministically). This requirement imposes a stronger constraint than does the need to avoid ambiguity. • ·T he syntax is straightforward. We want to be able to write a grammar that serves to document the syntax in a way that is readable by programmers,

The jssue of ambiguity is particularly important and it is enlightening to contrast English with useful artificial languages in this regard. For example, while English does not allow the usc of parentheses to force a particular parse structure, most programming languages do. While English does not exploit rules like operator precedence, and in~tcad allows great flexibility in organizing sentences, most programming languages do exploit such rules and are defined by grammars that force a single interpretation in -all cases. So. while many English sentences are highly ambiguous. most programming 'language statements are not. Contrast: The hoy and the girl with the red wagon bought a pencil and a book with a · lloppy cover, with

17

+ 12*(4•8)*4 + 7.

The E.nglish se~ten~e is a~biguous given any reasonable grammar of English. The arithmetiC _expr~ss10n ts ambt_g~ous g~ven some arithmetic expression grammars. But, as we _saw m. Example_ 11.19. 11 ts stratghtforward to design an unambiguous grammar for anthmet.Jc expressiOns. Unfortunate!~, some conv~nient programming language constructs present challenges to the dest~n of unambtguous grammars. The most common is the ; f statement that allows an optmnal e 1 se clause. We discussed this problem, generally known as the

886

Appendix G

Applications: Programming Languages and Compilers

dangling e 1 se problem. in Example 11 .20. Recall the following statement that we prese nted there:

if cond 1 then if cond2 then st 1 e1 se

st~

The problem is that. if we use a straightforward grammar that makes else clause~ optional. then this statement has two parses (and thus two meanings):

if cond 1 then [if cond 2 then st 1 else st 2] if cond 1 then [if cond 2 then st 1] e 1 se st2 The designers or any programming language that has this construct must solve the problem in one of two ways:

Rely on delimiters to disambiguate nested if statemt!nts. Languages such as Algol .6t;, Modula-2, Ada, Lisp, and Scheme take this approach. For example. in Algol 68, one would write:

•

if i "" 0 then if j • 0 then X

:= 0 I* In Algol fl8. each delimiter x had a matching close I* ddimiter .tR. So if/ fi.

fi else x

:= 1

fi Or, in Scheme. one would write:

(if(= i 0) (if (• j 0) (set! x 0)) (setl x 1)) It is clear, in both these cases. that lhe single else clause! (which sets x with the first if. •

to 1) goes

Dispense with delimiters and substitute an arbitrary decision that can be encoded in the grammar. Languages such as C and Java take this upproach, which we illustrated with a fragment of a Java grammar in Example 11.20. Of course. the main drawback to this approach is that programmers may not always be aware which arbitrary decision will be made~ they may write their Ct)dc assuming that something different will happen. So this approach can lead to programme r l!rrors, which can sometimes be caught by audit rule checkers ~ .

So there are cases like: the dangling else problem that are known to create ambiguity. And there are examples of grammars that can be shown (as we did in Example ll.IR) to be unambiguous. Unfortunatdy. the undecidability results of Section 22.5 make it clear that there can exist no gencrallools that can tdlthe difference. So, in particular, there exist no general tools that can: • •

decide whether a proposed language is inherently ambiguous. or decide whether a proposed grammar is ambiguous.

G.4 Compilers for Programming Languages

887

G.4 Compilers for Programming Languages The job of a compiler can be broken down into the following pieces: • lexical analysis. • syntactic analysis, • code generation and optimization, and • error checking, which must be done at each step of the process. Both lexical analysis and syntactic analysis are driven primarily by the theory that has been presented in this book. whidt alsu ldls us something about what kinds of error checking arc possible. In addition, the computability results we have presented have implications for our ability to design effective optimizers.

G.4.1 Lexical Analysis The job of a lexical analyzer is to transform a string of input characters into a string of tokens (typically corresponding to· the smallest meaningful units in the language). The character patterns that correspond to the allowable tokens are generally described with regular expressions. See Section 1S.l for a discussion of how lexical analysis is done and the tools that are available for building lexical analyzers.

G .4.2 Syntactic Analysis The job of a syntactic analyzer is twofold: •

to transform a sequence of tokens into a parse tree that represents the structure of the input unc.l that can be used as the basis for generating code7 and to check for errors.

•

We ·ve seen that most of the syntactic structure of most programming languages can be dcscrihed with a context-free grammar. But there are features, for example type constraints. that cannot. So one approach might be to move outward in the language hierarchy. For example, we might use a context-sensitive grammar instead of a contextfree one. That would solve many of the problems, but it would introduce a new one. Recall. from Chapters 24 and 29, that the best known algorithm for examining a string and c.lecic.ling w~ether or not it could be generated by an arbitrary context-sensitive grammar takes t1me that. is exponential in the length of the input string. Some programs are very long and compliers need to be fast. so that is not an acceptable solution. As a result, the way practical syntactic analyzers work is to: • •

exploit a context-free grammar that describes those features that it can. use a deterministic parser SU<:h as one of the ones described in Sections 15.2.3 and

•

augment.. the parser with specific code, for example a symbol table. to handle the noncontcxt-free features of the language.

15.3.3.

888

Appendix G

Applications: Programming languages and Compilers

G.4.3 Optimization Optimizing. compilers play an Important role in the dcvelopmc•H of modern software. But, unfortunately. the undecidability results that we have discussed, particularly the ones we summarized in Section 21.5. describe -some c,leor limits on what these compilers can do. For t:xamplc. consider the prohlcn'l of dead code elimination. Programmers do not intenlionallv write code that can never bt: reached at run time. But programs that have been mound for a while tend to accretc such dead code as a result of changes that affect overall control flow. Is it possible to build a compiler that checks for dead code and simply eliminates it? ·n1e answer is no. and it follows directly from Theorem 21.13, which tells us that the language { < M. q> ;: Turing machine M reaches q on some input} is not in D.

G.4.4 Compile-Time Error Checking Jt is safer and substantially more efficient to detect errors at cum pile time. rather than waiting until run time to do so. The theory that has ht!cn presl.!ntcd in this hook pr
Errors that Can be Caught by a Context-Free Parser Some errors arc easy to detect. TI1cy can be caught h} a context-free parser because they result in strings that ar~ outside \he language that is generated by the grammar on which the pars~r l)pcratcs. Ftlr example. givl.!n the expression grammar that we have used throughout this book. the expressions i d i d and i d ++ are syntactically ill-formed. Ill-formed strings present u challenge to parser designers. however. ll is generally unsatisfactory to find the first error in a program. stop. and rep,)rt that the parser failed. Instead the purser should try to find n point (perhaps a statement boundary), from which it can partially start over. TI1cn it can continue reading thl' rest of the program and checking for additional errors. To start over, the parser needs to figure out how much input to skip and how much of what is on its stack should be pnppcd and discarded. See [Aho, Sethi, ~nd Ullman l9Kc"lJ for a discussion of variuus ways of doing this.

Error Questions that are Decidable but not by a Context-Free Parser As we ~ave. already discussed, contcxt-f~ce grammars are unahle to capture the type constnunts unposed by mosl programmmg language~ However. all o£ the following questions arc dccidahlc. •

Given a program P. arc all the variahll!s in f1 declared'?.

'

Given a program P. arc all the variables in P used only in operations that are defined for thdr type'! Given a program P. do all the function calls in P have the correc1 numhcr of nrgumems?'

•

G.4 Compilers for Programming languages

889

Undecidable Error Questions Unfortunately, as we saw in Chapter 21 and, in particular in Section 21.5, there are other questions about the correctness of programs that are not decidable. The most basic is: L Given a program P. does P halt on all inputs? So no compiler can offer to find all infinite loops in programs. But there are others. For example. we showed, in Theorem 21.13, that it is undecidable whether a program reaches some particular state (place in the code) on any input. The question is also undecidable if we ask about a particular input or about all inputs. So all of the following questions are undecidable. 2. Given a program P and a variable x, is x always initialized before it is used? 3. Given a program P and a file f, does P always close /before it exits? 4. Given a program Panda section of codes within P, iss dead code (i.e., code that can never be reached)? Some other undecidable questions include: S. Given a program P and a division statement with denominator x, is x always nonzero when the statement is executed? 6. Given a program P with an array reference of the form a[i], will i, at the time of the reference, always be within the bounds declared for the array? 7. Given a program P and a database of objects (i, does P perform the function f on all elements of d? We will show that question 5 is undecidable. The proofs of questions 2, 3, 6, and 7 are left as exercises in Chapter 21. THEOREM G.1 uooes a Program Divide by Zero?" is Undecidable Theorem: The language L2 = { < M , s> : s is the statement number of a division statement to be executed by Turing machine M and, whenever M executes statement s. the denominator is. nonzero} is not in D. Proof: We show H s L2 and so L2 is not in D. Define:

R()

=

1. Co~struct the description of a new Turing machine M#(x) that, on mput x, operates as follows: 1.1. Erase the tape. 1.2. Write 1v on the tape, 1.3. Run M on w. 1.4. X= 1/Q.

z.

Return ).

890

Appendix G

Applications: Programming Languages and Compilers

{ R• ...,} is a reduction from H to L~. If Omde exists and decides L2. t~en C = -,Oraclt•( R( )) decides H. R and .., can bt! impl~mentcd as Tunng machines. And Cis correct. Note that there arc no explicit division statement~ in M (since division is not a primitive Turing machine operation). So the only division statement in M# is in step 1.4. Thus:

If < M. w> e H: M halts on ·w. so Mi makes it to step I .4. where it attempts to divide by 0. Oracle rejects. so C accepts. • If f H: M does not halt on w. so M# gets stuck in step 1.3. It never executes step 1.4. so, trivially. it is true that on all a ttempts rhe denominator is nonzero. So Oracle accepts and C rejects.

•

But no machine to decide H can exist, so neither does Orm:le. While. as we have just seen. there ate program errors that cannut he guaranteed to be caught and there are program properties that cannot be proven to hold. there are many useful situations in which it is possible Lo prove that a program meets some or all of its specifications. We return to this topic in the nc::xt chapter. where we will discuss a variety of tools that support both programming and softwart: engineering.

G.S Functional Programming and the Lambda Calculus In the 1930's. Alonzo Church. Alan Turing. and others were working on the problem that had come to he known as the Entscheidungsproblem. They sought an answer to the question. " Does there exist an algorithm to decide whether a sentence in first-order logic is valid'!" They all realized that to answer the question. particularly in the negative. they needed a formal definition of what an algorithm was. Turing's proposal most closely matched the procedural approach to computing that se~:mcd natural to most ~arly programmers. Thus the theory that we have heen discussin~ is based primarily on Turing machines. But Church's proposal, the lambda calculus, laid the groundwork for un alternative approach to programming that has had an important influence on the modern programming language landscape. ln this approach. called functional programming, one defines a program as a function to be computed. rather than as a ~t:quence of specific operations (possibly with side effects) to be perf,·,rmcd. In 1960.John McCarthy puhlishcd a paper (McCarthy 1~60} in which he described Lisp, a language that was directly inspired by the lambda calculus. Today Lisp .Q is the second oldest surviving programming language. (The oldest is Fortran.) Lisp remains the programming language of choice for many kinds of symholic computing applications. It is the platform that supports a variety of production tools. including Emacs Q a flexible and extensible text editor. knowledge-based systems like Scone c and KM Q, the music composition tool Common Music ~~and a user cxt~ nsion la nguage forth~ popular computer-aided design tool AutoCAD ~. just to name a few. It also inspired the design of an entire family of functional programming languages. including modem languages like ML and Haskell Q. 1

G.S Functional Programming and the Lambda Calculus

891

The easit!st way to begin to understand Lisp is to consider some examples, but o.ne caveat is required. Lisp is no longer a single language. It is a whole language family, each membe r of which has a different syntax. For illustrative purposes. we will use an easy-to-read syntax close to that of early Lisp systems; the various modern Lisp dialects will differ in some details. We begin with a simple expression.

(LAMBDA (X) (TIMES X X)) This expression defines a function of one argument that returns the square of its argument. We often want not just to define functions but to give them names by which they cap be referred. We can assign our function the name SQUARE by writing tbe following.

(DEFINE (SQUARE (LAMBDA (X) (TIMES X X)))) Since that syntax is clunky, there is an easier one.

(DEFUN SQUARE (X) (TIMES X X)) In this alternative syntax, it is no longer necessary to write LAMBDA explicitly. DE FUN takes three arguments: the name of the function. a list of the names of the function's arguments, and the body of the function. Named functions can call themselves recursively. So we can write the following.

(DEFUN FACTORIAL (X) (COND ((EQUAl X 1) 1) (T (TIMES X (FACTORIAL (SUBl X)))))) Read this definition as foll ows: Define the function FACTORIAL of one argument X. To compute it. evaluate a CONDitional expression with two branches. If X equals 1, then return 1! Otherwise (written as T, standing for True), return the result of multiplying X times (X - 1)! The FACTORIAL function is in many ways a toy~ it could easily be implemented with a loop in most programming languages, including modem dialects of Lisp. But it illustrates, for a simple case, the power of recursion. We'll mention less trivial applications shortly. In the early dialects o[ Lisp. there were only two kinds of objects: primitive objects, called atoms, and lists. Numbers. strings. and Booleans were all atoms. In this view, anything with an internal structure is a list. A list is written as a sequence of objects enclosed in parenth~scs. So. for example. (A B C) is a list that contains three atoms. A list may contain anothe r list as one of its clements. So, for example, we could have (A B (C D)), which is a list of three elements, the last of which is a list of two elements. The following list corresponds to a complete binary tree with a root node labeled Aand a height of 3. 111c first clement of each list is the label of the root node to which the list corresponds. The next two elements describe the node's two subtrees.

{A (B (D E))(C (F G))) Notice no~ that the definition we wrote for the FACTORIAL function is a list (with suhlists). In Lisp. programs are represented as lists. the data type that Lisp is best at

892

Appendi!l G

Applications: Programming Languages and Compilers

manipulating, as we ·n see he low.To par-se a program is ~nsy: The tree structure of a program is exactly its structure as n set of nested lists. So it is straighttorward in Lisp for programs to manipulate and modify other programs. Early Lisp programmers took advantage of this and wrote functions that explicitly manipulated the code of other functions. we·ll see one example of this below. a lthough il is now generally regarded as had software practice. Butt he fact that Lisp allowed programs to access other programs and the environments in which they were executed led to the development. within Lisp. of arguably the most powerful macro facility in any modern prog.ramming language. Modern dialects of Lisp have evolved suhstantially more sophisticated data typing systems than the simple. atoms and lists model that McCarthy introduced. But the notion that programs (functions) am data objects remains a key feature of the language and a rm1jor source of the llexibility that gives Lisp its power. In the Lisp programming environment. computation occurs when an expression (an atom or a list) is evaluated. Constants (including numhers. as well as the Boolean constants T and F) evaluate to themselves. Variahlcs evaluate to their values. Lists are cv~luatcd by treating the first cll.!mc nt as a function and the remaining clcmcnrs as the arguments to which the function should be applied. So. for example. we might write:

(FACTORIAL 5) This expression. when evaluated. will apply the FAGORIAL function to the value 5 and return 120. Before a function can he applied to its arguments. each of them must be evaluated. This wasn•t obvious in the (FACTORIAL 5} case because the atom 5 evaluates to 5. But if the variahlc X has the value 3. thcn (FACTORIAL X) will return 6. Lists can he written. as we have heen doing. as constants that arc specified by the programmer and they can he read as input. Or they can be construch~d within a program. Lisp provides a set of primitives for constructing lists and for taking them apart. The function LIST takes any number of arguments and puts them together to make a list. So we could write the following.

(LIST 'A (FACTORIAL 5)) When evaluated, thnt expression will huild a list with two clements. The first will be the symhol A. (Remember that arguments arc evaluated hcfo rc runctions are applied to them. The quote mark suppresses that evaluation. If we had omitted the 4uote mark in front of A. A would have been treated as a variable and cvalutlh.!d.lllcn its value. whatever it is, would have become the first clement or the new list.) The second element of the list will be the result of evaluating (FACTORIAL 5). So the new list will he (A 120). The function CONS (for constructor) adds an clement to the front of an existing list. So we could write the following expression. which will ~t:turn the list (8 A 120).

(CONS '8 '(A 120)) Lists can he broken apart into their pieces using two primitive functions whose names are historical accidents: CAR returns the first clement o( the list it is given. CDR returns its input list with the first element remun:d . So:

(CAR '(8 A 120)) evaluates to 8. (CDR '(8 A 120)) evaluates to (A 120).

G.S

Funct,ional Programming and the Lambda Calculus

893

In most programming environments, the semantics of the language are implemented in a black box runtime environment that cannot be accessed at the program level. This is 1\'t true in Lisp. Functions don't just look like lists. TI1ey tire lists and operations can be performed {111 them. The most important operation thal can he performed on functions is evaluation. Lisp provides functions, including EVAL, APPLY, and FUNCALL, that can be invoked from within any Lisp program. These functions explicitly evaluate expressions. For example. EVAL takes a single argument and evaluates it. To see how it works. suppose that we want to read in P. a polynomial function of one variable. and then apply it to se~eral values. Then we might input the following list. corresponding to the polynomial7r - 2x + 3. (PLUS (DIFFERENCE (TIMES 7 (SQUARE X)) {TIMES 2 X)) 3) Note that functions in Lisp are written in prefix notation. (Modem Lisps- use the more compact symbols+. *. and- though.) Suppose that the variable P now has that list as its value and the variable X has the value 4. Then the following expression will return the value 107. (EVAL P) Functions can also be passed as· parameters to other functions. Suppose. for example. that we want to write a program that takes two inputs, a function and a list. We want our program to apply the function to each element of the list and return a new li:;l with the rcsulling values. For example. if given SQUARE and the list (4 3 7), it should return ( 16 9 49). The Lisp function MAPCAR does exactly this. Given a function F and a list L. it first applies F to the first element of L. namely (CAR L). Then it applies it to the second element of L~ namely (CAR (CDR L)). And so forth. It re~ turns a new list that contains the values that it produced. So we can write the following exprcssic·m. (MAPCAR F L) If F has the value (LAMBDA (X) (* X X)) and l has the value (4, 3, 7). rhe result of EVALing our expression will be ( 16. 9. 49). Suppose. on the other hand,.that F has as its

value a function that returns. for any letter in the Roman alphabet. its successor (letting the successor of Z be A). And suppose that L has the value (C A T). Then EVALing (MAPCAR F L) will produce (0 B U). Modern dialects of Lisp provide an even more powerful mechanism for treating functions as first-class objects. They enable programs to construct and exploit closures, i.e., function definitions coupled with evaluatilln environments that bind values to variables. To sec how closures can be used, consider the following example. BOSS wants a list of candidat~s whose score on some dimension called KEY is at least THRESHOLD. To get the list. BOSS executes (GETSOME THRESHOLD). GETSOME considers. many criteria and has access to various sources of candidates. Each such source maintains its own list of possible candidates and it accepts two inputs, the number of candidates desired and a function (of a single argument~ that describes a test to be performed on candidates: only those that pass the test Will be suggested. Let's call one of the sources WELL GETSOME can be defined as follows. •

894

Appendix G

Applications: Programming languages and Compilers

(DEFUN GETSOME ( THRESHOLD)

/* C(lnsiJcr other things. (WELL

K l '(lAMBDA (X)

(AND (TESTl) (TEST2) (> (KEY X) THRESHOLD)))) f
)

When tht! expression (WELL .... ) is evaluated. the first thing that happens is that its ur~umcnts are evaluated. Its first argument evaluates to the value of the variable K.lt is ~tl:isumcd to he a numher. Its second argument begins with the symbol #' , which is shorthand for n function (called FUNCTION) that forms a closure hy capturing the current values of all of the free variables in the enclosed expression. In this case. there is a single such variable. THRESHOLD. So the closure that is formed contains the function descrihcd by the LAMBDA expression plus the current value of THRESHOLD. Without closures, one could imagine simply passing THRESHOLD as a nul her argument to WELL. But WELL may be called by many different kinds of customers. It doesn't know what THRESHOLD is. All it knows how to do is to select candiuatcs it wants to recommend and tht:n apply a single test to SC!c which ones will he acceptable to its customer. Its customer (in this case GETSOME) must therefore describe all of the tests it cares about as a single function that can he applied to a candidate. We 've now seen enough to be able to comment un some l)f the key ideas that underlie= Lisp and that play a key role in modern high-level programming languages: In Lisp. the must important data .structure is the list. Lists can bt: constructed by progra ms at run time. Their sizes cmd structures need not be dcclurcd in advance. It is very easy tu write programs that manipulate uynamic lists anu trees. So. rorexample. we were able uhovc to read in a polynt,mial function of arbitrary length and then EVAL it. ·n1cre is no need tn declare in advance what thc size of any structure i!'.'lllUs Lisp hau to provide run-time stnragc ullocatinn and garbage collection. (We should point out here. though, that Lisp was not the first Jist processing language. That lith:! belongs to the li>L family uf languages o. ) • In Lisp. t)ne describes a computation as a function to ht: cvnluntcd. While Lisp was not the first list-processing language. it was the first functional one. Lisp provided a few primitive functions and it nllnwed programmers to ddine new ones. To make this possible, Lisp introduced:

••

conditional expressiolfs: l11e Lisp COND function. which can have any number of branches. is the precursor of the modern i f-then-e 1se or case statement. We take such control statements for granted. hut at th~ time thut Lisp was first dcscrihed. Fortran, for example, only haJ a conditional go to statement. • ncur~ion: Another rcasun that Lisp imroduccu run-time storage allocation and garbage collection wns to make recursion pnssihlc.

•

G.S

Functional Programming and the lambda calculus

895

• functions as first-class objects: Functions can be manipulated by other functions and passed to other functions as arguments. In early implementations of Lisp. the runtime manipulation of functions was possible because the language was interpreted, rather than compiled. The job of the interpreter was to transform a list into something executable and then run it. While this execution model was flexible. it was also slow. Modern implementations of Lisp also provide compilers. with the consequence that. while Lisp maintains its flexibility, it no longer necessarily incurs a runtime performance penalty. •

Lisp functions are lists. a data type that Lisp programs can manipulate. Because they arc lists. they can be stored within other data structures and they can be parsed easily nt either compile time or run. time.

In defining Lisp. McCarthy did more than just create a new and arguably convenient way to program. He also made clear the connection between the power of the new languctgc and the fundamental notions of computability as investigated by Turing and Church. ln particular, McCarthy showed that the class of functions that can be computed in Lisp is exactly the computable functions (i.e.• exactly the set that can be computed by some Turing machine). llle Lisp language. as originally described by McCarthy. evolved into a family of dialects that became the programming languages of choice for the development of many kinds of artificial intelligence (AI) systems. The next few examples illustrate some of the reasons that Lisp was. and is. so well suited to the needs of AI programmers.

EXAMPLE G.S Search: Exploiting Recursion and Function Objects Lisp's natural control structure is recursion. This contrasts with the iterative control structures that were the mainstay of other early programming languages. What many AI programs do is search and search can easily be described recursively. To evaluate a situation to see whether it can lead to a solution to a problem, we generate all of the situations that can be reached via a single action from the current one. If any of them is a goal. we have found our answer. Otherwise, we call the evaluation procedure recursively on each of the successor states. In Section 30.3.2. we describe A*. a general-purpose, best-first search algorithm that does this. and in N.2.5, we describe minimax, an alternative that is tailored specifically to searching two-person game trees. While recursion is key in implementing almost any search algorithm, another feature of Lisp is also important in implementing best-first search algorithms like A* and minimax: The input to all such programs is a problem defmition, which must .contain two .ru~c~ions: successors, which computes a node's successors and ass1gns costs to mdtv1dual moves, and h, a heuristic evaluation function. In Lisp. it is easy to write A* or minimax once and then to pass it as parameters, the functions that it needs. '

896

Appendix G

Applications: Programming Languages and Compilers

EXAMPLE G.& Representing Parse Trees as Lists To see why list-like structures that need not be declared in advance are useful in AI. consider the natural language (NL) understanding problem. An NL understanding program must accept as input one or more sentences. In Lisp. this is easy. The input text can be represented as a list of words. So assuming that we have defined symbols that correspond to the words in our dictionary. we might, for example, have:

(the smart cat smells chocolate) An early step in understanding sentences is usually to parse them. Recall that, in Example 11.11, we showed the following parse tree:

s VP

NP

~

v

I

Adj.~

Nomit1al

I

I

Adj

N

I

the

NP

smart

cat

smells

I

chocolate

This tree can easily be represented as a Lisp list. in which each node is a list. The first element of the list is the label a ttached to the node. The remaining elements are node·s subtrees. So we have:

(S

(NP (the) (Nominal (Adjs (Adj (smart)))(N(cat)))) (VP (V (smells))(NP(Nominal (N (chocolate))))) )

EXAMPLE G.7 Representing Logical Formulas as Lists In many task domains, knowledge can be represented as sentences in first-order logic. These sentences, in turn, can easily be represented as Lisp lists. For example. the sentence: ~x

(3y (P(x)

A

R(x)- Q(y)))

can be represented as the list:

(FORALL X (EXISTS Y (IMPLIES (AND (P X) (R X)) (Q Y)))) The sentence can then be evaJu~ted by recursively evaluating its subexpressions.

G.S

Functional Programming and the Lambda Calculus

897

EXAMPLE G.8 Automatic Programming In Lisp, there is no distinction between programs and data. Lisp programs, like everything else. are lists. This turns out to be particularly useful if one wants to huild programs that can write or modify other programs. So, for example. one subrield of AI is called automatic programming or program synthesis Q . Its goaJ is to automate the task of writing code to solve a problem whose specifications have been provided. Of course. the problem of deciding what code to write remains hard. hut in Lisp it is straightforward to build up a program by composing lists. And the code that is built in that way can be run. as part of the coding and debugging process, in exactly the same way that any Lisp expression is evaluated.

EXAMPLE G.9 Learning to Improve Performance An important characteristic or an intelligent system is its ability to learn and to. improve its performance on the basis of experience. If a performance program is written in Lisp, then a learning program. also written in Lisp, can modify the performance program using Lisp ·s basic list operations. For example, suppose that we want to build a program that evaluates its environment and. on the basis of what it sees. decides how to perform a task. We might write such a program in Lisp using the following structure:

(COND ( (AND (conditionl) (condition2) (condition3)) ( (OR (condi tion4) (conditionS))

(action A)) (action B))

)

Now suppose that we want to learn to perform the task better. There are several things th~t we might w~nt to be able to do. We might discover a new special case that requares a new a~tton. The alternatives that are listed in a Lisp COND statement are evaluated m the order in which they are written. So we could describe L~c sp~cial ~ase by adding a new branch at the beginning of the COND expressaon. producmg:

(COND ( (condition6) ( (AND (conditionl) (condition2) (condition3)) ( (OR (condition4) (conditionS))

(action C))

(action A)) (action B))

)

898

Append!)( G

Applications: Programming Languages and Compilers

EXAMPLE G.9 (Continued)

Note that we'rl! not simply claiming that a programmer can make this change. A Lisp program that notices that the change is necessary can use CONS to add the new condition to the front of the list of branches. Or we might want to generalize the behavior of our program so that it works in some additional environments. One wav to do that would be to change the AND on line 2 to an OR. Another way would h.c to remove one. or more of conditions 1, 2, and 3. These changes can eas.. ily be made by the learning program to the list representatio n of the performance program. Then the new program can be run and evaluated to see whether the change improved its performance.

EXAMPLE G.10 Procedural Knowledge ·nu! cor~ of any AI program is its knowledge base. Some knowledge can naturally be thought of as declarative. For example. John's phone number is a string of digits. his age is a number. his mother's name is a string, and his birthday wish list is a list of ubjl!cts. But in real applications. the values for even these simple attributes, much l~.:ss more complex ones. may not always be known. Instead, what may be available are procedures for computing values as they are needed. Thus it may make scnsc to store. as the value of John's phone numher. a function closure that was created in a context in which il was known what city he lives in.The function searches tha t city·s phone book to find John's numher. Similarly. the value for J<>hn 's birthday wish list might be a procedure that executes in an environment that knows his hubbies and his favorite foods.

As originnlly ddincd. Li.sp was a purely functional language. All computation ochy cvalu~•ting functions. ll11!re existed no opcrations that caused side effects. Most mndcrn dialects have been extenJcd to allow side.! effects in vurious ways. In addition to input and output functions. there are typically functions that assign ~alues to vnriahles and functions that destructively mollify lists (as opposed simply to creating new ones). There may also be a way to ucscribc a sequence of actions that should be pcrf(,rmeu. So mo~t modern Lisps are not purely £unctional. But there are arguments for purely functional programming. In particular. when side effects an~ allowed. constructing correctness proofs may be hard. ln the vears since McCarthy's original d!!scription of Lisp. a variety of oth~r purely functinnai languages have hccn d~.: ~incd . ·nlt:sc languagc_s. of which llal·k rll o is prohably the most widely used. uwc Hn mtcll~ctu a l debt to L1sp. as well as to other developments in the area of high- l~.:vd programming language design.

curr~d

p.PPEND

X

H

Applications: Tools for programming, Databases and Software Engineering he format structures that we have been discussing have inspired the design of many different kinds of tools that programmers use every day. We have already discussed the design of high-level programming languages and the construction .o f compilers for them. In Appendix 0. we'll discuss the use of regular expressions in modern programming environments. In this appendix. we'll briefly describe. some other kinds of uscl'ul tools whose design is rooted in the theory that we have described..

T

Proving Correctness Properties of Programs and Hardware Consider the problem of proving that a particular software program or hardware device correctly implements its specification.1n the rest of this discussion~ we will use the term ..svstem" to dl!scribc hoth software and hardware systems. . . l£ tht: "nswcr to the Entscheidungsproblem that we introduced in Chapter 18 had been yt!s (in othe.r words, if there did exi:~t a procedure to determine whether an arbitrary ~cntcncc in first-ord~r logic is valid), then it might be possible to:

1. Write n first-order logic sentence that corresponds to the specifications for a .system.

z.

Write an()thcr first-order logic sentence that describes what the system actually docs. An effective way to do this is in two steps:

2.1. Define a set of first-order logic axioms that describe the primitive operations that a system can perform. For example, we could describe the behavior of an individual gate in a hardware circuit or an individual statement in some particuhtr programming language.

900

Appendix H

Applications: Tools for Pr<;>gramming, Databases and Software Engineering

2.2. Derive from those axioms- und the 'definition of a particular system (its logic design or its co<.lc) the required se ntence that <.lcscrihcs the system's behavior 3. Build a theorem proving program that could determine! whcthl!r tne sentence from step 2 entails the sentence from step 1. (In othe r words: Given a system that behaves as <.lcscrihed in the sentence from step 2. must the sentence from step 1 be true? Put another way; Does the system satisfy the specification'?) Bul. as we saw in Chapter 19. the answer to the Entschcidungsprohlcm is no. That result, proved indcpcndc.mlly by Turing and by Church in the: mid 1930s. coupled with the Incompleteness Theorem published by Kurt Gt"l<.lel at about the same time. dashed the hopes of mathematicians that they might find a completely syntactic basis for mathematics. It also means that it won't be possible to huild a completely automatic. general -purpose. first-order logic-basco verification system that cnn b~ guaranteed to hull and return Tnw precisdy in case the target system satisfies its specification and Fal.,·e otherwise. Early interest in the Entscheidungsprohlcm was motivated hy a concern with issues in mathematics and philosophy. Now, with the advent of mod~rn computers. we have a new and more practical need Jor a syntactically-bas~d theorem-proving method: We build huge and complex pieces of hardware an<.l software and we trust them to perform critical functions. If we could build programs that could produce formal proofs of the correctness of those critical systems. we would have an increased basis for the trust we place in them. (We say an increased basis for the trust, rather than total trust, because there would still be issues like the extent to which the formal specification corresponds to our goal for our systems. the correctness of the proofgenerator itself. and limits to our ability to describe, all the way down to the electrons. the behavior of the hardware on which our systems run). Fortunately. the negative results of Church and Turing do not doom all efforts to build mechanical verification systems. It is true that we showed. in Chapter 21. thut there arc some program properties. including some that could be part of many rl!asonablc specifications. that are undecidable. These pmpcrties include: •

Given a program P. does P halt on all inputs
• •

Given a program P. does P ever output anything'? Given two programs P1 and P2• are they equivalent'!

Nevertheless. there are useful correctness pn_ ) pcrtics that can he proved. at least of some programs and devices. For exampk. while it is not possible to decide whether an arhitrary program always halts. it may he possihh! to prove that a particular one does. To construct such targeted proofs. we require a logical language that can be used to represent specifications and to describe the hchavior of systems. In particular. we must find a logical language that meets all of the following rc4uircmcnts. •

ll is expressive enough to make it possible to encode both specifications and descriptions of system behavior.

H.1

Proving Correctness Properties of Programs and Hardware·

901

• Its decidahilily properties are strong enough to make it useful. • The complexity of its decision procedures is acceptable for the size problems we wish to solv~. These issues trade off and there does not appear to be a single approach that worst best for all kinds of problems. But there are two general approaches, each of which has proven to be effective for some classes of important problems: •

deductive verification systems, in which steps 1 through 3 are done but step 3 is typically only semiautomatic-a human user must guide the theorem prover~ and • model checking systems, which are usually fully automatic but are limited to rea·soning about systems that can be described with finite (and thus decidable) mo4els.

H· 1.1 Deductive Verification Deductive verification systems find proofs in much the same way mathematicians do. The core of all such systems is a theorem prover that begins with a set of axioms and then appli~s rules of inference to derive conclusions. The theorem prover may also be augmented with a set of conventional programs that perform tasks such as the computation of standard arithmetic operations. Effective verification systems must cope with two realities of the ta'\k: •

expressively powerful logical systems (including standard first-order logic) are undecidable, and

•

the number of legal proofs grows exponentially with the length of the proof so, t:vcn when a proof exists, a brute force approach to finding it will take too long.

Modern deductive veriticatjon systems Q solve those problems by choosing a carefully designed logical language and then exploiting an interactive theorem prover. A human user guides the theorem prover in one or more of the following ways. .•

The user describes the steps (lemmas) that should be solved on the way to a com-. plete proof. Ir the theorem prover is unable to complete a step, the user can provide additional information.

•

111e user tells the theorem prover what substitutions to perform as the variables and constants of one expression are matched against those of another.

Interactive verification systems of this sort have been used both to find faults and to prove correctness in a wide variety of critical applications. Some examples Q include.: • •

ll1e lliscovcry of llaws in the design of control software for an observatory. ·n1c analysis of cryptographic protocols (e.g., Bluetooth) for security properties.

But the fa~t that pcopl~ must he part of the verification process has meant that the sprca_d of. this approach mto practical system construction has been limited by the ~car~Jt~ ol people who understand the required logical language and who are skilled at pnw1dmg thl.! asstshmce that the theorem prover needs.

902

Appendix H

Applications: Tools for Programming, Databases and Software Engineering

H.1 .2 Model Checking Suppose that the system (software or hardware) whose correctness w.:: would like to prove can be modeled with a finite numher of states. Perhaps (us is often th~ case with hardware) it was originally designed as a finite state machine. Or perhups it was d~signcd some otb~r way but its state can be descrihed by a finite numher of variuhks and each of those va.nables can take on a tinitc numher of values. In the latter case. it is straightforward. to bu1ld a finite state mnchine that ucscribcs the operation of th&: system. Many concurrent systems, for example. can he modeled in this way sin<:t! tht!rc
H.1

Proving Correctness Properties of Programs and Hardware

903

• Sis a finite set of states designed, as described above. to correspond to the possible assignments of values to the variables of Y. • S11, ;~ subset of S. is the set of start states of Y. • p is a non~cmpty set of atomic propositions that describe properties (such as the fact that the variable x = 0) that may hold in the various states of S. • R is a transition relation. It is a subset of S x S. The pair (q 1• q2) e Riff Y can go di~ rcctly from tit to q2• Since Y does not hah. it must be the case that, for every state q, inS, there exists at least one state q2 such that (q 1, q 2) e R. Note that we do notre~ quire that R be a function. So it is possible to model systems whose behavior is nondeterministic. • L is a function that labels each state in S with the set of propositions that nre true in it. SoL maps from S to 'lP (P).

EXAMPLE H.1 Modeling a Simple, Two-Switch System Consider a very simple system with two switches. One is the tt switch and it can be on or off. The other is the blc switch. It can be off or it can be thrown to b or c, but that can happen only if it's currently off. Once it's ever thrown to c. it can't be changed. llu: a switch can only go off if c is on. The system starts out with the ll switch on and the blc switch off. We can model this system with the Kripke struc~ ture M = ({{a}, {a, b}. {a, c}, {c} }, {{a}}. {a, b, c}. R, L). where L assigns to each state the labels we're using as the state's name and R is given by:

A con~pu~ation_ 0 ~ a system Y: described by a Kripke structure M, is a path through the states of M. Smce computattons don't end. any such path will be infinite. Since a ~talc ~ay ~~ve ~c~rc.. than. one succcss~r. we. can describe all paths that Y could follow. from some start state q. ~sa computat•on tree rooted at q.

904

Appendix H

Applications: Tools for Programming, Databases and Software Engineering

EXAMPLE H.2 The Two-Switch System's Computation Tree Consider again the system we described in Example H.I. Its computation tree. starting from state {a} contains:

Steps two and three in using a model checker are to state the specification and then to show that, on every computation path. the system model satisfies the specification. The technique that is used to perform step three depends on the language that is used to define the specification in step two. We'll consider two approaches: •

Use a temporal logic to define the specification and apply one of a family of model chc:ckinA algorithms to compare the specification to the Kripke structure that models the system. Describe the specification as a Btichi automaton. convert the Kripke structure that models the system into a second Btichi automaton. and use operations on automata (complement. intersecti'on. emptiness checking) to decide whether the system satisfies the specification.

•

We first consider writing specifications as logical formulas. Typically. the specification for a system Y imposes constraints on the compulational paths that Y can follow. For example, we might want to guarantee that Y never enters a state that corresponds to the situation in which x = 0 andy = 1. Or we might want to guarantee that. once x becomes 0. y never does. To facilitate stating such requirements. the logical language that is used in model checking systems is typically some form of temporal logic in which it is possible to describe constraints on the future states of the system giver: its current state. Formulas in temporal logic may describe properties that must be true of states. including properties that must be true along paths that can emerge from those states. There are two main kinds of temporallogi·cs that are used in model checkers: •

Linear time logics, in which there is always a unique future. and

H.1

•

Proving Correctness Properties of Programs and Hardware

905

Branching time logics, in which, given a particular moment in time, multiple futures are possible. Branching time logics typically provide quantifiers that can range over paths. Common quantifiers are: • A (for all computation paths), and • E (for some computation path, i.e., there exists some computation path).

Temporal logics provide operators that can be applied to propositions. The following operators are present in the branching time logical language CTL• Q, and are typical: • G P, which holds iff Pis always true (is true globally), • F P. which holds iff P will eventually (at some time in the future) become true, • X P, which holds iff P holds in the next state, • P1 U P2, which holds iff P2 eventually becomes true and, at every state until then, P1 is true (P1 until P2), and • P1 R P2, which holds iff P2 holds in every state up to and including the frrst state in which P1 is true ( P1 releases P2).lt is possible that P1 may never become true, however.

EXAMPLE H.3

Some Simple Specifications of the Two-Switch System

We return again to the system we described in Example H.l. This time we'll consider some possible specifications and see whether the system satisfies them:

ave EG(av c)

This holds in aU states. (There exists a path such that a v c always.) This holds in aU states.

EFc EGc

(There exists a path such that eventually c ) This holds in aU states. ·

~ere exists a path such that c always.) This holds only in {a, c} and {c}.

EcRa

(~ere exi~ts a path where a until released by c.) nus holds m all states except {c}.

906

Appendix H

Applications: Tools 1or Programming, Databases and Software Engineering

Now we hcwe. for a system Y. a Kripke structure M that descrihes Ys implc.!mentatio n and a temporal logic formula j'that describes the requirements (specifications) for Y. The final step in determining the correctness of Y is to decide whether the implementation conforms to the specification. We would like to prove that there is no path thro ugh M that fails to satisfy [.If, o n the other hand. there is such a pa th. we would like to report it. The fact that model checkers can do more than "just say no'' is one reason that they are particularly useful in practice: A countt:rcxample tells the system's developers exactly where the system can fail and thus points the way to solution to the problem. Furth~r. if a specification requires that the re exist a path with some desirable property. the model checking process will find and repnrt such a "witness." The most straightforward algorithm!> for modd checking work with an explicit representation of the Kripke structure M = (S. S11• R. P. L) that describes the implementation. The idea is that we will consider the states in S and we will annotate each of them with those suhformulas [rom the specification f that cnn be shtlWn tu hold in it. The annotation process begins with the labels that are already attached to the states by the labeling function L. Then it considers the subformulas in .1: starting with the most primitive, builds up longer and longer annotations. a nd propagates them through M. lf all subformulas hold in all start states. then all computation paths satisfy j:

EXAMPLE H.4 Evaluating a Specification of the Two-Switch System We 'II continue with the same example system. Suppose that the specification for it is EG c. (In other words. from the start state there exists a pa th along which cwill eventually become true and stay true.)

• c holds in state {c }. • EG c also holds in {c }. • Thus EG c also holds in {a. c}. •

Thus EG c also holds in {a}. which is the only start stale, so this implementation satisfies EG c.

H.1 Proving Correctness Properties of Programs and Hardware

907

111C detail!; Of ~my model checking algorithm depend on the temporal logic language that is usctl for the specification. But, for example, for CTL~, the language we discussed above, there exists n model checking algorithm and its complexity is O(l/1· (lSI + IRI)), where f is the formula that describes the specification. So it is linear in both the size of the formula f and the size of the Kripke structure M. For large systems. though. this isn't good enough because the number of states in M may be 0(21d), where vis the numher C>f variables in the system that M is modeling. To solve thi.s prnhlcm we need a technique that docs not require the explicit construction of M bcfon: we can start. Suppose that instead of describing a system Y as a set of states and transition~ hctwccn them. we could deS(:rihe Y as a Boolean function.Thcn we could use an ordered binary decision diagram (OBDD).as described in R13,as an efficient way to represent Y. To start, we'll descrihc each state as the Boolean function that describes the condition under whkh l' is in that stute. So, for example. suppose that there are three atomic propositions (·n1, 112, 1~~) in our mudd. 1l1cn we'll represent the state shown in Figure H.l(a) as 111 A -.·v2 A 113• Now consider any transition in a Kripke structure. as for example the transition from state ( l) to state (2) shown in Figure H.l(b). We can think of this transition as a relation between the sets or propositions that are true in state (1) and those that are true in state (2). To describe it that way, we need a second set of proposition names. We'll use the original set A tu describe the- propositions that hold in state (1) and a new set A' to descrihc those that hold in state (2). Then we can describe the transition relation using its characteristic function (i.e•• a Boolean function whose value is True for each element of the relation and 1-"a/.'ie otherwise). Using this idea, we construct. for the single transition shown ubove. the Boolean function:

From the functions that describe individual transitions, we can construct a single function that is true for an entire transition relation.IL is true whenever any of the transition relations is true. So. for the simple two-state. two-transition structure of Figure H.l(c). we can construct the Boolean function:

(b)

(c)

FIGURE H.l States can be described as Boolean expressions.

908

Appendix H

;Applications; Tools for Programming, Databases and Software Engineering

This representation doesn't look smaller than the original stale model. but that's primarily because the original state modd was alrcauy small. In cast.!s where the state model is enormous.thl! OBDD representation of the corrcsponuing Boolean function is often small and it is possible to construct that representation directly from Y's description (as a logic circuit or ~~sa program). without tirst creating the Kripkc struc.: ture (and thus representing all possible combinations of vari
EXAMPLE H.5 A BOchi Automaton for a Simple liveness Property The following Btkhi automaton corresponds to a simple livcncss property. As in Example 5.39, we use the atomic propositions: C Ru (proc:e.'i.\'i1 is in its critical region) and CR 1 (prm:ess 1 is in its critical region). This time. we require that prm:es.'i0 eventually enter its critical region:

Next we need a Bikhi automnton that corrcspunus to
H.1

Proving Correctness Properties of Programs and Hardware

909

EXAMPLE H.6 Converting a Kripke Structure to a Buchi Automaton Returning again to our two-switch example, we.can co~vert the Kripke st~ucture to a Buchi automaton as shown here. As we did m labeling the states, we wtlllabel the transitions with the propositions that must be true. All others must be false. So. for example, a 1\ b should be read as equivalent to a l\ b 1\ -c.

Biichi automaton:

Kripke structure:

Given two BUchi automata,

Bsi'EC•

which corresponds to Y's specification. and

Bu.tl'• which corresponds to the Kripke structure that describes an implementation,

we're ready to decide whether B1,.,p satisfies B sPEC· We proceed as follows: Construct BuM)• a Btichi automaton that accepts the complement of L(Bsr£d· So BBAD accepts cxuctly the strings that violate the specification Next, construct B8orH• a BUchi automaton that accepts the intersection of L(BoAD) and L(BIMP>· Finally, test whether L(ll11aru) is empty. If it is not. then there are computation sequt:nces that are possible in the implementation but that are not allowed by the specification. If it is empty. then there arc no such computations and we have proved that the system satisfies its specificHtion. Note that this procedure works because the class of languages accepted by BUchi automata is closed under both complement and intersection and there exists a decision procedure for the emptiness question. It is also possible to skip the complement step if the user enters the negative specification directly. Using. these techni4ues ~. model checkers have been used to: • •

Prove the correctness of general-purpose processor chips. Prove the correctness of special-purpose processors such as for video game consoles.

910

Appendix H

Applications: Tools for Programming, Databases and Software Engineering

•

Prove that a logic optimizt:r has not altered the funct.ionulity of a circuit. This is done by showing that the functions for the two ci rcuits (the original one and the optimized one) are identicul.. • Prove the correctness of network protocols. including, for example. the allernating bit protocol described in I.J.2. • Prove the correctness of critical real time systems such as the controllers for aircraft and space exploration vehicles. For cxa;nplc. the SPIN 0 . model checking system found five previously undetected concurrency errors in the plan execution module of the controller for a space-craft that NASA launched in ll)l)~!

H.2 Statecharts: A Technique for Specifying Complex Systems Consider the follow~ng way of dividing systems into two important classes (as described in Harel 19R7)):

•

Reactive system.r are dri\'cn by (possibly asynchronous) sequences of external events. So, for example, the telephone system, your watch, your em. your microwave, your Web search engine, and your operating system are all reactive systems. Reactive systems typically have little control over the sequenc-es of inputs that they may receive.

•

Transformational systems, on the other hand, typically have· more control over their inputs.Tht!y accept inputs in particular forms and compute functions of them. For example, a payroll system accepts an input file and outputs a st!t of checks.

While the distinction between these two kinds of sysh:ms is not hard and fast. it is useful. In particular. it highlights one reason that designing reactive systems is hard (the fact that arbitrary sequc11ces of inputs must be handled properly). And it suggests a way to huild tools that are particularly well-suited for the design of those systems: Let the tool provide explicit support for describing the way that the syst~m·s state changes as inputs are received. ln Chapter 5 we used this approach to system dt!sign when we huih finite state machines. But what about real systems with real complexity'! Can they too he modeled with a finite number of states'! It turns out that often they can. And. in other cases. they can he modeled with a finite number of states plus a set of variables that can take on arbitrary values. A family of design tools based on statecharts 0. us dcscrih(!d in IHarelllJX7} and [Harcl and Politi 199X). exploits this ohscrvation. A stauchtut is u hierarchici\lly struc-tured finite state machine. The hierarchical structure nu1kcs it possibt~ tt): •

view a system design at whatever level ol' detail is ncccssilry.

•

describe a system design using fewer transitions than would he required in a flat structure. This happens hecausc a single transition from a par~nt state implicitly describes a whole family of transitions from its child states.

Statecharts differ from finite state machines. cts we•ve been using them. in one· addir.innal important way: Suppose that a complex reactive ·s ystem is made up of a set of

H.2 Statecharts: A Technique for Specifying Complex Systems:

911

components that act independently (or mostly independently) of each other as they respond to different kinds of input signals. For example. imagine a cell phone that must simultaneously listen for incoming calls while allowing its user to manage the local phoncbook. 1l1e "st.ate~· of such a .s~stem must.reflect the states of all o~ its. components. Rather than forcmg an exphc1t enumeratiOn of all such state combmauons as a way to describe the overall system's state, statecharts allow the specification of parallel state sets. A complex system can then be described as being in multiple states at once and the total numher of states that are required in the description becomes the sum of the numbers of th!! separate states. rather than their product. Statt:charts have been widely used in a variety of software design contexts. including real-time systems. simulations, and user interfaces. Statechart capabilities are. now part of many software design and implementation systems. including general-purpose tools such as the Unified Modeling Language (UML) Q , as well as specialized languages that have been crafted to support the design of particular kinds of reactive systems. For example. SCXML ~. is.a statechart-based tool that supports the design of voice. and multimodal interfaces. The details of how one specifies a set of states vary from one system to the next. ~vpically there exists both a graphical language and a text-based
cmy /1111/tm /lll!tltt•tl

\ setlmtton pusllrd tli.~playillg

(lltJrm·bt'<'fliiiJ; tl/111., tri~~:,•r ~

\

f'IGt.JRE lf.2 A simple. top-level model of a watch.

sercing

dcme btlttcm p11slleci

I

912

Appendix H

Applications: Tools for Programming, Databases and Software Engineering

llfllrm·lwepill$:

tm.v hutton flll.~/rl'tl ~tting

FIGURE H.l Zooming in to the alarm-hccping stale.

One might easily take issue with several things ahout the design that we have just described. For exampJI!, perhaps. if the set butt()n is pushed from the alarm-beeping state, the wntch should go directly to the selling state. The point of this example is not to argue for this particular design. It is to show the way that a stntechart makes clear what a particular design is and what decisions were made in constructing it. Of course. to build a watch. we need a more detailed design than the one we have just presented. To provide it. we must zoom in to each of the top-level states. Zooming into the ala rm-beeping state. we might see the statech,ut shown in Figure H.3. Now we see that this watch has two separate alarms. either of which may be set. We've used the: symbol a1 to mean that the first alarm has been triggered and a2 to mean that the second one has. Notice the way in which we used the statecharrs hierarchical structure mechanism to reduce the number of transitions. compared to the number that would be required in a flat machine: lltc transition from the alarm-beeping state back to the displaying state docs not need tt) be duplicated for both of the alarm sutlst~tes. Instead it is attached once to the parent state. Transitions are assumed to be inherited. unless overwritten. from parent state to child states. Now suppose that the watch has a background light. which can be in one of two stutes. on or off. By defa ult, it will be orf. but it will go on if either the light button or the set button is prt!ssed (the latter on the assumption that the user may need the light in order to sec well enough to perform any settings). So now. at any point, the watch is in some state within its main controller. as sketched in Figure H.2 , and it is in one of the tight stc1tes. We could model that by creating a second copy of the main control box. with one copy corresponding to the light being on and the other corresponding to it being off. But that doubles the number of states. Instead. we can exploit the ability of statecharts to represent (nearly) orthogonal state sets. Orthogonal sets will be separated hy a dashed line. If a model contains orthogonal state sets. it should be read to say that. at any point. the system is in one state from each of those sets. The new model, shown in Figure H.4. has five states. That's only one fewer than we would have had if we had enumerated all the combinations. But imagine a more realistic model. in which each of the components had containl!d 1.000 states. Re presenting all combinations of them would require 1.000,000 states. Using orthogonal state sets. as we just did. th::n model would be ahlt! be_ ~cscribcd wi~h j~st 2.00C~ states. Note. though, that we have not thruwn away the ah11tty tu dcscnhc mtcracllons between/among

H.3

dtl\' lmtrr.lllf"ISII~·d

lllimll·l"'''l'ing

\

913

set l>lllltm pmlwd ,Jisplttyins

~

setting tlmw lmlltm pu.~lwd

11ilmu triggt•r

--------

Model-Based Test Case Generation

ufllrm trigg<'i'

I

------- ------~------------ ~ -~~- ~-----~-------------

.Vt'tfJIIIItlll fJIISfw(/, /ig/11 /.11111011 J>WI/rt't/

nfj'

WI

tfllllt'

FIGURE H.4

D~scrihing

/1/ttrtm fWSflccf. J0 !lt'I'CJIIds d11p.~ed

the watch as two orthogonal state sets.·

orthogonal states when they occur. So, in our simple example. a single event. set button pus/led. can cause a state change in both components of the model.

Model-Based Test Case Generation We've now scc.n two uses for models that describe the desired behavior of systems that we wish to build: • •

Models can be used to verify the correctness of an implementation. Models can be exploited as design tools.

We'll now bricny mention a third: •

Models can be used to generate lest suites. 'l11c goal of lest Ci\se generation Is to construct a set of tests that. collectively. increase

our confidence that the behavior of a system under test conforms to some set of desired properties. Suppl)Se that we have a modd that describes those properties. Then it is possihlc to exploit that model in an automated tool for generating test cases. A variety of tedmiqucs can be used to dn this Q; the details depend. among other things. on the formalism in which the model is written. We'll mention just one idea here. Suppose that we have 1\ finite model of the sort that can be used as input to a model checker. If the model checker discovers a counterexample (i.e., an execution path that does not satisfy the specification). we know that there is a bug and can go about fixing it. But the model checker will typically find many paths. called witnesses. that do satisfy the specific.ation. Unfortunately. we are still not certain that no bugs occur along those paths ~mcc. the system n~odel is almost always an abstract description that ignores dctmls .ut the system's Implementation. ll is those details that need to be checked by testmg. So one wa~ tn generate test cases is to choose inputs that force the system under test down the Witness p~nhs.

914

Appendh; H

Applications: Tools for Programming, Databases and Software Engineering

H.4 Reverse Engineering Engineers start with spt:cifications and build artifacts. Reverse engin~crs start with artifacts and att~mpt to reconstruct specifications {and. often. various levels of structures that implement those specifications). One c'm try to reverse engineer just about anything. A common quip is that physics is an attempt to reverse engineer the uni~erse. Molecular biology is an attempt to reverse engineer the genetic code. Software ptrates reverse engineer their competitors' code. Why do reverse engineering'? Among the possible answers to this question are: • •

We just want to know. Physicists prohahly relate to this one. If we understood an artifact better. we could usc it more effectively. Physicists and ene.inecrs relate to this one. But also consider a pi~:cc of software for which no one ha; vet bothered to write a manual or a piece of software with undocumented feature~. Suppose. for example. that we want a systc;:m we arc building to share files with another system but the group that buill the other system never described the internals of their file structun.!. If we could reverse engineer that other system, we could discover how its file structure works. ·• We've got an artifact that's broken and we wunt to unde rstand its structure so we can fix it. This one drives research in molecular biology. It also comes up a lot with legacy software. which can easily become obsolete cvt!n if it doesn't directly .. break ... For e xample. the number of bits allocated to some field may no longer be enough. But we don't know what the consequences would be if we changed it. •

•

We've gut an artifact that is old and clunky. We want to r~place it with a newer, sleeker version but first we have to figure out exactly what it does. This one comes up all the time with legacy software; We want to steal our competitors' ideas. ll1is is why we have patent law.

We'll focus here on the specific problem ol' reverse engineering nf software. The problem is one of analysis and the artifacts we need to unalyzc;: arc strings. So this seems like a m\tural application for many of the ideas that we have been discussing. We'll brietly mention two techniques, one based on extending our notion ol' regular expressions, the other on the use of island grammars.

H.4.1 Hierarchical Regular Expressions

In Section 11 .9. in our introduction to Lo;land grammnrs. we sketched sumc of the problems that arise whe n we try to analyze legacy software. In a nutshell. any approach that requires exact matching will probably fail hccuusc of errors. the interleaving of different languages. dialect differences. and irrelevant code. among llthcr things. Further, there are application~ that. by their nutur_e . need to h: fust and cheap. For cxnmple. suppose that we are trymg to analyze code m order to ftgure out how expensive it would be to make some proposed change. We require that this feasibility an~slysis be cheaper and fast\!r than the more complete analysis that will prohuhly he rcquirt!d if we decide to ahead with the update. What we need is a good t~chniqut! for what is oftcn callg~ lightweight a11alysis. e A robust lightweight analys~s tool n~cds to he ncxihlc. It needs to he able to find instances nf pntterns that descnhe the pans of the cmJc that matter. And it needs to hP

H.4 Reverse Engineering

915

able to ignore the rest. Regular expressions are very good at this. For example, we could write the regular expression: (a U b)"' aaabba (a U b)*. which will match any occurrence of aaabba, while skipping over everything else. But, in a realistic software engineering environment, the regular expressions that we would have to write to get the job done become too complex to work with. One approach to solving this problem {Murphy and Notkin 1996) is to organize th~ regular expressions hierarchically. In this approach. we still allow only regular e~press1ons that can be compiled into finite state machines that can. in turn, be used to tmplement the actual search process.1bis means that the search process can be fast. So the use of regular expressions that we arc suggesting here contrasts with the use of extended regular expression notations, for example in Perl and in much of the Unix world. To see wh~1t hierarchical regular expressions can do. consider the following example (from [Murphy and Notkin 1996}):

[ ] \( [ { }1 ] \) [ { ; }1 ] \{

\( [ { }1 ] \) To rend these patterns. note two conventions: Expressions enclosed in square brackets arc optional. Reserved tokens, like brackets and parentheses, can be quoted using\, Tiu.: job of these two regular expressions is to extract static can relationships among functions. 'Otc first pattern looks for function definitions. It will match an optional type statement. fuiJuwcd by a function name. an open parenthesis. an optional list of formal parameters. a close parenthesis, an optional list of type declarations for the format parameters (c!ach tcm'linatcd by a semicolon), and an opening curly brace. which marks the beginning of the function body. The names in angle brackets just give names to the tokens that they match. Once these pieces have been matched, we assume that a function body comes next, foJiowcd by a closing curly brace. We want to find, within that function body, instances of calls to other functions. We don't care about any other code. So we use the second pattern, which is a daughter of the first. Once the first pattern matches, the second nne may also match. But, at the same ti01e, additional instances of the first pattern w.iU also be sought. H isn't necessary, for example. to find the matching right curly brace first. So the pattern matching is mhust, even in the face of mismatched delimiters in the code.

H .4 .2 Island Grammars WhiJc reg~Ja~ expressions are useful. they lack, for example. the ability to match properly nested dehnutcrs. And they don't describe a way to build anything like a parse tree of the code ~ragmcnts thctt do match. Context-free grammars do both of those things. But parsing w1th context-free grammars, as generally described (for example in Chapter 15), is not roh~!it. Parst:rs must find exact matches between input string~ and grammars. To solve the varaous problems that are generally faced in reverse engineering on the other hand. we require pa~crs ~hat are r~1bust in the face of aU of the issues that ~e mentioned above. So one aden as _1° us~ asland grammars, of the sort that we described in Section 11.9. As part of that dascussJOn. we ~ketched a simple island grammar modeled after one in 2001). Its purpose JUst lt'ke th t f · l ..1v•.councn .• . . . . ' . a o the regular expresstons we presented ab&.WC, ts to fmd funcllon mvocattons But b • t S. 0 f '' ecaUSe ·IS1~nn.1 or~mm~R ~r~ V~nan

916

Appendix H

Applications: Tools for Programming. Databases and Software Engineering

context-free grammars. it can find expressions with balanced delimiters and it can build parse trees of those expressions. Island grammars have proved useful Q in analyzing both old code (legacy software that may be written in obsolete languages and that has mutated. over the years. beyond recognition by its original writers} and much newer material. in particular World Wide Web pages. where straight text is typically interleaved with code in one or several programming and markup languages.

H.S Normal Forms for Data and for Querying Relational Databases In Section 11.8. we introduced the idea of a normal form and we mentioned two useful normal forms for context-fre.e grammars: Chomsky normal form and Greibach normal form. Throughout the rest of Part Ill we used those forms on several occasions to make it easy to. define algorithms that operate on grammars. We also introduced restricted normal form for PDAs and used it to simplify the design of algorithms that operate on PDAs. Later, in Section 28.4, we introduced two normal forms for Boolean formulas and we expioited them in our proof that SAT (the language of satisfiable Boolean formulas) is NP~mplete. But the idea of a normal form as a way to l!iimplify the design and implementation of an algorithm is useful in a much wider variety of contexts than those. For example, notmal forms are widely used in the design both of databases and their interfaces. In this section, we sketch one way in which a normal form can be used in the design of a graphical user interface for relational databases. Programmers can write database queries in programming languages such as SQL (which we discuss briefly in Q.l.l ). But nonprogrammers also use databases. They need an interface tool that is easy to use and they are typically able to get by with substantially less expressive power than languages like SOL offer. The Query by Example (or QBE) grid was proposed. in [Zioof 1975], as a tool for such users; it has since been implemented in commercial relational database systems Q. The QBE idea is simple. Imagine a grid such as the one shown in Figure H.5(a). The column headings correspond to fields in database tables or in other queries. A user creates a grid by dragging field names into the grid; each name creates a new column. So the grid we just considered could have been built by a user of a database that records a company's suppliers. along with each supplier's products and their prices. Once a grid with all the required fields has been created. the user can write a particular query by inserting values into the cells of the grid. So. for example. one could write the simple query shown in Figure H.5(b). The constraints in the non blank cells in a row of the grid are ANDed together to form a query. So this grid corresponds to the query .. Find all records where Category is fruit and Supplier is Aabco."' ' Disjunctive queries can be constructed by using more than one row. The constraints from multiple rows are ORed together to form a complete query. So the grid shown in figure H .S(c} corresponds to the query, "Find all records where Category is fruit or Category is vegetable." ANDs and ORs can be combined. The constraints from each row are first ANDed together. Then the rows are 0 Red together. So. for example. consider the query:

(Category-fruit AND Supplier-Aabco) OR (Category-vegetable AND Supplier2Bortrexco).

H.S Normal Forms for Data and for Querying Relational Databases

917

It can be written as the QBE grid shown in Figure H .S(d). But now consider the query:

(Category-fruit OR Category=vegetable) AND (Supplier-Aabco OR SuppliersBortrexco). If we try to write this query directly in a QBE grid, we realize that, because the QBE interpreter first ANDs all constraints within a row and then ORs together all the rows, every QBE query is effectively in disjunctive nonnal form. In other words, each query is a disjunction of subexpressions each of which is a conjunction of primitive constraints. But, to every logical expression, there corresponds an equivalent expression in disjunctive normal form. (We proved this claim for Boolean logic as Theorem B.3. The corresponding claim for first-order logic can be proved similarly.) So we can rewrite our query as:

(Category=fruit AND (Category-fruit AND (Category-vegetable (CategoryKvegetable

SupplieraAabco) OR Supplier=Bortrexco) OR AND Supplier=Aabco) OR AND Supplier•Bartrexco).

From this form, it can easily be written as the QBE grid shown in figure H .S(e). The QBE grid is a simple structure and it is easy for people to learn to use. It is more expressively powerful than its obvious structure predicts because disjunctive normal form is. just as its name suggests, a normal form. In other words, while not all logical expressions are in that form. all of them can be converted into an equivalent expression that is.

E

Supplier

Category

Price

(a) Category

5ruit

I

Supplier Aalxo

Price

(b) Category

Suppher

Pnce

fruit 'Vegetable

,..

(c) Category

t"('ruit t""vegc table

Suppher Aalxo Bortrexco

Pnce

(d)

,..

Category

'fruit "fruit rvc:actable rv-egctahle

-

Supplier Aabco Bortrexco A11bco Bortrexco (e)

Price

FIGURE H.S Representing

queries in QBE grids.

APPEND

X

Applications: Networks The theory that we have described in this book is useful in describing, at many levels, the structure of networks and the way they can be used. We'll introduce a few of them: •

the definition of network communication protocols.

• • •

monitoring and maintaining networks. exploiting network resources for problem solving-the Semantic Web. and network security (or the lack of it)-hnckcrs and virusl!s.

We'll discuss the first three of these issut!s in this chapter. We' ll talk ahout theremaining. one in the next chapter, in the larger context of computer security.

I. 1

Network Protocols The job of a network is to enable efficient and reliable cummunication between hosts. To mllke any kind of physical network suitable [or practical usc rc4uircs solving all of the following problems: • •

error control: Messages may be corrupted or lost or reordered as they are being sent. flow control: The receiving host may not be able to process messages as fast as the sending host can send them.

•

bandwidth limitation: The network itself has. a limiL un how fast data cnn be transmitted. If data are sent faster than they can be transmitted. they will be lost. So it is particularly important that the network never be forced to sit idle since idle time throws away capacity.

To solve these problems requires the definition of un~ or more communication pro10cols, i.e., shared conventions for the transmission of data in one direction and acknowledgements (that the data have heen received) in the other direction. Rather than attempting to describe all of the re4uircd functionality as a 5inglc protocol. it is now common practice to provide a protocol stack. Protocols at layer " of the stack make

918

·1.1

Tht lA)'ttS ond Thtir Responsibilitla

Network Protocols

919

Exumplt Protocols

Applic:ttion layer: . Supports :tpplications such as the World W1de Web and cm:til.

HTIP. SMTP. FfP

Transport layer:

TCP.UDP

Transmit~ complete

mc!lsages between upplication clients and servers.

Network layer: Rduys messages through a series of switchc.~ trouters) From source to dcstinntion.

lP

Link layer: Transmits messages from on!! .node to the next.

Ethernet

l 1 1wsicnllavcr~

Copper wire. coaxial cabll!, radio frequency

·Transmits hits ncross a physical nc,:twork.

FIGURE 1.1 The Internet protocol stack.

use of the functionality provided at layers n-1 and below. For example, the Internet protocol stack has rive layers, as shown in Figure 1.1. Many kinds of communication protocols can usefully be modeled as communicating. finite stale machines that never halt. Each process (machine) simply loops forever. sending and receiving messages. So. more precisely. the models that we are about to. build are Biichi automata, as described in Section 5.12, but without the distinction be~ tween occepting and nonaccepting states. In the rest of this section. we will show automata that correspond to the -explicit communication actions that are required by a few important network communication protocols. Note that, in aiJ of these models. the finite state ~tutomata will capture just the communication state of the corresponding processes. Additional state is required to encode the data that are being transmitted. The most hasic protocol we can imagine is iJlustrated in Figure 1.2. The horizontal axis corresponds to time. The sender simply sends data messages (indicated by the boxc.s lahch!d D) whenever it is ready to do so. The hope is that the receiver receives them (at some time after they are sent). Sender

Receiver

FIGURE U

A vt~ry simple protocol,

920

Appendix I

Applications: Networks

(S)D

Accl.!p.l d:tt:t

Deliver Jnta

(R)D

FIGURE 1-' s~ndcr and recdvcr nmdcls for the simple prutocul.

Finite state models of the sender and the receiver using thfs protocol are very simple. In constructing these models. we assume that there is a higher level process on one side that invokes the sender when it has data to send and a higher level process on the other side that will he notified hy the receiver whenever data havt.! arrived. Although we won't handle this part of the process in the models we art! about to build. we can note that the sender will maintain a FIFO (first-in. first-out) queue of messages that it has been told to send. Each time it is ready. it will remove and send the message at the head of that queue. In writing our finite state models. we will use (S) to correspond to the sending of a message. and (R) to correspond to the receiving of une. We'll useD to correspond to a message containing data. (In other protocols that we arc about to dcscribe. there will be other kinds of messages as well.) So we have the sender and n:cdvt!r models shown in Figure 1.3. l11c sender waits until it is given data to send. At thut point, it changes stale. sends a data message. and then returns to the waiting state. l11e receiver waits until data arrive. When that happens. it moves to the active state. in which it delivers lhe data. When it finishes. it returns to the waiting state, This simple protocol is efficient. ·nte ·s ender is free. to exploit -all the bandwidth of the network by continuously sending data. But this protocol fails h.l addrt!ss either of the first two concerns we mentioned above. •

If a message is··c orrupted or if i't simply !'ails to arrive.. there is no mechanism by which the sender can discover the problem and rdransmiL

•

H the sender is sending messages faster than the receiver can rctric.ve them, process them. and clear its buffers. then data will be lost. In this case. the sender needs to slow down. But. again, there is no mechanism for telling the sender that there. is a problem.

1.1.1 Stop-and-Wait A family of protocols called Automatic Repeat reQuest (or ARQ) pmtocols huve been designed to solve the two problems that we. have just dc~o;crihcd. In an ARQ protocol the receiver communicates hack to lhe sender and the sender exploits that communi~ cation to help it determine wh\!n a failure has occurred and a message should be retransmittetl. A simple suhramily of A RO protocols is called Stop-and- Wait.

1.1

Network Protocols

921

Sender

Receiver

FIGURE 1.4 The ARQ protocol.

A very basic Stop-and-Wait protocol is illustrated in Figure 1.4. Now there are two kinds of messages-data messages (labeled D) and acknowledgement messages ('labeled ACK). The sender waits for an acknowledgement that one message has been received before sending the next one. With just one additional state in each model, we can describe the behavior of senders and receivers using this new protocol. as shown in Figure 1.5. The Stop-and-Wait protocol that we have just described solves the flow control problem that existed for the simpler case. The sender wilJ never send a second message until it knows that the receiver has successfully delivered the data from the first one. And it solves one error control problem: If a data message is lost, the sender wi11 retransmit it after it times out waiting for an ACK.lf a data message is corrupted and the receiver can tell that (for example by using checksum bits), the receiver can simply fail to send an ACK. the sender will time out, and the message will be resent. There are also variants of the Stop-and-Wait protocol in which explicit negative ACK messages (NACKs) are sent when a message arrives corrupted.

Sendtr:

Rtctivtr:

Accept data

(R)D

Active (S)D

Timeout NecdACK

FIGURE 1.5 Sender and receiver models for the Stop-and-Wait protocol.

922

Appendix I

Applications: Network~

Receiver FIGURE 1.6 What happens when a delayed message t:vcnlually shnws up and its ACK is confused with another.

But other error control problems remain:

•

•

If a data message arrives successfully hut its corresponding ACK message gets lost, the sender will time out and then resend the data messag~:. But then the receiver will r~ceive two copies of the same message. ll has no way to know that it has just gotten a second copy of the first message rather than u first copy of a next message. Suppose that the sequence of events shown in Figure 1.6 occurs. l11e first data message (labeled 0 11) is delayed until a her the sender times out waiting for it to bt! acknowledged. So that first message will be resent. h arrives and an acknowledgement is sent and received. So the sender sends a second data message (labeled 0 1). It gets lost. But. meanwhile. the original copy of the first message arrives and is acknowledged. The subscripts in the figure are just to t!nable us to envision the events. There are no subscripts attached to any of the mt!ssagcs. So th~ sender. when it gets a second ACK, thinks that its second message was r~ceivcd. It goes on to send the third one.

1.1.2 Alternating Bit Protocol Nut ice that if subscripts. of the sort we used in the last example. wen! actually present in data messages and in ACKs, we could solve hoth of the Wait-and-Sec protocol problems that we just described. The next protocol that we·ll describe. the Alternating Bit protocol Q, doesn't add arbitrary subscripts. It docs. h()WCvcr. add a single control bit to each data message and to each ACK. The bit will alternate values with each transmission. By ctmvcntion. the receiver will flip the bit hdorc sending an ACK, so the message that acknowledges the rccdpl of D 0 will be ACK 1 (indicating that the sender's next data message should bt:: D1) and vice versa. Figure 1.7 (a) illustrates the straightforward case of the Alternating Bit protocol. A troublesome case. like the one we showed in Figure 1.6. will he handled by the Alternating Bit protocol as shown in Figure 1.7 (h). The second ACK 1 is simply discarded as redundant since the sender already knows that was received. Thl.! sender will timeout waiting to get an ACKu (acknowledging receipt <1f Dt). So it will (correctly) resend 0 1• This same mechanism makes it possible for the receiver to tell when duplicate messages are rec~ivcd. Whenever this happens, thl.! second cupy will simply be discarded. To describe the behavior of senders and receivers thnt exploit this new protocol requires two copies of each of the states that were needed for the Stop-and-Wait protocol. One copy corresponds to handling data whose con1rnl hit is 0; the other to handling

D,,

1.1

Network Protocols

923

Sender

(a)

Sender

Receiver

FIGURI-: 1.7 The Alternating Bit protocol.

data whose control bit is 1. The new models are shown in Figure 1.8. Both the sender's and receiver's start state is waitingo. The Alternating Bit protocol does not solve all error control problems. For example it dOt:s not address the problem of messages that are received but corrupted. It also won't work if a message is delayed long enough that its parity matches that of a more recently transmitted message. But its most serious problem, as a practical protocol. is throughput. As with any Wait-and-See protocol. the network must sit idle while the sender waits for the receiver to process a message and for an ACK to be sent and received.

Sliding Window Protocol wasted handwidth problem can be solved by a more sophisticated ARQ technique. th~ Sliding WindtJw protocol, that assigns sequence numbers (rather than alternating hits) to the messages that a sender sends to a receiver. As before, data messages are initittlly entered into a FIFO queue that is maintained bv the sender. They are assigned sequence numbers as they enter the queue and they "wm be transmitted in the order in which they enter the queue. Any specific use of the Sliding Window protocol begins by choosing a window size ""·The window is then placed over the first w messages. We'll say that those messages are in lh~ send wi?~oK'. The send~r may send (without waiting for acknowledgements) any message that1s m the send wmdow. The send window can. in turn. be shifted to the right as ACK messages are received. 1lle Sli~in~ Window protocol is used for sending messages on the Internet. It is illustrated m Flgur~ 1.9. Each box corresponds to a data message to be sent from the. sender. ttl the recc&ve~. Messages that have been sent are shown in the sender's queue with d1agonul hatch hnes. Messages that have been received are shown with hatched

11H!

924

Applications: Networks

Appendix I

rereplyingu

timed out41

(S)ACKo

Timeoul

Accept

Accept

Deliver

data

data

duta

(S)Do

Deliver data

(S)ACK 1

Timeout

timed out1

(R)00

Rercplying1

FIGURE 1.8 Sender and receiver models for the Alternating Bit protocol. Tht seredtr's qutut:

I•

I• Acknowledged

•I•

•I

Send winduw

~ s ls+tls+l Unacknowledged~

t

S\:nding

I•

I•

Delivered

Receive window

•I 1I "" I Next expected

FIGURE 1.9 The Sliding Window protocol.

Received (uut uf order)

•I

1.1

Network Protocols

925

lines in the receiver's queue. The sender begins transmitting by sending, in order, the messages in the send window (i.e., the first w messages). It will not wait for an acknowledgement of message 11 before sending message n + 1. It will, however, expect to be told of the arrival of all data messages it sends and it will resend any message on which it times out before it receives an ACK that acknowledges that message. In any cumulative acknowledgement protocol, of which the Sliding Window protocol is one example, an ACK, message acknowledges receipt of all data messages numbered upton - 1. So. as shown in the diagram. the receiver may have received some messages that have not been acknowledged; they won't be until all the messages before them in the sequence have been successfully received. This means that. if the sender receives an ACK, message, then it knows that all messages numbered up to n - 1 have been received. At that point, the send window can be slid to the right so that the lowest numbered message it contains is n. Each time the window slides, the sender may resume sending messages. It need only stop and wait when it has sent all messages in the current send window. For a more formal treatment of cumulative acknowledgement protocols. including Sliding Window, see [Gouda 1998}. The Sliding Window protocol cannot usefully be modeled as a finite state machine because of the need to store the message sequence numbers. (Of course, if we assume a maximum word size for storing those numbers, we could build a corresponding FSM, but it would be too complex to be useful as an analysis tool.) As we'll see in the next section though, we can continue to use finite state machines .as tools for describing higher-level protocols. including ones that are used on the Internet and that exploit the Sliding Window protocol. We 'II simply take advantage of the fundamental structure of a protocol stack and treat the action of correctly sending a message as an atomic event without worrying about how it happens.

... 14 .

TCP In the Internet protocol stack, the transport layer sits immediately below the application layer. So the transport layer protocol is invoked by application protocols such as HTTP. SMTP, and FTP. A practical transport layer protocol must be efficient and it must address some issues that we have not considered up until this point. For example, it must enable data messages to be sent in both directions between two hosts. The transport layer protocol used by the Internet is the Transmission Control Protocol (or ~c_P_). A TCP connec~ion is established by a three-step handshake procedure: One host tmtlates the connectton, the other acknowledges it, and the originator then confirms the acknowledgement. Once a connection is open, data can be transmitted between the two hosts until it is closed. Internet standards are defined by a set of documents called RFCs. The functional specification of TCP ~an be described as a simple finite state transducer, shown in figure 1.10. ex~ctly as.•t appears in [RFC 793] Q as Figure 6. The model ts descnbed, again in [RFC 793). as follows: A connection progresses through a series of states during its lifetime. The states are: LISTEN. SYN-SENT, SYN-RECEIVED. ESTABLISHED FIN-WAIT-1 FIN-WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT. ~nd th~

926

,Appendix 1

Applications: Networks

---------\

·~--------+ I

I

CLOSED

+---------+<---------\ ·~

1

passive OPEN j ------------ I create TCB I

\

CLOSE

rev SYN

snd SYN, ACK

+---.-.-.---<-+

~~~o

rev ACK of SYN

SEND

-------

'' \

-------------x

I

SND SYN

I V

+---------+·

l

v

V

+---------+ I FINWAIT-2 I +---------+

rev FIN ~ ------

snd AC~

I

I

I

V

-tr-------·--+ SYN SENT

I. I :

'I

'+-------...--+

-----------

I

snd ACK

ESTAB

CLOSE ------.snd FIN

I

I I

t-cv FIN

I 'I \

snd ACK

I

rev FIN

\

CLOSE

v

+---------+ I

CLOSING I

+---------+ rev ACK of FIN I -------------- I

TIME WAIT

-----"""- I snd FI~ V

+---------+ I LAST-ACK I

·+ ------- ---+ rev ACK of FIN 1 Timeout =2MSL -----------.,._-- 1

v -----------+---------+ delete TCB

""-------------~--------->I

CLOSE WAIT

+---------+

l

I

snd ACK

"

+--------'+-

--~--------~------->1

1-~----------------

I rev ACK of FIN I -------------V X

I I

+---------+

1<-----------------

<+---------+

rev SYN, ACK

l

I

\

·------------------.>1. I

I

\

+-----'----+

CLOSE

I ------I snd FIN

I

I

I

\

\

---------- I delete TCB I

rev SYN

1

FIN WAIT -1

'\ \

:<--------------------~-nd-~~K--------~-------------: 1-----------------·- ------------·------1

1

I

I

I

\

1

+.--j-"------+ I

1

I

I

I<---------------

snd SYN

\

CLOSE

+---------+

----------e'-

create TC8

\

+-.--------.+ LISTEN

\

\

- ·--------delete TCB

v

I

\

\

active OPEN -----------

)(. v +---------+

t------------------>1

+----------+

I +----------+ CLOSED

FIGURE 1.10 A finite state. transducer model ofTCP.

fictional state CLOSED. CLOSED is fictional because It represents the state when there is no TCB (Transmision Control Block). and therefore. no connection. Briefly the meanings of the states are: • LISTEN - represents waiting for a connection request from any remote TCP and port. • SYN-SENT- represents waiting for a matching connection request after having sent a connection request.

1.2

Modeling Networks as Graphs

927

•

SYN-RECEIVED - represents waiting for a confmning connection request acknowledgment after having both received and sent a connection request.

•

ESTABLISHED - represents an open connection, data received can be delivered to the user. The normal state for the data transfer phase of the connection.

•

FIN-WAIT-1 -represents waiting for a connection termination request from the remote TCP. or an acknowledgment of the connection termination request previously sent.

•

FIN-WAIT-2 - represents waiting for a connection termination request from the remote TCP.

•

CLOSE-WAIT - represents waiting for a connection termination request from the local user..

•

CLOSING- represents waiting for a connection termination request acknowledgment from the remote TCP.

•

LAST-ACK - represents waiting for an acknowledgment of the connection termination request previously sent to the remote TCP (which includes an acknowledgment of its connection termination request).

•

TIME-WAIT- represents waiting for enough time to pass to be sure theremote TCP received the acknowledgment of its connection termination request

•

CLOSED - represents no connection state at all.

A TCP connection progresses from one state to another in response to events. The events are the user calls, OPEN, SEND. RECEIVE, CLOSE, ABORT, and STATUS; the incoming segments, particularly those containing the SYN, ACK, RST and FIN flags: and timeouts. . . .an t h'as d'tagram has a Ia be I of the form . Each transat&on . , where lS



the event that causes the transition to occur and is the action that is executed when the transition is taken. The diagram ignores error conditions and other actions that are not directly connected to the state changes.

Modeling Networks as Graphs It is natural to .model a network as a graph in which the processors correspond to vertices and the hnks correspond to edges. As soon as we do that, it becomes clear that many of the problems that we need to solve when we build and analyze networks correspond to the graph problems that we discussed in Part V. We'll mention a few examples here. Consider the problem of designing a physical network that connects a set of points. We want to find the cheapest way to build the network. We can show that there is an efficient algorit~m for solving this problem. Let G be a graph whose vertices correspond to the pomts and whose edges correspond to the costs of laying cable (or wires,

928

Appendix I

Applications: Networks

or whatever) between pairs of points. Recall that a spanning tree T of G is a subset of the edges of G such that:

• •

T contains no cycles. and every vertex in G is connected to every other vertex using just the edgt:s in T.

If G is a weighted graph. then the cost of a spanning tree is the sum of the costs (weights) of its edges. Define a trt:e T to be a minimum spanning tree of G iff it is a spanning tree and there is no other spanning tree whose cost is lower than that of T.ln Section 28.1.6, we described the minimum spanning tree problem as tht! language MST = {< G. cost> : G is an undirected graph with a positive cost attached to each of its edges and there exists a minimum spanning tree of G with total cost less than cost}. We showed that MST is in P. We described one efficient technique. Kruskal's algorithm, for finding minimum spanning trees. The cheapest way to build n network that connects the points in G is to lay cable along a minimum spanning tree of G. So the network design problem can be reduced to the minimum spanning tree problem. Since we have an efficient way to solve MST, we have an efficient way to design our network. Nt!xt we consider the problem of finding the optimal route for a message through a network. Again we'll describe the network as a graph G. Let the vertices of G correspond to network nodes and let the edges correspond to network linKs. We can reduce the message routing problem to the problem of finding the shortest path, from source to deMination, through G. In Section 2K7.4. we described the language:

•

SHORTEST-PATH= { ; G is an unwcighted. undirected graph. u, and v are vertices in G. k ~ 0, and there exists a path from 11 to lJ whose length is at most/<}.

We showed that SHORTEST-PATH is in P. Unfortunately, SHORTEST-PATH is not exactly what we need to solve the message routing problem because it is stated in terms of unweighted. rather than weighted. graphs. We need to usc weights to describe the costs of the individual network links~ But. as we mentioned in Section 28.7.4, there also exist efficient algorithms for finding paths through weighted graphs. Next we consider the problem of checking a network to verify that all links are working properly. The shortest way to traverse all the links in a network is via an Eulerian circuit. Recall that an Eulerian circuit through a graph G is a path that starts at some vertex s. ends back ins, and traverses each edge in G exactly once. In Section 28.1.5 we described the problem of finding an Eulerian circuit as the language: •

EUlERIAN-CIRCUIT= { : G is an undirected graph and G contains an Eulerian circuit}.

We showed that EULERIAN-CIRCUIT is in P. St'l there exists an efficient algorithm for solving the link checking prohlcm. So far. our theory has yielded positive results for the network rrohlcms we have wished to solve. Unfortunately. that is not always so, as we 'll see in our final example. Consider the problem of finding a minimal set of network nodes at which we can place monitors so that we can observe the status of every network link. Again, we·u

l.l Exploiting Knowledge: The Semantic Web

929

describe the network as a graph G. whose vertices correspond to network nodes and whose edges correspond to network links. Recall that a vertex cover C of a grap.h G with vertices t.! and edges E is a subset of v with the property that every edge m E touches ~Jt least one of the vertices in C. We can reduce the problem of finding a minimal se.t of monitor sites to the problem of finding a smallest vertex cover of G. In Section 2K6.5. we described the vertex cover problem as the language:

•

VERTEX-COVER= { : G is an undirected graph and there exists avertex cover of G that contains at most k vertices} •.

Unfortunately, we showed that VERTEX-COVER is NP-complete.So it is unlikely that there exists.an efficient algorithm for solving it •.

1.3

Exploiting Knowledge: The Semantic Web Networks enable two or more computers. and their userS. to communicate. The World Wide Web enables millions (possibly billions) of computers. and their users. to communicate. Hard problems get solved by building software layers on top of the fundamental conununicatinn protocols that we have already described. Hypertext structure turns a .~el of documents into a web of documents that people can explore. HTML, which we describe in 0.1.2. is a standard hypertext markup language that makes the documents of the world accessible to the people of the world. But what about making information available to the programs of the world'? It is no longer possihle for people to manage the amount of information that is available on the. World Wide Web. We need programs to help. It is common to describe such programs .as ime/ligent agellls. We (people) can t!xploit the contents or the Web because we can read text, interpret tables. recognize images. and watch videos. In other words, we assign meaning to Web objects; At some point, it may be possible to build automated agents that can read the current contents or the. Wc.b in much the same way people do. In Appendices Land M we descrihe just H few or the research questions that mu~t be solved to make that happen. In the meantime, if we want automated agents to work for us on the Web. we must annotate the Web with meanings that arc stated in machine-usable fonns. And then we must provide a set of inference rules and proccdure(s) for exploiting those meanings to solve problems. As an example of what we might like un agent to be able to do. consider the Web pages for two local quilt guilds, shown in Figure I.ll. By reading these pages.. people wo uld he able to answer questicms !Such as:.

•

Find quilt guilds in Texas. The fact that Hometown is in Texas can be gleaned from the· map and. since Texas is the Lone Star State, Nimble Fingers is in Texas. Find quill guild~ that meet during the day.

• •

Find u list uf email addresses for presidents of quilt guilds in the American southwest_ Find quilt shows this fall in Texas.

•

J
930

Appendix I

Applications: Networks

Hmn~tnwn

Nimble fln~n Quilt GuUd

Quilt Guild

Gtlab:

Our t;OIIh •r~ 111 ·~

W.- ruc.-lal \llun• nn 1ho: firs& Mnnnlh al lh!: u•n~ Riv~r R~.-m:~lion (\ml~r. Clic:k lu'o:l urumap. Slou,. ..... TI'UI

~.---.

•

-

•

Our sn:a~ 10

Sepl.ll-10

Cl!klmur qulllin* in &he lo~~~: S1nr Se.tc.

C.t'nl~r

Nt_.

AI 1111: DowftiMI'II Coa\'ftllloll

~IOIICII'c MMIIa~

Tlll:iday. S.:pl. 26 al 7.30 Ri\"C:I"1iok Communi!)' C't'ntu ()"'"n; Pr~~~nl ..,

FIGURt: 1.11 Twn4uilt guilll Weh page~

··quill guild.'' ··quilt show," and "Texas... To reason correctly about these questions requires having effective access to the contents of the rd~vant Web pages. It also requires having hackground knowledge about what the terms on the pages .. mean." For example. it's necessary to know that 10 a.m. is ''during the day" but 7:30p.m. is not, that Texas is in the American Southwest. and that Septcmher is a fall month. The Semantic Web o !Berncrs-Lec. Hendler and Lassila 200l} is a vision for the transrormution of the World Wide Web into a knowledge store (or knowledge base) that supports the construction of agents that can answer questions like the ones we just considered. Tu make the Semantic Web u reality requires the solution uf a host of technical probh:ms. We focus here on two: •

common description (markup) languages. If knowledge is to be! used. its structure nnd its meaning must be described. If knowledge is to he shared, one or more :ttamlmd description langungcs need to be defined.

•

an inference engine. If knowledge is to he used. there must exist some technique(s) for reasoning with it so thut facts can be comhincd to soh•c a user's problem.

Issues that we have discussed in this book play important roles in the design of solutions to both of these problems. In particular: •

•

to solvc the common description languages problem requires that we: •

design one or more languages that ure expressively adequate for the job yet re-· tuin the dccidability and tractahility properties that we need.

•

exploit formal techniques for describing th~ languages so that users around the world cun share them.

l
In the rt!sl nf this section we"ll sketch the definition of a layered set of languages that can be us~d tu ddinc Web objects and assign meaning tn them. Each layer will he able

1.3

Exploiting Knowledge: The Semantic Web

931

to exploit the capabiliti~s of the layers beneath it. One way to think of the languages that we arc about to describe is. that, while most current Web languages (such as HTML) are designed to support a common way of displaying content on the Web. the new language~ must be designed to suppon a common way of alllomated reasoning with that content.

1.3.1 The Language of Universal Resource Identifiers (URis) Any language for describing the Web must start with a sublanguage for describing the Web's fundamental units, as well as other. non~Web objects that relate to Web objects in useful ways. Call these resources. A Web page can be uniquely identified by its Web address, stated as a URL (universal resource locator). A URL contains an access method and an actual path that will find an object on the Web. But what about things that aren't Web pages? For example, we might want to be able to say that the creator of a particulur Web page is a person (not a web location) whose name is Chris. who lives in New York. and who works for Jingle Co. To do this, we need a way to refer to Chris, New York. and the Jingle Co. To mnke that possible, we'll define a new language. the language of universal re... source identifiers. A univ~nal resourc~ id~ntifl~r (or VRJ) specifics a resource. Some VRls actually describe how to find their associated resource on the Web. For example, every \J RL is also a URL Other kinds of URis simply provide a hhook" that enables ·statements to be made about the resource. whether we know how to find it or not. A URI (as its name suggests), identifies an object. That object may be a file or some other structure that contains smaller units. In that case, we may want to be able to refer to those smaller units individually. We use the fragment notation to do that. A fragment is the symbol #, followed by a fragment name. So, for example, if http://www. mystuff. wow/ products. html contains descriptions of all my prod~ ucts. then http: I /www. my stuff. wow/ products. htmllwi dget might point directly to the description of my widgets. The syntax for the language of URis Q can be described with a BNF-style. context-fn:e grammar. (ln fact, this language is also regular and so it could be defined with a regular grammar or a regular expression.) We'll use here the convention that all special characters are metacharacters rather than terminal symbols. So, to use one as a terminal symbol, it will be quoted. We '11 also use the common convention that any sequence that is enclosed by a [ 1pair is optional. Recall that I separates alternatives. and Klcene star means zero or more occurrences are allowed. A complete grammar for URis on the Web Q is too long to present, but a simplified excerpt is shown us follows:

-+ '(••#"1 - <.Scheme> .. :.. ("?''}.

-+ ftp 1 http I https I mai 1 to I news 1.. .. - "II" l

...1/"l < Pmlr-Absolute> I

932

Appendix I

Applications: Networks

! < Pmh-Roorle."is> < Am/writ y > -+ [< User-info> ..@''j < llo.'it> [..:" < Port > ) < Ptllh-Ainwlme>-+ [("'/" < ScKmem> )*> ) -+

··r

EXAMPLE 1.1 Parsing URis Using the grammar excerpt that we just presented. we can produce the following parse tree:

http

//

www.automatatheory .org/Partii.htmllregular

1.3.2 RDF Now that we have a way t.o name Web objl!cts. we need a markup language that can be used to describe their properties so that they can be cxploih:d. We'll call such descriptions metadata, i.e.. information about other data sources (e.g.. Web pages). There is today no single standard mctadata language c for the Web. What we'll do here is to describe one interconnected family of such languages. All of them are evolving standards maintained by the World Wide Web Consortium (W3C') c. The bottom layer of this language family is RDF c (Resource Description Framework). Each RDF statement is a triple that asserts a value for some property of some Web resource. Re member that a Web resource is anything thnt can he named with a URI, so it could be a Wch page or something (like a person. a city. a business or a product) that is external to the Web but to which we can give u name. Taken tugether. a set of RDF statements can be used to describe a set of relevant prupcrth!s of some useful collection of Wt!h resources. Typically such an RDF description will describe how some collection or Wc:b pages relate to each other and how they relate to some collection of external objects. Like any language. RDF has a syntax and a vocahulttry. Both arc substantially more Ot:xible than in any of the languages that we've considered so far.

1.3

•

Exploiting Knowledge: The Semantic Web

933

The syntax: We'll begin by describing RDF syntax abstractly in terms of triplc:s. The meun.ing of an RDF c~pression is defined by a semantic interpretation function that applies to triples. But we also need a ..concrete" syntaxj.e., a form for writing stri'ngs that describe those triples. At least two such forms are in common use. Ea~h of them comes with a compiler that maps strings to triples. The Web commumty .docsn't have to ngree on a concrete syntax, as long as it agrees on the abstract one. Till! vocabulary: It is completely unrealistic to assume that all users of the Web will want to agree on a single vocabulary to be used in describing Web resources and thl!ir properties. Anyone who wants to do so can define an RDF vocabulary and place it in a resource somewhere. Then anyone who has access to that resource can usc one of its terms by referring to the resource and then to some specific term.

•

Every RDF .statement is a triple. So it has three parts: a subjc•ct (the thing about which a statement is being made),, a prt•dicme (a property or attribute that the. subject possesses), and an object (the value of the predicate for the subject).

• • •

The meaning of each triple is that subject has a property named predicate and the value. of that prnperty is o/Jject. The meaning of an RDF expression is the assertion of the conjunction of all of the triples that it describes. So RDF is a logical language with limited expressive power. In particular, •

all predicates are binary. (In other words. each predicate relates exactly two things:· its subject and its object). This is not a real limitation. though, since other predicates can be converted into sets of binary ones.

•

the only logical connective is AND (since an RDF expression is just a list of triples, all of which are asserted to be true). Neither disjunction nor negation is allowed.

So, ignoring all issues of syntax. and even of how entities get named, we might use RDF to specify triples like: •

(mywebpagt~. (.'Tl'lltt'll-by. nw).

• •

(me.live.'l-in.dryo[Au.'itin) .. ( 1-/om~town Quilt Guild. organi:zcrlicm-focus. quilting).

At its core. RDF is thus a very simple language. One uses it to write triples. There is, however, one way in Which its definition is more complex than is the definition of many other kinds o.r formal ..language.s. RDF is u language {or describing properties of objects un the Web. Sn t_here IS no. central con_trol of what objects exist or what they are called. RDF mu~t provJde a nummg convention that handles the distributed and dynamic na'turc of t~mgs on th~ Web. Its solution to this problem is to use URis as names. Specifically, subJects. prcdtcates, and ohjects are described as follows. •

Subjects: RI?F stat,cmcnts ~ue "about'' resources. So the subject of every triple must be uliRl ,.wt~h one exceptton:There may be "blank" nodes that exist only as placeholders wtthm RD: exprc_ssions. lbese nodes are "blank" in the sense that they have no name outsu.le the Immediate expression in which they occur. Blank nodes

934

Appendix I

Applications: NetworkS'

may be the subject of an RDF statement. So. for example. we may want to say that the Web page 11' was created by someone who lives in Maint! but we don't know who that person is. We can create a blank node_: 1 and then say: ( 1tl, created-by.._: 1)

(_: 1 , /ives-in. sltJtenfMtJint!)

Predicates: It is tempting to allow (as suggested by the examples that we have presented so far) simple predicate names like lives-ill , crt•med-l>y. wnrks-for, etc. But doing so would pose two problems:

•

• Where would we define lhe meanings of those strings'! • The (world-wide) community of World Wide Web users will never be able to agree on a set of predicate names. It must he possible for smaller communities of users (including communities of one) to dcl'ine the predicates they need without having to worry ahout what everyone else has done. RDF solves both of these problems by requiring that l!v~ry predica te be a URI. Anyone who can define a URJ can define an RDF predicate. We'll say more shortly about how this system works. •

Objects: An object may be a named resource (specified by a URI). a blank node. or an element of a primitive data type such as string or integer. We'll notice, by the way, that strings get used as objects much less frcqut:ntly than one might think. For example. we won't want to say that Chris lives in " Maine:· Chris clearly doesn't live in a string. We'll want to say that Chris lives in a statt! whose name is .. Maine....

Since URis play such an important rule in RDF.Ict '~ say a bit mure about them before we show a few concrete RDF examples. Entities and predicates in RDF arc numcd by URis. So they can he defined (using the RDF Schema language that we'll describe later) by anyone who can create a Web resource. Then they can be used by anyone who has access to that resource. A single Web file typically contains the definitions of a cnllection of related things, each of which can he uniquely identified within that file by a fragment name. So, suppose that we have defined a set of terms to be used in describing craft orgunizations. Then individual predicates might have name-s like:

http: I /WNW. myi sp. net/ regusers/ mytown/ me/ craft-stuff#organization-focus http: I /WNW. myi sp. net/ regusers/ mytown/ me/ craft-stuff#meeting-place Next. we observe that URis (like the ones we just \\ITOlc) teml to be long. But many of the ones a particular RDF expression uses may share a common prefix like:

http ://www. myisp. net/ regusers/ mytown/ me/ craft-stuff# Users don't want to have to write that whole string every time they write a subject predicate, or object in an RDF expression. The solution to this prohlt!m is the use

of

1.3

Exploiting Knowledge: The Semantic Web

935

namespaces (as they are defined in the markup lang~age XML):To defi~e a na~espace~ we simply associate· some (generally short. mnemomc) name Wlth a stnng that ts a pn::fix of some set of URis that we want to use. Using XML syntax (which is one common way in which RDF is writtc.n), we can define a namespace by writing, for example: xmlns:crafts="http:/ /www. myisp. net/ regusersjmytown/me/ craft-stuff#" This XML namcspace (ns) definition maps the string crafts to the long URI shown to the right of the e<.&ual sign. RDF then allows the use of what it calls qualified names or QNAMES whenever a URI is required. A QNAME bas the form:.

":" RDF will form a full URI from a QNAME by appending to the string to which maps. So. having defined the namespace crafts as we just did, the QNAME crafts: meeting-place is equivalent to: http: I /www. myi sp. net/ regusers/ mytown/ me/ craft-stuff#meeting-place n1e definition of the RDF language says nothing about what vocabularies can be used in RDF expressions. Whenever a URI is required. any syntactically valid URI is. acceptable. We'll have more to say later about how RDF vocabularies (of predicates and things to which predicates apply) can be defined (for example, how we could have defined the crart organization vocabulary). For now, though. we'll just mention that there are some public vocabulary definitions ~ that are commonly used in writing RDF expressions. Each of them has a URI and each of those has a standard namespace definition. As a shorthand, we'll use those definitions. but remember that in real RDF code you must cxplidtly define each namespace first. The namespaces we'll use are: •

rdf- contains terms that have sp~cial meaning in RDF.

•

rdfs- contains terms that have special meaning in RDFS (RDF Schema). a language. that we·n describe below. RDFS is a language for describing RDF vocabularies..

•

owl -contains terms that have special meaning in the inference language OWL. that we'll <,lcscribe below.

•

de- contains a vocabulary called the Dublin Core. The Dublin Core vocabulary has been designed for attaching common kinds of metadata to document-like network objects. So it contains. for example, the predicate$ Title~ Creator, Subject, Pub 1i sher, Language, and so forth.

•

foaf- co~tains a vocabulary caJled Friend of a Friend (thus foaf). The foaf vocabular~· cont.mns terms th~t are useful for describing people. So it contains. for example, the prcd1cates name. t1 t 1e, homepage, interest, web1og. and schoo1Homepage.

So. for example, if we wrh~ de: Pub 1 i sher as a predicate. it is simply a shorthand fo~ :1 ~RI that. happc.m~ to pomt to a place on a Web page defined by the Dublin Core lmtmttve: As ll turns out, on that Web page is some machinc·rcadable information (written _m the bm~Ui:\ge R~F Schema. to be described below) that can help an agent actually mlcrpret. 10 a uselul way, a triple that uses this predicate.

936

Appendil'. '

Application$: Networks

We've said lhat we can think of the abstract syntax of RDF as a set of triples. But what about its ··concrete syntax''? In other words. what sequence of symbols must users write if tht!y want to define an RDF expression'? The answer is that users can exploit anv concrete syntax for which a translator into ahstract syntax exists. One approach is to ·use the simple triple language Notation3 (also called N3) ~.We illustrate that next.

EXAMPLE 1.2 Writing RDF in N3

A very natural way to write RDF is to u&e the triple language N3. In this example, we will use seven namespaces. In N3, namespaces are defined with the @prefix command. So we ·n write those first. then we'II list triples.,

@prefix @p :-efi K @prefix @prefi I( @prefix

rdf: , de: :t:.lht•locarinn·of the Dublin Cart- clefinitimrs>. foaf: -.­ food db: . mystuff: •. @prefix my co:
nry company>. @prefix p1aces; .

myco:bigreport myco:person521 myco:person521 myco:person521 fooddb:chocolate fooddb:chocolate myco:person521

dc:Creator rdf:type foaf:firstName mystuff:favoritefood rdf:type fooddb:caloriesperounce places:birthp1ace

myco: person521. foaf:Person. "Chris". fooddb:chocolate. fooddb:food 150. [places:cityplaces: Boston; places:stateplaces: MA],

What we've said here is that a particular report called big report in myco was create-d by someone whose identifier in myco is person521. That someone is a person in .the sense defined in foaf. (To say this. we used the RDF-defined predicate type which relates a thing to a class of which it is a member.) PersonSZl's first name is th~ string "Chris". Chris's favorite food (in the sense. in which I defined it in mystuff) is something that is defined in fooddb and called choco1ate there. Chocolate's calories per ounce (in the sense defined in fooddb) is the number 150. Finally, person521 was born in an unnamed place, indicated by an unnamed structure in brackets, whose city is Boston and whose state is MA (both as detined in places).

1.3 Exploiting Knowledge: The Semantic Web

937

A common alternative to N3 is RDF/XML !iil, which exploits the markup language XML that we describe in 0.1.2. 'This form is attractive to many users because they are already familiar with XML and XML parsers. are. readily available.. Whatever concrete syntax is used to descnbe 1t, an RDF expresston corresponds to a list of triples. So another natural way to think of it is as a labeled, directed graph. The vertices of the graph correspond to entities (on the Web or in the world). The edges name properties that those entities can possess.

EXAMPLE 1.3 Representing RDF Triples as a Graph Here's the graph that corresponds to the triples that were defmed in Example 1.2:

dc:creator

~

foaf:firstNaM

It is perhaps more obvious in the graph representation than it was in the triples form that whenever an RDF expression exploits a term that has been defined in some other namespace, it connects with what is already known in that other namespace. So. for instance, we wrote the triple that asserted that chocolate has 150 calories per ounce. If that fact were already asserted in fooddb, we could have used it without explicitly mentioning it. Application programs can query RDF descriptions by writing their own code or by using any of a number of q~ery languages. many of which are very similar to the ones that are used to query relational databases. The results that are returned in response to such .que~i~~ will be a set of.Web resources that match the query. So, for example, we'd like • Chns to be returned m response to a query that asked the first name of the creator of bi 9 report. We ~d also like to be able to reason in less trivial ways with the facts

938

Appendix I

Applications: Networks

that have been provided. For example. l might want to find other people who live near me and who like high-calorie food. In the next section we'll talk about the issues that arise when we attempt to define reasoning engines that work with RDF. RDF expressions describe properties of various kinds of resources. But where do those RDF expressions reside on the Web? There are at least two possible answers. They may sit in separate files with distinct URis that name thcm. l110sc files may be private or they may be publicly available and searchable. Another possibility. though, is that they may be embedded inside the objects that they describe. It is becoming in· creasingly common for Web languages to provide a way to incorporate metadata.such as RDF descriptions. inside objects that are written in those languugcs. Lefs now return to the question of RDF's vocabulary. As we said in the introduc· tion to our discussion of RDF. a key featu re of the language is that there is not one universal. fixed vocabulary. There are some standard vocabularies but anyone is free to design a new one (such as fooddb or the one that describes my company). Then any number of vocabularies may be combined to form a single RDF description. To use a vocabulary. all that is required is that you know the URI for its definition. So the next question is. "What docs an RDF V()cahulury definition look like and wh1:1t does it contain?"The answer is that it contains descriptions of classes of ohjects and the properties that those objects may have.. In the next four sections, we will describe a family of languages in which such descriptions may he written. An important point about all of these languages is that. when we use them. we don't just write a list of terms. We take at least a first shot at defining the meanings of terms by relating them to other terms.

1.3.3 Defining an Ontology and an Inference Engine What RDF expressions do is to describe objects and their properties (which we called predicates when we were describing triples). Each RDF property corresponds to a relation that has a domain and a range. The domain is the set of things to which the property may apply. The range is the set of vulucs that the property may have. So. for example. we might want to define the property org - name. whose d<)main is the set of orgnni7.ations and whose range is the. set of strings. To define a property. we need to define its domain and its range. In order to be ahle to do that, we must also be able to define classes, such as organizations. And we need to make sure that these definitions are constructed in a way that makes it possible to reason with them. For example, we must be able to answer the question. "Docs nhjcct A. with properties p 1 and P'1 satisfy the description provided by query Q. which specifies properties P, and P4T' Note that it's not sufficient to do the trivial check of determining whethe~ p1 is identical to P_,. and P2 is identical to P-t. For CXitmplc. P1 might he the property of ?e~ng a golden retriever and P3 might he the propert y of being n dog. Clearly p sat1 tsftes P_,. even though they are not the same. So we must cnnstruct definitions that support the kinds of inference that we want to he able to do. At this point. anyone who is familiar with objcct·oricntcd programming. databases or artificial intelligence (AI) observes. ..This is famil iar territory.'' Object-oriented programming languages. database schema languages and AI knowledge representation languages are all designed to allow the definition of classes and their properties.

H

Exploiting Knowledge: The Semantic Web

939

In ohjcct-oricnted progmmming. the most important properties are methods, i.e., procedun~s th(lt c'm be applied to members of a class. In the case of database schema languagt:s unu AI kno~lcug~ representatio.n l~nguages. ~e~larative properties play a more importunt role. ·n1e JOb ot these propcrttes ts to permtt mference about both classes and their clements. Th~ design of a family of representation languages that can be used to define the meaning of RDF expressions is based on a long tradition pf work in the tradition of declarative representation languages. But the World Wide Web environment creates some new challenges. For example, inference engines in many database environments can reasonably make what is called the closed worltl a.uumption. They assume that all the relevant objects and their properties ~tre present in the database. If an object or a property is missing from the database. it is presumed not to exist in the world. For example. a program that is querying an airline database can assume that, if a Olght is not listed, it doesn't exist. In the distrihutcd knowledge environment of the World Wide Web. however. the dosed world assumplion is rarely justified. For example. if 1 query the Web with my uncle's name but fail to find him mentioned. it doesn't mean that he doesn't exist. we·u say that. in the Web environment. we must often make the open world assumption. Another important difference between most database and artificial intelligence systems on the one hand. and the World Wide Web environment on the other, is that no one im.lividualor organization has sole responsibility for defining the system. We will considc.r S<)tlle of the important implications of this difference below. An omo/ogy. as used in the context of an automated reasoning system, is a formal specification of the objects in a domain and their relationships to each other. An ontology consists of a set of classes, typically arranged in a subclass/superclass hierarchy. It may dcscrihc !ipeci(ic individuals that are members of those classes. It typically describes. properties. both of clusscs and of individuals. Those properties may include quite general ones. like part-of. as well as more specific ones like calori esperounce. The properties themselves lypically have properties. including, at a minimum. their domain and range. So un ROF vocabulary is an ontology. Despite the rich corpus of work that has been done in the broad area of knowledgebased systems. the architects of the Semantic Wch raced some hard choices as they set about the task of building. an ontology language that could support both defining the Wch's ohjccts and reasoning with them. The perfect language would be expressively powerful. de lined by a dear formal semantics. decidable. computationally tractable, and easy for people to use. Unfortunately. the key results that we presented in Theorem 22.4 and lll~!orem 2K.16 ngain rear their ugly heads: •

First-order logic is attractive bc:cause it has a clear. formal semantics. It is expressively powerful ~nough for many tasks. While it isn't expressively perfect, other. more powerfullog1cs are even less computationally tractable than first-order logic is.

•

Bu~. l'irst-ord~r logic is n~1t, in general. decidable (Theorem 22.4) and it is possible to dcttnc thconcs that are mcomplet·e (as shown by GOdel).

•

And even Boolean logic appears, in general, to he intractable (Theorem 2S.16).

Su .ct~mp:o~n.ises. ar~ ~equired: l~~re isn't one set of compromises that is perfect for ull tasks. Yet -st.mdards are .crucml d the pate· ntt'al of the Wcb as as hared resource ··ts

940

Appendix I

Applications: Networks

going to be exploited. Three decisions have made it possible for the task of defining a standards Q for the Semantic Web to proce~.:d :

St.1 of representation

• There won't he a single ontQiogy. For all the reasons that we have already mentioned. individual users and user communitits will he free to define ontologies that suit their needs. To make this possible. though. there must he a standard ontologydefinition lunguage(s) that thuse users can exploit. • Languages will be ddinl!d in layers. Rather than waiting until all of the issues and tradcoffs can he resolwd, standards for parts of th~ pmhll:m will be released as they an~ agreed upon. We've already descrihcd the lowest lt:vcl. thc URI language, and the second level. RDF. After a brief introduction to description logics. we will sketch the next two layers•. RDFS and OWL. • At the hight!stlevcl(s). there is simply nu one right answer for all pmhlcms. So alternative, hut upward compatible languages will he pnwidcd. Users who stick to the less expressive subsets can be assured dccidahility and. in som~ cases. tractability. Users who choose to use the more expressive langu&~gcs will he responsible for finding domain-appropriate compromises that deal with th~ dccidability and tractability issues that those languages p r~scnt.

1.3.4 Description Logics The ucvclopment path that we 01rc about w describe. from RDFS to OWL. Is based on a knowledge representation framework called den·riplion logic (or OL) 15• Most DL languages arc suhlanguagcs of first·orucr lugic. tailored to the task of defining and reasoning with classes. instances, ~nu their properties. The most important reasoning tlpcrat ions arc typically: •

suhsumption checking: Given the definitions of two classes. dues one subsume the oth..:r'! (\h&ss A subsumes class 8 iff cvc:ry d e men\ or /J is an clement of A. Stated another way. A suhsumes JJ iff A is at least as general ns ll.) • clnssificntion: Given a set of classc~. arrange them , based on their uescriptions. into a ~ubsumptiun graph. Think of a suhsumption gntph m; a subchtss/supcrdass hierarchy. We showed one inslanc~ nf such a graph in Example A.7. • r~aliznti o n: Given an ontology and a description of some particuhu entity in terms of a s~t uf properties. find th\: classes l<> which the entity hdongs.ln other words. finu the classes whose ucl'initiuns arc C'.msi!'tcnt with the definition of the c:ntily. • cunsistency/satisfiahility checking: Given~~ set uf two or more descriptions. are they consistent'! Alternatively. could there exist a noncmpty set of objects that salisfies all of those descriptiuns'? Note that incunsistcn~-y checking is a special case of subsumption cht!cking in which the proposed suhsumcr is the empty set. Any set that is subsumed hy th~ t.>ntpty set is also ncccssurily cmpt ~. 1

~Fe~r a t'tlmprdtcnsi\'C trcmmcnl ttl Jc..criplion lugic'l. ~cc l li.u11.lcr. c~ ul . lUII~ J. purticulmly the first chapter:

!Nardi and Hradtman 21MHI.

1.3 Exploiting Knowledge: The Semantic Web

941

The details of the definition of a DL language matter. Depending on how classes, properties. and instances arc allowed to be described, a DL logic may: •

share the problems of full first-order logic. In other words, it may be undecidable and it may allow the definition of theories that are incomplete.

•

be decidable. but appare.ntly intractable.

•

he decidable in polynomial time.

So what is the right DL language for the Semantic Web'? There is no one right answer. The tradeoff is between expressive power and the desirable computational propcrti~:s of dccidability and tractability. Then what should a standards committee do? We won't attempt to answer that question. But we will describe what the Semantic Web ·c onunith!es did do: worked for several years without agreeing on a single common language. But they needed to release a ~tandard in order to allow work on defining useful ontol()gie~ to proceed. Sn they rdeased the definition of RDFS (a Resource Description Framework Schema language). Oversimplifying a complex issue, RDFS is merely a subset of the language that everyone knew would eventually be required. But it is a subset that people could agree on after only a few years of discussion.

•

·nH!V

•

•

Meanwhile. work continued and a sequence of other languages were defined and standards for them were released. The ideas from some of the earlier such langu,lgcs evolved into the OWL family of languages. OWL has three dialects that r&mge. from t!xprl!ssively weak and computationally easy to expressively powerful und computationally undecidable. Users are free to choose to use exactly the lan'"' guag,e features that they need and no more.

ln the next two sections. we'll sketch the definitions of RDFS and the three OWL dialects, showing how the issues of computability and tractability influenced those definitions.

RDFS: A Resource Description Framework Schema Language The Rest)urcc Description Framework Schema language. RDF Schema, or simply RDFS. permits the definition of RDF vocabularies. i.e., classes, instances, and properties that correspond to relations among them. RDFS programs are written in RDF syntax. (I n..othc~ wurds. they consist of sets of triples.) They may exploit the constructs that tire dclu~~d m the RDF namespace (rdf), as well as the concepts that are defined in the R~I-S namcspace (rdfs). Tiley may also, of course. exploit constructs that are defined many other namespace for which a URI is known. The mechanism hy which classes and properties are defined in RDFS differs in one impurtan.l way ~rom the mechanism by which they are defined in most object-oriented systems.•mcluJmg. m~st datu base sc~ema languages. In those systems. the focus is on classes. Somcon~ dcfmes a class by listing its properties. In RDFS, on the other hand,

942

Appendix I

Applications: Networks

the focus is on properties. A user defines a property by specifying (using the domain and range propt!rtics defi11ed below) the classes to which it applies. The advantage of this property-oriented approach in the distributed environment of the World Wide Web is that classes are not ''owned.. by particular users. One user may declare a class and the n define some properties whose domain and/or range is equal to that class. Another user. working on a different problem. may define new properties that apply to that same class. So. for example. one user might define the property ca1ori esperounce with domain fooddb: food and range number. Someone else, taking a completely different point of view. might define the property when i nseason, also with the domain fooddb: food but this time with range ti meofyear. While one user owns the URl for food db: food. neither user ··owns" the definition of the class food. Both of them may use each other's properties if they want to and if they know the URis of the resources in which the properties are defined. RDF and RDFS. between them. define some fundament<~ I classes: •

rdfs: c1 ass: The class of all classes. The members of u class arc called its instances. Every class is an instance of rdfs: class. (So. in particul<~r, rdfs: (:1 ass is an instance of itself.)

•

rdfs: resource: The class that contains everything that an RDF program can talk about. Every other class is both an instance and a subclass uf rdfs: resource. which. in turn, is an instance of rdfs : class. In other ontologies. this most general class is typically called something like .. thing'' .

•

rdf: property:1l1e class of properties that can he used to define classes. instances, and other properties.

•

rdfs: 1i teral :The class of literal values such as strings anclnumbcrs.

RDFS distinguishes between a class and the set of its instnnces. Su, for example, the class that contains all cats that reside at the White House may be different from the class that contains all cats owned by the President, even if those two classes happen to contain the same instances. RDF and RDFS provide some built-in properties (i.r.:.• inst ances of the class rdf: property) that can be used in class and instance definitions. 'lltcsc indude: •

rdfs: subC1 assOf: Relates two classes. If A is n suhdass of H. then every instance of A is also an instance of B. The subClassOf property (relation) is transitive. so if A is a subclass of B. and 8 is a subclass of C. then ~very instance of A is also and instance of C.

•

rdf: type: Relates an instance and a class. If A rdf: type 8 . then A is an instance of B. In other ontologies. this property is often culled instance-of.

RDF.s (unli.ke.RDF) all~wS' users to define new pr~lp~rties hy. in turn. defining the~ properttcs. ButlHn propcrhes that can be used to ddmc othc::r properti~s include: •

rdfs: s~bPropertyOf. s.o. f~r example. co1 or might be declared to be a subproperty of phys1 ca1 characten st1 c. The subPropertyOf property (rehuion) is transitive.

1.3 Exploiting Knowledge: The Semantic Web

943

rdfs: domain: If P rdfs: domain C. then the domain of the relation (property) Pis the class C. So the only resources that can possess property P are instances of the class C. In terms of triples. this means that the subject of any triple whose predicate is P must he an instance of C. • rdfs: range: If P rdfs: range C, then the range of the relation (property) Pis the class C. So all values of the property Pare instances of the class C. Again, in terms of triples, this means that the object of any triple whose predicate is P must be an instance of C.

•

RDFS also provides mechanisms for defining: •

Containers: opl!n l!nded structures that contain other resources. Containers may be viewed as (unordered) bags (i.e., sets with duplicates allowed), as sequences (where order matters). or in some other way.

•

Collections: closed structures that contain other resources. A collection is described as a list of its members..

An RDFS program. just like an RDF program, is a list of triples. The meaning of such a program is to assert the conjunction of the assertions made by each of the individual triples. So. viewed as a logical language, RDFS is first-order logic with some very important restrictions. including: •

Only binary predicates can be represented as triples. Properties that might natural· ly be stated in lirst-order logic using unary predicates are typically described in RDFS using the rdf: type property. So, for example, instead of saying food (drocolme). in RDFS we say chocolate rdf:type food. -

• 111c only logical connector that is allowed is conjunction. (And it is not written explicitly. Rather all the triples are interpreted to be conjoined.) In particular. negation, disjunction. and implication cannot be represented. • ·There are no explicitly quantified variable~ So RDFS, while useful. is inadequate for many practical representation and reasoning tasks. To support those tasks. additional mechanisms must be provided. The OWL family of languages. to be described next, provides some of those mechanisms. On the other hand. RDFS. as a logical language, allows the specification of theories that arc incomplete and undecidable. One culprit is the fact that a class may also be treated as an individual. So a set (a class) may be both a subset of and an element of an,. other set. Without a distinction between sets and elements, definitions, such as this one,. known us Rus!;ell's paradox. are possible-: Let S be the set of all sets that are not members of themselves. Is S an element of S? l11e answer to this question cannot be yes. since if S is an element of S, it fails to meet the requirement for membership inS. But the answer also cannot be no, since if S is not an clement of S then it does meet S's requirement for membership und it must therefore be inS. 'lbe design of the OWL family of languages also addressed the desire to eliminate this problem.

944

Appendix I

Applications: Networks

1.3.6 OWL OWL. like RDFS. is a language for publishing and sharing ontologies on the World Wide Web. OWL is designed to support both the construction of ontologies and the implementation of reasoning engines (theorem provers) that can reason with the knowledge that its ontologi~s contain. Building an OWL ontology for a particular task may be substantially easier than building that ontology in a more traditional environment because it is rarely necessary to start from scratch. Many OWL ontologies Q alrc.!ady exist on tht: Web and new ontologies can be buill simply by adding to the existing ones. An OWL ontology.justlike an RDFS one, is simply a set of RDF triples. What OWL offers. that RDFS doesn't. is primarily: •

the ability to express more complex relationships aml)ng classes, and

•

the ability to specify more precise constraints on classes and their properties. So. for example. in OWL, one can:

describe constraints on the numbe-r of values that a property may possess foro\>. jects or a particular class. For example, a person can have only one mother but any number of children. In OWL. a property is .. functional .. iff each subject (i.e.• an element of the property's domain} has at most one value for the property. A property is "inverse-functional,. iff each object (i.e.. an elem(!nt of the property's range) may he the value of the property for no more than one subject. So. for example, USsoci a1sec uri tynumber is a functional property of people (since each person has only one). It is also inverse-functional since each number is the social security number of at most one person. • describe constraints on the values that a property may possess for objects of a particular class. (Note that this is different from specifying the range of the property since the range e~pplies to the property regardless of the class of the individual to whom the property applies. So. for example. the value of the mother property for a person must he a person. while the value of the mother property of a cat must be a cat. and so forth.) • write statements that enable the system to infer that any object tluu possesses a given set of property values is necessarily a member of some class.

•

•

define new classes in terms of existing classes by using the operations union. intersection. and complement. assert that two classes are necessarily disjoint.

•

define a class by enumerating its clements.

•

declare thnt a property is transitive. symmetric, one-to-one. one-to-many. or manyto-one.

•

assert equality and inl!quality.

•

ln providing these: ahiJities. OWL must give all its users the power they need for the applications they arc building while making. as ~trongly as po~siblc, guarantees of completeness, dccidahilily and tractability.

1.3' Exploiting Knowledge: The Semantic Web

945

Since expressive power trades off against comp.lcteness, deci~ability and tractabi~­ ity, there is no single language that can meet all of these go~ls for all users (and l~etr applications). So OWL is not a single language put a fanuly of upward compatable languag.es: •

OWL Full is the most expressively powerful of the OWL dialects. But the expressiveness comes at a price. A particular theory that is expressed in OWL Full may be consistent. complete and decidable, but there is no guarantee that it is. In this sense. OWL Full is analogous to tirst-order logic: There are first-order logic theories. such us Prcsburgcr arithmetic. a theory of the natural numbers with p/11s as the only operator. that arc compktc and decidable. But not every first-order logic theory is. 1t is known that no decision procedure for OWL Full can exist.

•

OWL DL (where the DL stands for Description Logic) is a compromise between expressiveness and desirable computational properties. More specifically, OWL DL is intended to bl.! a maximal subset of OWL Full for which not only a complete inference procedure but also a decision procedure is known to exist. OWL DL supports all of the language constructs of OWL Full. but it imposes some constraints on how they arc used.

•

OWL Lite is expressively the weakest dialect of OWL. As a result. its inference pro... ccdurc guarantees better worst-case performance than do the corresponding procedures of its Jll()re expressive cousins. It is designed to support the definition of a straightforward ontology based on a hierarchy of classes and subclasses.

OWL Full is n superset of RDFS. OWL Lite and OWL DL are both supcrscts of a subset of RDFS. In pmticular. while RDFS allows an object to be both a class and an insumcc, neither OWL Lite nor OWL DL does (and so the Russell paradox would not be l!Xpressible in them). OWL DL achieves its completeness and decidabitity properties by imposing constrnints, including the following. on the way that the OWL vocabulary may be used. •

Type separation is enrorced (as mentioned above). A class cannot also be an instance or a property. A property cannot also be an instance or a class.

•

A property must either be an objectProperty (I.e~. a relation between instances of two classes) or a data type property (i.e .. a relation between an instance of a class and an RDF literal or a built-in XML Schema data type). lt may not be both.

•

No cardim\lity constre~ints can he placed on transitive properties or their inverses or any or thl.!ir supcrpn>pcnies.

•

Statements about equality and inc.q uality can only be made about named individuals.

The fact that OWL DL is decidable (and thus consistency checking is possible) makes it u useful tool in domains where consistency is critical. In the next section we'll mention 01~c system. Galen. that takes advantage of this ability. OWL L1te uses a subset of the OWL vocabulary. It imposes all the same constraints on vocabulary use that OWL DL does. And it imposes additional constraints, including:

•

The o~ly val~c~ that ~r~ allowed. as cardinality constraints on properties are 0 and 1. So. for example, ll ts not poss1ble to say that the number of members of a soccer team must be at least 11 .

946

Appendix I

Applications: Networks

•

Classes cannot be defined using the union or complement operator (applied to other classes). So.l'or example, it is not possible to defint! a etas.'\ commonpet as the union of the dog and cat classes. • Only explicitly named classes and properties may be used with the intersection operator to form new classes or properties. • Clas.-;cs cannot be defined hy enumerating thdr elements. So, for example. it is not possihle to define the class dayofweek hy listing the! seven days of the week. • •

Classes cannot be asserted to be the same or to be disjoint. It is not possihlc to assert that any instance of a given class must have some particular value for u given property. So. for example. it is not possible to require that every element of tht! class USci ti zen must have the value United States for the ci ti zenof property.

1.3.7 Exploiting The Semantic Web Metallata resources on the Web nre growing daily o .. Many are based on RDF, RDF Schema, and OWL. as we have described them here. Many arc bHsed on other representational systems.. Some use other languages derived from description logic: others use languages that are more similar to relational databases. All he core of many of these efforts is the development of common ontologies that enable entire communities of users to share Web resources. We've already mentioned a r~w of these ontologies. for example the Friend of a Friend (foaf) and Dublin Core vocahularic~ We'll mention a few more here. The need to share data is acute and well-understood within the biomedical research community. Several ontology-construction projects have hecn driven by the need to make this possihle. For example: •

The objective of the Gene Ontology (GO) Consortium l:l is to address the need for consistent descriptions. in a species-inllcpcnllcnt way. of gene products in different databases. To meet this need. the GO project has dc\'doped three ontologie.s, each of which enahlcs the descripti{1n of gene products from one important perspective: i'!ssociutcd biological processes, cellular components, and molecular functions. GO exploits both XML and RDF/XML. • The objective of the GALEN ~ project is make it easier to build useful clinical (medical) applications by providing a common knowledge hase of clinical terminology. The core of this knowledge hasc is an ontology that is intended to contain all and only the sensible medical concepts. Dic\ionarics (in many languages) can then connect words to concepts. The first implementation of GALEN was built using a representation language that was designed explicitly to support th\! GALEN project. The GALEN ontology has since hcen translated into OWL. More specifically. it exploits OWL DL. Because OWL DL is decidable, it is pl'lSsihlc (although it may require running overnight) to answer the question. "Is this Vt!rsion of the GALEN ontology consistent'!'' Consider the problem of adding location information to various kinds of Web resources. For example. a user might like to find only thos\! hlogs that originate in some

1.3

Exploiting Knowledge: The Semantic Web

947

particular location. There exists an RDF vocabulary Q for describing basic geographical information in a way that would make that possible. Consider the wide range of physical devices that are used to access the World Wide Web. They range from computers with large display screens and full sound capability to handheld devices that are used in settings where sound is not appropriate and bandwidth may be limited. Further, the users of those devices have their own preferences for things ranging from language to font size. One way to make it possible to customize information delivery for all of those situations is for clients to describe, for example in RDF, their characteristics and preferences Q, Thesauri categorize words using relationships similar to the ones used in many useful kinds of ontologies. So it may be natural to represent thesaurus information in an ontology language such as RDF or OWL. Wordnet .Q is a large, online lexical reference system that organizes English nouns, verbs, adjectives and adverbs into synonym sets that correspond to underlying lexical concepts. Those lexical concepts are. in turn. organized hy relations such as lzypernym (more general concept),hyponym (more specific concept), and htzsinstance. The Wordnet lexicon has been encoded in RDF/OWL to make it accessible to a wide assortment of other metadata projects.

APPEND

X

J

Applications: Security

I

n the modern world. security is an important feature of almost every system we usc. We protect physical locations with locks and burglar alarms. We protect com~ pulers with safe operating systt!ms and sophisticat~d virus checkers. We protect sensitive communications by encrypting them. The theory des,·ribec.i in this book has something to say about all of those techni4ues.

J.1

Physical Security Systems as FSMs lmagine a conventional intrusion-detection security system of the sort that is found in all kinds of huildings. including houses, offices, and banks. Such systems can naturally be modeled as finite-state machines. Some intrusion-detection systems arc complex: They may. f
query slut us

FIGURE J.l A sin'lplc physical security system. 948

J.2

Computer System Safety

9441

FIGURE J.% The code-entering fragment of a security system.

the system). disarm (turn off the system). query the status of the system. reset the svstem. open a door, activate the glass-break detector, and 30 seconds elapse. The job ~f this machine is to detect an intrusion. So we have labeled the states that require an alarm as accepting states. State 6 differs from state 1 since it displays that an alarm has occurred since the last time the system was reset and it will not allow the system to be armed until a reset occurs. A realistic system has many more states. For example.. suppose that alarm codes consist of four digits. Then the single transition from state 1 to state 2 is actually a sequence of four transitions, one for each digit that must be typed in order to arm the system. Suppose. for example, that the alarm code is 9999. Then we can describe the· code-entering fragment of the system as the DFSM shown in Figure 12. Note that we have not specified what happens if the query button is pushed in states A-C. One of the questions that that system designer must answer is whether the query function is allowed in the middle of an arming sequence.

j .. 2

Computer System Safety Consider a complex computer system. It includes files that some users, but not others, have access to. It includes processes (like print pay checks') that some users are allowed to run but most are not. Is it decidable whether such a system is safe? For example is it decidahle. given the operations that are possible in the system. whether an unauthorized user could acquire access to the paycheck printing system'? The answer to this question depends, of course,. on the operations that are allowed. To build a model of the protectinn status. of a system. we'll use three kinds of entities: •

Subjects: Active agents. generally processes or users.

•

Ohj~cts: Rcsour~es that the agents need to exploit. These could include files. processes, de~1ccs. ~lc. Notace that. process~s .can be viewed both as subjects (entities capable of domg thmgs) and as obJects (enhtaes that other entities may want to invoke),

•

Rights: Capabilities tha.t agents may have with respect to the objects. Rights could include read access. wnte access, delete access or execute access for files. execute access for processes. edit. compile, or execute access for source code check or change access .for n password file. and so forth. ·•

950

Appendix

l

Applications: Security

pi'Ot:t!SSJ process 1 process1 pro("t!SS.l proass~

processz

execute:

procnsJ

PTIKI!S$~

CXCC:Uh!

execute

Jilt!, uwn

read read

e:~~ecute

execute execute

filtl

own ex~.-cute

read

FIGURE J.3 A simple acces.<; control matrix.

We can describe the current protection status of a system with an acc~ss control matrix. A, that contains one row for each agent and one column for each protected object. Each cell of this matrix contains the set of rights that the agent possesses with respect to the object. Figure 13 shows a simple example of such a matrix. The protection status of a system must be able to evolve along with the system. We'U assume the existence of the following primitive operations for changing the access matrix: •

Create subject (x) records the existence of a new subject .t. such as a new user or process.

• • • •

Create object (.t) records the existence of a new object x. such as a process or a file. Destroy subject (x). Destroy object (x). Enter r into A[s, o] gives subjects the right r with respect tu object o,

•

Delete r from A[s. o] removes subjects's right r with respect to object o.

We will allow commands to be constructed from these primitives, but all such commands must be of the following restricted form: command-name(x 1• x2, •••• x,) = if r 1 in A( ... ~ ... ] and r2 in A[ .... ,_. ~ ·I and

r; in A( .. , •. •. } then operation 1 operation2 opcrationm ln other words. the command may check that particuh1r rights arc present in selected cells of the access matrix. If all conditions arc met. then the operation sequence is executed. All the operations must be primitive operations as defined above. So no additional tests, loops. or branches are allowed. The parameters x., .t2•• • •• •t, must each be bound to some subject or some object. The rights '•· r, arc hard--coded into the definition of a particular command.

'2· ...

J.2 Computer System Safety

951

Define il protection framework to be. a set of commands that have been defined as described above and that are available for modifying an access control matrix. Define a prmection system to be a pair (init,frnmework). /nit is an _initial configuration that is described hy an access control matrix that contains various rights in its cells. Framework is a prot~ctiun framework that describes the way in which the rights .contained in the matrix can evolve as a result of system events. In designing a protection framework, our goal is typically to guarantee that certain subjects maintain control over certain rights to certain objects. We will say that a ri~ht has leaked iff it is added to some access control matrix cell that did not already contain it. We will say that a protection system is safe with respect to some right riff there is no sequence of commands that could. if executed from the system's initial configuration, cause r to he leaked. We ·n say that a system is unsafe iff it is not safe. Note that this def· inition of safety is probably too strong for most real applications. For example, if a process create.s a file it will generally want to assign itself various rights to that file. That ·assignment of rights should not constitute leakage. It may also choose to allocate some rights to other processes. What it wants to be able to guarantee is that no further transfer of unauthorized rights will occur. That more narrow definition of leakage can be' descrihcd in our framework in a couple of ways, including the ability to ask about teak~ age from an arbitrary point in the computation (e.g.. after the file has been created and assigned initial rights) and the ability to exclude some subjects (i.e.• those who are .. trusted") from the matrix when leakage is evaluated. For simplicity, we will consider just the hasic model hc.re. •

Given a protection systemS = (init. framework) and a right r. is it decidable whether Sis safe with respect to r?

It turns out that if we impose an additional constraint on the form of the commands in the system then the answer is yes. Define a protection framework to be mono-operational

·iff the body of each command contains a single primitive operation. The safety question for mlmo.opcnltional protection systems is decidable. But such systems are very limited. for ex~tmple, they do not allow the definition of a command by which a subject create~ a file and then gives itself some set of rights to that file. So we must consider the question of decidability of the more general safety question. Given an a!hitr~ry ?rotection systemS = (init.framework) and a right r. is it decidable whether SIS sate wtth respect to r? Now the answer is no, which we can prove by reduction [rom H,, = { :Turing machine M halts on e}. The proof that we are about to present ~as origin~lly given in ~H~rrison. Ruzzo, and Ullman 1976), which was conct:rncd wtth protection and sccunty 111 I he specific context of operating systems. It is also presented. i~ the ~arger context of overall system security, in [Bishop 20t13]. The key 1deas m the proof are the following: •

ll is possible to e~code the configuration of an arbitrary Turing machine Mas an access control matnx we'll call A. To do this will require a set of "rights" as follows: • One l'or each element of M's tape alphabet.

•

One for ~ach state of M·These must be chosen so that there is no overlap in na.mc~. With the ones that correspond to tape. alphabet symbols. Let q be the "nght that corresponds to any of M's halting states... f

952

Appendix J

Applications: Security

We call these objects •·rights" (in quotes) because. although we will treat them like rights in a protection system. they are not rights in the standard sense. Th~y do not represent actions that an agent can take. They are -simply symbols that wtll be manipulated by the reduction. Each square of M's tape that is either non blank or has been visited by M will correspond to one ctll in the matrix A . The cell that corresponds to square; of M's tnpe will contain the .. right .. that corresponds to the current symbol on square; of the tape. In audition. the matrix will encode the position of M's read/write head ~d its state. It will do that by cClntaining. in the cell that is currently under the read/wnte head. the ••right" that corresponds to M's current state. •

lt is possible to describe the transition function of a Turing machine as a protection

framework (a set of commands. us described above, for manipulating the access control matrix). • So the question... Does M ever enter one of its halting. states when started with an empty tape'?" can he reduced to the question. " If A starts out representing M's initial configuration. does a symbol corrcsponuing to any halting state ever get inserted into any edt of A']" In other words. ··Has any halting stute symbol leaked?" So. if we could d«!cide whether an arbitmry protection system is safe with respect to an arbitrary right r. we could decide H ~:. But we know. from The!orem 21.1. that H~ is not in D. The only question we arc asking about M is whcth'-!r or not it halts. If it halts, we don't care which of its halting states it lands in. So we will begin by mooifying M so that it has a single halting state q_r. Tht! modified M witt enter q1 iff the original M would e nter any of its halting states. Now we can ask the sp~.:cific question. "Does q1 leak?" To make it easier to represent M's configunttion as nn access control matrix, we will assume that M has a one-way (to the right) infinite tape. rather than our standard, two~ way infinite tape. By 1l1corcm 17.5. any computation by a Turing machine with a twoway infinite tapt: can be simulated by a Turing mnchim: with a one-way infinite tape, so this assumption docs not limit the generality of the result that we are about to present. To see how a configuration or M is encoded as an Hcccss control matrix. consider the simple example shown in Figure J.4 (a). M is in state q~ and we assume that it started on the blank just to the left of the beginning of the input. so there are four nonblank or ex~ amincd squares on M's tape. This configuration will be represented as the square access control matrix A, shown in Figure J.4 (h). A contains one row and one column for each t<:1pe squares that is nonhlank or has been visited: Notice that. primctrily. only cells along A ·s majm diagonal contain any rights. The cell A(i. i} contains the '' right"that corresponds to the contents of tape square i. Since the rcau/write head is on square 3. A(3. 3] also contains the •·right" corresponding to the current state. lf~· The only other .. rights" encode the sequential relationship among the SlJUarcs on the tape. Jf s; immediatdy precedes sl' then s; "owns'' s ·. Finally 1 the cell that corresponds to the right-most nonhlank or visited tape square contain~ the "right" t'lld. It remains to show how the ope.ration of M cun be s·imulatcd hy commands that mouify A. Given a particular M . we can construct a sc.t of such commands that exactly

J.2 Computer System Safety

0

a

b

b

.St

.s,

0

953

0

.sl

s..

S.s

()Wif

a

.Sz .SJ

s,.

own b,qs

OWII

b, end (b)

FIGURE JA Representing a Thring machine configuration as an access control matrix.

mimic the moves that M can make. For example, suppose that, in st~te q5 reading b, M writes an a, moves left, and goes to state q6 • We construct the followmg command:

stateq5 readingb(xll x2) = if own in A[x1 • x2] and

then delete q5 from A[x2 , x2J delete b from A[x2, x2]

/* This command can only apply to two adjacent tape squares. where /* the one to the right is currently under the read/write head and M i s in q5 , and /* there is a b under the read/write head /* Remove the old state info /* and the current symbol under

the read/write head. /* Write the new symbol under the read/write head. /* Move the read/write head one square to the left and go to state q6 • We must construct one such command for every transition of M. We must also construct commands that correspond to the special cases in which M tries to move off the tape to the left and in which it moves to the right to a previously unvisited blank tape square. The latter condition occurs whenever M tries to move right and the current tape square has the "right" eml. In that case. the appropriate command must frrst create a new object and a new subject corresponding to the next tape square. The simulation of a lUring machine M hegins by encoding M's initial configuration as an access control matrix. For example, suppose that M's initial configuration is as shown in Figure J.5(a). Then we let A be the access control matrix shown in Figure J.5(b). There are a few other details that we must consider. For example, since we are going to test whether q/ever gets inserted into A during a computation, we must be sure that

954

Applications: Security

Appendix J

a

0

0

b

0

(a)

s,

"•.St

Sz

O.CJu

SJ

OW/I

a

till'I/

b. t•ntl

SJ

(h)

FIGURE J.! Enc
q, is not in A in the initial configuration. So if M starts in q1; we will first modify it so that it starts in some new state and then makes one transition to t/ 1~

Notice that we have constructed the commands in such a fashion that. if M is deterministic. exactly one command will have its conditions satisfied at any point. If M is nondeterministic then more than one command may match against some configurations. We can now show that it is undecidable whether. g.i\'cn an arbitrary protection system S = (init. framewnrk) and right r. Sis safe with respect tor. To do so, we define the following language and show that it is not in D: •

Safety = { : Sis safe with respect to r}.

THEOREM J.1 "IsS is Safe with Respect to r?" is Undecidable

Theorem: The language Safety= {: Sis safe with respect tor}

is not in D.

Proof: We show that He - { : Turing machine M halls one} ~ Safety and so Safety is not in D because Hr. isn't. Ddinc:

R()

=

t. Make any necessary changes to M: 1.1. H M has more than one halting state. then add a new unique halting state q,r and add transitions that take it from each of its original halting states to q1. 1~ If M starts in its halting state 'lr· then create a new start state that

simply reads whatever symbol is under the read/write head and then goes to q1• 2. BuildS: 2.1. Construct an initial access control matrix A that corresponds toM's initial configuration on input e. 2.2. Construct a. ~et of commands, as dcscrihcd a hove. that correspond to the transtttons of M. 3. Return .

J.3

Cryptography

955

~ R• ....,} is a reduction from He to Safety. If Oracle exists and decides Safety. then = ,Orm:le( R( )) decides H 11• R and ...., can be implemented as.a Tu~­ ing machines. And Cis correct. By definition,S is unsafe wi~h respect to qtiff qf1s not present in the initial configuration of A and there extsts some se~uence of

c

commands in S that could result in the initial configuration of S bemg transformed into a m:w configuration in which q1 has leaked. i.e., it a~pe~rs. in .s~~e· cdl of A . Sine\! the initial configuration of S corresponds toM bemg m tts mtttal configuration on a hlank tape, M does not start in q1, and the commands of S simulate the moves of M. this will happen iff M reaches state q1 and so halts. Thus: • If e HI): M halts one, so q,eventually appears in some cell of A. Sis unsafe with respect to q1, so Oracle rejects. C accepts. • If fl. HI): M does not halt on tv, so q1 never appears in some cell of A. Sis safe with respcct to q1, so Oracle accepts. C rejects.

But no machine to decide He can exist, so neither does Oracle. Does the undcch.bthility of Safety mean that we should give up on proving that systems arc safe'! No. There are restricted models that are decidable. And there are specific instances of even the more general model that can be shown to have specific propcrtit!s. This result just means that there is no general solution to the problem.

J.3

Cryptography Effective encryption systems. or the lack of them, have changed the course of history. Modern techniques for encoding sensitive financial information have enabled the explosion of electronic commerce. Throughout history, the evolution of cryptographic systems has been a game of cat and mouse; as code breaking techniques were devel~ped. new encoding methods had to be developed. Before computers. any useful cryptographic scheme was necessarily computationally trivial. It had to be because both senders and receivers implemented their algorithms by hand. With the advent of computers. things changed. Senders and receivers. as well u~ cncn1ics and eavesdroppers, all have access to substantial and equivalent computational resources. But computational complexity is still important. What is now required is a scheme with two properties: There must exist efficient algorithms for encoding a mes.-;agc and fur decoding it by the intended recipient. And there must not exist an efficient algorithm for decoding the message by anvone else. The facts about computing with prime numbers. as we describe them in Part ·v. provide the basis for a system that possesses both of_ these properties. In ~111 hut. the stmplest cryptographic systems, the algorithms that will be used, both for cncryp~ton a.nd decryption. are fixed and known to everyone. But those algo-· rithn1s take two mputs, the text to be encoded or decoded and a key. In a symmetric /uy system~ . scn~~r a.~d receiver use the same key. Symmetric key systems su(fer from the rollowmg r•tl,\lls .. No message can be sent unless there has been some prior agreement on"' key. Even •f there has been such an agreement. if the same key is used over

956

AppendiiC J

Applications: Security

an extended period of time. an eavesdropper may he uhle to inf~r the key and break the code. For example. the eavesdropper might he a hie to collect statistics on the f~e­ qucncy of letter combinations in the encoded text and compHrc them to frequenc1~ in typical une ncoded texts in order to infer relationships between the two. But m order to change keys. there must be some way to tra nsmit new keys (securely) between senders and receivers. Public key systems. first introduced in the 1970s. get around all of those problems. The most widely used public key system is the R.SA ufgorithm IRivest. Shamir and Adleman L97~J . Following convention. we'll assume that Boh and Alice wish to exchange secure messages and that Eve is a ttempting to eavesdrop. We'll call the original (unencrypted text) the plailttext and the encrypted text th~ ciJJhertext. The most general way to describe RSA. and related algorithms. is as follows: Assume that Alice wants to send a message to Bob.l11en:

1.1. Bob chooses a key. private. known only to him.'This key may need to possess some specific mathematical properties in order to he effective. so Bob may need to exploit a function dwose that guarantees to return an appropriate private key. Bob exploits a function fto compute his public key, p11hlic = f(privare). 1.2. Buh publishes puhlit (either completely publicly or by sending it. unencrypted, to Alice), 1.3. Alice exploits Bob's public key to compute ciplwrct!xt public) and she sends dplwnexl to Bob.

= enaypt(plaimext,

1.4. Bob exploits his private key to compute pluilllt'XI

= th•crypt(dphtertext, private). Ln order for this last step to work. encrypt and deCIJ'P' must bed~

signed so that one is the inverse of the other. If the re exist efricient algorithms for perl'orming all four of these steps, then Bob and Alice will he a ble tn exchange messages. Bu1 what al'lout Eve'! Might she also be able to decrypt Alice's message'! We assume that Eve knows the algorithms encrypt and decrypt. So she could easily eavesdrop if she could infl.!r Bob's pr ivate key from his public one or if she could compute decrypt without knowing Bob's private kev. The RSA algorithm exploits the mathc:matical properties of modular arithmetic a~d the computational properties of prime numbers to guarankc that Buh and Alice can per· form their tasks efficiently but Eve cannot. Alict: uses the RSA algorithm to scm.l a mess
1. Bob constructs his public and privntc keys: 1.1. Boh chooses two large prime numh~r~ p ~mc.lq. From them. he computes n = p • q. 1.2. Boh finds a value t such that 1 <: ~ < p • t/ and g cd (e. (p _ 1). (q - I))= 1. (In other words. he finds nn e such that e a d (p - 1) • (q - 1) arc relatively prime.) n

J.3 Cryptography

957

1.3. Bob computes a value tl such that c/• e (mod (p - 1) • (q - 1)) = l. In RSA terminology, this value tl. rather than the original numbers p and q, is rererred to as Bob's private key.

2. Bob publishes (11. t•) as his public key. 3. Alice hrcaks her message plt~imext into segments such that no segment corresponds to a hinary number that is larger than 11. Then. for each plaintext seg· mcnt. Alice computes ciphertext = pluimext' (mod 11 ) . Then she send dphe1:tex1 to Boh. 4. Bob recreates Alice's original message by computing plailllext = ciphertext'' (moo 11).

TI1c RSA algorithm is effective because: •

The functions C'm:rypt and decrypt are inverses of each other. The proof foHows from Euler's gcncntlization of Fermat's Little Theorem (as described in Section 30.2.4 ). ll1e generalization is called Euler's Toticnt Theorem or sometimes just Euler's Theorem ~.

•

Boh can choose primes eft1ciently using the following algorithm: 1.1. Randomly choose two large numbers as candidates.

1.2. Check the candidates to see if they are prime. This can be done efficiently using a randomized algorithm. as described in Section 30.2.4. There is a tiny chance that u nonprime could be thought to be prime, but the probability of this hap· pcninp. can be reduced so that it is substantially lower than the probability of a transient hardware failure causing an error in the transmission process.

t.J. Repeat steps l and 2 until two primes have been chosen. By the Prime Number Theorem ~ . the probability of a number near x being prime is nhnut 1/ln x (where In is the natural logarithm, i.e.• the log base 2.71828 .. . . ol' x). So. for example. suppose Bob wants to choose a 1000 bit number. The probahility of a randomly chosen number near 211llKl being prime is about 116lJ3. So he may have to try 1000 or so times for each of the two numbers tha1 ht! needs. •

Bub can check gccl efficiently (using Euclid's algor,thm, as described in Example 27.6),so he can compute e.

•

Bob can compute ll efficiently. using an extension of Euclid's algorithm that exploits the 4uotients that it produces at each step.

•

Alice can impl~ment e~1crypt efficiently. It is not necessary to compute plai11texf and then take tts rcmamder mod n. Modular exponentiation can be done directly by successive squaring. as shown in the example below. Similarly. Bob can implem~nt tlecrypt efficiently. Eve can't recreate plaimcxt because:

• •

•

she cun't simply invert t!llCI)•pt because modular exponentiation isn't invertible. S~tc cnuld try every candidate p/aimext and see if she gets one that produces c1plwrtc•xt. but there are too muny o( them for this to be feasible.

958

Applications: Security

Appendix J

•

sh~ can't computed from11 and e. No cfticient ul~flrithml't)r factoring 11 into P and q is known.su she can·r solve the prol.llcmthat way. And if then~ were some o~her way for her to cumputc d dtkiently. that algorithm could he used as an df14:1ent ulgurithm for computing p and tJ. Aml again. no su\:h dTicicnt algorithm is known.

EXAMPLE J.1 The RSA Algorithm We can illustrate the RSA algorithm with a simple message from Alice to Bob. In practice, messages will he longer and keys should be large numbers. We'll use short ones here so thut it is easier to see what is going on.

t. Boh is expecting to receive messages. So he constructs his keys as follows:

= 19 ilnd q = 31. He computes = p • q = 5N9. 1.2. He rinds an(? that has no common divisors with lS • 30 540. Thee he selects is 49. 1.3. He finds a value d = 1069. Notice that 1069 • 49 = 52.381. Bob needs to assure that the remainder. when 52.3XJ is divided hy 540. is 1. And it is: S2,3Xl = 540· 97 + I. Bob's private kt.!y is now )(1()9.

1.1. HI.! chooses two prime numbers,p 11

=

2. Boh publishes (5Nl), 49) as his public key.

3. Alice wishes to send the;! simple message '"A ... The ASCII code for A is 65. So Alice computes 654t> (mod S~N). She does this without actually computing 6549• Instead . she exploits the following two facts: 1t+. i

= n' ·ni.

(" • m )(mod k) = (" (mod k) • m (mod k ))(mod k).

Combining these, we have: ni +l(modk) = (1i(modk)·n1(modk))(modk).

So. to compute 65 4'~. first observe that 49 can he expressed in binary as 1HXXll. So 49 = J + 16 + 32. Thus 654~ = 65 1 + lh • -' 2 • 1l1e following table lists the required powers of 65: 65 1 (mod 5S9) = 65. 651 (mod 5H9) = 4225 (mod 589) = 102. 654 (mod 5K9) == 1022 (mod SH9) = 10404 (mod 5X9) = 391. 6511 (mcx.l5X9) = 391 1 (mod 5X9) = 152~1 (mod 5X9) = 330. 65 1" (mod 5X9) = 330~ (mod 5N9) = 108900 (mod 5Kl)) = 524. 65~2 (mod 589) = 5242 (mod 5~9) = 274576 (mod SNY) = 102.

J.4

So we have that: 6549 (mod 589)

Hackers and Viruses

959

= 65° + 16 +32) (mod 589). = (651 • 65 16 • 6532)(mod 589). = ((651 (mod 589)) • (65 16 (mod 589)) •

(6532 (mod 589)))(mod 589). = (65 • 524 • 102)(mod 589). = ((34060 (mod 589)) • 102)(mod 589). = (487 ·102)(mod 589). = 49674 (mod 589). = 198. Alice sends Bob the message 198.

4. Bob uses his private key (1069) to recreate Alice's message by computing t98Ul6
f-or the details of the mathematical claims that have just been made. as well as some additional points that should be considered in choosing good keys. see any good cryptography lx1ok. for example (Trappe and Washington 2006}.

J.A

Hackers and Viruses In this section, we'll brieny touch on two other network security issues:The first is virus detection. We'll sec that the undecidability results that we proved in Chapter 21 tell us that the definitive virus detector cannot exist. The second involves the difference between the average-case and worst-case time complexity of some important algorithms. This difference may allow hackers to launch denial of service attacks and to observe •·secret" behavior of remote hosts.

J.A.1

Virus Detection Given a known computer virus V, consider the problem of detecting an infection by V. The most straightforward approach to solving this problem is just to scan incoming messages for the text < V>. But viruses can easily evade this technique by altering their text in ways that have no effect on the computation that v performs. So, for example, source code. could be modified to add blanks in meaningless places or to add leading Os. to ~umbers. Execut~ble code could be modified by adding jump instructions that JUSt JUmp to the next mstruction.

960

Appendix J

Applications: Security

So the practical virus detection problem must be stated as. "Given a known virus V and an input message M, does M contain the text of a program that computes the same thing V computes'!" By Theorem 21.8, we know that the equivalence question is undecidable for Turing machines. Using that result, we showed. in Theorem 21.12. that the equi\'alence question for arbitrary programs is also undecidable. So there exists no algorithm that can. in the general case. decide whether a program P. contained in some message M. is equivalent to a given virus V. So we can't solve the virus problem by making a list of the known viruses and comparing new code to them. What about going the other way? Suppose that, instead of making a list of forbidden operations. we allowed users to define a "white list" of the operations that are to be allowed to run on their machines. Then the job of a virus filter is to compare incoming code to the operations on the white list. Any code that is equivalent to some allowed operation can be declared safe. But now we have exactly the same problem. No test for equivalence. exists.

J.4.2 Exploiting the Difference Between the Worst Case and the Average Case Some widely used algorithms have the property that their worst-case time complexity is significantly different than their average-case time complexity. For example: • Looking Up an entry in a hash table may take, on average. constant time. But if all the entries collide and hash to the same table location. the time required becomes O(n) where n is the number of entries in the table. •

•

Looking up an entry in a binary search tree may Lake. on average O(log n) time. But the tree may become unbalanced. In the worst case, it becomes a list and lookup time again becomes O(n) Matching regular expressions (often called regexes) of the sort that are supported by Unix utilities and programming languages likt: Perl may take close to constant time on average. But these regex languages allow expressions that arc not regular expressions in the sense in which we defined them in Chapter 6. Any of the regular expressions that we considered there can he converted to a finite state machine that can be guaranteed to perform a match in linear time. But the added flexibility that is provided in the practical tools (sec Appendix 0 for a description of one of them) means that languages that are not regular can be defined. So no finite state machine can be built to accept them.ln the worst case, matching some of these patterns may require a backtracking search and so the time required may he exponential in the length of the input string. Hackers can exploit these facts Q. For example:

•

One way to launch a denial of service attack against a target si te S is to send to sa series of messages/requests that has been craftt::d so that Swill exhibit its worst-case performance. If S was dt!signed so that it could adequately respond to its traffic in the average case. it will no longer he able to do so.

J.4 Hackers and Viruses

•

961

One way to get a peek inside a site S and observe properties that were not intended to be observable is to time it. For example, it is sometimes possible to observe the time required by S to perform decryption or password checking and so to in[er its private key or a stored password.

A PPEND

X

K

Applications: Computational Biology roteins and DNA. the huilding blocks of every living organism. can naturally be described as strings. In Section Ul.2.9, we dcscrihed an experiment in DNA computing. In it. synthesizt!d DNA molecult:s an: treated as strings and operations on them are used to solve a simple graph search problem. Of more practical interest, at lcusr so far. is the fact that significant sets or real DNA and protein molecules can be moddcd as languages. So. not surprisingly. several of the techniques (including FSMs, regular expressions. HMMs. and context-free grammars) that have heen described in this book play an important role in modern computational biology. ln Section 5.12.1, wt: dcscribc!d the use of an H MM to model a problem in population genetics. In this chapter. we will discuss several other application areas.

P K.1

A (Very) Short Introduction to Molecular Biology and Genetics We begin this chapter with a very short introduction to the biological concepts that are required for an understanding of the way that the computational models we have discu~scd arc being used by hiologists. We skip many important details. For more information . follow the Web links suggested here, or consult !Alberts ct al 2002} or any good modern text on molecular biology.

K.1.1 Proteins Proteins arc the building blocks of living organisms. A prot~ in is a large molecule that is composed of a sequence of amino acids. There are 20 amino acids .Q that occur in proteins. They arc shown in Table K.l . along with their standard. one-le tter symbols. Amino acids are typically divided into classes: hydrophobic (h-phob), hydrophilic (h-phil ). and polar. with the polar molecules further divided into positively (pos) and negati vely (neg) charged.l11c class to ·which an amino acid helungs can have an effect on its function in a protein.Table K.l shows the class to which each amino acid belongs.

962

1(.1

,-Table K.l 1-;tmino act'd

Amino acids. .'tym Class

1--p.tanine p.rginin~

.,Asparagmc p.spa~tic acid cystem~

01utanune oJutamic acid otycinc }-listidi~e

tsoleucmc

A R N 0

c Q

E G H I

h-phob pos h-phil neg h-phil h-phil neg h-phob pos h-phob

A (Very) Short Introduction to Molecular Biology and Genetics

Amino Acid

Sym

I l Leucine K Lvsine Mt!thionine M Phenylalanine F p Proline Serine s Threonine T Tryptophan w Tyrosine v Valine v

963

Class

Pattern Symbols

Symbotl

h-phob pos h-phob h-phob h-phob h-phil h-phil h-phob h-phil h-phob

Aspartic acid or Asparagine Glutamine or Glutamic acid any amino acid

s.

z X

L---

It also shows a set of symbols that are sometimes used in specifying patterns of amino acid sequences. Amino acids share a common chemical structure. Each contains a carbon atom, to which is attached an amino group (NH2), a carboxyl group (COOH), and a side chain, also called the functional group. Amino acids contain different functional groups, and it is thut part of the molecule that causes them to behave differently. Amino acids combine to form proteins when the amino group of one amino acid molecule bonds with the carboxyl group of anuth~r. releasing one molecule of water (H20) and forming a bond called a peptide linkage. For example, three amino acid molecules, joined by two· peptide linkages would look roughly as shown in Figure K.l (ignoring the details of what the peptide linkages, shown as ovals, actually look like, and letting the '?'s represent the functional groups of each of the amino acids). The part of each amino acid that remains after peptide bonds have been formed is called an amino acid residue. So. to be exact, a protein is a sequence of amino acid residues. For simplicity. however, proteins are often described simply as sequences of amino acids and we will follow that convention. The sequence of amino acids that makes up a protein is called the protein'sprimary structure. If each amino acid is assigned a distinct symbol, as shown in the table above•. then the primary structure of a protein can be described as a string. So, for example, the string QTS corresponds to the sequence Glutamine. Threonine, Serine. Notice that the two enJs or the sequence illustrated below are different. There is an amino (NH 2} group on one end and a carboxyl (COOH) group on the other. If we adopt the conven-· tion that a protein molecule will be described with its amino group on the left, then there is a unique string that corresponds to each protein. Proteins vary in size. lbe smallest ones may contain fewer than a hundred amino acids. The largest mny contain thousands. A typical protein may contain between 300 and 500 of them.

rJlh

-c-~-~-<@§O:H I

?

•I

I

,,,

•

~-<;-COOH

----

1

'?!'!'

FIGURE K.l Putting amino acids togethea· to form proteins.

964

Appendix K

Applications: Computational Biology

While: a protein can he described as a two-dimensional (primary) structure that is simply a chain of amino acids. every physical protein also has a three-dimension~} structure that is formed as the amino acid chain folds and wraps around itself. Tius three-dimensional shape is called the protein's secondary structun Q. The secondary structure! of a protdn is determined by its primary structure. as well as by environmental factors. such as temperature. Each protein has a natu ral secondary structure. and it must exhibit that structure in order to perform its func.•tion within a living organism. Sometimes. when the structure is brokcn. as for example. by changing the temperature. it can be rebuilt if the natural environment is restored. Sometimes. however. it cannot For example. the proteins in a cooked egg cannot be ••uncooked" and returned to their natural structure. Proteins that have formed abnormal secondary structures will typically behave abnormally. For example. it is believed that an accumulation of abnormally shaped proteins called prions is responsible for causing mad cow disease. llle work of a protein is done at some number of specific locations called funcllonal sites. It is there that other molecules can attach to the protein. In order for a protein to do its job. it must be folded so that its functional sites are exposed and the sites themselves must exist (i.e.• contain a ·sequence of amino acids with the ch~mical properties that are necessary for whatever job the site is required to pcrfonn). But it turns out that some variations in the exact amino acid sequc.nce that makes up a protein molecule can be tolerated without affecting the ability of the protein to function correctly. Such variation can be intnxluced by mutations. So if we examine a particular protein, for exampJ(! the blood protein hemoglobin. in multiple organisms, we will find similar but not identical molecules. The similarity among related molecules may be able to be described as a set of motifs. where a molifis a relative&y short region that has been conserved (left unchanged) by the evolutionary process. If the same motif occurs in very different organisms. then it is likely that it is significant. in the sense that it corresponds to a sequence whose structure is necessary in order for the protein to function properly.

K.1 .2 DNA DNA is the blueprint for living organisms. Each molecule of DNA is composed of two strands. held together by weak hydrogen bonds and arranged as a helix. Each of the stmnds is made up of a sequence of nucleotidcs. each of which in turn is composed of three parts: deoxyribose (a sugar). a phosphate group, nnd one of four bases, shown in Table K.2, along with the symbols that arc used Ill represent them. The four hasf!s are divided into two chemical classes. alsu shown in the:: table below. Each base has a complement, which is the other base in the same class. So A and Tare complements, as arc C and G. When a double strand uf DNA is examined as a sequence of hasc pairs (one nucleotide from each strand). ea~h base is paired with its compte~ mcnt. Figure K.2 shows a fragment of a DNA molecule. Table K.2 The nuclcotides that make up DNA. T\•pt Base S\·mbol Adenine Thymine Cytosine Gunnin~

,.

A

('

(j

punnc purine pyrimidine pyrimidine

K.1

A (Very) Short Introduction to Molecular Biology and Genetics

965

FIGURE K.l A DNA double helix.

111e sequence of hase pairs in the DNA of an individual is called the individual's genome. Since the DNA of individuals from the same species is almost identical, the genome of nn individual can be considered t.o be representative of the species, so we can also talk about the genome of a species. DNA molecules encode the program (set of instructions) that an organism uses to manufncture the proteins that it needs. Since a protein is a sequence of amino acids. the program that builds it can be encoded as a sequence of subprograms, one for each amino acid in the sequence. There are 20 amino acids and only four different nucleotides. so it takes a sequence of three nucleotides to specify a single amino acid. Such a sequence of three nucleotides is called a codon. There are 43 = 64 different codons and only 20 amino acids, so there is redundancy in the way that amino acids are specified. Some amittO acids arc described by more than one codon. So. in particular. note that some changes to a codon will have no effect on the protein that the codon defines. A sequence uf couons that contains the blueprint for a protein or some other important molecule (such as RNA) is called a gene and is said to cock for that protein or molecule. The DNA of an individual is organized as a set of double-helix strands called chromosomes. The human genome, for example, is arranged into 46 chromosomes. Sexually reproducing organisms are diploid. meaning that the chromosomes in all but the egg and sperm cells occur in pairs. Each organism inherits one member of each pair from each parent. Generally both members of the pair contain the same sequence of genes. although there may be exceptions. For example. humans have 23 chromosome pain;, of which 22 are matching. In addition, females have a pair of X chromosomes, while males have one X and one Y. Differences between individuals within a species are the result of differences in their genes (as well as differences in environmental factors). When a gene occurs in more than one form. those forms are called alleles. So. for example in humans. there are three alleles (called A. B. and 0) of a gene that codes for an important blood protein. Each person possesses two genes for this blood protein (one from each parent). Those two genes may be the same o~ the~ may he different. So each person's genotype (the actual genes they possess) for th1s trmt.n~ust be one of the six values: AA,AB,AO,BB, BO,and 00. (Order doesn't matte r.) lnd•vtc.luals with two identical genes. (i.e.• two genes that correspond to the same all de) are called htmror.ygo11s with respect to that gene. Individuals with two different genes arc call~d heteroz.J'gous with respect to that gene. ·n1c .ohservable lratts of ~m indi~idual represent its phenotype. The phenotype is dctermmcd by the genotype m a vanc.ty of ways. Sometimes a single gene is responsible

966

Appendix K

Applications: Computational Biology

for determining a trait. Sometimes several genes play a role. Sometimes one allele ~s dominant while others are recessive. In that case. in hewrozygous individuals, the domtnant allele is expressed and determines the observable trait, while the recessive allele has no affect. although it can be passed on to offspring. For example. people have a gene that determines whether their earlobes will he attached to their skull or hang freel). The free carlohes allele is dominant and the attached earlobes allele is recessive. So anyone with attached earlobes must be homozygous and possess two genes for attached earlobes. Fortunatdy. many disease-causing alleles. for example the one that causes cystic fibrosis. are recessive. But some, for example the allele that causes Huntington·s disease, is dominant. Not all observed traits are determined by the simple dominant/recessive model. For example. in the case of the ABO blood proh:in. none of the alleles is dominant. Any individual who possesses an A gene will have red blood cells with antigen A on the surface. An individual with a B gene will have red blood cells with antigen Bon the surface. A pctson with one of each will produce both antigen A and antigen B. A person with ndther the A nor the B gene (i.e... someone who is homozygous with two 0 genes) will pro<.luce neither antigen. S.o there are four phenotypes: A (corresponding to the genotypes AA and AO), B (corresponding to the genotypes 88 and BO), AB (corresponding to the genotype AB). and 0 (corresponding to the genotype 00). While genes are the key to the function of DNA. most of the DNA that is present in the chromosomes of living creatures codes for nothing. For example. about 97% of the human genome is noncoding. A small amount of that no ncoding DNA appears to serve some function. for example in regulating the activity of the coding regions. But we do not know what. if any. function is served by most noncoding DNA. Noncoding DNA is important when we compare DNA sequences across rclutcd organisms since mutations in nonessential DNA can occur without arfecting the fitness of the organism. So. whi le functional DNA sequences are more likely to be conserved across individuals within a species and across related species. other segments may vary, sometimes substantially. These variations make it possible tu do DNA testing to identify individuals. They can also be used to infer genetic closeness of spccics:·nte more changes there are in the DNA sequences, the longer ugo the species shured a common ancestor.

K.1 .3 RNA RNA is chemically very similar to a single strand of DNA.l11ere are two important differences: •

The four bases that are present in RNA nucleotidcs ar~ adeni ne (A). guanine (G), cytosine (C) and uracil (lJ). The first three are also prc~cnt in DNA. The last. uracil. occurs in place of thymine. C and G are complcmt:nhuy Uust as they are in DNA). A and U are also complementary.

•

RNA nuclcotidescontain a different sugar molecule (ribose) than do those in DNA. ln a living cell, RNA plays several important roles. including:

•

Messenger RNA transports the encoding of a protein from the cell 's nucleus (where the DNA is) to the site of protein synthesis.

•

Transfer RNA transports individual amino acid molecules to the building site dur-· ing protein synthesis.

K. 1 A (Very) Short Introduction to Molecular Biology and Genetics

967

FIGURE K.3 An RNA molecule folding.

•

Ribosomal RNA makes up a substantial part of the ribosomes. the cell's protein factories. Catalytic RNA functions like an enzyme and is involved in a variety of cell functions.

•

RNA molecules. unlike DNA ones, do not form douhle strands. But a single RNA stntnd dues fold around itself. creating a secondary structure that is important to the function of the molecule. In particular, if two subsequences that contain complementary bases fold so that they align next to each other. they form hydrogen-bonded base pairs in much I he same way that two DNA strands do. We call these bonded subsequences stems. The umtligned subsequences between the stems will then form loops, and unaligned subsequences at the end wilt simply hang out as tails. Conliidc:r. for example. the RNA sequence AAGCCACUCUAAAUCUCCUGCCAGUCG. lt is likely to ((lid to form the slructure shown in Figure K.3. The lines indicate hydrogen bonds hetween complementary bases in a stem. In this example. a stem containing six base pairs has formed. Stems generally arrange themselves in a helix. in much the same way that the paired strands in a DNA molecule do.

~('. 1.4 Genetics and Evolution The gcnomcs of living creatures are under constant evolutionary pressure from three natural stochastic processes: • •

mutation. which occurs when DNA is imperfectly copied during reproduction, natural selection. which occurs when fitter (i.e., better-adapted) individuals have higher ~urvival and reproduction rates than their less well-adapted cousins, and • genetic drift .Q , which occurs when the relative frequencies of competing alleles changes. either as a result of sampling bias or as the result of random (i.e.• not based on fitness) events. Sampling bias is particularly likely to occur in small populations (and m~ny species ~xist in relatively small. isolated populations). Suppose that there cxtsts a gene With two alleles. a and A. that occur with frequency .5 each. Assuming .scx~al rcprod~ction, one gene from each parent is passed on to the next ge~c~auon. fh~t gene 1s chosen a~ random from the two that the parent possesses. Jt IS likely that m the n~xt .generatton, the relative frequencies of the two alleles will not ~c e~actly .S. but wtllt~stead be something like .4955 or .5045. At the next generatu.m. 1t could become shghtly more weighted. And so forth. If it ever reaches 011. there is no going back. Genetic drift also occurs when some individuals are selected more .hccause they are lucky than because they are fit. Suppose. for example, that there 1s Cl flr~ th·"'. mgcs ~hrough. n large portion of the natural habitat of a population and all the mLhvtduals m the hrc-ravagcd area are destroyed. A small group that

968

Appendix K

Applications: Computational Biology

happened to be lucky enough to be outside the burned nrca is left to reprod~ce. There is no guarantee that the distribution of allc:les in that population is identical to the distribution in the original larger group.

K.1.5 Summary The computational theory that we have described in this book is well-suited to biolog· ical applications for two important reasons:, • proteins. DNA. and RNA can straightforwardly bt! represented as strings. • naturally occurring stochastic processes apply to th~m.

K.2 The Sequence Matching Problem There now exist sophisticated techniques for analyzing DNA and protein molecules and for determining the sequence of amino acids or nuclcotides that they contain. This process is called mapping or sequencing. In 2003, the human genome project Q com· pldcd its g\>als of describing the approximatc:ly 3 billion base pairs and identifying the approximately 30.00() genes that make up human DNA. The genomes of other organ· isms. ranging from the bacterium £. <:oli to chimpanzees, have also been mapped. as have many of the proteins that are found throughout nature. Consider a set of relatc:d organisms. They will share many of their proteins, as well as much of their DNA and RNA. But, when molecules arc looked at as sequences of amino acids <.lr nucleotidcs, they will be similar but not identical for two reasons: •

Mutations can occur during reproduction. Those changes will. in turn. cause changes to the proteins that make up the organism. resulting in individual differences within a species. as well as the more significant differences that can he observed across species. •, Many mutations have no effect on the function of the protein. DNA. or RNA mole· cutes that they modify. So they may be passed on. without effect. to the descendants of the original organism in which they occurred. Thus even very similar organisms may pnsscss different DNA and difl'ercnt protdns. · Proteins are very long molecules and DNA strands are even longer. So, to analvze th~m. it mukc:; sense to break them apart into shorter (hopefully signil1cant) subsequences. There arc a variety of techniques available for doing this. The important thing about these techni4ucs is that they cut up long sequences in predictable ways. For exam· pie. there is a family of enzymes called restrictiun (!nzymcs, each of which cuts doublestrnnded DNA only in places that contain a particular nucleotide sequl!nce. So if similar DNA molecules are suhjected to the same proccs.'ieS. they will produce similar sequences. Comparing DNA. RNA, or protdn sequences can help tt) answer the following kinds of questions: •

Given an organism (from which we have a DNA. RNA. or protein sequence). how is it related to other organisms'! The closer the match between the sequences. the more clnscty related the organisms are likely to be.

K.2 The Sequence Matching Problem

969

Given a DNA. RNA, or protein sequence, what function does it perform? What parts of the sequence are important. in the perforn:tance of the function? If the ~­ quence is very similar to sequences m other orgamsms, or even other molecules m the same organism. and we already know what those molecules do. we may have an answer. Or. if we find similar sequences in other molecules, perhaps it is possible to figure out what all of them do by looking for similarities in what we know about all of them. • Given a DNA. RNA. or protein sequence from a diseased organism and one from a healthy organism. what is the difference? This may help us understand the cause or a potential treatment for the disease.

•

There now exist large databases Q of known protein, DNA, and RNA sequences from known organisms. But a substantial computational problem remains: comparing sequences to each other. Generally the goal of such comparisons is to find related sequences that are evolutionarily as close as possible to each other. In other words. we want to find sequences from which the current sequence could have evolved with the smallest number of mutations. Because the sequences can be described as strings, another way to describe the problem is as string matching. Because related sequences are not necessarily identical. the problem is one of approximate string matching. For a good introduction to the variety of computational techniques that have been developed to solve this problem, see (Durbin et al. 1998]. A II of these techniques rely on the notion of alignment.l\vo or more sequences are aligned if they are arranged in a way that minimizes some notion of evolutionary distance. One strategy. for example. is to maximize the number of positions that contain the same amino acid or nucleotide. But each alignment algorithm exploits its own specific measure of closeness. For example, some rate pairs of amino acids as either identical or different, while others consider amino acids in the same class (as shown in the amino acid table we presented above) to be more similar than ones in different classes. Some alignment algorithms are global: They try to align entire sequences so that as many symbols as possible match. Other algorithms are local: They try to find smaller subsequences that match exactly or almost exactly, even if the rest of the alignment produced by that match isn't very good.

EXAMPLE K.1 Aligning Amino Add Sequences Consider the four amino acid sequences:

AGHTYWDNR, AGHDTYENNRY, YPAGQDTYWNN, AGHOTTYWNN In this simple case. a straightforward alignment is:

AGH T YWDNR AGHDT YENNRY YPAGQDT YWNN AGHDTTYWNN

970

.Appendix K

Applications: Computational Biology

Similar (but not necessarily identical) sequences can be aligned as shown above. Such sequences are probably related (both genetically and functionally). But they may differ as a result of three kinds of mutations: •

substitution: For example. in the third sequence above. Q has been substituted for H. In evaluating the closeness of an alignment. some substitutions are typically assigned higher distances (and so alignments that incluc.lt! them are ranked as less close) than others. Sometimt!S the distance is based on chemical and structural propertie5 of the corresponding amino ucids or nuclcotiucs. But in one common scheme, the PAM Q family of distance matrices for amino acids. the distances are based on the probability that one amino will replace another during t!Volution.llle Q/Hsubstitution in the third sequence has very low evolutionary distanct! (i.e.• it is very likely to occur). while the E/W substitution in the second se4uencc has u high one.

• •

deletion: For example. the 0 is missing from the first sc4ucncc above. insertion: For example. an extra T has been inserted in the fourth sequence above.

ln the rest of this section, we consider a collection of techniques for solving the fol· lowing problems:

1. Given two sequences. find the best alignment between them. 2. Given one new sequence and a database of known sequences, find the known ones that are most likely to b~ related to the new one.

3. Given one or more patterns that describe related fumiJies of sequences. compare the pattern to an individual sequence or to a database of sequences and find close matches.

K.3

DNA and Protein Sequence Matching Using the Tools of Regular Languages We begin by describing three techniques that can be used lo solve alignment problems involving proteins and DNA! •

Deterministic FSM!) are used in BLAST. a very fast query engine that operates on huge databases and solves problem 2 above.

•

Regular expressions can be used to specify motifs. or patt~rns that describe a related set of scqucmccs. TI1csc patterns ate used to solve problem 3 a hove.

•

Hidden Markov models can be used both for pairwise matching (rroblem 1 above). as we ll as to model known families of sequences and to compute the probability that other sequences are related to that family (problem 3 above).

All of these techniques make the assumption that the mutations that caused the variation among related sequences occurred. for the most part. indepcnc.Jcntlv of each other This independence assumption makes it possible to rely on techniques th.at are based 0~ models of regular languages. In Section K.4. we will consider phenomena, such as se· quence evolution and seconc.Jary structure prediction of RNA, in which distant parts of

1<.3

DNA and Protein Sequence Matching Using the Tools of Regular languages

971

a sequence. interact with each other. To solve such problems it will be necessary to use more powerful formal structures.. such as stochastic context-free grammars.

1(.3.1 Finite State Machines in BLAST ·n 1e first problem we will consider is the following: Given a protein or DNA sequence, find other sequenc-e s that have high-scoring local matches with it. The BLAST Q (Altschul. Gish , Miller. Myers, and Lipman 1990) search engine is widely used to solve this problem. ll1ere arc several versions of BLAST. Some of them now do global as well as local matches.l11e BLAST family of search engines uses a variety of heuristic techniques to search huge databases and find the sequences that are most likely to be biologically significant matches for the query string. The core of the original BLAST system is a three step process:

1. Select a reasonably small number w (usually between 4 and 20). Examine the query string and select the substrings of length ·w producing local matches with it.

th~tt

are good candidates for

2. Using the set of substrings found in step 1 (called words). build a DFSM that can he used to scan a database of known sequences and identify those sequences that have high-scoring local matches with one of the words. Run the resulting DFSM nguinst the dutnbase and find the sequences that match. 3. Examine the matches found in step 2 and see if it is possible to extend any of 1hcm to build longer matching sequences. Assign scores to all of those extended matches and return those sequences with a local match score above some predetermined cutoff. The implementation of step 2 can take advantage of the observation that we made at the end of Section 6.2.4: Given a finite set of keywords K, it is straightforward to build a DFSM that matches aH instances of clements of K. If we had to view the set K as an arhitmry regular expression. build an NDFSM. and then convert it to a deterministic one. the construction step could take lime that grows exponentially in the size of K. But a v
The

~,

3 .2 Regular Expressions Specify Protein Motifs Given " collection of proteins that are already known to be related and a sequence alignment of them, we can define a motil' a conserv · ac1·ds, th a t 'J • . e d sequence o f'ammo

972

Appendix K

Applications: Computational Biology

we mav know or hypothesize corresponds to some function of th~ proteins in which th~se sequences occur.

EXAMPLE K.l Detecting Motifs Suppose that we have the following fragments (which are smaller than the ones that are gene rally considered. but they illustrate the idea):

YYNKNR ESGHDT M DTTTTTSWQS RGSDTTT PDMT AGP TT WRNT KQGEDTT DGMT AGMDTT KPQT RWNS M D T

1 2 3 4

5 6 7 8 9

This example appears to contain a short motif, shown in hold. Notice that small variations (in particular. things that we beli~ve do not affect function) may be allowed.

Once we have defined a motif. we woulll like to search to find occurrences of it in other protein sequences. So we need a notation in which to describe! the motif. Regular expressions are often used to do this. Not all systems us~ ¢~a\:l ly the same: regular expression synhtx. hut most .Q usc somethin!t similar to the synlax of Perl (Appendix O) or Python.

EXAMPLE K.3 Describing Motifs w ith Regular Expressions Continuing with the example we started above. we see that: •

TI1e motif begins with G or with M. but if it starts with M then the M is the first element of the sequence.

•

Position 3 is D. but it is optional.

•

Position 4 is T. which may be· replicated some number of times.

•

Position 7 can be anything except P (purine). It would be hard to hypothesize this from this small a sample, but such an ohscrvation could b\! part of a motif that was dcrivell from a larger sample. so we include it here to illustrate the idea. P('lSition R must be K. S or T.

•

K.3

DNA and Protein Sequence Matching Using the Tools of Regular Languages

973

EXAMPLE K.3 (Continued)

To describe this motif with a Peri/Python-style regular expression, we would write!:

CGI(AM)) X 0? T T* X X (AP] (KST] xis a shorthand for {ARND .•. ). so it matches any amino acid. The character A means the left end of the sequence if it occurs outside []. Inside O, it means not one of the characters listed.

The cxuct syntax that is used to specify these patterns varies across systems. For ex~ ample. l'rosite Q, a database of motifs, uses a different format, but the structure of the pattern expressions is the same.

1( .. 3.3 HMMs for Sequence Matching While regular expressions are a good way to define motifs that are derived from a set of very similar sequences. the technique degrades when the set of sequences becomes fairly diverse. As new sequences are added to the set, more and more special cases must be included in the regular expression. and there is no way to indicate that some of thcm

*<.w:ction 1><.vectim1'!> * But we immediately have two problems. both of which are fundamental to the use of context-free grammars:

Both occurrences of .~c!clion 1 in part2 must he based on the same: theme as section, in part1• Similarly for the two fragments Jabded .vt•aion 1. So. ignoring the musical variations that are Hllm\ed. we have the same prohlcm that we hm.J in trying to write a conh.:xt-fre~ grammar for WW = { 1l''ll': 'We {a. b l *J nnd for describing the type checking. constraints imposed by programming languages like Jnva. • There ar~ what linguists call suprasegmcntal properties (like intonation) that are not capturcu by the breakdown of' the piece into sections. l'art 1 starts in a home key. There is then movement away rrom the home key through section~ and section3. In part 2• th..: motion is in the other direction. back to the home key by the end of the second occurrence of sec.·tion 1. These larger prupcrtics cannot be described by a context-free g.rummar that must generate each suhconstituent independently of all the others.

•

As a result. most grammars of music exploit more powerful rormalisms. primarily ones that are c4uivalcnl in power to unrestricted granunar~ and to Turing machines. When rules can cuntain multiple symhols in their left-hand sides. contextual construints can be described. We ·n mention one example. taken from a much larger body of' wurk whose goal is to dcscrib\!. for Wt.'Stern musicians. nonwestern musical traditions. Formal systems have been appealing in this arena since people arc working in traditions with which they arc not familiar. It thus becomes important to construct theories that can be validated without appeal to intuitions that arc probably not well formcll . l Kippen am.l Bel 19921 presents a grammar that dcscrihes the rules by which impwvisation is done in North Indian tabla drumming. One uf the constituents in the grammar is named V1• lt can be realized in one of three ways. numed cllrcr.ti. and -. It turns out that the way it is realized is dcp~ndent on its context. So the grammar contains rules such us the ones shown in Figur~ N.J. Kippcn and Bel call these rules "contt:xt-scnsitive:· as indeed they are. But the overall grammar th
CtlniCXIS that !ti.:O\!r
contexts 1hat general~.: tllw:

C\11\tcxl ~

tllm \ ' 1 tllm-t dlwti tllrn t/1111 \1 1 V, -t tllw II \I I etc,

kt V 1 tr -t Ia dlw rr kt \' 1 V I -t 1.:1 t/1111 V,

tllw V 1 lr -+ t/1111 • lr '"'" ~~, t llltl -+ cllJa • dh11

FIGURE N.l

i!IC~

Contcxt-dcpcnd~:nttahla

Ihat generate-:

etc.

rules.

N.1

Music

1033

The language of music appears to share many structural features with natural languages and many of those features are hard to describe without substantial formal power. Linguists have; over the years. defined grammatical frameworks that have the formal ~xpressive power of unrestricted grammars but that offer particular kinds of representational advantages for describing linguistic data. Musicologists have adopted many of those.! formalisms and put them to their own use. Starting in the mid 1960s. the transformational grammar formalism described in [Chomsky 1965] became popular. The key idea in transformational grammar is that a sentence has a deep structure that corresponds naturally to its meaning. It also has a surface structure (namely what we hear or read). While these two are not the sarne, there is a systematic relationship between them. A transformational grammar, then , has two parts: a set of context-free rules that define the deep structure of a sentence and another set of transformational rules that describe how the deep structure is to be realized as a linguistic form. It is the existence of the transformational component that gives these grammars the expressive power of an unrestricted grammar. Transformational rules can be used to do many kinds of things. For instance. they can move constituents around. A simple example from English illustrates this idea. Consider the sentence, ..What did Mary see in the park?'' The object of the verb .. see" is the interrogative pronoun .. what.'' It got moved from its natural place after "'see" to the sentence-initial position in which interrogative pronouns occur in English. Transformations that describe the relationship between an underlying structure and a surface form appear to be a natural way to describe many aspects of the structure of many kinds of music. This notion was articulated by Heinrich Schenker Q (see, for example, ISchenker 1935]) well before it became possible to write formal grammars that could be run on computers. So the transformational model has been used to descrihe many genres of music. For example,[Baroni et al 1984] describes work on grammars £or the melodies of Lutheran hymn tunes. the melodies of French chansons, and Bach's chorale harmonies.lCamilleri 1984] describes a similar effort to construct a grammar for the initial phrases of Schubert's Lieder. Other work has modeled folks tunes and jazz. Early efforts at defining grammars of music were conducted by hand. People examined music, attempted to extract recurrent patterns, and then wrote grammars that descrihcd those patterns. But if the goal is to construct a grammar that accurately describes the patterns that characterize an existing corpus of work, then it makes sense to usc as large a corpus as possible and to extract the grammar rules automatically. Once the grammar has been written, it can be used as the basis for an analysis uf the existing corpus or as a way to generate new works. Many genres of classical Westcs:" tonal m~sic. have been analyzed in this way. Twentieth century jazz has been extcns1vely studted m order to extract rules that describe successful improvisation strategies. One of the most widely used musical pattern extractors and composers is David Cope's EMI system (Cope 1996). lt has been used to write music in the styles ·o f Bach. B~cthoven. Mo~art, Stravinsky, Gershwin, Joplin. and Cope himself. EMI represents Jts .gr~mmars ~~a formalism called an augmented transition network (or ATN ). A~ A1 ~ IS c~senttally a finite state machine except that arbitrary computational aclltms. mcludmg rc1:ursive calls to the state machine itself, may be performed

1034

Appendix N

.Applications: Art and Entertainment Musi'c and Games

as tests on any of the arcs that connect one state to another.. ATNs bave the same

computational power as do Turing machine~. Rule-based systems that do not behave exactly as grammars do have nlso been used both to analyze and to generate music.This approach is appealing when dealing with musical genres that appear to be describable by some fixed set of rules. Some kinds of early Western music. for example. have that property. Sixteenth century countc.!rpoint is a good case study. Fux's Gradm: ad PtlmtJsswn. published in 1725. offered a set of rules and guidelines that characterize species counterpoint IFux 1725]. (Schottstaedt 1989] de~ scribes a program for generating counterpoint according to the rules that Fux laid out. Associated with each rule is a penalty that shouh.J attach 1o nny composition each time it breaks the rule. So, for example, Schottstacdt lists the following rules and their penalties: Parallel fifths arc not allowed. (Infinite penalty.) Avoid direct motion to a perfect fifth. (Penalty is 200.) Avoid unisons in two-part counterpoinl. (Penalty is 100.) Avoid tritoncs ncar the cadence in Lydian mode. (Penally is B.) SchoustaedCs program composes counterpoint hy· running a best-first search. process through a space of po~sible compositions. Jt uses the penalties as its heuristic function for evaluating competing notes. Another example of a rule-based system that composes in a well-understood style is CHORAL (Ehcioglu 19lJ2]. which creates bar~ .monies to accompany the melodies of Bach chomles. One way to represent production rules is as a Prolug program (as described in M.2.3 ). For a description l)f this approach to dcscrihing music. see [Schaffer and McGee 1997). A simple example shown there is the following rule, which encodes the definition of a passing tone:

nonharmonic_tone(passing_tone) approached(step), resolved(step), · registral_direction(same).

....

Other approaches to writing musical grammars have·also bt!en exploited. For example. L-systems ~::. can he used to model a process in which hasic musical characteristics, like pitch and duration, arc encoded as symbols and then transformed into music.

N.2 Classic Games and Puzzles By the late 1940s. the idea of a stored-program, electronic computer had been born and development efforts were well underway. By then. also, Alan Turing and others had ~egun talking about com~uter che~s. At the start of the 1950's. Claude Shannon. someumes called the father of mf~rmat10n theory, puhlished the first paper (Shannon 1950) on computer chess. By the m1d 1Y50s. there were. several programs that played at 1 t partial games of ches.~A few years later,AJlen Ncv.'Cll and Herbert Simon. pioneerS in the t~as _ new field of artificial intdligcnce, wrote, "Within ten years a computer will be the wor~;s

N.2

Classic Games and Puzzles

1035

chess champion. unless the rules bar it from competition," [Simon and Newell 195K]. The rules imposed no such bar. But a computer did not beat the human W~)rl~ chess champion until l997, when Deep Blue '-!! beat Garry Kasparov. Why dtd tt take so long? The theory or complexity that we describe in Part V does not apply directly to chess. Nur docs it apply to Go. Suc.Joku. or most of the other classic games and pu~­ zles with which we are familiar. It doesn't apply because most of those games have fixed-size hoards and the theory we have built describes complexity as a function of increasing problem size. But chess and its cousins are hard for the same reason that the sort of combinatorial problems to which our theory does· apply are hard: There does not appear to exist an algorithm for solving the problem without searching a large space ur individual moves. For example, in the middle of a chess game, the branching factor (the number of alternative moves) is about 35. It is typically estimated thM playing master-level chess requires looking ahead about eight moves. So choosing a move using lookahead requires examining about 35~ ~ 2 • 1012 moves. For the game of Go q, the situation is even worse. The branching factor is greater (over 300 at some points in the game) and it is substantially harder to write an evaluation function that can examine an intermediate board position and determine how good it is. While tables of opening and closing moves can help reduce search (at least in the case of chess). no one has yet [ound a way to avoid search in the general cas~ of these classic games. Complexity theory can help to explain the observed difficulty of writing programs to play these games if we generalize the games to boards of arbitrary size. This isn't a totally off-the-wall idea. For example. one early ( 1956) "chess"-playing program used just a 6 x li board in order to make the problem tractable. Using this idea, we have derincd. for example. the language that contains descriptions of generalized chess configurations from which the current player has a guaranteed win.

•

rJ . 2.1

CHESS = { <'h>:· b is a configuration of an n x guamnteed win for the current player}

11

chess board and there ·is a

Nim In Exmnple Zl.5. we considered the game of Nim. We can· describe Nim as a language as follows.

•

NIM = { < 1>> : his a Nim .c onfiguration {i.e.~ a set of piles of sticks) and there is a ,guaranteed win f(.l f the current player}

R!.!callthat. .in ~x~1mple 21.5, we described a straightforward algorithm for deciding, in any game of .Nun. whether the current player has a move. that leads to a guaranteed win. Our algonth~n docsn•t ~e~uire actm~lly searching the space of possible move sequences. It proves that NIM 1s m I' (and.m fact, in L). People get bored pretty quickly with Nim (particularlv once they know the trick).lt appears that, for games and puzzles. gremer complexity a desirable feature.

is

1036

Appendix N

Applications: Art and Entertainment: Music and Games

N.2.2 NP-Complete Puzzles The generalizations of many popular puzzles have been shown to he NP-complete. We'll mention a few of them here.

Sudoku Sudoku o is typically played on a Q x 'l grid. 'Illc goal is to complete a partially filledin grid. such as the one shown in Figure N.2. The rules of the game require that, in any solution. each of the digits I through 9 must occur exactly once in each row, in each column. and in each marked 3 X 3 subgrid. ln order to he able to talk about the complexity of a Sudnku problem as a function of the size of its input. wc generalize the stamlurd Sudoku game to an 11 x n grid, where n is a perfect square. We then require that each of the numbers from J ton occur in each row. in each column. and in each of the " suhgrids divided as above. Then, to turn the problem into a decision problem. we restate it as follows. •

SUDOKU = { : h is a configuration of an " x under the rules of Sudoku}

11

grid and b has a solution

The problem becomes one of deciding whether a solutiun cxists. Cle.arly completing the grid can he no easier than deciding whether such a completion is possible. Sudoku is typical of a large class of one-person games or puzzles. It can be solved by a straightforward search process in which it suffices to find a single solution. In the case of Sudoku. we need to find a single way of filling in the grid that meets the requirements of the game. Of course. experts lt!nd to do very little actual scnrch as they exploit a variety of heuristics that prune the space of alternatives, often to the point that no search is required. Those heuristics break down. however, for larger versions of the puu lc. Whether or not heuristics work to simplify the problem some of the time. the search approach suggests a straightforward way to show that SUDOKU is in NP: It can be verified in polynomial time hy a deterministic Turing machine that considers a proposed solution and simply checks that all the constraints arc satisfied. Sudoku is a variant of an older puzzle called Latin squures o. which considers only the row and column (but not the subgrid) constraints.l11c prohh:m of dl.!ciding whether

5

')

4

:!

1

k

4 ~

7

..

2

2

"'

fl

s

h

3 X

h

3

1 fl

X

\)

:i

4

FIGURE N.2 A game of Sudoku.

N.2 Classic Games and Puzzles

1037

c a b a g e d

0

g

FIGURE N.3 A crossword grid.

an instance of ann x 11 Latin squares puzzle has a solution is NP-complete [Colbourn 19X4). SUDOKU has been .shown [Yato 2002] to be NP-complete by reduction from the Latin squares problem.

Crossword Puzzles Next we consider the problem of constructing a crossword puzzle. More specifically, define: CROSSWORD-PUZZLE-CONSTRUCTION = {: it is possible to form a valid crossword puzzle by filling every blank square of 8 with a symbol in 'I in such a way that every contiguous string of letters, running both horizontally and vertically, is a word in W}.

•

For example. if n is 3. all squares of Bare blank. and the list of words is age, ago, beg, cab, cad, dog. then it is possible to construct the grid shown in Figure N.3. CROSSWORDPUZZLE-CONSTRUCfiON is NP-complete. (See [Garey and Johnson 1979).)

Instant Insanity The last example that we'll consider is the Parker Brothers® puzzle known as Instant Insanity® ~.1l1e puzzle consists of a set of four plastic cubes. There is a set of four colors c. and each of the six sides of each of the cubes is painted one of the colors in C. A solution to the puzzle is an arrangement of the cubes into a single column in such a way that. on each of the four sides of the column. each color in C appears exactly once. We can describe lhc generalized puzzle as the language: •

INSTANT-INSANITY = {: the blocks in 8 can he stacked in a single vertical column and, on each of the four sides of the column, each color in C appears exactly once}.

INSTANT-INSANITY is NP-complete [Garey and Johnson 1979].

N .. 2 .3 Two-Person Games But now consider two-person games, like chess and Go (and checkers and backgammon and so forth). Suppose that we have the fragment of a game tree shown in Figure N.4. To make it easy to follow the discussion that we are about to present, we·n assume that we always evaluate ~mves from a single perspective. We'll pick the perspective of the first player. whom w~ 11 call playert. Let's say that, at A, it is player1•s turn. Player1 will consider its allcrnat1~es a~d attempt to find the best one. We'll call this a maximizing step. But thl!n somcthmg d1fferent happens at the next step. Player2 gets to choose a move. So suppose thnt pla.ver1 chooses to move to C. Then player2 will choose moveD orE or

1038

Appendix N

Applications: Art and Entertainment: Music and Games

tn:IX

mill

max FIGURE N.4 A game tree when there are two players, a

maxunizcr am.lu minimizer. une of thdr alternatives. When it doc-S ~o. it will chl)OSc the-worst move (from player,·s perspective). So we'll call this a minimizing level. Then it's player 1's turn again and it will ngain attempt to maximize. and so forth. Now suppose that phl)'t!t1 is considering a proposed move scqut!nce, say A , C, E.G~ Play£•r1 can't verify that it can win by choosing to go from A to C just by examining that sequence. Maybt: A. C, E.G docs lead l<) a win. But if A , C, D leads to a loss, then player1 cannot be guaranteed a win just by choosing tll move to C. The problem is that the second move isn't under its control. 'S o CHESS is not obviously in NP since we no longe r have a simple verifier for it (of the sort that we have for SUDOKU). Another way to think about the reason that we have failed to show CHESS to be in NP is to think ahout a nondeterministic dcciJcr (as oppllSt:d to a deterministic verifier). Recall how a nondeterministic Turing machine works. ll accepts a string iff there is any path that accepts. But now consider the prnhl~m of deciding whether there is a guaranteed win for p/ayer1• When it is playr!r1·s turn. it suffices to find any move that is guaranteed to he a win for it. But. when it is plt~yer~·s turn. it is necessary to show that et•ery move that p/C1yer2 might chouse l!Uarnnlccs the win we seek for player1• So we can't solve thl! CHESS prohlcm hy (nondetcrministicatly) finding a single path that leads to a win , There arc. however. two w~1ys we can solve it. TI1c first is to conduct. deterministically, a depth-first search of the tree of possilllc moves. Each branch ends whenever one side wins or a draw is Jeclared. As the search hacks up. it can compute win/lose/draw values for each intermediate nodt! once the vulucs for all of its daughter nmJes arc known. At each maximizing step. the win/lose/draw value is the maximum of the values of the daughter nodes. At each minimizing step. it is the minimum of those values. If the starting nodt! gets assigned the win value. then 'it corresponds to a guaranteed win for plt1yer1and the algorithm will accc.:pt. Othe r'Wise. it will reject · Let's analyze the space complexity uf this approach. The search retluire!; C'J(r-12) space to store one hoard configuration. Th
N.2. Classic Games and Puzzles

1039

time as they were the first time. So the length of any search path is bounded by the number ot' bml~d configurations it may encounter. How many such configurations are there? That depends on exactly what we mean when we say that we g~neralize the ga~e to an 11 x 11 board. If we just add board squares but do not add any p1eces, then f(n) lS a pol~~ nomiul. If we take that definition for CHESS, then it is in PSPACE. And, in that case, 1.t can be shown to be PSPACE-complete. lf, on the other hand, we make the perhaps more reasonable assumption that the number of pieces grows with n, then f(n) grows exponentially with 11 and CHESS appears not to be in PSPACE.It is, in that case, EXPTIME-complctc [Fraenkel and Lichtenstein 1981]. An ann lysis similar to the one that we just did for CHESS can also be applied to the game of Q. First we must generalize the standard game to a board of arbitrary size.. poing that. we can define the language:

Go

•· GO = { : ''is a configuration of an n X n Go board and there is a guaranteed win for the current player}. GO is PSPAC'E-hard (Lichtenstein and Sipser 1980]. so no efficient algorithm for it is likely to exist. Saying anything more precise about the computational complexity of GO is complicated hy the fact that the rules of the game vary and the details of the rules appear to affect GO's complexity class. For example. using Japanese rules and the simple "ko" rule (which makes it illegal to make a move that causes the board configuration to return to its immediately preceding configuration), GO is EXPTIME-com· plcte l Robson 19N3). Using some other rule sets, the complexity class of GO remains an open question. It is worth noting. however. that while. CHESS and GO do not appear to be in PSPACE, there are two-person games that are. They are games for which it is possible to place. a polynomial bound on lhe. number of moves that can occur in one game. So. for example. Amazons, Hex, and Othello are all PSPACE-complete Q.

rJ .2.4 Alternating Turing Machines An alternative approach to analyzing languages like CHESS is suggested by the observation that. at alternating levels of a game search tree. we need to ask, "Is there at least one winning path from here? .. and then. "Are all paths from here winning?" Define an altemlllillg Turi11g machine to be a nondeterministic Turing machine with one difference: Whenever a nondeterministic choice is made, the machine specifies the condition under which it will accept. It may choose to: • •

accept whenever at least one daughter path accepts, or accept only if all daughter paths accept.

Note th~~t it can. make di~ferent ch?ices at different branch points. We can easily build an Hltcrnut~ng Tunng ma~h~ne. l? decade CHESS. At maximizing levels. it suffices to fmd .011 c accc~tlllg path. At I?tmmtzmg levels. all paths must win. What as the co~lplcxlty of !he alternating Turing machine that decides CHESS? To ans~cr that ~uesllOn. we begm by defining a set of complexity, classes for alternating Turang mm:hmcs: ·

1040

Appendix N

Applications: Art and Entertainment: Music and Games

AP (alternating polynomial time): For any language L. LeAP iff there exists some alternating Turing machine M that decides tin polynomial time. APSPACE (alternating polynomial space): For any language L. Le APSPACE iff there exists some alternating Turing machine M that decides L ·in polynomial space. AL (alternating logarithmic space ): For any language L, L e ~L iff there exists some altl!rnating Turing machine M that decides L. in logarithmic space.

• • •

A stgnificant result is that alternation buys l'inc complexity class lChandra, Kazen a nd Stockmeyer llJX 1). More specirically: • • •

AL = P: Alternating l'ogarithmic space is exactly as powerful as dctcnninisti!: polynomial time. AP = PSPACE: Alternating polynomial time is exactly as powerful as polynomial space. APSPACE = EXPTIME: Alternating polynomial space is exactly as powerful as exponential time..

Without assuming a polynomial bound on the number of moves that may have to be considered. we know that CHESS is EXPTIME-compldc. So, given the result we just described. we can conclude that it is also APSPACE-complctc. Similarly. since, using Japanese rules and the simple ko rule. GO is EXPTIME-complctc. it is also APSPACEcomplete. Alternating Turing machines are more useful. however. in cases where the complex.. ity of a problem is not already known. When the ullernating Turing machine model naturally matches the structure of a problem (as it does in chess, for example), it may be easier to determine the problem's alternating Turing machine complexity than it would be to determine its complexity with respect to the standard model. But then. using the result we just described. its standard complexity class can be inferred.,

N.2.5 Game Programs that Win: The Minimax Search Algorithm So far. we have been considering ways of constructing an ~~ act unswcr to the question. ··Can plt~yer 1 win'!" and we've seen that it is hard. But the analysis that we have done suggests that there may be a way to get an approximatt: answer and, at the same time, play a very good game of chess or checkers. The backbone of most programs that play two-p~:rson gmnes is the minimax algorithm (so named because it allcrnate.ty minimizes and maximizes the values of the .moves that arc considered). In principle. minimax could be used to search a complete game tree. following each move sc4ucnce until it c,nds in a win. a loss. or a draw. In practice. however, complete trees are too large for that to be feasible. For example Claude Shannon (mentioned above as the author or the first paper on computer ches~ in 1950). estimated that a typical chess game takes 40 moves (hlr each player) and that there is an average of about 30 choices at e<1ch point. Su the total numhcr or moves that would be examined in a complete. game tree would be about (30 • 30)~1' ::; 10''~. Modern estimates put the size of a chess game tree at ahuut 10 1zJ. For comparison. the num·bcr of atoms in the observabk universe is c~tim~1ted to he ubout tuN,

N.2 Classic Games and Puzzles

1041

max

min

FIGURE N.S A game tree in which a static evaluation function has been applied to each leaf node.

So practical programs look ahead as many moves (typically called ply) as they can. giv~n the time they are allotted. They examine the game configurations that rcsull. applying to each a heuristic function, generally called a static evaluation Junction. that measures how promising the configuration is. Then they choose a move based (Hl those scores. When mininwx is used in that way, it becomes a heuristic search technique of the sort described in Section 30.3. Like other heuristic search algorithms. it is an approximation technique. It isn't guaranteed to return an optimal result. But optimality isn't generally required in games. It suffices to find a move that is good enough to win. To see how mi11imax works. consider the two-ply search tree shown in Figure N.5. Bdow each leaf node is shown the value of the static evaluation function applied to that node. We. will assume that all evaluations are done from the perspective of playerl, whose turn it is to move from position A. High scores are good for pluyer1• ·n,c. job of minimax is to generate this tree and to apply the static evaluation function to each leaf n chooses from position A is a maximizing player. the player who chooses from B is a minimizer, who will choose to go to G. which makes B's score 2. Positions II and I send their scores up to C. Again. this is a minimizing level, so the choice from Cis I and Cs score becomes -3. Positions J and K send their scores up to D, whose score hccoml:!s -9. From A, then. there is a choice of three moves: B (with a score of 2). C (with a score of -3), and D (with a score of -9). From A. the maximizing player will choose 8, which sets A's score to 2. Figure N.6 shows the tree after all scores have been propagutcd.

max

min f'IGURE N.6 A game tree in which values have been propagated up the tree.

1042

Appendix N

Applications: Art and Entertainment: Music and Games

The version of minimax that we will describe searches a game tree. It will not check to see whether any of the configurations that it examines have already been considered somewhere else in the tree. It is possible to implement a graph-based version of the algorithm that does notice and collapse subtrees whenever possible. MinimtlX will exploit three functions that encode the facts about the game wear~ playing: • The function move-gen(nmle) returns a set of nodes that correspond to the game configurations that can result from making a single move from the configuration stored at node. It may implement a legal move generator or it may incorporate addition al heuristic information and generate only moves that appear plausible from the current position. • The function srmic(node) returns the result of applying a static evaluation function to the configuration contained in node. For chess. for example. static might be a measure of piece advantage. mobility, ~nd control of the center. We will define static so that it always evaluates from the perspective of plt1yer1• • The function tleep-enouglz(notle. deptlz-su-fur) returns True if the path that currently ends at node is as deep as we want to go in the search and Fal.\·e otherwise. In its simplest form, cleep-erwuglr just counts to some fixed depth limit and then returns True. In more sophisticated implementations. it considers additional factors. For example. it's not a good idea to stop searching at a point that is likely to be half-way through a piece exchange. Each node of the tree that minimat explores will have three fields: po.'iition (a description of a game configuration).score (if it is known), and be!il·succ(•s.wr (if one is known). A straightforward way to describe minimax is as a recursive procedure. To choose a move from position A . minimax will invoke nwve-gerr(A ). Then it will call itself on each of the resulting nodes. Whenever a branch gets deep enough. swtic will evaluate its final node and the resulting value will be passed back up to the parent node. If minimax is called from the perspective of a maximizing player, then each call it makes on successor nodes will be from the perspective of a minimizing player. and vice versa. Minimll.t will not return a value. Its job, given A. is to fill in A's sc:ore. And. unless A is at the last ply of the search tree, it must also rill in A 's be.vt-:rucces.'iar. To implem~nt these ideas, we'll define a proccdurc.gmne-secm:lz, whose job is to create the first invocation of mininwx. Game-searclr can be called with a starting game configuration. or it can be called at any point during a game. in which case it is given the current configuration as input. It calls minimtlX with its input configuration, a depth (so far) count of 0. and a persp~clive.that of play~r 1 • When Rtmrt!-.'it'tlrch ends, best-succes.'iUr of the starting node will be filled in with the move that should be chosen (if there is one). We can .stale gmne·.'fettrch as:

game-search(current: configuration node) = Call minimux(ntrrem. 0. pluyer1) . Then minimax is:

minimax(position: config~r.ation node, cleptlr: integer. per.'ipec:tive: player) = t. If tlet•p-errough{posttton, depth) th~n set position's score to .'itatic(position) and return ..

N.2

Classic Games and Puzzles

1043

2. Set sucn'.'i.WT.'i to the set returned by move-gen(position). J. tr .wt<"C"f!.'i.WJr.\· is empty then there aren't any moves to make so set position's scur~ to statk(position) and return. 4. If sttcces.wrs is not empty then examine them and choose the best. To do this. (.}o: 4.1. For each clement move of successors do:

Call minimax(move, depth

/* Fill in a score for move.

+ 1. opposite(perspective)).

4.2. Considering the scores of all the moves in successors:

If per.'ipecti'lJe = plcryer1• setdwsen to an element of successors with the highest .'icore. If pers peel i'Ve =· pia yer2• set chosen to an element of successor.~ with the lowest :'lcore. 5. Set positio11•s ~core to the score of chosen. 6. Set position,s lu•.w-sun:esse>r to chosen. Heuristic in[urmation plays an important role in minimax since it enables the algorithm to choose a move without having to search all the way to the end of the game. And il's possible also that heuristics are embedded in move-gen. But minimax is still required to search all the subtrees that move-gen creates. Now recall A*. the heuristic search algorithm that we introduced in Section 30.3.2. Jt exploits heuristic inforn1ation that enables it to ignore large parts of its search space. We'd like a way ror mi11imax to do the same thing. For example. look again at the game tree of Figure N.6. Once the subtree rooted at 8 has been examined, millinwx know:s thnt p/cl)'tt'l· a maximizer, is guaranteed a move, 8, with a score of 2. But it continues and looks at the alternatives in hopes of doing better. It considers C next. und first notices that C has the successor H. whose score is 0. At this point. with· out looking at uny of H's altermllives. pltl)'er2, a minimizer, knows that it is guaranteed that Cs score cun he mnde no higher than 0. Player, already knows it can get a 2 from B. Suit can decide immediately that it won't go to C. It doesn't have to examine I or any othl!r successors that C might have. We,ll ca112 an alpha cutoff lt corresponds to the lower hound that a maximizing player can count on. Of course, in our simple example. using an ulphu cutoff lets us skip expanding just a single node. But suppose that the search were g,oing to eight ply. TI1en using the cutoff would save expanding a possibly large subtree under/. Next, notice that, if we extend the tree another ply. we can see that it is possible to exploit a second threshold, which we' ll call a beta cutoff lt corresponds to the upper bound that a minimizing player can count on. ·~c minima.\·-lt'it~-a-/J-pr~ning ~lgorithm takes advantage of the alpha and beta cutorfs that w~ have J~st dcscnbed.lo see how it works. consider the game tree shown in Figure ~.7. ll~c c~mcc of a move from Dis made by a maximizing player. soD gets n score ol 5. wh1ch 1s passed back to B. 'lhe minimizing player who chooses from B is thus gm~rantccd a score .or.~s. So hl'ta is set to 5 and passed down to E. The choice I'ron• £ es made hy a maxmnzmg player. J gets a score of 7, which is passed back to E. E is till'~ guanmtccd H score nf >7. But the IJ£'ta value at f.' is S.cuu esponding to the fact

1044

Appendix N

Applications: Art and Entertainment: Music and Games

max

min

(5)

(2)

(7)

(2)

X

X

FIGURE N.7 A deeper gam\! tree. in which a/p/J(J anJ ht•ta cutoffs can be used.

that the minimizing player above E is guaranteed a score of no more than 5 by going to D. Because 7 > 5. a beta cutoff occurs aml the rest of the successors of E need not be considered because the move t'rom B to E will never he chosen. At this point. we know that B's score is 5 and it is passed buck up to A. So we know that A is guaranteed a score of ~s. That becomes the value of ulpha. lt is passed down to C. The choice at F is mac..le by a maximizing player, so F's score becomes 3.lt is passed up to C. SoC is now guaranteed a score of ~3. But A is guaranteed a score (as reflected in alpha) of at least 5 by going to B. Because 3 < 5. an alplw cutoff occurs and the rest of the successors of C can he ignored because the move from A to C will never be chosen. Summarizing. we see that alpha cutoffs correspond to guarantees for maximizing playl!rs. They get set (to reflect the best option so far) at maximizing levels and they get usec..l to cut off search at minimizing levels. Similarly. beta cutoffs correspond to guarantees for minimizing players. They get set at minimizing levels and they get used to cut oU search at maximizing levels. Both cutoffs must he providl!d at both levels. The order in which moves are examined now matters. So we will let mcwe-gen return an ordered list, rather than a set. of moves. As before, we'll assume that the player to move first is a maximizing player. Since no information is available yet.llfpha at the! top node starts out as the minimum value that static can compute. Similarly, beta at the top node starts out as the muximum value that static can compute. We can describe 1{cmre-seurch-a-{3 and minima.\·-wirh-a-{3 as follows: game -.~em·clr -a- f3(mrre111: configuration

node) 1. Return minimax-witlr-a-fj(currc•nt.

o.. player1• minimum value Jtlllic can l:ompute, maximum value ~·wtic can compute). minimax-witlt-a-fj(puJition: configuration node. depth: integer, perspective: player

11lphn. bew: integers)

=

•

1. H deep-fiWllglr(position. depth) then set po.ttirivn's score to stmic(position) and return.

N.2 Classic Games and Puzzles.

1045

z.

St!l sm·ces.mrs to the list returned by move-gen(position). 3. lf :waessor.~ is empty then (there aren't any moves to make so) set position's sc:ore to sratic(posirion) and return. 4. If suc:cessor.'l is not empty and perspective = player1 (maximizing) then do: 4.1. Set clwsen to the first element of successors. 4.2. Do the following for each element move of stlccessors. stopping if a cutoff occurs: Call milli11WX(move, de pch + 1. opposite(perspective), alpha, beta).

If mow's sc:ort > chosen's score then set c:lw.ven to move. lf nrove ~'l score> beta then all other elements of successors can be skipped. Cut off and exit the loop. If move's score > alph.tl then set alpha to move's score. 5. U sw·ces.mrs is not empty and perspective = player2 (minimizing) then do: 5.1. Set dwsen to the first element of succe.uors. 5.2. Do the following for each element move of succe.~ors, stopping if a cutoff occurs: Call mi11imax(move, depth + 1, opposite(perspective), alpha, beta). If move's score < chosen's score then set chosen to move. lf moves score s alpha then all other elements of successors can be skipped. Cut off and exit the loop. If move's score < beta then set beta to move's score. 6. Set pmdrion's ,.;core to the score of chosen.

7. Set position's best-successor to c.:hosen.

l11c c.liff~rcnce in performance between minimax and minimax-with-a-{3 depends on the tmlcr in which moves are considered. If the best moves are considered last, minimax-witll-a-13 cannot prune any subtrees. It gets its best performance when the best moves arc always considered firsL Of course, if we always knew the best move before we searched the tree. we wouldn't need to search at all. But it is possible to use heuristics. such as the function .vtmic, to make informed guesses about the order in which to consider moves. Without using alpha-beret cutoffs. minimax must search O(b"). nodes, where b is the average branching factor of the tree and n is the depth of the search. U the hcst move is always considered first. then minimax-witlz-a-{3 searches CJ(/,"12 ) nodes (Knuth and Moore 1975}. So, in a fixed amount of time, it can search to twice the depth that minimax can. Neither heuristic search nor the use of cutoffs alters the fact that games like chess and go arc hard, as suggest~d by the fact that their generalizations are EXPTIMEcomplct~. But a chess-playmg program has beaten a reigning world champion. No Gu-playmg program has .Yet come close to doing that. Why does Go appear harder than chess'? The answer 1s that constants do sometimes matter. There came a time when there exis~cd exactly enoug~ computing power to search a chess game tree fast enough to wm some games agamst a champion. The game trees for Go (which is

1M6

Appendtll N

,,J,

t\J)J\.',all\ plu\eJ on,, ltl h~ 1'1 ~ •• .arc hu,htc r .• nJ Jc
,.,

h

r"'"""'

N.3 Interactive Video Games llucc: uf the tcc h ru4uc:' th .at \Ate h .aH Jt"u'"·J en thl\ h--.•" .uc """'d~ u~J an th~ '"'\M\• ..uu\."tton ••I mh:t.a\.'11\c (umpuh: r ~ ''""' '-\\ ~,.· II f1l4.' flhon c.kh ••f thnn "udl) . .,ut then: '"much murc th.1t ,,uJ - I''' ·• nh•f '-' uomph: l.-c n'''c: '''"-u''''"' ''' th~ and ••th&."r h:c hnhl'h:'- \C C .an~ ~~~...s '"'"'"- 'Lkh '" I' 1\;•mp.mJ.arJ ~~~\. un the r"k ,,( :uta· f•~a .tl Jill&.' lla ~"' n"c m g.unc dn dt~pmcnt

'""'''hot.·

N.3.1 finite State Machines Hmtc ''·''"' m.h: hanc' .arc u~J tn 11 \.Uac t ~ "' "" ·' ' ' '" ,k ftn \' lh•- t h.l\hU ,,1 mtc~tl\'c: )!.unc'- In~~~ . we rnc:nll~tn tlnc c\llmpk. th"· u~ uf .an I ' ' ' h• ,k\Cltl"C the hch.a''''"" o( .a \U('(cr · rl.a~ln~ ruh.•t In tha' ~·,·tum . "- c ' ll m c nh••n .-nuth•·• \l .an~ mtcr;adi\C ~.arne' au: '" ucturcd ,., luutt· ~t;atc nuu h10\"'\. ( Htcn the\ cue de· tcrnuna,llc m.a.. h•nc'\. I he aJ\,ant.a~c ••f Ul ~~h '' llwt the\ .:uc ptcJa~t:ahk lha' mcaM th.tt hum.an pb~ c " ~'" lc..rn hu"' the ~un c, "'"'~ .an.J thcu pcrt,trm• .an..:c rd.ah~c tu the m.adunc'- C)th,·r ~am c' U"C tlttflJdc tmuu't" 1l\C&r *han· tagc L\ th.at the~ arc unprc,Jt&.'1 .thlc ~.the~ tc.ttc tn11t c p lo1u,ahk ' 't~"-ncnh and IU'C thu.' more ••Jlf"CO!hfll( tu pi a ~ In etthcr ca,c , the 't:tte" of the m .tdunc '''"'" 't,.•n•l t•• \ltu"t'''"' ttnn 10 \Aohtc h :an .t~l.· nt 1\ J,untc lut nut JtttnJ{l \A•mc thm ,: I he '"I' Uh lH the ffiA~hanc ~-,. fC\f'I(IO.J tn C\' Cflh Uut . IU\t hlc the \lfl J.[ k c h.:U.ICtc: n th~t "'"'the anruh " ' th~ Sift\• plc l.tn ~Ud\!C · fC ll)tOillll~ ffi,lchtn C\ "-C h.a\ C' ~ 4.' 0 huahhi\IC , the tni'Ub thai corrc"pt'nJ tu ~.amc cH' nh c.an t\C r c prc~· nt c J "' ~ htutc "<'I ... 1 he,,.~ .. ,~h. "hen u-.cJ tn th" '""IC' \1 , ''nut h• J,·~aJC' a l"n ,; u;a~,, hut rothcr tu ''"'~"be: the t'Ch.t\hlf uf the ~ .tm c ~l f''"l~ •h " " ,h.•ulJ ' 4 \ tho~t • c a rc U\lllJ hnat~ ,lah: tt .an,,IUCCf\ r.athc:t lh ,an n HalhiOl:\.. lh11t t c tmtntil••~ \ ' '- h••'A C'Io"C f , tAfcl) U'<~ tft lhtS ,nnt&.'\1 In h ~ urc SM. •c 'h••• c\.tntplc In m1 .. t tc .1l .:~n-.c' lh.c n.tftrl.au:r ch~ar-.'1~n I~ I"< ') .uc cunlft~llcJ "" ~ \Cl uf I ' ' "· c"" h ,.f • h" h ~., •hn ,me _.,('C"Cl uf ~""-\'• Jut 1-ur C\.tmpl.: . unc nliJthl two '"'P''""hk h•t • r.-.r--n '-C.'"""~ "' aon< mt~hl ch,~"'C :a t.u~ct . ;1nJ .,. thuJ nutothl \'untrul 0\o \ c m c nt In aJ..Jtlttllft rcalt,hrC 4 c1rl "'""'' uU~n c\.pltnt rr.•hahJith~' ~'-~IIC't;alcd ••th the ";J"'thttn" In Uk h nU
"'',,.,.""I''"'"

I'''"

t\·r''""'.

''m""'•'"

"tn'

,,r

N.3.2 Heuristic Search and the A• Algorithm C'un"Jcr the rruhlcm of comruting the mo\"Cm,·ntr- of the nonrlaycr characters (NPCs) an an lnlCUk... I\'C ttamc. One \'el)• ~mrle arrn'M'h to SOI\·ing this problem is just to let the ~hRf'k... cn nltt\'C an •hal arrears tO be the COrt"CC'I direction and hO('C that they don't run lnlu C~l«let. Ocner all~· • ~ner arrruach il to rtan a route in advance. taking ad\·an· tattc of maa,. and uther terrain informallun. Ucca~ rath• arc ~ang cnmruted an real time. u a game ii being rla~·cd. • ·ant an dhnent nlt:unthm fur romrutantt the ~~ path to be followed for each player. The A• nltturtlhm. •• dc~-n~d an Sccunn :14U.2. is • ·iddy used to d() this Q . Recall thAI the A• alttunthrn imrlements a kind of heuristic sc:arch. It makes usc of h\o ktrh uf ,.._,., anfurmahon:

•·e

'"'"l'

lllc that arc a•\4-.iatcd •ith e•i\ nl the c1p:raton that ttcneratc new states: In the ft.Amc cn\1fnnmcnt. the wmrlnt ···~· to define thc1ic ~ u. the ac1ual distance Uawkd If we dtt that. •'C11 find the ~ortcst path frum the current stale to a goal. Uut ctthcr fK1nn hntdc1r. dU.Ian« can .,... added. Rw eumrk. IUO\'n that go lhrout:JI danlt"f'tU' tcmh~' an he ~'-wtlflcd ha~hcr ""'t than mm-n that go through areas that arc kn•••n tu ~ ._fc.. Mu\~ that f'C\'Cal to the enemy our kno•·lcdtee of the lc¥· raan ant,..- a'''ttned ha,:hcr co.b than Rkl\'0 that gi\'C nothintt •••ay, and so fonh. • I:.•Untatc~ ut future co•t~ a1 pro\'idcd h~· the hcuri!.tic function: These: arc guess· e' 11"-•ut hnw n'uch 11 cml to mo\'c from the current fltate to a goal. The ~im· rlnt • ay to nlllke •uch C~hmatc• ll to UM: lOme kind of JComelric di,lllnCC. If

•

,..,u

1048

Applications: Art and Entertainment: Music and Games

Appendill N

travel in any direction is allowed. Euclidean distance (the length of a straight l~e between two points) may be a good (and admissible) heuristic. U travel Is restncted to roads that fom1 a grid. then Manhattan distance may be a good (and also admissible) he uristic. The Manhatta n distance hctwec:n two points (x 1• Y1) and (.tz, .Yz) is lx1 - x 21+ ly1 - .\1:! 1. In other words. it is the distance that must be traveled in the plane if no diagonal moves are allowed. h is named after the arrangement of (most of) tht! streets in the borough of ManhcHtan in New York City. Manhattan distance is often used in games that dun 't use roads hut that model their environments as square or hexagonal grids. Of course. as in dcscribjng re~tJ costs, cost estimates can also include estimates of factors other than distance. For example. paths through mountainous terrain may cost more than flat paths. Dangc:rous paths and thusc that require the expenditure of scarce resources will also have high cost. In real-time games, path-finding must be done very efficiently. It may be more important to find a good path quickly than to find an optimal path several minutes from now. So inadmissible heuristic functions that do a good job of pruning the search space arc fairly widely used.

N.3.3 Rule-Based Systems that Control NPCs in Interactive Games In M.3. we sketched the origin of rule-based systems as descendants of one of the earliest formal models of computation. We also described their basic architecture. Rule-based systems (or RBSs) deserve me ntion in the context of game development because they have proved to be useful for describing the behavior of nonplayer characte rs (NPCs) in interactive games. Rules can be used to define a variety of kinds of behaviors. For example. they can be u~cd for proble m solving, in much the same way they are used by expert systems. There might. for instance, be rules such as:

If: The laser is gone, Then: Someone else has it It is night and the powcr is out, Then: The enemy cannot see me. If:

Rules can also be used to define behaviors. ngain in much the same way in which they are used in expert systems. There might. for instance. he rules such us:

lf:

I have just hit a wall,

Then:

Turn to the right and keep walking.

If:

A grenade has just been thrown,

Then: Duck.

N.3

If:

Then:

Interactive Video Games

1049

I have a shotgun and I do not have ammunition for a shotgun, Find ammunition.

Rule-based systems of varying degrees of sophistication are being used to build NPCs. Simple systems consist of sets of rules such as the ones we just described. Another approach is to start with SOAR (described in M.3.2), a comprehensive, rule-based cognitive architecture whose goal is to model human behavior. The idea is that agents based on SOAR will exhibit convincingly human-level behavior.

APPEND

X

0

Applications: Using Regular Expressions atterns are everywhere. Regular expressions dcscrihe patterns. So ifs easy to see that regular expressions could he usdul in a wide variety of applications. We have already discussed some important uses of regular expressions, including the description of lexical analyzers that are used by context-free parsers and the description of protein motifs that are to be matched against protein sequence databases. In Q.l.2, we'll describe their use in defining XML document types. In this appendix, we briefly highlight the use of regular expressions in programming environments and, more broadly. in computer system tools. A quick look through the manuals for many programming languages (such as Perl Q. Python Q, and Java Q), as well as systems utilities (such as the Unix file searching program grep Q or the mailing list management system Majordomo Q ). will turn up a chapter on regular expressions. (or "regexes"). But we must be careful. While these systems share a name and some syntax with the pattern lunguage that we described in Chapter 6 and they were certainly inspin:d hy that language. they are, both formally and practicany. quite different. In these systems, ·it is possible, as we'll see below, to write regular expressions that describe languages that are not regular. The added power comes from the presence of variahlcs. whose values may be strings of arbitrary length. For example, consider the regular expression language that is supported in Perl. Table 0.1 shows some of the most important constants and opt!rators in that language. Notice that all of the operators that exist in the regular expression language of Chapter 6 are present here, although union is represented differently. Some new operators, such as replication. word boundary, and nonword houndary. are simply convenient shorthands for patterns t~al can easily be writh:n in our original language. Much of the rest of the syntax IS necessary because of the large character set including nonprinting characters. that may occur in real h!Xts. But the most importan; difference b~tween the two la~guages is the ability to assign a value to a variable. Then the vanable can be used m a pattern and we can rc4uirc that it match the same substring each time it occurs.

P

1050

Applications: Using Regular Expressions

.-'fable 0.1 1-"

Syntax

abc

albic a*

a+

Regular expressions in Perl.

Name Concah:nation Union (Or) Kleene star At least one

p?

a{n.m}

a*'?

1051

Replication Parsimonious

a+?

Description Matches a, then b. then c, where a. b, and c are any regexs Matches a or b or c, where a, b, and care any regexs Matches 0 or more a's. where a is any regex Matches 1 or more a's, where a is any regex Matches 0 or 1 a's, where a is any regex Matches at least n but no more than m a's, where a is any regex Turns off greedy matching so the shortest match is selected

,

Wild curd Left anchor Right anchor

\d \D \VI

,w \S

,,.,, ,s \'t

,f \b

'b

,s ,o Vtnn

'""" ,ex v:har (tl)

Matches any character except newline Anchors the match to the beginning of a line or string Anchors the match to the end of a line or string Assuming a collating sequence, matches any single character in range Assuming a collating sequence. matches any single character not in range Digit Matches any single digit, i.e.• string in [0-9] Nondigit Matches any single nondigit character, i.e.• [.... 0-9] Alphanumeric Matches any single ..word" character, i.e., [a-zA-Z0-9] Nonalphanumeric Matches any character in [....a-zA-Z0-9] Whitt! space Matches any character in [space, tab. newline, etc.) Nonwhite space Matches any character not matched by \s Newline Matches newline Return Matches return Tab Matches tab Formfeed Matches formfeed Backspace Matches backspace inside[] Word boundary Matches a word boundary outside [] Nonword boundary Matches a non-word boundary Null Matches a null character Octal Matches an ASCII character with octal value nnn Hexadecimal Matches an ASCII character with hexadecimal value nn Control Matches an ASCII control character Quote Matches char. used to quote symbols such as . and \ Store Matches a•.where a is any regex, and stores the matched string in the next vanable Varia hie Matches whatever the first parenthesized expression matched Matches whatever the second parenthesized expression matched For all remaining variables

~~·--~--------~----------------------------~

1052

Appendix 0

Applications: Uslng Regular Expressions

u i!\ possible to write many useful regular expressions. withuut exploiting variables. as we can sec in the next two examples:

EXAMPLE 0.1 Spam Detection The following regular expression matches the subject field of at least some email messages that are likely to be spam:

\badv\ (?ert\) ?\b

EXAMPLE 0.2 Email Addresses The following regular expression scans text looking for valid email addresses:

\b[A- Za-z0- 9_%-]+CO[A-Za-z0-9_%-]+(\.[A-Za-z]+){1,4}\b But. using variables. it is possible to do things that would not be possible without them. as we can St!C in the next two examples.

EXAMPLE 0.3 WW Recall the language WW = (ww: 1.o e {a. b }*} . The following regular expression matches all and only strings in WW: A([ab] *)\lS The pattern [ab] * can match any string of a's and b's. Whatever it matches is stored in the variable 1 (because [ab] * is the first pattern in parentheses in the expression), 1l1en \1 will match a second occurrence of the same string of a's and b's. The anchors " and S force the pattern to start at the beginning of the target string and end at the end of it. So this pattern matches all and only strings in WW.

EXAMPLE 0.4 Finding Duplicated Words Suppose that we want to proof read some text that we are writing. One common error is to duplicate a simple word like the. The following regular expression matches duplicated words:

\b( [A-Za-z] +) \S+ \1 \b

Applications: Using Regular Expressions

1053

By using variables. it is possible to define languages that are not regular.lbis means that it is no longer possible to compile an arbitrary r~gular expression into a deterministic finite state machine that can dccidc whether the expression matches against an input string. While every deterministic finite state machine runs in time that is linear in the length of its input. we cannot make this claim for a regex matcher when variables are allowed. In fact. it can he shown that regular expression matching in Perl (where variables arc allowed) is NP-hard. To see why search may be required. consider again the rcgcx of Example 4n.3. When it is attempting to match against a string w. it may have to try each position in was it searches for a place to stop matching [abler and ·start matching \1. Most regular expression languages. including the one we just described in Perl. !.\Upport nt.1t just string matching but also string manipulation. For example. it is easy to write a Perl expression that works like a production in a Post system (see Section 18.2.4 ). The first part of the expression defines a pattern to be matched and the second part describes a string that should be suhstituted for the matched substring. The Perl syntax for string substitution is: s.,ariahle = - slrc•gexlre.mltJ; When such a command is executed, the first substring in $variable that matches rel-!c'X will he replaced hy rc.>:m/r.lf the symholg (for global) is inserted after the last I. all instances of n•~:ex will be teplaced hy l'l'.'mlt. EXAMPLE 0.5

Deleting Duplicated Words

Continuing with the duplicated word example, we might want to write a substitution command that deletes the second occurrence of any duplicated word (plus the white space in between the words). We could do that. assuming a text string in

$text, al\ follows: $text=- 5/\b([A-Za-z]+) \5+ \1 \b/\1/g;

EXAMPLE 0.6

A Simple Chatbot

Regular expression substitution can be used to build a simple chatbot. For example. supp~se that whenever the user types an expression of the form 15 . we want our chatbot to reply with the expression Why is ? So, on input, ..The food there is awful." our ~hat hot wou.ld reply, "'~hy is th: food there awful?.. We could do this, again assummg that the mput text IS stored m the variable $text. as follows:

$text=- 5/1\ ([A-Za-z]+)\5is\s([A-Za-z]+)\.?S/Why ; 5 \1 \2?/;

f

Applications: Using Finite State Machines and Transducers

I P.1

n this app\!nclix, we illustrah! som~ examples of cmly (i.e., hdorc the advent of modern computers in the middle of the 20th century) finite: state m
Finite State Machines Predate Computers The history of finite state machines (also called finite

stat~:

autl\tnata) suhstantially predates the history uf anything we would now call a "computcr".'lllC Oxford English Dictionary !OED lYXlJilists the following among its definition!\ uf the word. ''automaton":

3. A piece of mechanism having its motive power so cnncc&\lcd that it appears to move spnnt an~o u s ly~ ·a machine that has within itself the power of motion under conditions fixed for it. hut nut by it' (W. B. Curpcntcr). In 17-lHth c~ applied to dot k~. watch~: s. ~tc:.. and

transf. to the Universe and World: nnw usually to figures which simulate the action of Jiving beings. <\S cluck·wurk mice. images which strike the hours on a clock. etc. Autnm:.Ha. in this sense. have l'
P.1.1 The Antikythera Mechanism The Antikythcra Mechanism. huilt in Gn:ece around 80 BC is perhaps thl.! earliest

knuwn example of a sophisticat~:d mcchanic:.1l automaton. Crafted in bronze. it contained <\t lea~t 30 preci~iun g.curs inside a wuodcn cm~c lhat was covered with writing. It wa~ discnvcn:d in tl)l)l . a$ part of a 'hipwrcck niT the Grc~k island uf Antikythera.

1054

P.1

Finite State Machines Predate Computers

1055

FIGURE P.t A fragment of •he Antikyt hera Mech
After ahout 2.000 years under water. the device is fragmented and corroded, as can be seen from tlu: photngr:-tph in Figure P.l.lllUs its exact function is unclear. 11 appears. hnwever. to have heen un astronomical calculator that was substantially more sophisticatc<.l than any others that arc known to have been built for at least another 1.000 years. Using modern techniques .Q, researchers have been ahle to ana· tyze the m!!.chanism ancJ to huilcJ a modd that describes n likely hypothesis for how the mechanism worked. Figure P.2 (a) shows a front view of that model; Figure P.2 (b) shows a rem· view.

p. 1.2 The Prague Orloj Another spcl'tm:ular example of an early automaton is the Prague orloj Q. shown in Figure P.3(a): l1le orlnj is an astronomical clock mounted on a wall of the old town city hali ."Jl1c original clt1Ck was buill in 1410. At that point. it consisted of just an astronomical

(h)

L'IGlJ IU~ I~ 'l\\'U views of ., "'tl·". · · ,• •• .oucn1 rccunstrut11on of the Anukythcra Nkch;lllism.

1056

Appendix P

Applications: Using finite State Machines and Transducers

(a)

(h)

FIGURE P.3 The Pragut: orloj.

dial. whose state was controlled by three gears (with 365, 366
A set of four figures that represent threats to the city: a skcl\!Lon representing death, a Turk, a miser. and vanity. While these figures do not move, they do have moving parts. They are shown. next to the original astronomical dial. in Figure P.3(b).

•

A set of four figures that represent virtues: an angel. a chromcler. a philosopher, and an astronomer. These figures do not move at all.

•

The twelve Apostles.

As the hour is about to chime. the skeleton tolls the bell. Then the Apostles parade through two small doors above the original dial. Then. as the clock chimes. the Turk shakes his head, the miser watches his bag of money. and vanity admires itself in the mirror. lbc Prague orlog. as well as other early mcchanicul clocks. is typical of a simple class of finite automata that accept no input except the passage of time.

P.1

Finite State Machines Predate Computers

1057

FIGURE P.4 An abacus.

p. 1.3 The Abacus The abacus .Q. shown in Figure P.4. on the other hand, does accept input. In one form or another, the abacus has been in use for over 2,000 years. It is a computer whose inputs (bead movements) correspond to the steps required to perform a calculation and whose state corresponds to the result of performing the calculation. Modern computers are. in some sense, finite state devices. since the actual universe (or at least the part of it that we can observe) does not contain an infinite number or molecules that could be used to encode memory. But we model them as Turing machines because there is no a priori upper bound on the amount of memory. New tapes or disks can always be provided. But the abacus is difrerent. When an abacus is built. the largest number it can record is fixed. So it truly is a finite state computer.

p.. 1 .4 Programmable Automata and the Jacquard Loom A loom is a finite state machine whose slates correspond to configurations of the warp thread~

A wcav('r works a loom by throwing a shuttle. wound with the weft threads. back and forth through the warp. By raising and lowering the warp threads, the weaver can create a pattern. The shuttle wilt fly below all raised warp threads and above all lowered ones. A simple pattern can be created from a two state machine: ln the first state. all the even numbered threads are raised. The shuttle is thrown. and then the machine enters the scc
1058

Appendix P

Applications: Using Finite State Machines and Transducers

FIGURE P.S

A Jacquard loom.

Each loop of cards defines a specific finite state machine with n states. where n is the number of cards in the loop. Every such machine has an input alphabet that consists of the single symbol next, which is executed by the weaver after each time the shuttle is thrown. Each time it gets the next signal. the loom moves from one state (warp configuration) to the next.

P.2

The Towers of Hanoi: Regular Doesn't Always Mean Tractable llte Towers of Hanoi problem Q was invented by Fram;ois Edouard Anatole Lucas, who published it in 1M~3 unc.Jer the name of M. Claus. To solve the problem, it is required to move a stack of disks from one pole to another while obeying a couple of simple rules. An example of a starting configuration is shown in Figure P.6.

t'IGURE P.6 lltc Towers nf Hanoi.

1'.2

The Towers of Hanoi; Regular Doesn't Always Mean Tractable

1059

Various stories have been crcat~:d to go along with the problem. One version is the following: 'In a monastery in India there are three poles and M golden disks. each of a different t.lianh.!t.:r. When God created the universe. he stacked the disks on thl' first uf the poles. with the largest on the huttom. The remaining disks were sta\:kcd in order of size, with the smallest on the top. l11e monks were given the task of moving all 64 disks to the last pole. But the disks are sacred, so then~ arc important rules that must be followed. Whenever a disk is removed from a poll~. it must immediately he placed on some other pole. No disks may he placed on the ground or hdd. Further.a disk may never be placed on top of a smaller disk. The monks were told that they must hcgin working immediately. taking turns anlUnd the clock. When they finish. the world will end. It is. in principle. fhlssihlc for the monks to accmnplish this task. The following simple prnccdur~ solves an ~nhitrary Tuwcrs of Hunni problem with ll disks:

/oweno.JIImwitn: p\)Sit ive integer) =

1. If 11

-

I then mow the disk to the goal pole.

2. Else:

2.1. Move the top

11 - 1 disks to the pole that is ·neither the current one nor the goal. 2.2. Move the bottom disk to the goal pole. 2.3•. ~·tovc then - .1 disks that were just set aside to the gmtl pole.

Fortunatdy. even it' the story of the monks and the end or the world is true. no one need wnrry. l Ising the procedure that we just described. it will take the monks 2~ - 1 moves 10 actnmplish the task. So. at one move per second, it would take 584.542J)4n.OlJO year:-.. 22X days. 15 hours. 14 minutes. 45 seconds. That's approximately 6 • 1011 years. The univcr:-.c. on th'-' other hmld. h&~s existed for probably abuut 12 • W'' years (since the Bit! Bang).

~Of t·o~tro;c. this analysis
Th('Y might. 4uitc rcas{umhly. louk t'or a nonrccursive solution. People are quite had at

maintaining ret·ursion stacks in their heads. So imagine the three poles arranged in a cirdc (whether they actually are or not ). 'lltcn let's say that the "next'' pole is the next one in do\:kwise mdcr and the "previous" pole is the previous one, again in clockwise order. A ch.:vcr monk might come up with the following solution: ..

/owc>r.,·ofl ltmoidrch•(m positive integer)· = I. Move 1he smallest disk to the next pole. 2. l!ntil all disks form a single stack on some pole other than the starting t)nc du:

2.1. Make the only lcgalmtwc that does not involve the smallest disk.

2.2. M,we th\! smallest disk to the next pole. If th'-' numhcr of t.li~~s is odd. this technique will mnvc ull of the disks from the starting pole to thl· nl'XI nnc. ll tl~c number of disks is even. it will move all of the disks from the ~tat1ing pule l\) the pi'CVIllUS t)llC.

1060

Appendix P

Applications: Using Finite State Machines and Transducers

This technique seems quite different from the first. It can easily be implemented, even by a child, for small values of n. But it actually requires exactly the same number of moves as does the recursive technique. And no better technique exists. So the shortest solution to the 64-disk problem is very long. Nevertheless. the system of poles and disks can easily be modeled as a nondeterministic finite state machine. The start state is the one in which all 64 disks are stacked on the first pole. The accepting state is the one in which all 64 disks are stacked properly on the goal pole. Because there is a finite number of disks and the position of each disk can be uniquely described by naming one of the three poles, the number of distinct states of the system, and thus of the machine we'll build to model it. is finite. Fmite but not tractable: This system has 364 states (because each of the 64 disks can be on any one of the three poles). The transitions of the machine correspond to the legal moves (i.e., those that satisfy the rule that aU disks must be on a pole and that no disk may be on top of a smaller one). Each transition can belabeled with one of six symbols: 12 (meaning that the top disk from pole 1 is removed and placed on pole 2), 13, 21, 23, 31, and 32. To make the machine as simple as possible, we have left out transitions that pick up a disk and put it right back in the same place. We can define the Towers of Hanoi language to be the set of strings that correspond to move sequences that take the system from its start state to its accepting state. The Towers of Hanoi language is regular because it is accepted by the Towers of Hanoi FSM as we just described it. And it is infinite. since there is no limit to the number of times a disk can be moved between poles. But the shortest string in the language has length 264 - 1 (namely the length of the optimal sequence of moves that solves the problem).

P.3

The Arithmetic Logic Unit (ALU) In most computer chip designs, the ALU performs the fundamental operations of integer arithmetic, Boolean logic, and shifting. The ALU's operation can be modeled as a finite state transducer, using either a Moore machine (in which outputs are associated with states) or a Mealy machine (in which outputs are associated with transitions).

P.3.1 An Adder As an exampl.e, cons~de~ ~ si~ple binary adder, shown in figure P.7.1Wo numbers can be added ~y ad~mg thetr dtgtts ~ght to ~eft. So~~ can describe an adder as a Mealy machine whose mput 1s a stream .or pa1rs of bmary d1gtts (one digit from each of the two numbers to be added). The m~chme has two states, one o~ which corresponds to a carry-in bit of 0 :and the other of whtch corresponds to a carry-In bil of 1. When the machine is r t 't ese .1 . . . . . ent ers thestate cor:respondmg to no carry ( t.e., a carry-m btt of 0). Thas simple one-bit adder can be embedded mto a larger system that adds numbers of any ftxed number of bits. li fO (J()j(l

Ol/1 lOll

00/l

01/0 lU/0 .ll/1

FIGURE P.7 A binary adder.

P.3 The Arithmetic logic Unit (ALU)

1061

p.3.2 A Multiplier Binary adders can also be used as building blocks for binary multipliers. Figure P.8 shows a schematic diagram that describes the behavior of an 8-bit multiplier. The multiplier can be implemented as the finite state transducer shown in Figure P.9.

Multiplicand

Done Control

Start

Shift Right

fiGURE P.8 A schematic diagram for an 8-bit multiplier.

Start 0/Shift

0/Shift

0/Shi(t

0/Shift

fiGURE P.9 The finite state transducer that implements the multiplier.

1062

P.4

Appendix P

Applications: Using Finite State Machines and Transducers

Controlling a Soccer-Playing Robot A finite state machine may not he the best way to model all of the complexity that is required to solve many kinds elf hard prohlems. But an FSM may be a good way to start. There exist good tools for building FSMs and for displaying them graphically so that their structure is obvious to their designers. The process of articulating the states helps designers understand the large-scale structure of their problem. Experiments with an FSM-based prototype can ht.! used to highlight those parts of the design that require more powerful capabilities. So a reasonable llevclopment methodology is: Build an FSM as a lirst shot at solving a pmhlcm. Run it. Decide where more sophistication is requi red. Add it. Experiment again. and so furl h. Lers look at an example of the successful use of this approach. We begin with a statement of the problem: ..The goal of the international RohoCup soccer initiative is. by 2050, to develop a team of humanoid robots that is able to win against the official human World Soccer Champion team. In some sense, the RoboCup challenge is the successor of the chess challenge (a computer beating the human World Chess Champion) that was solved in 1997 when Deep Blue won against Garry Kasparov.'' 17 There exist a number of diHerent Robo~up leagues that focus on different aspects of this challenge. Figure P.l 0 shows a Sonyc1.1' Aibd}'J robot. For several yl.!ars, one of the leagues was the Four-Legged League. in which teams of four Aibos pluyed on a field measuring nm hy 4m. The robots operated fully autonomously. So there was no external control ei ther by people or by computers. Consider the problem of designing the controller for a Four-Legged League team member. Clearly each robot must perceive its environment and then decide how to act. No simple controller will make a rohot competitive with a human player at either task. But a simple controller may provide the basis for a first-generation prototype. Figure P.ll shows a finite state machine .that was use~ to define th~ behavior of an attacking player for the Austin Villa teum Q m 2003.thc llrst year that at entered the Four-Legged com· petition {Stone et al. 2003].

fiGURE P.lO A Sony Aibo robot. 17Th is dcsc:riptinn

is a vcn· ~li!!htly c:~ited wrsion nf the dc~riptiun Oil th-: Rnht)(: · liJl 2liC.16 wchsite

Q.

P.4 Controlling a Soccer-Playing Robot

1063

The robot bas tllllltd

Walking To Unseen Ball

at least ~ withoul sccina Uic boU. aod tbt Global MIIP knows where the b&ll is.

Head Scan ~-----+-:---:--:--::---:-!~ Turning For Ball Enough time has elapsed For Ball wilhoul findin' the ball.

Recover From Kick

FIGURE P.ll A finite state controller for an Aibo soccer player.

The states of this machine correspond to simple behaviors. The transitions between the states depend on input from the robot's perceptual systems: vision and localization, 18 as well as its global map and its joint angles. The states can be described as foUows: •

Head Scan for Ball:This is the first of a few states designed to find the ball. While in this state. the robot stands in place, scanning the field with its head. • 'J\Jrning For Ball: The robot is turning in place with the head in a fixed position (pointing ahead but tilted down slightly). • Walking To Unseen Ball: The robot cannot see the ball itself but one of its teammates communicates to it the ball's location. Then the robot tries to walk toward the ball. At the same time, it scans with its head to try to find the ball. • Walking to Seen Ball: The robot can see the ball and is walking toward it. The robot keeps its head pointed toward the ball and walks in the direction in which its head is pointing. As the robot approaches the ball, it captures the ball by lowering its head right before making the transition to the Chin Pinch Turn state. • Chin Pinch Thrn: The robot pinches the ball between its chin and the ground. It then turns with the ball to face in the direction in which it is trying to kick. • Kicking: The robot is kicking the ball.

iifhesc: descriJMions ore slightly edited versions or the ones that appeared in (Stone et al. 2003).

1064

Appendix P

•

Applications: Using Finite State Machines and Transducers

Recover From Kick: The robot updates its knowledge of where tht! ball is and branches to a nother state. Which state comc~'i next dept:mls on the kick that was just performed. Stopped To See Ball: The robot is looking for the! hall and thinks it has seen it. But it still is not sure it has seen the ball. Possibly tht: vision system has returned a false positive. To verify tha t the hall is actually there. the robot momentarily freezes in pluce. Once it has seen the ball for enough const!cutive frames. it can take the transition to Walking to Seen Ball. If it fails to do that , it returns to its previous state. Note that the Stopped To Sec Ball state is not shown in the diagram. Instead the lahel"Ball is scen",just above the state Walking to Seen Ball, is a shorthand for the actual process of transition ing into Walking to Seen Ball [rom the three states ahove it. If. in one of those states. the robot believes it has seen the ball. it enters the state Stopped To See Ball. Then. if the condition for believing that the ball has actually been seen are satisfied, the transition continues into Walking to Seen BalL

•

To evaluate the conditions on the transitions of this FSM. the rohot controller exploits the following Boolean-valued functions: •

Balllost returns true if the robot is reasonably confident that the hall is lost. It is a sticky version of what the vision system is reporting. So. if Ball lost is true, then it will become false only if the vision system reports seeing the ball for several consecutive frames. Similarly. if Ball Lost is false. several consecutive frames of not seeing the ball are required for it to become true.

•

NearBall is used when the robot is walking toward the ball.lt re turns true when the robot is close enough to the ball to he able to begin capturing the ball with a chin pinch motion.

•

Determine AndSetKick is used when making a transition out of WalkingToSccnBall. It determines whether or not a chin pinch turn is necessary. It also computes the angle through which the robot should turn before it kicks. as well as which kick should he executed.

A p P E N D

X

Q

_Applications: Using Grammars

T

he Ox.fonl Engli.'ilr nkrimwr>' (2"d Edition. 1989) gives. as its fifth definition of the word "grammar". the following:

a. The fundamental principles or rules of an art or science. b. A book presenting these in methodical form. (Now ran:~ formcrly common in thc titles of books.) It goes on to mention the following examples:

• E. Newman's hook. The Grammar of Enwmology, 1856. Owen Jones's book. Grammar of Omamem. uno. W. Sharp. in Ru~clli. 18R2. said. "The young poet may be said to have reachcd the platform llt' literary maturity while he was yet learning the grammar of painting." • An article in Tire Listt•m•r. 18 September, 195M. said, "Reizenstein's dissonances do not make;: one 'sit up' in the way Haydn's do if we attend to his musical grammar:· • An articlt! in 71u! Times, 5 March. 1963. said ...The grammar of the film was estnhlishcd."

• •

We have been using a more restricted. tl!chnical definition of the term. But its wider use is not disc' mnccteo:d from our narrower one. Gram mars. as we have defined them. can be used to describe n wide variety of phenomena. We have already seen that context-free grammars cm1 he used to describe some or all of the structure of: •

Artificial languages that have heen designed to facilitate people's interaction with computers. For example. we've mentioned the programming languages Algol. Lisp. and Java.

•

Naturally uccurring phenomena. For example, we have considered written/spoken langucegcs such as English and Chinese. as well as other natural symbol systems such cas UNA and protein sequences.

In this appendix, we will consider other uses for context-free grammars. We will also mention the use of unc other formalism, the Lindenmaycr (or simply L) system.

1066

Appendix Q

Applications: Using Grammars

Q.1 Describing Artificial Languages Designed for Person/Machine Interaction Imagine that you need a Jan~uagc that can !'Upport p~r~onlmachinc communication or machine/machine C
Q.1.1 Query Languages Query languages allnw users tu write lngii:al expressions that dcscrihc uhjects that are to be retrieved. For ~xamplc. SOL is a wiJcly used qucry language for relational datahasc systems. A simple SOL query is the. following:

SELECT OISTINCTROW A.x, A.y, A.z FROM A INNER JOIN B ON A.x = B.x WHERE (((A.z) -=; "m") AND ((B.w) ="c")) OR (((A.z) = "n"))

There exist wntcxt-frcc grammar!'. that c.lcscrihc the syntax of the various dialects of SOL u . Rc)!.ular cxpn:ssiuns (or regular grammars') me nnt powerful enough because. «mwng other things, they canrw\ des~rihc the lang.ut~g~ of Boolean expressions. For cxamph: nolic~ the usc
E'flrcs,ih! power

O..·.:b i•\11

pn~cllurcs:

c"i~t.:ncc

!{~:gula r l:.m)!Ui!J!l'S

CuntcXI ·Ircc la ngu a!!C'~ D~::cidahlc

lunguagco;

t'l(;U R 1:: Q.1

~

.. ' . , Qt-- t

:e:e-

1t ..

~.,.

•• a

( '11mparing lllfCI! dassco, of language~

..

lkcisiun pnx:cllun.-s:l

.. .., ...

I

.. ¥ .

-1

c flidc n~v ~

l

Q. 1 Describing Artificial Languages Designed for Person/Machine Interaction

1067

Q.1.2 Markup Languages: HTML and XML Markup Jangu~lgcs allow users to annotate documents with tags that identify functional components within u document. By defining standard. agreed-upon tags. markup languages allow multiple users and application progrmns h .) perform operations on the same document. for example. a menu document might contain units that are marked with a price tag.Then a mt>nu formatting and printing pmgrmn cuuld right justify each price on its line. And a restaurant-cataloguing program could extnu:t the prices from the restaurant's men u and use them to assign a price category to the restaurant. The Hyperte"t Markup Language (HTML) Q powers the World Wide Web hy providing a standard langungc in which hypertext documents on the Web can be described. An HTML document is simply a text string. but that text string describes a set or structural clements. each of which is delimited by a starting tag and its matching closing tag. The text between the starting and closing tags will be displayed according to the definition of the dcnu.:nt ch•ss thut is delimited by those tags. Since elements may be nested within other elements. HTML is not regular (for the same reason that th~ language of balanced parentheses isn·t). It is context-free and t:an be uescribed with a t:Outcxt free grammar. Each dement dt:tinition defines a new kind of delimiter (a matched pair of tags). A syntactically valid HTML text must nest the delimiters correctly.

EXAMPLE Q.1 A Grammar for a Fragment of HTML Consider the following syntactically Jegal fragment of HTML. This fragment contains a ul or unordered list (generally displayed with bulJets before each item). The list contains two items (each marked as an 1 i. for list item). the first of which contains a nested unordered list:

• ltem 1, which will include a sublist
• First item in sublist
• Second item in sublist
<1 i>Item 2

This f~ag~cnt ~uul~ have' ~en.gener~ted by the fol1owing context-fret: gra~­ mar (w.htch tgnorcs many details, mcludmg the fact that an li can occur only anside a hst): HTNLtext -

E1e~rent

Element -

UL 1 LI 1

HTMLtext 1

2

/* Text is a sequence

of elements.

• • • (and other kinds of elements that are a 11 owed ; n the body of an HTML document)

1068

Appendix Q

Applications: Using Grammars

EXAMPLE M.1 (Continued) UL -

HTMLtext

;• The

and

tags must match.

LI -

HTMLtext

;• The

and

tags must match.

In HTML, the set of legal tags (e.g.. ul./ul . l i.aml /li) is fixed and determined by the language d~:signers. s.., it can he huilt into a grammar such as the one we just de~ scrihcd. But the idea of annotating text with structural tags is useful in all kinds of con~ texts. not just the display of text on the Web. To exploit this idea. it's necessary to allow users to define new tags to suit their needs. The Extensible Markup Language (XML) o docs exactly that. Users write definitions nf new document types by specifying the set of legal elements and the tags that delimit thcm.ll10sc clements can then he prot:csscd hy application programs. So some tags may be used to indicate how an clcm~nt should hi! displnycll (as in HTML). But others cnuld be used to define fields in a llatahasc. to provill~ a basis for sorting the clements. and so un. In 1.3.2. we mention RDF. a lunguagc for annotating Web resources so th include problem-specific clements and tags.

-

EXAMPLE Q.2 Writing a Document Type Definition

Suppose that we want to define a document type thul will he used for homework assignments handed out to a class. Each such document may contain some or all of the following fields: sequence number. title, due date, body. We can describe this class of documents with the following DTD:



1 / ' ..

#PCDATA (parsed character della ) is a built-in type.

)>

So a homework document may he composed of four kinlls of elements. Each element_ will be. delimited with one of the four tags. seq. tit1e. due. and body. that we JUSt dcftned. An example of a homework dcxumcnt thut is consistent with this definition is:

2

Q ..1

Describing Artificial Languages Designed for Person/Machine Interaction

1069

tlue-+ \#PCDATA~ /due> body~ \#PCDATA

Some XML parse rs. called validating parse rs. check documents to m ake sure th t they contain o nly the ele ments that are denncd in the current DTD. But. for efficie a there are also nonvalida ting parsers. which check o nly that the core syntax has b~~ followed and that tags arc properly nested.

Q.2 Describing Naturally Occurring Phenomena

1071

FIGURE Q.2 A picture drawn with a graphics metalanguage~

XML has been used to define specialized markup languages for hundreds.. and probahly even thousands. of specialized application environments Q .

Q.1.3 Graphics Metalanguages: SVG Pictures.. like the one shown in Figure 0.2. do not look like strings. So sets of pictures do not ~ppcar to he languages. But programs that draw such pictures must be told what to do. Strings in graphics metalanguages provide such instructions. So while the pictures arcn"t strings. th<::ir descriptions are. Many graphics languages exist ~.some as stand-alone languages and others as extensions of more general programming languages and environments. With the advent of the World Wide Web, came the need for standards in this arena, as in many others. l11c Scahtblc Vector Graphics (or SVG) language is one proposed standard. SVG Q. is a language for describing two-dimensional graphics (including interactive and animated graphics) in XML. The following SVG program drew the figure shown above:





SVG can he described with a context-free grammar.

Describing Naturally Occurring Phenomena Many kimJ~ of naturally occurring phenomena can usefully be described using grammars of vanous sorts. We sketch two of them here. We should point out before we.start, thou~h. that. th~re many kinds of naturally occurring phenomena that cannot easily be dcscnhcd Wtlhm the context-free framework. In N.l.2 we briefly mention some of the reasons that they are rarely used in music, for example.

2 .1 oance Q.

Dancers move in three dimensional space and 1· n 1· c t f d r· . . • 1. . • , . tme. on ext· ree grammars e me tangu.lg~;s t Mt a rc composed of one-dunens1·onal t · B h · b ·d · h . s nngs. ut sue stnngs can c use to dcscnhc t e esscnttal characteristics of many kt'nd f d . ·r b d fi · s o ances t we start y e mmg

1072

Appendix Q

Applications: Using Grammars

a set of primitives (which we'll then usc as the alphahet for our grammar) that corre· spond to the hasic moves that a dancer might perform.

EXAMPLE Q.S A Grammar of the Foxtrot When dancing a foxtrot, one can take either slow (S} steps {that take two beats) or quick (0) steps (that take a single beat). Each step may start with either the left (l) or the right (R) foot. And each step may move forward (f). sideways (W), or hackwards (B). or it may close (C). i.e .. hring the feet together. These basic symbols can he combined to form an alphahet that corresponds to the individual steps that can make up a dance. One popular foxtrot form is called the box rhythm. The following grammar rule (slightly adapted from the larger grammar described in IHerbison·Evans 2006) Q ) describes it (for the man):

B _... LS RQW LQC RS LOW RQC The dancer takes six steps: slow. quick. quick. slow. quick quick. The man al· ways starts with his left foot, then alternates feet.

Q.2.2 The Development of Plants In Section 24.4. when we introduced Lindenmayer systems (typically called just L·systems), we pointcu out that they were created by the hiologist Aristid Lindenmaycr as part of hiS work on plant development and growth.lbere we showed " simple L·systcm thal described highly stylized tree structures. But much more realistic !.\tructun:s can also be produc;ed Q.

References Other text hooks on automata theory. language theory, and the theory of computation also cover the material described in this hook. See [Sipser 2006). [Hopcroft. Motwani and Ullman 2001]. [Lewis nnd Papadimitriou 1998}. and lMartin 20031. Many of the exampl~ and proofs in this book arc. modeled after those presented in these earlier texts. References that exist on the Web are indicated with the symbol Q . Current links to them can be found on the Web site for this book. in the final section entitled Bibliography. Ackermann. Wilhelm, 1928. ··zum Hilbcrtschen Autbau der Reellen Zahlcn," Matlrematical Attnu/J, 99, pp. llS-133. Translated as ··on llilbert's construction of the real numbers," in From l·i·,~gl! to Godel: A Source Book in Mtttlremalical Logic 1879-1931 , Jean van Hcijcnoort ted). Harvard University Press.l967. Adleman. Leonard M .. 19'J4. "Molecular Computation of Solutions to Combinatorial Problems'' &;!•• Sciellc.'e, 266. pp. 1021-1024. Adleman, Leonard M., 1998. "Computing with DNA" Q.,Scientiflc American. August. pp. 34-41. Agrawal. Manindra. Nceraj Kayal,and Nitin Saxena,2004. ·'PRIMES is in P.. Q.Amzals of MatiJemalk.~. 160(2). pp. 781-793. Aho. Alfred V. and Margaret J. Corasick, 1975. "Efficient String Matching: An Aid to Bibliographic Search." Comm. ACM. 18(6), pp. 333-340. Aho. Alfred V., Rnvi Sethi, and Jeffrey D. Ullman. 1988. Compilers: Principles. Techniques. and Tools. Addison-Wesley. Alberts. Bruce. Alexander Johnson, Julian Lewis, Martin Raft'. Keith Roberts, and Peter Walter, 20()2. Molec·u/ar Biology of tire Cell, Fourth Edition, Garland Publishing. Altschul S. F.. W. Gis~. W. Miller, E. W. Myers, and D. J. Lipman, 1990...Basic Local Alignment Search Tool' Q,Juumal of Molec:ular Biology, Oct 5. 215(3), pp. 403-410, Andcrsun,John R. and Chrislian Lebiere,l998. 1'11e Atomic Componcms ofThouglrt, Erlbaum. Anderson. lt~hn ~·· Danie_a Bothell, Michael D. Byrne. Scott Douglass. Christian Lebiere. and Yuhn Om. 2004.' An Integrated Theory of the Mind'' n p . , 1 ·1 ·. 1R, · w 111(4) pp. 1036-to6o. ·' !'!!, ~}C 10 ogtw cvtc • • Appel. Kenneth and W()lfgang Haken, 1977, .. Scit•m;jic: Amc!rinm. 237. pp. 108-121.

n 1e SoluftOn o f 1he ~our-Co 1or Map problem."

Baader, Franz. Diego Calvancse. Deborah McGuinnes, D· . ,.

2003.71re' Ot•saiption Logic Hmulbo0 k

.

.

S. . amde Nardt,and Peter Patel-Schnetder.

• Cambndge

University Press.

1074

References

Backus. John. l980. ··Programming in America in the t950s-Some Personal Impressions" g, in A 1/iswry of Compllling in the Twrlllietll Century. Nicholas Metropolis and J. Howlett (ells). Academic Prc:ss. pp. 125-135. Bar-Hillel. Yl!hoshua. M. Perles. nnd Eliahu Shamir. 1%L "On formal Properties of Simple Phrase-structure Grammars.·· Zeit:r,:llrift fur Pllonc!tik. Spraclrwisse11scllaft und Kommmrik.atit>llsforsclrwrg. 14. pp. 143-172. Baroni. Mario and Laura Callegari. 19S4. Musict41 Grammars and Complller A11ulysis. Florence: LeoS. Olschki. Baroni. Mario, Rosella Brunetti, Laura Callegari, and Carlo Jaroboni. 1984. "A Grammar for Melody. Relationships between Melody and Harmony," in !Baroni and Callegari 1984], pp. 201-21M. Baum. Lennard E.. Ted Petrie, George Soules and Norman Weiss. 1970. "A Maximization Technique Occurring in the Statistical Analysis or Probabilistic functions of Markov Chains," Amwls of Mmhcmatic'al Statisti<:.~. 41, pp. 164-171. Bentley. Jon. 2000. Progrctmming Pecrrl.'i, Second Edition

~.Addison- Wesley.

Berger, Bonnie and Thm Leighton. 199!t .. Protein Folding in the Hydrophobic-Hydrophilic (HP) Model is NP·complctc." J. Compuwtimwl Biology. 5( 1). Spring. pp. 27-40. Berger, Robert, 1966....Titc Undecidability of Lhe Domino Prohlcm," Mmr. Amer. Math. Sex:. 66, 1- 72. Bcrners-Lcc.lim. James Hendler. and Ora Lassila, 200 t. "The Semantic Web." Scientific American. May, pp. 34-43.

Bcmers-Lec:. Tim and Noah Mendelsohn. 2006. Tire Rule of uusr Power Q . Berlckamp. Elwyn R. John H. Conway. and Richard K. Guy, 1982. Winning Ways for your Mathematical Plays, volume 2. Chapter 25. Academic Press. Bishop. Mutt. 2003. Computer Sec..•ttrit.v, Addison-Wesley. Brachman, Ronald J. and I lector J. Levesque, 2004, KfluwletfJ:e Reprt.femarion a11d Reasotring. Murgun Kaufmann. Bryant. Randal E .• 1986. "Graph-based Algorithms for Boolean Function Manipulation" Q, IEEE Tramuctiom on Comptller.~. C-35-8. pp. fl77-691. Bryant. Randal E., 1992. "Symbolic Boolean Manipulation with Ordered Binary Decision Diagrams"~. ACM Computing Surveys, 24(3), September, pp. 293-318. Btichi. J. Richard, 1960a. ··on a Decision Methud in Restricted Second Order Arithmetic," Proc: of the lntematicmnl Congress on Logic, Metlwclology und Plrilo:wplry of Science. Stanford University Press. pp. 1-11. BUchU. Richard. llJ60b. ··weak Second-order Arithmetic and Finite Automata," Grwullag. Mmlt. 6. pp. 66-92.

z. Math Logik

Camilleri. Lelio. 1984. "A Grammar or the Melodies of Schubert's Lieder," in [Baroni and Callegari 1984]. pp. 229-236. Cantor. David G.. 1%2...On the Ambiguity Prohlcm in Bnckus Systems.'' J. ACM 9(4 ) pp 477~79. • • . Carroll, John A.. 1983.-An Island Parsing Interpreter for the Full Augmented Transition Net k Formalism." Proce~dings of the First Con.ference (If 1lw European Clrctpter ~ tlr~ A ~or·. I I 4mgm.mcs, . . . p·tsa. Itu Iy. pp. 101-105. . J 5soct. ution fior ( .ompwmumu

0

References

1075

Champandard. Alex J.. 2004. AI Game Developmem: Symlwtic Creatures witlr LearnitrB and Reactive /Jelutviors. New Riders. Chamlra. Ashok K.. Dexter C. Kozen, and Larry J. Stockmeyer, 1981. "Alternation," Journal of tlw ACM, 2~( 1), pp. 114-133. Chomsky. Nuam. 1956. "Three Models for the Description of Language,"I.R.£. Transactions 011 Information Theory, vul.IT-2, September, pp. 113-124. Chomsky. Noam. 1957. Synlilctic Structures. Mouton. Chnmsky. Noam and George A. Miller, 1958. ''Finite-state Languages." Information ami Control. I. pp. 91 - 112. Chomsky. Noam. llJ59. "On Certain Forn1al Properties of Grammars", Information dncl ComroJ, 2(2), pp. 137-167. Chomsky, Noam. 1962. "Context-free Grammar and Pushdown Storage," Quarterly Progre.ras Report. 65. pp. 187-194, M.l.T. Research Laboratory in Electronics, Cambridge, Mass. Chomsky. Noum and M. P. Schutzenberger, 1963. "The Algebraic Theory of Context-free Langun~cs," Computl'r Pmgramming and Formal Systems. North Holland, pp. 118-161. Chumsky, Nnam. I%5.Aspect.tt ofthe Theory of Symax. MIT Press.

Christufidcs. Nicos. 1976. "Worst-case Analysis of a New Heuristic for the Traveling Salesman Prohkm: · Tcchnic/ Matlrmwtic.f, 58. pp. 345-363. Reprinted in [Davis 1965}. C larkt:. Edmund M. and E. Allen Emerson. 1981. ••Design and Synthesis of Synchronization Skeletons Using Branching Time Temporal Logic," Logk of Programs: Workshop, Yorktown lieigllt.'t, NY. Let:ture Nores in Computer Sdtmce 13l,Springer, pp. 52-71. Clarke. Edmund M., E. Allen Emerson, and A. P. Sistla, 1983. ·"Automatic Verification of Finitestate Concurrent Systems Using Temporal Logic Specifications.'' Proc. of tire 10'1' Annual ACM Symposium on Principles of Programming Languages, January, pp.l17-l26. Clarke. Edmund M., Orna Grumberg, and Doron Peled. 1999. Model Checking, MIT Press. Clocksin. Willium F. and Christopher S. Mellish, 1981. Programming in Prolog, Springer Verlag. Colhourn. Churlcs J., 1984 ...The Complexity of Completing Partial Latin Squares." Discrete Applit'd Mutlwmmics, 8, pp. 25-30. Cook. Stephen C,l97L ..l11e Complexity ofTheorem-proving Procedures,'' Third ACM Sympo.ttium on '11wnry o[Compwing.ACM. pp. 151-158. · Coppersmith. Don and Shmuel Winograd, 1990...Matrix Multiplication via Arithmetic Progression," Jmmwl nf Sym/Jolit: Computation, 9(3). pp. 251-280.

Cc.tpc. David. 1996. E.tperimems in Mu.'iical lmelligence,A-R Editions, Inc. Computer Music and Digital Audio Series. Vol. 12.

'

copeland. Jeffrey ;md Jeffrey Haemer, 2001. "Nonsense" Q, SW Expert, October, pp. 32-35.

Corm~n. 11mmas . H .• Charle~ E. Leiserson. Ronald L. Rivest, and Cli(ford Stein, 2001. Jmrotllu·twll ICJ Algcmtlrms. Second Edition. MIT Press.. culik, Kttrclll. 19%. "An Aperiodic Set of 13 Wang Tiles." Disc. Muth. 160,245-251.

1076

References Oantzig. George B.• Ray Fulkcrson and Selmer M. Johnson. l'J54. "Solution of a Large·S\:ale Traveling Salesman Problem," Opertllions Research. 2. pp. 393-410. 1

Davis. Martin D. and Hilurv Putnam, 1960...A Computing Proccdurc for Quantification Theory," Jmmwl of the ACM. 7(3).pp. 201-215. Davis. Martin D.• 1965. Tit«' Untleciduble. Raven Pre!."S. Ocdckind. Richard. IXSK. Wo.t .v imlund was sol/en di~ Zahh•1r? F. Viewcg. Braunschweig.Trans· hued by W. W. Beman and W. Ewald. in Ewald. William Bragg (cd.), From Kom to 1/ilhert. A Surm:e Book in tlrt' fmmdtttion s < 'J' Mmlrrmmic.'i, vol. 2, Oxford University Press. pp. 787-832, 19Y6. Dijkstra, Edsgcr, EWD-1248 g._ Durbin, Richard. Scan Eddy. Anders Krogh. and Gracmc Milchison. 1991i. Biological Sequence Ana(\·.~i.'i: ProiJabilistic Mmlels of Proteins ami Nttdt•ic Acids: A tutorittl introduction to lridtl£•n MMkm' models ami other fJroba/Jili.'itk nwtlt•lling npprnaclres in C'omputational .vrqm•11ce mwlysis. Cambridge University Pres!;. Du. Ding-Zhu and Kcr-1 Ko. 2001. Problem SolvillR in Auto,wta, L.cmguages, and Complexity, John Wiley and Sons. Earley. Jay. 11J70...An Efficient Context-free Fc:hruary. pp. 94-102.

P<~rsing Algorithm" .~.

Comm. of the ACM, 13(2),

Ebcioglu. Kemal. 1992. "An Expert System for Harmonizing Chorales in the Style of J. S. Bach." in Mira Balahan. Kemal EbciQglu and Otto Laske (cds.). Umh•r.muuliug Music with AI: Pc~pectives 011 Musk Coguirirm, AAA I Press/MIT Press. pp. 294-334. Ehrcnfcucht. Andrzcj, Juhani Karhumaki. and Gncgo17. Rozenbcrg. 19K2. ''The (Generalized) Post Correspondence Problem with Lists Consisting t'lf Two Words is Decidable,., Theoret. Compur. Sc:i.. 21 (2 ), pp. 119-144. Epp. Susunna. 2003. DiJcn•tf! Mmlrematics wit!J Applicatimu·. Brooks Cole. Evey. RJ.. 1963. "Application of Pushdown Store Machines." Proc:eetli11gs of tire 1963 Fall Joint Computt•r Couferem·e, llp. 215-217,AFIPS Press. Fischer, Michael J. and Rabin, Michael 0 .. llJ74. "Super-Exponential Complexity of Prcsburger Arithmetic." Complexity of Crmrpwmion Pruce(•di11gs oj'11 Symposium irr Applied Mathematit'.'i of the America" Mt~tlremmiclll Society ami tile Society j'or Industrial and Applietl Matlrematic.t. lleld ;, Nc•w YtJrk, April 18-/IJ, /973 (Ed. R. M. Karp). Providence. Rl: Amer. Math. Soc.• pp. 27-41. Floyd. Robert W., 1962. "On Ambiguity in Phrase Structure languages." Cumm. ACM. 5( 10). pp. 526-534. Forgy. Charles L., 1982. ··Rete: A Fast Algorithm for the M~ny Pattern/Many Object Pattem Mntch Prohlcm,'' Artifida/Jmelligeuce, I9. pp. 17-37. Ful(. Johann Jost!ph. 1725. The Smtl:\' of Counterpoint from Jolwmr Jo.'ieplr Frt:t's Gradus ad ParllttJ.\'11111 . translatt!d and cdttcd hy Alfred Mann. W. W. Norton and Company. 1971. Fmenkel, Aviezri S. and David Lichtcnstdn. 19tH. ~computing a Perfect Stratcgv for x 11 . ·nmtc Exponcntta · I m · 11 ,"" J. Cumbiuatoriul Theory Series· A 31 n Cht!SS Reqmrcs JY9-214. • ' pp. Gardner. Martin, JQ70. "Mathematical Games" C..Sdt•utifit· Amc-rinm. 223. pp. 12~123. October.

References

1077

Garey, Michael and David Johnson, 1979. Compmers and /ntrac1ability-A Guidt> to tire Theory of NP-complt>tencss. Freeman. Gillman. Leonard and Allen J. Rose, 1984. APL: An Interactive Approach. John Wiley & Sons.

Ginsburg. Seymour and H. Gordon Rice, 1962...Two Families of Languages Related to ALGOL," J. ACM. 9(3), pp. 350-371. Ginsburg. Seymour and Gene F. Rose, 1963. "Some Recursively Unsolvable Problems in ALGOL-like Languages." J. ACM, 10(1). pp. 29-47. Ginsburg, Seymour. 1966. The Mathematical Tlreory of Co11tt'.tt·Free Latrguages. McGraw-Hill.

Godel, Kurt, 1929. "0ber die Vollstandigkeit des Logikkalkiils,'' doctoral dissertation, University of Vienna. 'GUdel. Kurt. 1931. "Uber rormal unentschcidbare S~lze der Principia Mathematica und verwundter Systeme, I," Monatslle.ftc fiir Mmlzcmlllik wul Plry.vik. 38, pp. 173-198. Translated as ''On formally undecidable propositions in Priucipia Mathemillka and related systems, I," in From Frege to Gode/: A SCJurce Book in Matlre11wtica/ Logic 1879-1931, Jean van Hcijenoon (ed). Hurvard University Press, 1967. There is also a translation into English and ml'ldern notation by Martin Hirzel Q . Gouda, Mohamed, 1998. Elements of Network Prmocol Design, Wiley lnterscience. Greibach. Sheila A .• 1965. ''A New Normal Form Theorem for Context-free Phrase Structure Grammars.'' J. ACM, 10(2), pp.l75-195. Greibach, Sheila A .. 1981. ··formal Languages: Origins and Directions." Annals of tire History of Computing, 3( 1). January. 1981. Gries. David. 1989. Tht> Science of Programming. Springer. Harel. David. 1987. " Statecharts: A Visual Formulation for Complex Systems," Science ofCompwer Progmmming. 8(3). pp. 231-274 g _ Hare I. David and Michal Politi, 1998. Modeling Reactive Sy!t'tt>ms witlr Statecl1am: 11re Stalemate· Approach, McGraw-Hill. Harrison. Michael. Walter L. Ruzzo, and Jeffrey D. Ullman. 1976...Protection in Operating Sys· tems.'' Comnt. of the ACM,l9(8).August.pp.461-47l. Harrison, Michael. A .. 1978./mroduction to Formal Langm1ge Theory. Addison-Wesley. Hart, Peter E .• Nils J. Nilsson. and Bertram B. Raphael. 1968...A Formal Basis for the Heuristic Determination of Minimum Cost Paths." IEEE Transaclions on Systems Science and Cyhemt-ticj· SSC4, (2). pp. 100-107. Hart, Peter E.. Nils J. Nilsson. and Bertram B. Raphael. 1972. ·•correction to ·A Formal Basis for the Heuristic Determination of Minimum Cost Paths.'" S/GART Newsletter, 37, pp. 28-29. Hartmanis. Juris and John E. Hopcroft, 1968. "Structure of Undecidable Problems in Automata Theory," in Proc. Nimh Symposium Switching and Automata TIJeory. IEEE pp. 327-333. •

Hartmani~ Ju~s.. and Ri~~ard E. Stearns. .1965. "On the Computational Complexity of Algonthms, Trmrsactm11s of tlte Amertcan Matlrematiclll Society, 117. pp. 285-306. Herbison-Evans, Don,2006. "A Revised Grammar for the Foxtrot" Q . Hi Ibert. David and Wilhelm Ackermann, 1928. Gnuu/ziige der lheoretischeu Logik, Springer-Verlag.

1078

References

Hiller. Lcjaren and Leonard Isaacson 1959. E:cperimcmwl Music. McGraw-HilL Hiller. Lcjaren and Ro~rt Baker. 1964. ··Computer Cantata: A Study in Compositional M~:thod." Pcrspe<.·tires New Mm•k. J( I). pp. (l2- 90.

n.r

Hiller. Lcjaren, 1972. Compllf£'1' PmgramJ U:H•d to l'mdll('(! tlw Cimrpositiou 1-/PSCHD. :cch?i· cal Rcporl Nu. 4. Nntit)lllll Sch~nce Foundation Proj~ct Nu. GK-14191. State Umverstty ul' New York. Buffalo. Ht)
Natural Languttgr l'rvcessi11g, Compuwtimwl

UnRui.'ltic.~

mul .~j>e~dr Recognition Q,

Prentice-Hall. Kaplan. Ronald M.. and Martin Kay, 1994. "Rt:gular Models of Phonological Rule Systems," Compltlaticmctl Lingui.,·tic:s. 20(3). pp. 33 1-378, written in 19g{), Karmarker. Narcndra. 19X4. "A New Polynomial-time Algmithm for Linear Programming;\ CcmrhimltCJricu. 4(4). pp. 373-395. Karp. Richard M.• ltJ72. "Rcducihility Among Comhinaturial Pmhkm~'ln Ccmrple:.:it_v of CompUler Computmion.o;. Proc. Sympos. JBM Thumus J. Watson R~s. Center. Yorktown Heights. N.Y., Plenum, pp. 85-103. Kurlluncn. Laurie and Kenneth Beesley. 20
References

1079

'Kieene. Stephen C., 1956. ••Representation of Events in Nerve Ne_ts and Fini~e Au_tomata,'' in C. E. Shannon and J. McCarthy, Auromata Studies, Pnnceton Umversny Press, PP· 3-42. Kleene. Stephen C., 1964./ntroduction to Metamathematics, Van Nostrand. Knuth. Donald E.,l968. "Semantics of Context-free Grammars," Theory of Computing Systems,. 2(2). pp. 127-145. Knuth, Donald E. and Ronald W. Moore, 1975. ••An Analysis of Alpha-beta Pruning,'' Artificial lntelligence. 6(4), pp. 293-326. Knuth, Donald E .• James H. Morris, Vauglln R. Pratt, 1977...Fast Pattern Matching in Strings," SIAM Journal of Computirzg, 6(2), pp. 323-350. Koskenniemi. Kimmo,l983. Two-Level Morphology: A Gerzeral Complltational Model for WordForm Recognition and Production. PhD thesis. University of Helsinki. Publications of the Department of General Linguistics. University of Helsinki. No. 11. Helsinki. Kruskat, Joseph Bernard, 1956. "On the Shortest Spanning Sublree and the Traveling Salesman Problem," in Proceeding~ of the American Mathematical Society, 7. pp. 48-50. Kuroda, S. Y., 1964...Classes of Languages and Linear-bounded Automata," Information and Control, 7, pp. 207-223. Ladner, Richard E.,l975. ··on the Structure of Polynomial-time Reducibility," JACM. 22(1), pp. 155-171. . Lagarias. Jeff, 1985...The 3x + 1 Problem and Its Generalizations" Q, American Mathematical Monthly, 92, pp. 3-23. Laird, John. Allen Newell. and Paul Rosenbloom, 1987. "Soar. An Architecture for General Intelligence." Artificiallntelligence,33, pp. 1-64. Landweber, Peter S., 1963, ..Three Theorems on Phrase Structure Grammars of Type 1," lnformation mtd Comrol, 6, pp. 131-136. Lawler, Eugene L.. Jan Karel Lenstra, A. H. G. Rinnooy Kan, and D. B. Shmoys, 1985. The Traveling Salesman Problem: A Guided Tour ofCombinatorial Optimizarion.John Wiley & Sons.. Lee, Lillian, 2002. "Fast Context-free Patsing Requires Fast Boolean Matrix Multiplication" Q, J. ACM, 49(1). pp. 1-15. Lesk. Michael and E. Schmidt. 1979. Lex-A Le_Tirnl Analy~er Generator Q, In UNIX Programmer's Manual. Holt, Rinehart. and Winston. Levin. Leonid A., 1973. ··universal Sorting Problems," Problemi Pereclachi ltzformatsii, 9(3), pp. 265-266. Lewis. Harry ~-and Christos H. Papadimitriou, 1998. Eltments of the Theory of Compattation, Prenttce-Hall, Lichtenstein, _David and Michael Sipser. 1980. "GO is Polynomial-space Hard," JACM, 27(2), Apnl. pp. 393-401. Lindenmayer•.Aristid, 1968. "'Mathematical Models for Cellular Interactions in Development:• Journal ofTJreoretical Biology 18, pp. 280-315. Linz, Peter, 2001. An introdrcction tu Formal Languages and Automata, Jones and Bartlett. Mabalanobis. P. C .• 1940. "A Sample Survey of the Acreage Under Jute in Bengal" Sankllyu 4 pp. 511-530. . • '

Manning,PChrist~phe;: and Hinrich ScbUtze, 1999. Foundations of Statistical Natural Langt~age rocessmg ee, The MIT Press.

Markov,Andrei.l'JSl. "Theory of Algorithms,.. Trud)• Mat.lttst. Steklov.• 38, pp.l76-189.

1080

References

Markov. Andrei A. and N.M. Na~orny. 19Xlo!. The Tlwmy uf.A/goritltms (English translatioJ1), Kluwer Academic Puhlish~:r~. Martin, Juhn C.. 2003.. /mroducrimr tn Ltm1:11t1ges tmd tire 11wory o[Complltttlion. McGraw-Hill. Mutiyascvich. Yuri. 1970. ··Enumerable Sets are Diophantine:· Doklady Akademii Nauk SSSR, 191. pp. 279-282. t 970. English translation in Soviet Mathenwtifs. Dok/atly, vol.1l, no. 2. McCarthy, John. 1960. "Recursive Fum:tiuns of Symhulic Expressions" Q. Cmnm. A.CM., 3(4), pp. IR4-ll)5. ML-c'ulloch. WHrrcn S. and Walter Pins. 1943. "A Logicul Calculus of the Ideas Immanent inNervous Activity," Bull. Mtttlr .. Bioph.v.~ic'l, 5. pp. J 15-133. Mealy. George H., llJ55. "A Method fur Synthesizing Se(iuential Circuits," Bell System TeciJ11ical Jvuma/,34{5). pp. 1045-1071). Menger. Kul'l. 1932. "Das botenprohlcm,'' in K. Menger (cd). Ergebnisse eine.\' Matlrematischeu .Kolloqllillnu 2, Teubner. Leipzig. pp. 11-12. Miller. Gary L.. llJ76. "Riemann's Hypothesis and Tests for Primality." J. Comp. Syst. Sci: 13(3), pp. 300-317. Minsky. Marvin L., I%1. "Recursive Unsnlvability ot' Post's Prohlcm of ' tug' and Other Topics in Theory o( Turing Machines.·· A 1111. of Math, 74. pp. 437-455. Misra.Jcty. IY96. ''Coloring Grid Points. Without Rabbits or Snakes:"

~.

Misra. Jay, 2004. Theory in Programming Pructice 0.. Muon~n.

Leon. 211{11. "Generating Rubust Parsers Using Island Grammars.'' Proc:eetlings of tire 8 '1' Workiug Crmj'er~nce on Revn.w: Engineering. IEEE Computer Society Press, pp. 13-22.

Momc, Edward F., 1956. "Gedanken Experiments on S(!qucntial Machines." in C. E. Shannon und J. McCarthy. Aruvmma Stmlhw. Princeton University Press. pp. 129-153, Moore. F. Richard. Element.~ uf Compwer Mw.'k. Prentice-Hall, 1990. Morris. Robert. and Cherry. Lurindn L.. 1975. "Computer Detection of lypograhica) ErrorS:' IE££ Trcms. On Pro[t'.'l.'limwl Conummit·atimr. PC-18, I. March, pp. 54~. Murphy. Gail C. and DHvid Notkin. 1996. ''Lightweight Lcxic:tl Source Model Extraction," ACM Trummclion.v on Sojiware Etrgilleering t.mcl J\lft'tlwclo/tJgy. 5(3 ), July. pp. 262-292. Myhill. John. 1957. "Finite Autom;lla Hnd the Reprcsentatinn of Events," WADC Technical Report 57-Cl24. Wright Puncrsun Air Force Base. Ohio. pp. I 12-137. Nardi. Daniele and Ronald J. Brachmnn. 2tKJ3. ''An Introduction tu D~scription Logics" Q in Franz Baader ct al (cds.), 11re De.w;riprimr Logic /lam/book , Cambridge Univer~ity

Press. Nnur. Peter (cd). 1963. "Revised Report un the Algorithmic Language ALGOL 60," CACM, 6. p. 1; Tire Computer Jmmwl. 9. p. 34lJ: Num. Mm/1 .. 4. p. 420.

Nemde. A nil. 1958. "Linear Automaton Transfurmations," Pmc..Amer. Math. So('.• 9. pp. 54 1_544• Newell. Alien. J.C. ~lu':· and Hc~bcrt Simon. 1957. "E~pi~ical Exploratit?ns with the Logic Theory Machmc. l'roceedmg.'l o.f the Westem Jmm ( (lttrpmer Ccmjc~reuce. 15. pp. 218239 Reprinted in Edward A. Feigenbaum and Julian Fcluman (ed!i.), Computers Tlwllglrl. McGraw-Hill, t%3.

alld

Newell. Allcn . l19lJO). Unified Tlworie:i oj' Cogt~iriun. Harvard University Press.

References

1081

Ochoa, Gabriella. 1998. ·•on Genetic Algorithms and Lindenmayer Systems." in Parallel Problem Solving From Nature (PPSN V).• Leclllre Notes br Conrpttter Science 1498. SpringerVerlag. pp 335-344. Oettinger. Anthony G .• 1961. '•Automatic Syntactic Analysis and the Pushdown Store," Struc. Lcmg. Math. Aspects; Proc. Symp. Appl. Math, 12, pp. 104-129. Ogden, William E. 1968. "A Helpful Result for Proving. Inherent Ambiguity," Mathematical' Systems 71u•ory, 2(3). pp. 31-42.. Parikh, Rohit J., 1966. "On Context-free Languages," J. ACM. 13(4 ), pp. 570-581. Paun, Gheorghe. Grzegor Rozenberg. and Arlo Salomaa.1998. DNA Computing: New Complll· ing Paradigm.'>, Springer. Pearl, Judea, 1984. Heuristics: lmelligeltl Search Strategies for Computer Problem Solvitlg, Addison-Wesley. Peter. R6zsa,l967. ReL'ursive Functions. translated by Istvan Foldes., Academic Press. Post. Emil. 1943. "Formal Reductions of the General Combinatorial Decision Problem,'' American Journal of Mathematics. 65.197-215. Post. Emil L.. 1946. ··A Variant of a Recursively Unsolvable Problem.'' Bulletin of tire Americatl Matlumuttical Society. 52, pp. 264-268. Post, Emil L.,.1947. "Recursive Unsolvability of a Problem ofl11Ue," J. Symbolic Logic. 1.2. pp. 1-11. Reprinted in [Davis 19651. Pullam, Geoffrey K. and Gerald Gazdar, 1982. "Natural Languages and Context-free l,.an-· guages." Linguistics and Plrilosoplzy. 4. pp. 471-504. Pullam. Geoffrey K., 1984. "On 1\vo Recent Attempts to Show that English is Not a CFL," Computatimwl Linguistics, l 0;3-4,Ju1y-December. Prusinkie.wicz. Przemyslaw and Arislid Lindenmayer 199(). Tilt> Algoritlrmic: Beamy of Pltmts. Springer-Verlag Q. Prcsburger. Mojzesz. 1929. "Uber die Vollstaendigkeit eines gcwissen Systems der Arithmetik ganzer Zahlen. in welchem die Addition als einzige Operation hervortrit~" Comptes R<.•ndu.'tl du I Congres de MatluJmmiciemi des Pays Slctves. Warsaw. Poland, pp. 92-101. Queille, Jean-Pierre. and Joseph Sifakis. 1982. "Specification and Verification of Concurrent Systems in CESAR," Proceeding.'! of the Fifth lmernalionlll Symposium in Programming. Le,·wre Notes in Comtmter Sciem:c 137. Springer, pp. 337-351. Rabin. Michael 0. and Dana Scott, 1959... Finite Automata and Their Decision Problems." IBM Journal of Research am/ Developmem, 3, pp. 114-125. Rabin, Michael 0.,1980. "Probabilistic Algorithm for Testing Primality,"Jounw/ of Number Theory, I2( l ), pp. 128-138.

Rabiner. Lawrence R., 1989. "A Thtorialon Hidden Markov Models and Selected. Applications in Speech Recognition'' ~. Proc. 1EEE, 77(2 ), pp. 257- 286.

Rado, Tibor. 1962. "On Non-Computable Functions," Tile Beli Sy.~tem Tedmica/ Joumal, 41 (3 ). pp. 877-884.

R~tner, Daniel ~nd Manfred K. Warmuth, 1986. '' Finding 8 Shortest Solution for the N Extcns1on of the 15-Puzzle is Intractable;' AAA/-86, pp. 168-172.

X

N

Reingold. Orner. 2005. "Undirected ST-connectivity in Lo)!-soace.'' Proc:eeditJP.~ of the 37'h Annun llf'&l "·····

•

•

1082

References

Rencldl. Paul, 2000. Tlri.~ is u ·ruri11g Muchille lmplmremc>tl ;,, C(mway:~ Gume of Life Q, RFC 7\J3, Postel. J....Transmission Control Protocol." RFC 793. Sept. 19tH c. Rice.. H . Gordun. 1953. ..Cins.'\Cs of Recursively Enumcrahlc Sl"\s and Thdr Decision Problems." Trtrmwctinn.r,; of tile Amerko11 Mathc•llutt;ctll Sudety. 1~. pp. 35~-366. Rich. Elaine! and Kevin Knight. IIJ91. Arri/idll/lmellige~~t:e, McGmw·Hill. Rivest. Ron. Adi Shamir and Leonard Adleman. 197K "A Ml!thc)d for Obtaining Digital Signatures and Public Key Cryptosysh:ms." Comm. ACM.21 . pp. 120-126. Roads. Curtis. 19~5. ''Research in Music and Artiriciallntelli~nce," 1\CM Complllitrg Surveys, 17(2). pp. 163-190. Roads. Curtis. 1996. Tire ComJmtc•r Mtt!til! Tlttorinl, MIT Press. Robinson, Julia B.. l, LNCS 250(). Spring~r- Verlag, pp. 43-W. Rosen. Kenneth H .. 20U3. Discrete Mm/Jemtttks.und I ts Applic:mion'i. McGraw-Hill. Russell. Stuart and Peter Norvig.2002. Artijkiullmelligelln~: A Mot/em Approach. Prentice-Hall. Savitch. Walh!r 1.. 1970. "Relationships Between Nondeterministic and Deterministic Tape Complexities,.. J. Computer mrd Systems Scit'llu.,-. 4(2). pp. 177- 1lJ2. Schaffer, John and Deroo McGee, 1997. Kt~owleclgt•-Basecl Progrcmrmill,f{ f()r Musir Research. A-R Editions. Inc. .Scheinbcr~. Stephen. 1960. ·'Note

on the Boolc11n Propcrtks of Context-free Lmguages."lnformution and Control.3(4), pp. 372-375.

Schenker. Heinrich. 1935. Der .freit• Satz (Frc!C' Cmupn.~ition). Originally puhlishcd in Vienna: Universal Edition. 1935. English edition, E. Oster (ed. and trans). Longmun.lnc.l979•. Schollstacdt, William, 19X9. "Automatic C(.)untcrpoint," in Max V. Matthews and John R. Pierce (~:ds). Cum•m Dirl!Ctimr-.'i in Computer Mu.' ik Rc!.~ellrdl . MIT Press. Schutzcnhcrgcr. Marcel-Paul, 1963. "On Context-free Ltmgunges and Pushdown Automata,!'lnrormution and Control, 6(3). pp. 241l-264. Searle. John, 1980. "Minds. Brains, and Programs," RL'Iwviora! ami Bmin Scit'IIC'f','i, 3. pp. 417-424. Senizergucs. Geraud. 2001 . ''L(A) = L(B)? D~cidahility Results from Complete Formal Systems,.. Tlreoreticul Compmer Sdenc:e. 251. pp. l- 16h. Shannnn, Claude E .• l94R. ''A Mathematical Theory of Communkat ion·· cui Juumal, 27. pp. 379-4..,~ and 623-n.'\6, July and October.

~.

lll•/1 Sywem Teclmi·

Shor. Peter. 1994. ''Polynomial-Time Algorithms for Prime fm:torization and Discrete Logarithms on a Quantum Computer" Q, Pmcc•t•tlin~.'l o.f tilL' 35'11 Ammal Symposium on Fmmtftllitm$ ofComplllt'r St:imn·. pp. 124-134. Simnn. Herhert A and Allen Newell. 1958. "Heuristic Prohlcm Solving: The Next Advance Operations Research.'' Op<•mtiom• Rr.'irctrddanuary·Fcbru;ary. pp. 1-10.

In

-References

1083

Sipser, Michael, 2006./mroduction ro the Theory ofComptttation. Second Edition, PWS Publishing Company. Skulem, Thoralf, 1928. ..Ober die mathematische Logik," Norsk Matematisk 1iclsskrift, 10, PP• 125-142. Translated as ··on Mathematical Logic." in From Frcge to Gocle/: A Source Book in Mmh~mmkul Logic 1879-1931. Jt;an van Hc:ijcnuor\ (ed). Harvard University Prcss..l%7. S<_,koi.Jod.2004. "An Intuitive Markov Chain Lesson from Baseball" Q .. Stock, Olivicro, Rino Falcone, and Patrizia Insinnamo, 1988. "Island Parsing and Bidirectional Charts," Procl!cdiugs 12'1' Conferellce on Computational Unguistic:s, (ACL). pp. 636-Mt . Stone. Peter. Kurt Dresner. Selim Erdogan. Peggy Fidelman. Nicholas Jong. Nate Kohl. Gregory Kuhlmann, Ellie Lin, Mohan Sridharan, Daniel Stronger, and Gurushyam Hariharan, 2003. "UT Austin Villa 2003: A New RoboCup Four-Legged Team.'' Technical Repon. The University of Texas at Austin, Artificial Intelligence Lab, UT-Al-TR-03-304, Oct~ bcr, 2003 Q. Strassen, Volker, 1969. "Gaussian Elimination is Not Optimal," Numerische Mutlrematik, 14(3), pp. 354-356. Sudkamp. Thomas A., 1998. Languages and Maclzitrts, Addison-Wesley. Sundberg, Johan and Bjorn Lindblom, 1991. ..Generative Theories for Describing Musical Structure," in l Howell, West and Cross 1991 ). pp. 245-272. Szelepcsenyi, Robert, 1988. "The Method of Forced Enumeration for Nondeterministic Automata," Al'ltl lnformmico 26, pp. 279-284. Taylor. R. Gregory, 198H. Models of Computation a11d Formal umguages, Oxford University Press. Thomas, Wulrgang. 1990. "Automata on Infinite Object.s." in Handbook ofTI!eoretit'al Computer Science, J"n Van Lceuwt;n(ed). MIT Press, pp. 135-19l. Trappe, Wade and Lawrence C. Washington, 2006. Introduction to Cryptography with Coding Theory. Prentice-Hall. Turing. Alan. 1936. ··on Computable Numbers. With an Application to the Entscheidungsproblem'' Q, Proceeditrgs of the London Mtulremmical Society. Series 2, 42, pp 230-265. Errata appeared in Series 2, 43 (1937). pp 544- 546. Reprinted in (Davis 1965}. Turing. Alan. 1950. "Computing Machinery a,nd Intelligence.. Q , Mind. 59, pp. 433-460. Valiant. Leslie G .. 1975. ~·General Context-free Recognition in Less Than Cubic Tune." J. Cnmputt>r 11ml Systmr Scie11ce.v, I 0, pp. 3~315 .

of

Viterbi, Andrew J., 1%7. '' Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm," IEEE Tratrsacrions 011 Information Tlzeory IT-13. pp. 1260- 1269. Wang. Hao. 1%1. " Proving Theorems by Pattern Recognition pp. l-42.

n;· Bell System Tedz. Journa/, 40,

West. Rnhert. Peter Howell. and Ian Cross. 1991 . "Musical Structure and Knowledge Represenllttion," in (Howell, West. and Cross 19911. Whitehead. Alfr~d Nort~. an~ Bertrand Russell, 1910. 1912, 1913. Pritrcipitl Mallremarica, 3 vols. Camhndgc Umversny Press.

1084

References

Wolfram, Stephen. 2002. A New Kind of Science

~.Wolfram

Media. Inc.

Yato. Takayuki and Takahiro Seta, 2002. "Complexity and Complch:ness of Finding Another Solution and Its Application to Pu1.zlcs~ ~.IPSJ S/G Nmes. 2002-AL-87-2. 3

Younger. Daniel H.. 1967. " Recognition and Parsing of Contcxt-fn:e languages in lime n ." Information uml Comml. 10(2). pp. 189-20R. Zloof. Moshc M.• 1975. "Query by Example." Proc. Natiomrl Computer Conft'retlc~ (NCC), AFI PS. 44. pp. 431-43H.

INDEX a-fJ pruning. 1043 'I'(L). 306, 313J~52 Nu. 7MS w-automatun. 11!.100 -.H. 439.445.44H,.&52 ,475, 4n. 4RI,4H2 _,H,. 477.494 1/'lr.· 579

g-loop. 73.82.188 ~1\.4.455

.... 728.764 ~r.M4

It-recursive function. 571.584 s-rulc.224 15-pu7.7.1e. 43, 732, 741 '2-Ct'F. Seco 2-conjuncliva nnrmal form 2-COLORABLE. 676. 693. 807 2-conjunctivc nonnal form. b74 2FSMs-INTERSECf, 712 2-SAT.674.691. 720 2-SAT-MAX. 691 3-CNF. Sl' e 3-conjunctivc normal form 3-COLORABLE., 676. 693 3-ct)njunctive normal form . 642,810 3-t•olfjtmcril't'Booleull, 690, 8 I 0 3-SAT. 642. 645. 646. 65~. 659, 66.'\, 674.810 ,3.\' + I problem. 428. 583 4-color prohlem. 676 4-COLORABLE. 676 A. 461. 4M2, 522 A,. 463. 482.484 A* algorithm. 734, 740. 744. ~95, l(J07.1047 AAU.• 463. 4S2. 484

A.ne.n· 478, 4S2 A t\'O:Y• 463. 4~2. 484 Aarons\ln. Scott. 743 abacus. 1057 absorption laws, 747,774 accepting hy a BUchi automation. 116 by a deh:rmimstic TM. 376 by a DFSM. 57.691 hy a nondctcmlinisticTM, .'\Rtl hy a PDA. 251,276,848 by an NDFSM. M access control matrix. 950

Ackermann. Wilhelm, 413. 569, 584 Ackermann's function, 569. 584 active tape, 366 ACT-V,l026 Ada,886 adder, 25, 1060 adjacency matrix. 592,759 Ad Ieman, Leonard, 582. 956 admissibility or a heuristic function. 737, 1048 of a search algorithm; 738 age of universe, 570.586, 598, 1059 agent lntl!lligcnt, 929. 1005, 1019. 1026, 1046 agreement in natural languages. 351 , 542.993 Aho.Aifred. l99.362.888

AI. &e artificial intelligence AL,I040 Alexander. Christophcr,127 ALGOL 58. tk~ ALGOL 60. 211.880.H81,882 Algol c~. 886 algorithm,36,411 , 792 algorithms 3-ccmjwu·ti,,eBoolean,810 A*,734 atmo.~toneEp.t, 226 huiltiFSMcano"i'"rllform,OS builtlgrmmnar. 271 builcikcywordFSM. 145 buildwwmbiggrammar, 302 cfgtoPDAbortomup, 263 ljgmPDAIIoeps, 317 tjgro PDAtoJ'down. 2(l2

deddeCFLitrfillile, 319 decideCFLusinJ~Grnmmar. 315

dt!C'iclrCFLusingPDA. 317 drcidcFSM, 188

tledtleregex, 188 df.wnsimulme. 81 disjmJ('/iveBoalt.a/1, 811 Eurlryparsc. 357 cmptyFSM, 190

emptyFSMcammicalgruplt, 189 t!mptyFSMgrapfr, 1~9 emptyFSMsinwlall!, 190 ep.v, 72

cqttaiFSMs. 193 Eulcritm. 628 {i11iteFSM. 192 jiuir~FSMgr"J'h . 191

fillittFSMsimulatt. 192 first ,362

follow. 362 forwartl. Ill fsmtorcgex. 143 f.smtoregcxllturistic. 137 gume-searclt. 1042 game-search-a- fJ. 1044 grmnmartofnn , 151 i11ji11iteFSM. 192 imerst't'rPDAmuiFSM. 291

K11WII-Morris-Pruu. 608 Krtt.fkal, 629 minDFSM. 92 mi11imniFSM.193 m i"i11wx. I040 minimax-wiriJ-a-fJ-prmrillg. 1043

MS1'tlecide, 631 mlf~mcm• vertamlsimttlate, 81

CKY.342

t~dfmuimttltlle. 73

t·omput~:>transitivtcfo.ntre, 778

mlfsmtatlj'sm,13.835
conjum:tivt•Booltllll, 809 CCIIIIICC'let/, 62..')

convenPDAtmletnomwlfonu, 852 collvtrtPDAtore3trictt'tl algoriJ/mrs, 266 COIIVtrttoCIIOIII.Iik}', 235 converlloclcw.w:form, K24 cmtvertwGreibadr, 842 cmwemmrorztt'rminal, 533

quicksrm. 12.'\, 744

rege:ctofsm. 135 remove£ps, 224

nmovelcoftrecursioll, 333

creat~:>OBDDfromtree. 817

removeLo11g, 238 mnoveMixcd, 237

dc•cideCFL. 315

renww~ UIIits. 236. 321

1086

Index

ulgorithms (nmtimu·tl) rt!mt•l't'liiiTl'llt'lwlolt•. 215

re.mll-1!· Honletm . ~ 14 rc.wlve· FO L . X31 simm iiiJWII!r . 394 .'llimplc:·rt•writc. 21~

tmaiFSM. 191

lmifl··for-rt'solllliull. K2R ~'itt·rhi. II()

IVit/!t>/11$. 21J7 ulignmc01 of prott'in sequences. %9

allele, 11)4. 965 almo~l satisfiahilily. 6"12 alphnhct. X

tllmo.ttnneEps. 226, 321 ATN St-r augmented transition network atoms in universe. 198. 5711. 10411 attachment amhiguity. 229. 241, 24R, 2J2. ~"\. 91J5 attrihut~ grammar. 540. 554,5XJ uu~mcntc:O transition network. 1033 Austin Villa. 1062 AutoCA D. KlJO automatic programming. 51.897 automatic reasoning. 1004 autumaton. I054 avcr;tge case pcr(ormance. 51J6

alpha·hcta pruning, 1043 11(1. 312

13
Alternating Bit pn1tocol. ~I. 12fi. 9ll alternating Turing machine. 11139 ALU lfl6(l Amazons. lll3lJ

Bach chorales. 1033. 1034 hack,Junmon. 1037 Backus N~ur furm.l9.2ll.lhl1. 11122 Backus. Juhn. KXO. AAI backward reasoning. 205. 4SO. 1010. Jfl25 Baker. Robert, 1030 Bal.llJ. 173. 2Ut.20M. 220. 226, 230. 247.252.277.313. 714 balanced delimiters. 246 Ntlanccd parentheses language. Sc>e Bal bar code. 97. 1211 Bar-Hillel. Ychnshua. 200. 36 I. 5M3 Baroni. Mario. 1033 Baum. Leonard E .. 200 Baum-Welch algorithm. IOIJ

amhi~tuity

in l'UnleXI·frcc gmmmar.;. llO. 337. .UX. 500. SOK Kli5. KX6

in English. 223. 2."\2. KX5. 995 in prn~ramming lun~uages. 222. 22lJ. 247. XXS in regular gramma~ 221 inherent. m .499 techniques fur reducing. 224 amhiguuus auachmcnt.l29. 247. 14K 232. KX5. 1N5 Ana•. 18. 2lJ. 17 t. 301.3116. 445. 5:!9. 7211 A"B"C'n, JO. 34.2711. 2X4. 37h. 420. 445. 511 .515. 52R, 541 , 55.'. 5"J0. 5%. f123 und dimination. 74N
ARPAhct. 11101 ARO protocol. 920 artificial intelligence. Cl.K27. MIJS.

binary adder. 25. 1060 binary decision trcc,816 binary funct ion. 770 hin:~ry multiplier. H161 hinilrYrelation. 757 binary scarch.lJf,(l

a!\.'ioc:iativity. 774 astronomical calculatur, 1055 astronumicul duck. I055 7taW

98. 'Jf(()

Big Bang. 570. 5R6. 597, 1()59 big-CUi99. 7\JS. AA9 bigwm model, 9R7 higram taggcr. tm2 hijl-ction, 773

1004. IIM)5, IC).t6

"··~···nnl ,,tt.- t l,-, nunan.~,.. ~'IN

Bel. Bernard. 1032

hcttr with ri0c. IJ'J7 Berger. Bnnnie.lJ77 Berger. Rubert. 583 be: lief reasoning ahout. 11108. 1(119 Bengal farmers. 5X7.. 742 Bentley. Jun. 9X7 Bt:rn..:rs-Lct:. 'lim .. 3J. I.J311 bcst•lirst search. 7J4. M\15 , 111~4 bidirectional finite statt: transducer.

~U

nlll.l

nVJ:Jl"Ti l

; I= l

BIN'- PACKING. M7.1l5K.693 hipartite graph. 671.863 hi~-ction of a ~rarh. 694 Bisho!'. MaU. lJ51 BNF. Set> Backus Naur form Buolcan expression grammar.247. J()(lfl

Boult!~tn

logic. 745.808 dccidahility. 495 normal furms. NUX resolution. Mli.IU2 satisfiubility. 746. Kl2, Stt SAT Butcnpmblcm. 742 bollom-up parser. 262. J4f. 544 houmh:d simulation. 190 BOUNDED PCP.49l.658.692 houstruphcdun . .502 BPI~ 725 Brachman. Runald.940, 1007 branching facwr nf a grammar, 119. ~13. 2RI .304,319.853

breadth-first search, 392. 616,137, ~6.802

Bryant. Randal. Kl7 DOchi automaton. l15. 126.200,

904.90X.919 DUchi. J. Richard. 200 hlliillfSMn mnllit-alfo rm. 95, 189, 193 huifllxrommur. 27\

buildkeyword FSM, 145. 200. ro7, 971 huiltl11namhiggrammar. 302 husinc:ss rule!> cnttinc, ltl22.1026 busy heaver function, 380,468, 561, 56!}. ~3

c. 13, 229. tui6 C+.,., 22~. 472

Cage. John. lll29 Cumillcri, Lcliu. 1Cl33 canonicul form , 94, 233 for Houkun expressions. 95, 811 fur FSM1.. 9S, IK9, 193

Cantor, Oa\'id,5~3 cardinality of a lang.uage. 14 nf a set. 7:'4. 788 or the .,;onlexl-frcc languages. 280 of the rc~ular languages, 162 or the so languagcs.43K Carmichael numher, 729

Cuncsian product, 757 t:t'ilillg. 3K cellular automaton. 414, 411. 544 55U.5H2 • certificate. b35

lndell

CFGAu.5tn. sm

mathemnti~a{ definition. 49,776,

cfgtoPOiiht11WIIIIIp. 262, '263.321

,fgtoPDAnoC!f1!i, 317. 321 cfgtoPDAwpclon·lz, 261 , 262,263, 316.321.504 CFL. Sf!t' cunt~:xt-rrc:t: lunguattc Champandard. Alex. 1046 Chandra. Ashok. 1040 characteristic (unction, 10.380.561.

754 chart parser St
spucc complexity closlM!s. 71R. 720

Chomsky normal form. 233.235, 24~.289. l13. 315.324,

342,361 ' 533. ~() Chumsky. Noam.12.200, 360,361, 539, 5~3.

800 programming language definition, 893 closure properties of Bilchi automata. 120,200, 'JO'} cuntcxt-free languages. 2~ 288. 295.308. 312.361 context-sensiti\·e languag">s. 533, 553, 583. 719 decidable languages, 438, 445 deterministic context-free languages, 297,312,846 finite languages. 41}. 52 infinite languages. 49. 52 PSPACE, 720 regular languagcs.65. 165.179. IS3 semidecidablc languages.43K. 445

LOJJ

diiiJ1.48.1RI , 312

CHORAL. 1034 Christofidcs. Nicos. 742 chromatic number, 046.677. ~ CHROMATIC-NUMBER. 646. 677.6."W chromosome. 965 C.hurch. Alonzo. 413,414.415.582.

NYU Church's thesis. Sl!f Church-Turing thesis Church-Turing thesis.4ll.413.424, 42(,,516 circuit Eulerian. 61:1. 643. 673. 6~l. '12M Hamiltonian. 591.637.657.663, M9. 673,743. cKV algorithm.St't' C'ocke-KasnmiYounp.cr algorithm Clarke. Edmund. 200. 902 clatL'it definite. 1012 Horn, 1012 in Boolean logic. 642. 809 in first order logic. M22 clause furm. 750. 822' cLIOUE.6J7,64U.643.646,657.691 clique detection. t137. 640. 64b, bS7. 691 dock mechanical, 1055 CJocksin. William F.. 1016

the dass BPP. 716 the dass L, 717 the class NP. 680. 691 the class P. 622. 691 CNF. St'e conjunctive nonnal form 2-C'NF. Set' 2-conjuuctivc normal form 3-CNF, St!t! 3-conjunctive nom1al form Cobol.241 Cocke, Juhn. :WI. 362 C.ocke-Kasami-Yountter algorithm. 315.324.341.362,614.

625. 1)()5 codomain of a function. 769 cu-dspe~ce. 718 coffc.c can problem. 795 co-rinite language. 694 cognitive modeling, 1022.102!. 1049 cognitive psychology. 1022. 102!, 1049 Cohn. Martin, I98 co-L. 717 Col bourn, Charlc:s. Hl37 coloring, Se-e map coloring colorless green ideas. ll. '197 Common Music. 8~ conunutativity, n4 compiler construction, S. 222. 323. 36(), 474,484,180 comfllement closure under. 65, 165, 289,297, 438.537,622, 680. 717.

71K, 719,726 or a language, 15.427,,476 of a set. 756

1087

..:omplctc graph.677 completeness logical. 751 of a set of inference rulel\. 747,75() with tl!spcct 1<.1 a world, 752

Completeness Theorem. 413. 497. 582.750.752 complexity logarithmic space, 714 multitape Turing machines. 388 nondeterministic Turing machines. 3~. 392. 595' polynomial spuce. 700 polynomial time. 604. 62l randomized Turing m11chines. 725 spncc,595,695 suhlinear space, 713 time. 595, 621, 71)8 complexity classes AL.St•rAL AP,SceAP APSPACE, See APSPACE BPP, Sre BPP

co-L. See co-L co-NL.Sec:co-NL co-NP. Se-e co-NP co-RP. St•e co-RP EXPTlME,Set> EXPTIME EXPTIME-complctc. See EXPTI ME-completeness L, SccL NL.See NL NL-C'omplcte.See Nl...completeness

NP.Se'' P NP-Complctc. See NPcomplctcness NPSPACE,See NPSPACE P. SeeP PSPACE, Sec PSPACE PSPACE-ct\mplete. See PSACE-complete.ness RP.St'e RP ZPP. See ZPP Complexity Zoo.S89. 743 COMPOSITES. 390,613, 673,728 composition of functions. 771 compusition or relations. 758 compositional semantics. 18, 128,544 compression algorithm, 876 computable function, 379,414,520. 557, 565,570 computation or a Turing machine, 370 - I I - - l:.'t''A C'J

1088

lndel<

comput<1fiun hiswry. SOO. 53M cumputatinnal binlogy. fl, 9~6. 962 Crmtfntlt'r Ctmllllll. I 0311

computer architecture:. 10611 game.6.7.1UJ. 734.•.lCN.l022, J(J26. 1046 n~twork. 918

~curity. h. 4K5. 6511.949. 955. 959

complltc•trcmsitil't't'lci.Wrt'. 77K concotcnntinn clusur~ under. IM. 2KM. 44.LS34 or languag~:s. I.S of strinp.s. 9 cnnccpt sub-.umption. 7M concurrent S)'litems. 95, 116,117.902 frNII/.~puc(•. 711:! configuration of a PDA.:!5t.l of a Turin~ machine. Jh'J of •m FSM. 57 connict rc:o.ulutiun. 1112S conjunctive normul form for Bunlcan lngic.642. H09. R13 fur firsHmJcr lu~k. Hll. IOI!tJ etmiuiiL'Iil'«' lloo/nm. X()')

ru-NL. 717 CONN E
co-NP.6N. 721i consistency. 751 cunstructiun proof. St•e pnKif hy cunstruction cont~:xt·frL'C gmmmar.207, 1Cl31, 11M. amhip.uity.l2f). 501<1 correctness pruuf, 21S desi!!ninp.. 212 for English. St•u English grammar for prup.rmnming lant'.UIIgcs. Kl:IQ fetr URls.lJ~I isl
normnl fclrm. 233 simplifying. 213 stochastic. 243, •n6.W7 .IJIJX rontcxt-frec langua~c. 29. 2!11 . 21111. 624. XH3. ~J. 1066

deterministic. 2f1Cl. 295. Jut. 33R.

J5U.K4h inherently amhi~uous.W. .301.4~ ll.(I)..U~ LR(k).J:\f)

um.Jccic.lahlc prupenics. 499 cuntcxt-free par!ling.:! 1~. 2hll, .'00, lll. fll3. 61 x. 625

context-free Pumping Thcmcm. St!l: Pumping Theorem. context-free cunlCXl•SCositi\'C grammar. SlU. 529. 1032

contcxt-scm;itive languattc. 526. .583.713.719 contradiction in Boolean logic. 74S in firsHmJcr logic. 751 contradict ion pronf. Srr pmor by contrddictinn t'mll:t•rt PDAtmlcmwrmaOimn. M52 rmu·ntPIJAtorr.wrictrt/.266. JU2., 321.155<) t:mnwuoCitomsky. 235.234, 321 coi/Vf!Ttlodtlll!it'fimn. X24 mtrverttoGrrihudt. .'21. 842. s-46 (fllll't'rtlollr.llllr!rlllilllli. 533

Conway. John. 414. 421.582 Cook. Stephen. 649. 743 Cclok ·levin 'fhcor~:m. 649. 6..'i0. 74.~. J(I(IIJ

Cope. David. 11133 Coppersmith. Dun. 344• .362. 62U. 743 Coppersmith· Winngrad algorithm. 344.362.6211.743 t•opyllmlrt!W'r.it'. 184.312

Corasick, Margarct.l99 Corman.lltoma.'i. 742. 743 co-RP. 726.72X correctness proof or 3 grammar.215.5 12 of a program. 22. II h. 473,4%, 792, N9X. 899 countabl~: ~t. 7K8 counwhly infinite set. 7X8. s,•t• proof hy cuunh,:rcxamplc counterpoint. 1034 t·tmml.. 5XO arcttt!OHDDfmmtrt>e. Nl7 Cross, lan. 11131 cross-54!riul dependency. 286. 21)2. 1J'J I crossword pur.~:le construction. 1037 cryptography. 23. 632. 637. 724. 12M. 731.901.955 cumulative acknnw1cdgemt:nt protocol.925 currying. 579 cut in a graph. 693 CYK algorithm. See Cockc-KasamiYoungcr algurithm D (th~ decidable languagc5). 31. 376.426.~35.4X2.565

DOL-system. 550

do Vinci. Leonardo, 1028· dance.to71 dangling else prohlem,129. 247, 2~.AA5.996

Oant1.ig. George. 678.. 742 database query language.l8,27, 336.916. t(X;b Oa\'is. Martin. 582.1009 de Morgan's laws.747. n6.ff». Rl2.K24 dead ccxlc elimination. 474. Ill dead stute. 59. 157 dc:cidahility. 31. 37.376.426,435,448 decid;~hility of Boulean 1ogic.495 context-free languages.315 context-sensitive languages. 528 first-order logic, 413,496. 827, 11Mit}.l020

regular lanttuages. 188 decidable langua1lc.31.376, 426, 435. 4g2. 565 decidable pruhlcm. 37, 448. 482, 951 cledtleCFt . 315. 322 tlt·c·irlc•CFI_mlpty. 319.322 rlw'd~CFI.mfinite. 319,322 tlc•dtlc•CFI.u.~i11gGrnmmar, 31!, 322, 323.fl2-' Jc>c·itieCFLusingPDA. 317.322,323, 327.624 tlc•drle'FSM. 188. 192 clc•c·itlt•regex. 11:i8 d(•c·itleSA Ttlt•termini.~tkally. 696 deciding hy a deterministic TM. 376 hy a nondetcrministicTM,390 decision prohlcm. 21. 36,593 decision procedure, 36,426 dccisinn rmcedures for BUchi nutomata. 121.909 cnntcxt-frcc languages, 314,499,

624 context-sensitive language$. 528. 537 rc~ular languagt.-s, 187,499,624, 712 decision trec. Xl6 decoding prohlcm in HMMs,108,

m

l)cdekind. Richard. 58.~ deductive verification, 901 Deep Blue. lll3S deep structure, 1033 dcfnuh reasoning. t 008 ddinitc clause. Hll2 degree of a vertex, 627

Index deobfuscation, 109 depth-first search, 327, 392.616 derivation left-most, 220 right-most, 220 with context-free grammars, 208 with context-sensitive gramman, 529 with regula.- grammars. 156 with rewrite systems. 204 with semi-lllUe systems. 523 with unrestricted grammars. 511 description logic, 940, 1008 deterministic bottom-up parser. 341.348 Bi.ichi automation, 118 finite state machine, 56. 199 Linden mayer system, 550 linear bounded automation, 528 normal form for PDAs, 302, 847 pushdown automation, 2,.~ top-down parser, 340 Turing machine, 36S, 595,621,700 oFSM, See deterministic finite state machine OFSM-ACCEPT. 691 DFSM-ALL, 691 diagonalization, 431 ,568,685,790 dialects of natural languages. 351 Dijkstra, Edsgar. 863 Dijkstra 's algorithm, 674 Diophantine problem, 488,679 diploid organism, 104.965 DIRECTED-HAMILTONIANCIRCUIT, 663 disjunctive normal form, 811,917 disjrmctiveBooletm, 811 distance Euclidean, 588,738, 1048 Manhattan, 1048 distinguishabilty of strings. 83 distributivity, 774 divide-and-conquer. 167, 451, 618, 701,700 division by zero. 889 DL. St!e description Iogie DNA.6, 19,107,964,968,970 pNA computing. 422,962 document type definition, 1068 domain of a function, 769 double negation. 747 dovetAiling, 441,459 dspace, 718 pTD. See document type definition

dynamic programming. t 10,341, 352,587.614

word list Wordnet, 947 entailment, 747,750, 812 Entscheidungsproblem, 413,432, 495,522.582,650,890, 899,1020 enumerable lexicographically Turing, 443

Earley algorithm, JS:Z, 362, 614, 995 Earley, Jay, 362 Earleyparse, 3!57, 99S Ebciojlu, Kemal, 1034 EBNF, See Extended BNF edge cover,675 EDGE-COVER, 675,690 Ehrenfeucht, And.rzej, 563 electronic music. 1028 EM, See expc:ctation maximization Emacs,890 email, 123, 143,919, 1052 Emerson, E. Allen, 902 EMI, 1033 .emptiness question for Bochi automata, 121 context-free languages. 318 context-sensitive languaFS. 538 regular languag.es, 189 SO languages. 463 empty clause, 814, 1013 empty set, 754 tmptyFSM, 190 emptyFSMcanonicalgraph, 189 emptyFSMgraph.l89 emptyFSMsimulate, 190 encoding, 22 for complexity analysis. 590 multiple values as a single string. 22,404 of binary addition, 25

Thring,440

enumerating lUring machines. 402. enumeration, 13,440.754, 789 lexicographic, 14,443 Epp, Susanna, 745 eps, 72 EqTMs. 464, 469, 482 equafFSMs, 193 equivalence classes, 764 equivalence modulo n, 728, 764 equivalence of states in an FSM, 90 equivalence question for BUcni automata.121 conlext-free languages, 320, 499, 503,553 context-sensitive languages, 538 deterministic context-free.

languages, 320

of context-sensitive grommars.S31

of graphs, 24, S9l of graphs in DNA, 423 of multiple tapes as one. 386 of natural numbers, 591 ofl\uing machines, 400 power of, 22 1bring machines encode themselves, 573 enCI)'Ption, See cryptography English, 12, 978,983 ambiguity, 223,232, 246, 995 grammar, 211, 219,232,542, 985. is context-free, 311,989 is not regular. 179, 183,983 longest sentence, 984 Markov model of, 987 morphological analysis. 99,978 parser, 350,994 rendomty generated. 243,987 CP.ma,.ti,-. ;nt _ _ ..__. .

,.

1089

regular languages. 192 SO languages. 464,959 equivalence relatiQn, 762 Ethernet, 919 Euclid's algorithm, 614. 957 Euclidean distance, 588,738, 1048 Euler, Leonhard. 626 Eulerian circuit, 6r1, 643, 673, 690, 928 Eulerian path, 627 EULERIAN-CIRCUIT,62'7, 643,, 673,690,928 eureka!, 733 evaluation problem in HMMs, 109, 999 Evey, R. J., 361 evolution, 967 excluded middle

law of. n9

existential generalization, 751 expectation maximization, 109 expert system,204, 1022, 1024, 10l6 Expr•111,228,263 EXPTIME, 643,687 Exn'lME-completeness. 688, 1039 EXPTIME-hardness. 688 Extended BNF, 881, 1069 Extensible Markup language,see XML finite state machine· .. -::

1090

Index

factorit.ation, 614. f\33. 731 , 958 Falcunc, Rino.3nl false negative prohahility. 725 fat~ positive prohahility,725 feature grammar. 540.554. 5K3. 'J94 Fermatliar.121J Fermat numhcr. 38. IM Fermat witness. 711J Fermat's Little Theorem. 729. 957 fihonacci sequence~ 4.~. 546. 806 FIFO queue plus FSM.414, 4l6 finite set. 7AA finite state autnmutun. Sttl! finite statc machine fini le st:He mach inc, 2~. 56, IIJIJ. 712. 9H). L>IQ. L>4H. %U,IJ7l ,. 1114n, 1053, 11154 al'lncus, 11157 Antikytheru Mechanism. 11154 equivalence with rcgular expressions. l33 Jacquard loom. 1057 nondetl!rministic.46. 66. 199 Pragu~: orlnj. 1055 Tuwcn- of lianoi, JOSH finite state machine simulator. XO finitc state transducer. 96.971. q78. 1(160 Jinitr•FSM .1'12 jillitC'fSM~:mph. llJI finitd'SM.~imulutr, 1'12 finitcncss question for contcxt-frec lunguap.cs.31R regular lang.uup.es. 191 semidccidable langunges. 483 fir.~l. 337. 362 ji~tdum.49.180.3()Q

rirst-tmlcr logic, 748. X2U. 101)7 Entschcidungsproblcm, SeC' Entscheidungspruhlem nurmal fnrms. R21 resolution. 827, I(JCI'I undecit.lahility. 4% first-order predicate cttlculus. Srt! first-order logic first-order predicate logic. Su firstorder logic Fischer. Michad. SXJ. 6XS fixcd point. 774 fixed-point theorem. 511 FLOAT.I44 Aoyd. Robert. 5M3

FNP.6S'J folding RNA. 213.967,975 FOL, Sec first-order lugic

fullow,JJ1.362 FOL,......."lrcm• 496. 6AA, 705.827 FOPC. Sec first-order logic Forgy. Charles. 1025 Fortran,3.141. HHI. 890. XIJ4 forward algorithm. Hl9. 111. 112. 614.'l74,999. 1U31 forward chaining. 1025 forward-backward algorithm. 109 four-color problem. 676 four-color theorem. 676 foxtrot. 1072 FP.Clm fract;~l, 547 Fracnkel. Aviczri. 11139 FSA.Sec• finite state m11chinc FSM. Sec finite state machine FSM-EMPTY.691 FSMs-JNTERSECT. 712 fsmton:~:e.t. 143 fsmwn•gexlrellristic, 137 FfP. 919 Fulkerson. Ray. 742 function. 761J #L-rccursive, 571 .584 computable. 379,414, 520, 5571 Sb5.570 computation by a Mealy machine. 97 computation hy a Moore machine.% computation hy a Turing machine, 379.557 computation hy grammars. 51M grammatically eomputahlc. 519 heuristic. 733. IOJ4,1041 . 1047 partial. SSS. 771 partialiL·recursive. 5/i partial recursive, 566 partially computahle. 379,557. 565 primitive recursive, 566. 583 f'-'Cursivc. 5M space-constructihlc. 719 static evaluation, 1041 time-constructible. M3 total. SSS. 771 functional programming. 415, 579, 890. M'JM functions un languages. 48, 179 alt. 312

clrnp.48 copymufreverst, IK4 firstdwr.~. 49

ma.t.ftrillg, 181

mitldrar. 184 middle. 312 nri:c.l82 prcf. 1&.'\ rcp/.485 slru.ffle, 184 suf/.183 twice. 184 Fux, J. J.• 1034 gadget.645 GALEN,946 Galois theory, 453 Galois. Evariste, 453 games 15-puzzle,43. 732.741 Amazons. 1039 backgammon, 1037 checkers.. 741), 1037 chess.7.648.687,740,1034,1037 computc:r.6. 7,101, 734.909,1022. 1026. 1046 crossword puzzles.l037 Go. 649. 687.740.1037, ltl9,1045 Hex.I039 Instant Insanity, 732.1037 interactive, 6, 7, 101,734,909, 1022.1026. 1046 Nim. 451. 621,720, 1035 11-puzzlc, 732 Othello. 1039 Suduku.52. 648,657, 743,1036 two-person. 648, 711, 1037 video.6,7. 101.734,909,1022, 1026.1046 game·.fturclt, I 042 game·st•tm:lr-a-J3, 1044 Garey. Michael, 673,712, 713, 743 gcd. See greatest common divisor Gene: Ontology Consortium, 946 generative capacity strong. l19, 232,241.986 weak. :19, 2..'\2. 241.986 genetic drift, 967 gcnctics.104, 964. 967 gcnomc,%5 genotype, 965 GG •• 503 Gillman. Leonard. 4 Ginsburg. Seymour, 361 . 583 Go.641J.687, 740,1037. 1039,1045 God.164 Gtxlcl numbering, 579 GOdel. Kurt, 412.498,579, S82, 750,752

Index Gilders Incompleteness Theorem,

Srt• Incompleteness 11tcort:m G {kfel's Completeness Theorem, Sl!t' Completeness 1lteorcm Goh.lhach's conjl-'Cture. ~6 Gun~tl c.

I02, I05

Gouda. Mohnmcd.925 grading pmgram. 464 Gradtt.YtLV Pttrlla:uum. 1034 grammar. 205 amhi~unus. 220

attribUII.!, 540. 554.583 context-free. 207 conh!Xl-scnsitivc. 529 cnrrt!C!Int•ss proof, 115. 512 dnncc. 1071 English. 211. 219, 232. 9K5. r~nture. 540.554.583. 994

furmal ism. 206.416. 510. 540, S44 HTML. I067 island, 241. Jlll. 915 Juva. 211. 229.247.351 . &!2. &'16 LL(k ). 337. J5tl music. 293,311.1031 normal forms, 232 OED definition. 1()65 phrase structure.360,Sil r~gulur. 155 right linear. 155 s~cification with BNF. 211.881 stochastic context-free, 243. 976.

99M Swiss German. 9112 transfurmational. 1033 ty!)l! 0. Sit. 540

unilicalinn, 541. 554, 583, 994 unrestricted. SIU, 583 gmmmttrwfsm. 157 ~rammatically Ctlmputable rum;tiun. 519 graph algorithms. 7. MS. 628.629 hipartite. 6T1. ~3 hist:ction. 694 chromatic numher. 646. 6T1, 6K9 cli'luc detection, 637,640, 646, fl57,tlt)l

coloring. 67(t. 611~. M7 complete, fl77 oonncct~d.24.622.62S.695

cut.693 ••tiiiP NWf'r f\7'\

encoding. 24. SCJl Eulerian circuit. 627,643.673, 690,928

Eulerian path, ft27 Hamiltonian circuit. 5!.1 I. 593. 637, 657.663.669, 673.74~

Hamiltonian path. 422, 637. 657 independent set. 645. 657. fl60. 678 isomorphism. 673 longest put h. 675, 6l)2 minimum spamsing tree, 6Z8. 643, 743, 91~

plannr,677 shortt:st path. 674. 732. 928 suhg.ruph isommphism. 657.673'ravcling salesman problem. 586. ~91 . 638.663,671 ,679.

6RIJ, 697,142

vertex cover. 660.675,692. 929 weighted, 628 graphics, 5. 1071 GRAPH-ISOMORPHISM. 673 greatest common divisor. 614,632,957 greedy algorithm. 630 Greibach normal form , 2.l4. 309, ~17.318.333.36 1 .839

Grcihnch. Sheila, 361 grep. 1050 Gries, David. 795 ground instance. 749. 822 Grumhcrg, Orna. 2tXl, 902 H.4%8. ~, 454, 482.501 ,523.556.

560,590,889 Hr,456,4~2. 498,951

H ~ANY• 476, 4H2.536

hackcrli, 959 Haken. Wolfgang. 676,744 HAu.• 461 , 479, 4~.484 hnlting.3l finite stale macbin~:s. 31 , 59, 369 linear bounded automata. S2K pushdown automata. 31 . 274. 369 Turing machines, 31 . 369. 426 halting prohlem. 31. 426,454, 598, 793 Hamiltonian circuit, 591.593,637, 657,(169. 1173. 743 directed graph. fl63 llamiltonian patlh, 422.637. 657 HAMILTONIAN·CIRCU lT.59J , 663. 637. 657,66~ . 67~. 743

HAMILTONIAN-PATH. 637.o57 HANy.4S9.482

1091

Harrison. Michael. 362, 951 Hart. Peter. 744 Hartmanis. Juris. 583,743 hashing. 596. 9fl0 Haskell. 41 5, 89(), R98 hearing aids. 1022 Hendler, James, 930 Herbison-E,•ans. Don. 1072 heterozygous organism. tn5. %:'i heuristic, 733 lunction, 733. 1034. 1041. 1047 search. 7, 731, 744, 895,HXl7, 1034, 1041, l047 Hex.1039 hidden Markov ntodel,106.614 for biological sequence matching, ~7J

for speech understanding. 999 of music, I 031 hierarchical regular expression. 135. 915 hierarchy theorems. 681 space.719 time. 644, 681.685 Hilbert. David. 413.4M8.582,583 Hilbert's lOth pr(,blem. 488.583 Hiller, Lejaren, l029, 1030 HMM. Sl!t hidden Markov model Hoare. C. A. R .. 744 homomorphism. 773 homozygous organism. %5 Hopcroft, John . .361 , 362, 583, 743.7.$4, 1073 Horn clause. 1012 Howell, Peter. 11>:\1

Ht'SC'HlJ. 1029 HTML, 5. 18,241, 1067 HTIP. 919 hypertext markup language. See HTML IBM 7090. 2

idempotence, 174 identity element. ns if statement, 229, 247. 248, 885, 894 1//inc· Suite for String Quune1, 1()3() immediate constituent analysis.360 Immerman. Neil. 718 lmmemlan-Szelcpcscnyi 'llteorem. 718 inbreeding. 104 Incompleteness Theorem. 41 2, 498,

579.582,752.900 inr.nno:illtPnrv '74\ I

1092

Index

INDEPENDENT-SET.64S.657.. MO.ii7R lndistinguishahilty uf strin~. A3 induction. 71'12 inl'~rcnce cn~tinc. 93R.

1024

Johnson, David. fl?J. 712. 713.743 Johnson. Selmct. 742 Juhnsnn. Stephen. 362 Jones. Kevin. Hl30 Jurafsky. Daniel. 361.362,583.978.

infinite c.lcsccnJin~· chain. 768 infinite set. 78R lnJinitt•FSM. 192

intix notation. 7711 'inherently amhiguous C'FL. 223. 33R. 4'19. ~!\. 8Xt'l

inherited attribute. 544 lnsinnamu. Pntrii'ia. ~61 Instant Insanity. 7J2. 1037 intcg~r programming. fl7~ INTEGERSlJM. 24. 26.27 intelligent ugcnt. 929. 1005. 1019. Hl2fl, 11'146

inlcructivc gr.~t>,·tPDAmufFSM.291. 321 imractuhility. 589.598. ii4l-l, n 1. 930. I 009, II)) I)

invnrianl lonp. St•t: Juur invarinnt inverse uf u functiun. 773 uf n relation, 7513 II~ 149. lJI'J IPA alphabet. 1001 I PL languages. X94 irrational numhcr. 7Rtl lsaa(;!;on. Leonard. 1030 island gmmm;tr, 241, 361, 915 island parsing.243.3nl isomorphism. 773 graph. 657.673 !'Ui'ograph. 657.673 iterative tlccpening,392.617.8.~. AA2 Jvcrsnn. Kenneth, 4 Jvcs. Charles. 1113ll Jacquard loom. 1()5? Jacquard. Joseph Marie, Hl57 .Java. 2 ll. 229, 247,)51.472, KR2. AA3. 8l-16. 1050 JCL.~

9N0,99'J

Karhumuki. Juhuni. 5B3 Karmurkar. Narendra. 67ft Karp. Richard. 743 Kasumi. Tmlao. 341, 3()2 Kasparov, Garry. 10~5 KB.St>t:' knowledge t:ln!iC' Khachian. Leonid. fl78 KhoussHinnv, Bakhadyr. 200 Kippen. Jim. 1032. Klecne slur. 8. 16 clnsurc under, 165. 2~'<. 445.535 in regular expressions.. 121) Klcenc. Stephen. 143, 1~}.5R.4 Klccni!'sThcorem. 133, 143.1lJt), 7tl7 Klccne's.NII·Il Theorem. Sett s-m-11 Thcnrcm. KM.X90 KNAPSACK. 638,657. 6QJ knapsack problem. 63R

Knight. Kevin. 1005 knowledge husc, 93(.1, HX17. 1015, 1023 Knuth. Donuld. 7U. 5X3. 60K, 743, 1045

Knuth-Mmris·Pratl algorithm, 70. 6U8.h20.743 Ko. Kcr-1,362 Knch island, 554 f{onigsbl!rg. Scv~n Bridges uf, 6Z6. fi'Xl

Kozcn. Dexter. 1041.1 Kripke structure. CXI2 Kruskal, Jos..:ph, 743 Kruskal's algurilhm. 629,743, 1)28 Kuipers. B1mjamin, 1021 Kuroda. S. Y.. 5R3 L. 714 L. NL'?. 715 Ladner. Richard, t172 Ladner's l11eorcm. 67Z. 743 utgarias. Jeff, 5R3

=

Laird.John,t02~

LALR parser, ~50 lamhda calculu~.413.414.582.R()() Landwehcr, Peter. 5g3 language. I() cardinality, 14 context-free, Sc•t context-free language

context-sensitive, See contextsensitive language decidable, St>e decidable language markup. Set markup language pmgrammingSt>e programming language query. Sre query language RE. St!c' recursively enumerable language recognition. 21.592 recursive. Sc·c recursive language• recursively enumerable. Set recursi\'ely enumerable lan!!uage regular, St•l! regular languagt: scmidecidab1c, See semidecidable language l)'pe O,Sc•e type 0 language type l . SI!e type !language type 2. St•e type 2 language type 3, Se,! type 3 language L1s Vl.'gas algorithm. 72.'i Lassila. Ora. 930 Latin squares, 11136 law of the excluded middle. 779 Lawler. Eu!!cne. 742 LBA. St'l' linear bounded automaton lt::lkagc. 951 L~hierc. Christian. ll)26 Lee. Lillian.31l2 left factoring, 336 left identitv. ns left recursive grammar rule, '332 left-most derivation, 220 legucy soflwan:. 242, 914, 916 lcgul reasoning, 1019. 1021 Leighton. Tom, 977 Lciscrson. Charles, 743 Lt!nstra. Jan. 742 Leonardo da Vinci. 1028 Lcsk. Michaci.J62 lcucr substitutkln, ()9, 166,289.321 closure under, lflb, 288 letter-equivalence of languages. 306 Levesque. Hector. HI07 Levin. Leonid. 64lJ, 743 Lewis. Harry.199,582,692, 1073 Undhlom, Bjorn, 1031 Lcx. 316. -'50. 362 lcxcr.326 lexical anul~is. 127. 145.222:, 325. 887 lcxicograph~c enumeration, 14. 443 lexicnp.raphtc order, 14 lexicographically Thring-enumerable language, 443 Lichtcnstc'in. Daviri tn.,n

Index

Life. game of, 414.411. 544 lightweight analysis. QJ4 Linden mayer system. 206, 414. 544, 583. 1034.1065. 1072 Lim.lenmayer, Aristid, 544, 5S3, tnn linear bounded automaton. 517, 583 linear programming, 678,742 linear set, 853 Linear Speedup Theorem, 599,744, 875

LINEAR-PROGRAMMING. 673, 678 Linz, Peter, 583 Lisp.51.24M,415,gt4.886,890.1006 literal in Boolean logic. h42.809 in first-order logic. 822 Jittlc-o, 6CMt. X75 Jittle-<>mcga, 600 L.L(k) grammar,337 language. 33S parser. 339 LMMI~· 65,69 logarithmic spa~ comph:xity, 714 space reduction, 716 Jogic. 745 Buolean, See Boolean logic first-order. See first-<>rder logic jn Al. l007 pr!!dicute. See first-order logic propositional. See Boolean logic temporal, 904 logic programming. 205,419. K27, 833.1015

l,.ogic Theori!it, 412. UHl8 tongest English sentence, 91W LONGEST-PATH, 675.691 longest-prefix heuristic. 349 1oom Jacquard. 1057 loop invariant, 215. 79S LR(k) ttrammar. 350 language. 350 parser.349 J..-systcm. Srt Lindenmayer system L1'. St!e ~ogic Theorist Lucas. Ednuard Anatole. 1058 tflachine learning, 897 Mctcrki.m m, 1030

Mahalanubis. P. C., 742 Mairson. Harry.l98

Manhattan distance.1048 map t:oloriog. 676. 693. 807 mapping reducibility. 455,464. 644 mapping reduction. 455. 464, 644 Markov algorithm, 414.418, 582 Markov chain, Su Markov model Markov model, 102, 200 higher order. lOS. 987 of English, 987 of inbn:eding. 104 of music, 1029, 1030 of weatht:r, 103 Markov. A. A .• Jr.. 418. 582 Markov. A. A., Sr.. 2(Xl markup languageS. 18. 30, 149, 930.932.935.937,1067 Martin. James, 361,362, 583. 978. 980,999 Martin. John. 362. 583, 1073 matching algorithm.1025 mathematical induction. 782 Mntiynsevich, Yuri. 48ij, 583 Matiyasevich's theorem. 488.583 Matlab.126 Coppersmith- Winograd algorithm. 344, 620,743 StraSSC1l's algorithm. 344. 362, 599.618. 621 . 743 MAX-BISECTION, 694 MAX-CUT. 694 maximal element. 767 ma.ntri11g, 52. 181. 184. 310, 3'11 McCarthy. John. 890 McCulloch, Warren, 199 McCulloch-Pitts neuron, 199 McGee, Deron.l034 Mealy machine. 97.199, 11,1(,() Mealy, George. 97, 199 medical databases, 946 medical reasoning, 827.946, lll24 Mellish, Christopher S., 1016 membership question for context-free languages. 314.315, 3t7 conteJCt-scnsitive languages. 528, 530. 537,713 regular languages. 188 SD languages. 431, 461 Thring machines,461 Mendelsohn, Noah. 33 Menger, Karl. 742 Mersenne numher. 781 Mersenne prime, 781 message routing. 674.928 MP.~nftn-r P..n"-1 ... -

., ...

1093

Mctaphone. 101 midcllar. 184. 555, 558

m;,/,ilt, 31 2 Millennium Problem~643. 742 Miller, Gary, 731 ,744 Miller-Rabin tesl, 731,744 MIN-CUT, 694 minDFSM. 91.95.191.607 minimal clement, 767 minimuiFSM, 193

minimalizahle function . 570 minimali7..ation of a function. 570 mininut:c, 740.895,1007.1040 miflinw..t -witll·a· J3·prtmifrg. 1043 minimi:r.ation of FSMs, 82. 193 of PDAs. 500.504 ofTuring machines. 475 minimum spanning trec.588,628, 643,742. 743, 92H Minsky. Marvin. 582 Minsky's Theorem, 582 Misra. Jay, 451. 863 mi.'t, l81. HW, 312,447 ML,415, 81)() model checking. 95. 116.200, 496. 641 .816.820,902.908.913 model of a wff. 749 model-hascd test-case generation, lJI3

Modula-2, 886 mudule~r arithmetic.6l4, 728,764,956 modulo equivalence. 728. 764 mudus poncns. 747.751 modus tollens. 747. 784 monO-Qpcrational protection frumework. 951 monotonic rcasoning.1008.1019 monolonicity of a heuristic function, 740 Monte Carlo algorithm. 725 Moonen, Lcon.242.361, 'H5 Moore machine. 96,199. 1060 Moore. Edward,%, 199 Moore, F. Richard. 1030 Moore. Ronald. 1045 morpheme. 978 morphologic-ell analysis. 99. 978.998 Morris. Robert. 998 Morris. James. 70. 608.743 most general unifier~830 motif protein, 14ft 964.971 Motwani, Rajeev, 743.744. 1073

1094

Index

MST. 630. 643. 928 MS1clt•t·i,Jt!. fl.ll Muller iiUtomuton, 21)() multiplier. IOCll Murphy. Gail. 915 music. R. 102, 2'13, 3 II. 1028 Mtt.~kul Din• (imm•. 11129 MUSICOMP. IIBU Mt~ikali.rclll!.t Wl'irjl!l.~pir/, 1029 mutation. 107.%4. 9M. %7. %S. ')73,975 mnllt:d exclusion, 117. :!IX). YON Mvllill. John. 21M I Myhiii-NcrmJt: Thcurcm. 90 NJ,IJJ(;

namcspilcc. 'J35 Nmdi, Duniclc. IJ411 n-ary function. 7711 n-ary rclutinn, 757 n·puzzlc. 732 naturilllanguagc ambiguity. St't' am"iguity in English dictionary. Sl't! Word net ~ramma r. Ser En!?.lish srammar p:~r.~er. St't p:1rscr. nut ural lan~uagc

processing. 5. 350. 8%, 97R, 11)(16 natural numhcr. 753 natural selection. %7 Naur, Peter. XXII. K81 NOFSM. Sr~ nond~h:rministic finite state muchine

II(05mt·tm~·,·rtlllldiiimulule.IU lldj'smsimu/(llr, 13.82, IHN. 275 lltlfsmlot(/~m. 75, 95, 165.191 . 19(),

199, 215. 295, 607. R35 ml.~pace, 718 negation as fuilurc.IOI9

negative Horn clause. IIJ13 negative literal in Boolean logic. Rll9 in first-orde r IO{lic. R22 Nc'lNDFSMs. 712 Ne11REGEX. 712 N~nK.Ic. A nil. 2CJO Nerude's Thcnrcm, YCI network

unalysis. 7. 627. MO. n7 design. 621J. 927 routing. h7.J. IJ2~ protocol. 5, IN. Rei. 126. 918 s~u rity. '1~9. 9.55

m:ural nctwork.l99 n~uron. nrtificial. ICJIJ

Newell. Allen. IIJI)g. 1112h. IOJ4 Nilsson, Nils. 744 Nim. 4SI, 623. 72U.I035 NL 71-J NL-comp1ctcness. 716 NL-hardncs.<;.. 716 NLP. St't' natural language proet.>ssing nude c
I(IJIJ nunplaycr charactcr.417. ICHfl nontcrminal alphuhc:l. 2US

nuntcrminal normal fnrm. 53.1 normal form. 232. 234. 241 J -conjunctive. 6-12. 810 Chomskv. 233. 2J5. 24H. 2lN. 313. .J IS. 324. 3.J2•.'61 , 533. 84() clause. 822. R.11 conjunctive fur Boolean logic. MIJ.809.1'11.1

conjunctive for rirst-ordcr lugic,

K12 deterministic. ~02. 847 disjunctive. 811. 917 for BliOic:.n h)~ic. XOCJ for context-rrcc grammars. 2~3

for context-sensitive grammars. 533 for datahasc queries. 916 ftlf' first-order logic. K21 fur pushdown automma. 266. 302.

M47 Greibach. 234. 3(1'). 317. 3 UU.lt 36J .S39

nontcrminnl. 533 prencx. 7116. 707. 821 restricted. 266, 3()2 Norvig, Peter. 740, 744. 10116 Nut:atiun3, ')Jfl Notkin, David. 'JI5 NOT-SIGMA-STAR. 712 NOT-SIGMA-STA R-SQUA KING, 713

NP.34. 6l3.6S8. 703.728 N l>c. •'H-e non player character N .,
OBDD. s,•t• ordered hi nary dccisinn Jiugram uhfuscntion. liN vbl,crvuhk universe, 198.436,570, 1(14()

11/trttinSclf, 475. 573.576,578 liCl'llr check. X21J Ochun. Guhricllu. 5R3 OCR. optical chornctcr n.·cuv.nitiun O'Dell. Mnrgarct. HM) Oc t tin~cr. Anthony. 361 Ol!dcn. Willinm •.3<,1 O!!dcn's Lemma, 303.312,361 Omega. ()(I() omc~:. ;IUiumatun. Sloc- w-automaton

s,.,.

c)( n )• .'l't•t• hil!-o

unc·tll·nnc functiun . 772 onto functiun. 772 ontnlo~y. 9)9, 944. lJ46. 997 open world assumption. 939 ~'Plical chaructcr recognition, 34. 107. fl74 optimi1.atinn prohlcm, 592 or introduction. 74M un.lcrcd binary dt:cision diagram, 95, 46. 4%, 642. 692, 721'

IU7,1J07 urdcrcd binary decision tree,816 ordered rnir, 157 uri~nmi , 4:'i3

OS/3flll JCL. 3 Othcllu. 11131J OWl.. IIJ. 944

P,34. 5XIJ.621.672.6XS.703.717. 72X P .. N.,'!, J4. 31J2 , !i37.~7.642 ~if!( A). St•t• power set PngcR<~nk , IU2. 105 Pu1Evcn. 29. 174. 210,244. 247. 255, 27'1. 2HO. 301.308.416 palindrome. 29. 174, 2111. 244,247, l55. 279.2S6, 301,308,

J13, -H6

PAM, 'J7U Papadimilriou. C'hristos. 199, S82. 61)2, 1073

Index

Parikh, Rohit, 361 Parikh's Theorem, 307.313,361,852 parity bit transducer. 97 parity checking, .5, 29, 59,79,144 parse table,339,341.349,350 parse tree, 218 represented as a list, 896 parser.218 bottom-up, 262,340, S44 chart, 352, 998 CKY,341 context-free. 218, 260,300,323,618 deterministic, 297.336,349 Earley, 352, 362 island, 243, 361 LALR,3SO LL(1),339 LR(k),349 natural language, 350, 983,994,998 predictive, 336 recurslvc-descent, 338 shift-reduce, 262,345 top-down, 261, 32.7, 544 parser generator, 350 part of speech ragging,l08,354, 981.998 partial p.-recursive function, 572 partial function,409,5S5, nt partial order, 765 partial recursive function, 566 partially computable function, 379,

551.565 partially ordered set, 765 partition of a set, 84, 638, 657, 691, 756,764 patent law, 473. 9l4 path Eulerian, 627 finding, 586,663, 674,714,116, 734,1047 Hamillonian, 422, 637, 657 in a directed gnph, 663, 716 in an undirected graph, 669,714 through a Thring machine, 370 pattern language. 127 p!iun, Gheorghe, 582 pCP. See Post Correspondence Problem pOA. Set pushdown automaton PDAMtN•504 pDAtoCFG, 271 peano arithmetic, 412,433,496.498,

688 Peled, Doron, 200.902 Perl.147.154.204.692.91~ ~

Perles. M .. 200,361, 583 P~ter, Rozsa, 584 Petrie, Ted, 200 phenotype. 965 phone,1000 phoneme. 1001 phonetic alphabet, 1001 Phonix,101 phrase structure grammar. 360, 511 physical security system, 6, 63, 126, .

948

pigeonhole principle, 86, 88. 170,

693,785 Pitts, Walter. 199 planar graph, 6TI plant de,•elopment. 204, 544,1071 poetry,l98 Politi, Michal, 910 polynomial space algorithm, 700 space complexity, 700 time algorithm, 604, 621 time complexity. 604, 621 time reduction, 644, 704 population genetics, 104 POS tagging, See part of speech tagging positive Horn clause, 1012 positive literal in Boolean logic, 809 in firsl-order logic, 82l Post Correspondence Problem, 489, 505,525,583,590.650 bounded version, 492,583, 658,692

MPCP,525.864 proof'in SO. 491 proof not in 0, 525,583, 864 Postnnachine,277,416 Post production system,414.416, 582,880,1022, 1053 Post, Emil,4l.5,416,489, 5M2, 583, 880,1022 power set. 756.790 Prague orloj, 1055 Pratt, Vaughan, 70, 608,743 precedence relation, 349 precedence table, 349 pred,580 predictive parser,336 pref, 183,312 prefix notation, no prefix ofa string, 10 prenex normal form, 706. 707,821 ft • .

.

1095

Presburger, Mojzesz, 583 Prim's algorithm, 743 primality testing, 7, 23, 38, 622.631, 643,690,728.744,956 primary structure of a protein, 963 prime factorization,614,633, 731. 958 ,prime number, 23, 38,631, 728,

779,956 prime number of a's.l75, 308 Prime Number 'Theorem, 957 Primea, 175, 308 PRIMES, 23, 38,591.632. 643,690,

728 primitive recursive function, 566, 583 Principia Matltematica,41'1., 1008 prior art, 473 probabilistic algorithm See randomized algorilhm· probabilistic context-free grammar, 243, 976, 997. 998 probabilistic finite automaton, 101 production system. Z03,416.418,

880,1022 profile HMM, 973 program. 792 program synthesis. 897 program verification, 23,116,473, 496.792. 898. 899 programming language, 5, 29, 30,

201.222,230.288,323. 351, 415,880 Ada, See Ada ALGOL 60 See ALGOL 60 Algol68, See Algol68 C, SeeC C++.SeeC.H Cobol, See Cobol Fortran, See Fortran Haskell, See Haskell IPL, See IPL languages Java, See Java Lisp, See Lisp ML,SeeML Modula-2, See Modula-2 Perl, See Perl Prolog, See Prolog Python, See Python Scheme, See Scheme, PTolog.205,419,516,10lS, 1034 proof by construction, 779 by contradiction.171. 281 . 4.1\?

1096

Index

by diilgonalization. 431. SM. fl85. 190 hy induction. 782 proper prefix. 10 proper subset. 766 proper substring. lU proper suffix. I 0 propositional logic. St't' Boolean logic propositional symbol. 745 Prosite. 973 protection framework. 951 protein alignment. %9 matching. 24. 107.. 145.968.970.973 motif, I-»>. 964. l>7J structure. l)h2 prutocol Alternating Bit.l«l, 126.911 ARQ.920 cumulative acknowledgcmtnl. 925 communicatinn. 62 Ethcrnct,IJI'I

F'TP. 9 JIJ H1TP.919 IP. 1411. 919 network. 5. IK. SO. 126.918 SMTP.919 Sliding Windt1W, 923 Stop-and-Wait. 920 TCP. MO. 919. 1J25 prnvef'OL . 497. R27 PrusinkiewicL, Przcmyslaw. SHJ PSPACE. 34. 537. 643. 688. 700. 717.1037 PSPAC.E-complctcness, 537.704, 1039 PSPACE-hardoe.'IS. 704 Pumping Theorem cunlcxt-frcc. IW. 2tn. 21)(). 292.. 303,305.307. 31l. 319, 3fll.H53,8M4 regular, t70.1~lJ.ll)J, Wlt20U punched card. IC)S7 pushdown automaton. 29. %49. 361 configuration. 2511 deterministic. 254.295 nondeterministic. 47.251.154 Putnum. Hilary. ICJ19 Psthon. 972. H150

QBE.Srt Query by Example QBf. 705. llll)tJ

quantified Boolean expression. 705 quantified Boolean ronnula. 706. 11109

quantifier exchange..751 quantum computing. 59K, 633 Query by Example. 916 query language. JX, 30.201.323, 336,\116. 937. 1066 queue plus FSM. 277. 414,416 qttiL'klrort. 7l3. 744

Ouiclle. Jean-Pierre. 902 Rabin auhlmaton, 200 Rabin. Michael,l99. 583.688.731, 744 Rahin-Miller test. 731.744 Rado.lihor. 583 railroad diagram. 8tl2 randomized algorithm. 7. 632.724 reactive system, Yto randomized Turing machine. 724 range uf a function, 769 Raphael. Bertram.744 RBS. !Wt' rule-based system RDF5.932 RDF Schema. 941 RDF/XML, 937 RDFS. See RDF Schema RE language. Set' recursively enumerable language Recursion Theorem. 475,573 n!Cursive function, S66 n:cursive function theory. S65 recursive grammar rule, 209. 332 recursive language. 376. 5M recursive-descent parser. 33R recursively enumcrahle language. 199.378.SM

reduce-reduce conllict, 348 reduction. 27,433.449, K62 for oomplt!xity. 644. 704 t'or um.lccidability. 452,494.498, 500.522, R64. HH9, 951 for unsemidecidabilitv, 476 logarithmic-space. 716 mapping. 455. 466.485. M4 polynomial-time, 644. 704 via c;omputation history. 500. S3K reflexive property of relations. 76() refutation complcteni!Ss, 81h, 827 refutation proof. &t proof by contradictiun rcgex. s" regular expression re}:tXW/mi. IJS. 1~

regular expression, 128, 199.204. 712, 10.'\U equivalence with FSMs.l33 for protein motifs. 972. hierarchical. 135. 9JS

in DTDs.1~9 in EBNF.881 in l~xical analysis. 325 in litthtweight analysis. 135.915 in Pert. 147.154.692,960.1050.1053 with squaring. 712 regular grammar. 155. 200.221.1031 regular language. 2l'l, S3. 60. l33.143, 162.198.199.279.624 regular Pumping The
replication of a string. 9 resolution. 750 in Buol~an logic, 691, 812 in first-m\ler logic. 497. 827, 1009, SLD. 834. 1014 rt•.wl•·t•·Brw/c:ull. Kl4 rtrs()/,,,._ fO L. 831 Resource Descripti<>n Framework, Set• RDF Resource Description Framework. Schema. See RDF Schema respectively. 311.991 restricted normal form for PDAs,

266. 302 restriction enzyme. 968 RETE. l025 reverse closure und~r. 165.288.445. S53 or languages. 17 ()(strings, 9 reverse engineering. 242. 914 rewrite system. 203.416.418, 510, 516.522.581,880.1022

Index

Rice. H. Gordon. 361.583 Rice's Theorem. 468.470.485 Rich. Elaine. )(Xl5 right identity. ns right linear grammar. ISS right-must derivation. 2211 Rinnooy Kan. A. H. G.. 742 Rivest. Ronald. 742. 743. 'l56 RNA.%6 folding. 213.967.975 Roads. Curtis. 1028 Robbins Algebra Conjecture. R27 Robinson. J. Alan. 827 Robinson. Juliu. 742 Robinson. Ruphael. 5.84 RoboCup.I062 rohot socc~r.6,59, 1061 Robson.J. M., ltl:\9 Roggenbach. Markus. 201) Rose. Allen. 4 Rose. Gene. 583 Rosen. Kenneth. 745 Rosenbloom. Paul. Hl26 route. linc.Jing. See pulh findin8 routing. 674.928 Rozen berg. G17.c&ort. 582. 583 RP. 726 RSA algorithm. 731 .956 Rubik's Cube. 52 rule or lcmll powcr.ll. tu31 rulc-ba!iCd system. 203. 416. 997. 11122. 1()31 , IC'l4NRUSsell. Bertrnnd. 412. IIX)8 Russell. Robert, lUll Rossell. Stuart. 740, 744. I006 Russell's f'llrRdox . 412, 94~ . 945

Roi'.zo. Walter. 951 sufcty. 951 Salomaa. Arlo. 582 Santa Claus. 164 SAT. 45.409. 4%. S90. 6.17, MO. M3. 649. 65U. bAA. 696, 70S, 743.~ 16

sAT solvcr,641 ,6.SO. Nl6 satisfiability in Bookan logJ:. 74fl. K12.St't' SAT in first-order lo~ic. 496, 749. 827 savitch. Walter, 744 savitch's Theorem, 643. 70U. 709. 714.744 Schaffer. John. 11134 scheduling problem, SR7. 641 . 645,

677,732, 733

Scheinberg. Stephen. 361 Schenker. Heinrich. Hl33 Scheme. 415. &:16 Schmidt. E.. 362 Schottstacdl. William. 1034 Schubert Lieder. 1033 Schutzcnbcrger. Marcel-Paul. 361, 583

Scott. Dana. 199 SCXML.9ll SD (the scmidccidahlc languagl.-s). 31.J78.427,435,4K2,5h5 SO/D. 436.616.895 search A*. 734, 740, 744. 895, UXI7, I047 avoidance by greedy nlgorithms. f13()

.

best-first, 734,895, }()34 hreadth-lirst. 392.616. 737, ~56. 862

by nmdumizwd algorithms. 724 depth-first. 327.392.616 heuristic,?. 731. 744,8'l5,1U07, JU34.1041 , 1047 in AI programs. 1nut. in game-playing,IU35.1037, 1040. 1047 in music composition. I034 in pattern malcthing. 60fl in primulity checking. 72~ in pw.zlc-solving.52.451, 741, 1()36 in SAT solving. 45. 641.6% in rcgcx matching, IU53 in solving the traveling salesmen problem. 5H6. fi39 simulating nundcterminism. 46. 3H9.fi33 in solving NP prohl~ms. 634 in theon:m proving, 'XH. I 009 iterative deepening, 392,617.857, 862

miniiiW.t. 1007.1040 secondary structure of a protein. %4 secondary structure of RNA. 975 security. 948 cryptography. 955 computer. 6. 485.650.949 network. 955,1)59 physical, 948 self inverse. 77S self-cmbedtling grammar.ZU9. 361 self-similarity. 546 semantic interpretation function. 18, ')}3

.

(Qr En~lish. 219.544,995. 999

.

1097

for regular expressions. 12R Semantic Web,5,7, 33. 929. 1005 semantics, 18 semidccidability. 31,37. 378.427.435 of ...,TJLES.4'J4 of first-order logic. 497 of the Post correspondence problem. 491 semidecidnble language. 31. 378. 427.435. 4S2. 565 scmidcciding by ll dclcrministic TM. 378 by a nondeterministic TM. 390 semidecision procedure. 40. 427.435 scmilincar. N53 semi-lbuc !>ystcm.511.523. 51:U. 5K:\ S\!ni1.er~ues. Gcraud. 320 sentence of 11rst-ordcr logic. 749 sequence alignment. 24.%9 set. 7!\3 complement. 756 intersection. 755 partition. 756.638,657.691.764 union. 755 set diffc:rence

closure undcr. l65. 28'J Seta. Takahiro. 743 Sethi. Ravi. 362.888 sct-of-supptu1 strategy, 815, li32.1014 SET-PA RTlTI ON, 638. 657 , b91 Seven Bridges of Konigst'lerg. 626. 690 Shamir.Adi. 956 Shamir. Eliahu. 200. 36l, 583 Slumnnn, Cl::mde. 9K7, IIJ34. 1040 Shuw.J. C.,JOOH shift-reduce conOict. 348 shift-reduce parser. 262, 345 Shmoys. David. 742 Shor. Peter, 743 Shor's al~orithm. 633.743 SHORTEST-PATH,674. 732,928 SHORTEST-SUPERSTRING, 658, 977 .Yiwffle. 184. 312 Sierpinsld triangle. 548. 550 Sifakis. Joseph. 902

simcumpttter. 394 Simon. Herbert. HXl8. 1034 SIMPLE-MAX-CUT. 694 simpiL'·rel\'rite. 204, 215 simplex algon thm. 67K Simulink. 126

1098

Index

SipseY. Michael. 2(11. 5K4. 744.726. )()~9. 1073 size of universe. 19N, 436. 571t. 1040 Skolem constant. X23 Skolcm function. 823 Skolem. Thoraf. R22 Skolcmization. 812. H24 SLD resolution.IS34. Ul14 Slidin!! Window protocol. 923 smn Theorem. se, s-m-11 Theorem s-m-n 'Theorem. 579, S!W SMTP.9llJ SOAR. 10Z6.1049 soccer-plnying robot. 6. 59. llt6Z sonata form. 10~ 1 sorting. 26. 723. 744 Soules, George. 2110 suund wave. I()(XI Soundcx, 99,100. 126. soundness of an inference 'rulc.747. 750 of resolution. Mlh.K27 space complexity. 595, tl'JS with rc:spcct lt) a world. 752 space hierarchy theorems. 719 space-constructi"lc function. 719 space-craft controller. 910 spam. 106. I
Speedup Theorem, 599, 744. 875 spell checking. 9xg SPIN, 9111 SOL. 91fl.l066 standards delinition of, 7. 925, '132, 939 statcchart. 79. 126.910 statement of first-onJer logic. 749 static evaluation function. Hl41 statistical AI,IU07 STCON, 716.717 Stearn,;, Richard E.• 743 Stein. Clifford. 742. 743 stochastic context-free grammar, l4l. 976. W7. 99M stochastic finite automaton. 101 Stock. Olivicro. 361 Stockmcyer. Larry. 1040 Stone. Pc:tcr. 11162 Strassen. Volker, 362, 611!1. 743 Strassen's algorithm, 344. 362. SIN, 618.621.743

string. 8' string search. 65. 69. 145. 6CI6. h20. 743 STRING-SEARCH. (1()6 strung gencmtive capacity. 119. 232,

241 , \}X(, strong induction. 785 SUBGRAPH-ISOMORPHISM. 657.673 -suhlinear space complexity. 713 suhsct. 754.765 SUBSET-SUM. 637. f\57.6\11 suhstring. Ul. ~05 subllumptiun. 766. Kl5. K.~2. 940 successor functit)O. 3X1. 519, 555. 5SR.5hll.571.572, 76~

Suc.Jkump.lltomus. 712.744 Suduku. 52. MX. 657. 74.:1, 1036 SlljJ,I83 sunix of u string. 10 Sundhcrg. Juhan. 1031 surface structure. 1()~3 SVG. 2<16. 1071 Swis.o; Gernmn gr.tmmar. W2 swilch statement. XX2 Sn-itdlt'd mr Budt. Hl28 symtx1lic modl!l chc:cking. 95. 1 th. 211().4%. MUll h. K20. 94t2. I.J( lti, 'J 13 ~')'mmctric

key system. 955 symmetric matrix, 761 symmt!tric property or relations. 761 ~yntactic analysis in compilers. 323, 8M7 in nuturallangut~gc proce.o;sing, 350.'JX3

synthcsi1.cd altrihutc. 544 Szclcpcsenyi. Robert. 71 K tahla urumm1ng. 1032 tag. system. 277.414.415. SKI. 5~2 tautolugy Boolc~sn logic. M«l. 705. 746. XI K firsHm.h:r logic. 749 ta.~icah ripoff prohlcm. h')2 Taylor. R. Gregory. 582 TCP. 80.1119.925 temporal logic. 904 term. 748 in expression grammars. 228 in first-order lngic. 74K terminal alphahct. 155.205,510 t'--sl case generation. 913.1JK9 tl!xt editing, 70. 1052

h!Xt- proccssin~ software, 989

theorem. 747.750 theorem prover. 901.944. 1008,1020 theorem pruving.450. 722. 814 applications. 820. 900,901.944, !IUS. 1010. 1020 Boolean. 743.H12.1008 first-order logic. 748 resolution. Kl2.827 thCOf)'. first·t>rdcr logical, 497,751 th'-osaurus. 947 Thcta.6UI Thomas. Wolfgang. 200 Thue system, 523 Thue.Axcl, 523 Tl LES. 493. 583. 650.693 tiling prohlcm. 492. 583.650,693 time complexity. 595.621, 798 time: hierarchy theorems, 644. 6MI.685

time-constructible function, 683 TM. S('t' Turing machine TMMIS• 475. ~2 TM RE(i• 448. 471.482 token.I45. 32S.352.882. 887 tor·d''"'" parser.261, 327.S44 total function. 555. nl totnl orllcr. 14.767 tow/FSM. l91

totally ordered set, 768 Towers of H~moi. 164.590,1058 lnn!!-uagc. 106() tracrahility. 7. 34.589.598,621, 9311, IIJ06 truining prohlcm in HMMs.109, 975.1001,1029 trnnsducer hidircctional. 98.980 finite state. 96.125.199.925.971. H14!,, 1054.1060 Mealy machine, 97 Moure machine. 96 stochastic. lUI tmnsformntionsl system. 910 tmnsfurmational grammar, 1033 transitive closure. n6 tmnsiti\•ity. 761 traveling salesman problem. 7. 586, 592.593,633.638.643, M7, M.l671 ,678,679, M9,697. 721.742 triangle inequality. 5ll8. 122.742 trigram mudcl. IJR7 tri~mm taggcr. 9N2

Index trisecting an ang,le, 453 TSP. See traveling salesman problem 'TSP·DECIDE. ~93 . 633.634.638. 643, 671, 679' 697

1\JringAward.4,199. 743.881 1\Jring machine. 30. 364 alternating. 1039 deterministic, 36!. 595,621. 700 encodin& of. 400 enumeration by. 440 enumeration of. 402 function computation. 379. 557 macro language,for 371 minimization, 475 multitape. 382 nondeterministic, 47,388, :59.5, 633,700 randomized. 724 universal. 404 Turing reducibility. 454 1\Jring test. 1004, 1021 Turinl, Alan. 413, 582. 1004, 1020. 1034 ')"uring·enumerable language, 440 1\Jring-recognizable language, 378 tutoring system,l02S ,wice.l84 twin primes. 184 two-person games. 641S, 711.1037 type 0 grammar. 511 type 0 language. 540 type I language, 540 type 2 language, 540 type 3 language, 540 &ype cbecking.20J.311.883.887 U (the universaiTM),400, 404.410 Ullman, Jeffrey. 361,362. 5S3, 743, 744,951,1073

UML. 9ll unary function. 770 uncountably infinite set, 792 undecidability. 37,448 of a simple security system, 951 of context-free languages, 223,499 of first-order logic, 413,496,827, 1009, 1020 of Hilhert's 10th problcm.488 of the halting problem. 429.454,598 of the Post correspondence problem. 864, 491

of tiling problems. 494 uni£ication. 542, 827 unification grammar, 541.554,583, 994 Unified Modeling Language. 911 tmify·for·rtsolution, 828 union closure under, 49, 165.288,298, 445.534 of sets, 755.775 unit preference strategy. 815, 832 unit production, 236,240.839 universal instantiation. 751 universal resource identifaer, 61. 247,931 universal Thring machine, 400, 404, 410 universe, age of. 570, 586,598, 1059 size of, 198.436. 570. 1040 unreachable variahles in context~ free grammars,214 unrestricted grammar, 414,418.510, 583.1032 UNSAT. 6110,816 unsatisfiability 10 Boolean logic. 680,746, 812 in first-order logic, 749, 827 unweighted graph, 628 URI. See universal resource identifier USAT,692 useless variables in context-free grammars.. 214 tJSTCON, 714 Valiant, Leslie. 362 VALID,496, 705.680 validity in Boolean logic, 496. 640, 680. 746 in first-order logic, 496,749 vending machine FSM, 5, 54, 144 Venn diagram, 755 \'erification of programs. 23, 116, 473,496,792,898,899 verifier. 635 vertex. cover. 660, 675. 692.929 VERTEX-COVER. 660.675,692,

929 video game, See games, video

vim.t,573

1099

virus detection, 463. 573. 9!9 Viterbi algorithm, 108.110.\12, 200.614,975.977,982,

999, 1031 Viterbi. Andrew, 200 W3C, See World Wide Web Consortium Wang tile. 491, 583.693 Wang, Hao. 583 Wang's conjecture, 494,583 Warshall's algorithm, 779 WcW, 216. 287,292,377.S28,6l7, 883,991 weak generative capacity. 219,232, 241,986 weighted graph, 628 Weiss, Norman, 200 well-formed formula, Set wff well-founded set 768. 793 well-ordered set,38. 768. 785 Wesl, Robert, lOll wff Boolean logic. 74S first-order logic, 748 Whitehead. Alfred North,412.1008 Winograd, Shmuel, 344.362,620,743 without$, m. 321 Wolfram. Stephen, 421, S82 word problem. 522 Wordnet, 947 working memory, 1024 World Wide Web. 5. 19, 22. 33, 43, 61, 102.105.24l,336.910, 916,919, 929,1005,101S,

1067, 1071 World Wide Web Consortium, 932 worst-case performance. 315,596,723, 724, 820. 945.959, 1020

ww. 30,279,192.409.416.418.425, 513, 553. 991. l037.. 10S2

XML; 19, 149,935,937.1068,1071 'iacc,327,350.362 Yato. Takayuki, 743 'ioungcr. Daniel. 341,362 zero of a function. 775 Zloof, MosM, 916 ZPP, 727

utomata, Computability, and Complexity Elaine Rich ~\ utomata,

Computability and Complexity illuminates the elegant theoretical

underpinnings of computing and brings theory to life by demonstrating its influence n modern hardware and software system design . Dr. Elaine Rich begins w ith fin ite state machines and regular languages, then turns to ontext-free languages and parsing techniques. Next, she introduces Turing machines nd equivalent models of computation, as well as the crucial question of undecidability. uilding on this material, she then considers theproblem of practical computability. Throughout, the author shows how key con cepts are applied, giving readers practical hsight for using computing theory in their own work. Applications discussed include: rogramming languages, compilers, networking, natural language processing, artificial telligence, computational biology, security, games, business rule modeling, markup anguages, Web search, and more. An appendix offers a detailed refresher on the athematical techniques associated with the principles of computing.

• Understand1ng computation, including decision procedures, nondeterminism, and functions on languages • The hierarchy of language classes: regular, context-free, decidable, and semidecidable languages • Exploring formal computation and the problems it can solve • Discovering fundamental limitations on what can be computed • Finite state machines, regular expressions, and reg ular grammars • Context-free languages and grammars, and non-context-free languages • Algorithms and decision procedures for regular and context-free languages • Turing machines and undecidability: from the Church-Turing Thesis and the Halting Problem to the Chomsky Hierarchy • Decidability and undecidability proofs • Analyzing complexity: time and space complexity classes

About the Author Elarne Rich. Senior Lecturer at The University of Texas at Austin, currently teaches Automata Theory, Artificral Intelligence, and other courses. She is the author of Artificial Intelligence, which has been honored on CACM's list of classic computer'science textbooks. She holds a Ph.D. rn Computer Science from Carnegre-Mellon Univers1ty.

ISBN-13: 978-o-13-228806-4 0·13·228806·0 ISBN-10:

9 0 0 0 0

m

> z Upper Saddle River, NJ 07458 www nrPnhall.com

9

780 13 2 2 88 064