X-Git-Url: http://lambda.jimpryor.net/git/gitweb.cgi?p=lambda.git;a=blobdiff_plain;f=zipper-lists-continuations.mdwn;h=fdfc2a556d1c36cacee72fd5dde5084223f14d68;hp=d13dc353d71e06e2aaa4446771ddf5df21360f2f;hb=db89aab6e40647f64f6f51ed6281c0cbce361550;hpb=b078fa71d02d362b0b9b763abc77f75d8f19e22e diff --git a/zipper-lists-continuations.mdwn b/zipper-lists-continuations.mdwn index d13dc353..fdfc2a55 100644 --- a/zipper-lists-continuations.mdwn +++ b/zipper-lists-continuations.mdwn @@ -5,15 +5,13 @@ continuation monad. The three approches are: -* Rethinking the list monad; -* Montague's PTQ treatment of DPs as generalized quantifiers; and -* Refunctionalizing zippers (Shan: zippers are defunctionalized continuations); +[[!toc]] Rethinking the list monad ------------------------- To construct a monad, the key element is to settle on a type -constructor, and the monad naturally follows from that. I'll remind +constructor, and the monad naturally follows from that. We'll remind you of some examples of how monads follow from the type constructor in a moment. This will involve some review of familair material, but it's worth doing for two reasons: it will set up a pattern for the new @@ -24,58 +22,64 @@ and monads). For instance, take the **Reader Monad**. Once we decide that the type constructor is - type 'a reader = fun e:env -> 'a + type 'a reader = env -> 'a then we can deduce the unit and the bind: - runit x:'a -> 'a reader = fun (e:env) -> x + let r_unit (x : 'a) : 'a reader = fun (e : env) -> x -Since the type of an `'a reader` is `fun e:env -> 'a` (by definition), -the type of the `runit` function is `'a -> e:env -> 'a`, which is a +Since the type of an `'a reader` is `env -> 'a` (by definition), +the type of the `r_unit` function is `'a -> env -> 'a`, which is a specific case of the type of the *K* combinator. So it makes sense that *K* is the unit for the reader monad. Since the type of the `bind` operator is required to be - r_bind:('a reader) -> ('a -> 'b reader) -> ('b reader) + r_bind : ('a reader) -> ('a -> 'b reader) -> ('b reader) We can deduce the correct `bind` function as follows: - r_bind (u:'a reader) (f:'a -> 'b reader):('b reader) = + let r_bind (u : 'a reader) (f : 'a -> 'b reader) : ('b reader) = We have to open up the `u` box and get out the `'a` object in order to feed it to `f`. Since `u` is a function from environments to -objects of type `'a`, we'll have +objects of type `'a`, the way we open a box in this monad is +by applying it to an environment: .... f (u e) ... This subexpression types to `'b reader`, which is good. The only -problem is that we don't have an `e`, so we have to abstract over that -variable: +problem is that we invented an environment `e` that we didn't already have , +so we have to abstract over that variable to balance the books: fun e -> f (u e) ... This types to `env -> 'b reader`, but we want to end up with `env -> -'b`. The easiest way to turn a 'b reader into a 'b is to apply it to +'b`. Once again, the easiest way to turn a `'b reader` into a `'b` is to apply it to an environment. So we end up as follows: - r_bind (u:'a reader) (f:'a -> 'b reader):('b reader) = f (u e) e + r_bind (u : 'a reader) (f : 'a -> 'b reader) : ('b reader) = f (u e) e And we're done. +[This bind is a condensed version of the careful `let a = u e in ...` +constructions we provided in earlier lectures. We use the condensed +version here in order to emphasize similarities of structure across +monads.] + The **State Monad** is similar. We somehow intuit that we want to use the following type constructor: - type 'a state = 'store -> ('a, 'store) + type 'a state = store -> ('a, store) So our unit is naturally - let s_unit (x:'a):('a state) = fun (s:'store) -> (x, s) + let s_unit (x : 'a) : ('a state) = fun (s : store) -> (x, s) And we deduce the bind in a way similar to the reasoning given above. First, we need to apply `f` to the contents of the `u` box: - let s_bind (u:'a state) (f:'a -> ('b state)):('b state) = + let s_bind (u : 'a state) (f : 'a -> 'b state) : 'b state = But unlocking the `u` box is a little more complicated. As before, we need to posit a state `s` that we can apply `u` to. Once we do so, @@ -86,8 +90,8 @@ is an `'a`. So we have to unpack the pair: Abstracting over the `s` and adjusting the types gives the result: - let s_bind (u:'a state) (f:'a -> ('b state)):('b state) = - fun (s:state) -> let (a, s') = u s in f a s' + let s_bind (u : 'a state) (f : 'a -> 'b state) : 'b state = + fun (s : store) -> let (a, s') = u s in f a s' The **Option Monad** doesn't follow the same pattern so closely, so we won't pause to explore it here, though conceptually its unit and bind @@ -97,7 +101,7 @@ Our other familiar monad is the **List Monad**, which we were told looks like this: type 'a list = ['a];; - l_unit (x:'a) = [x];; + l_unit (x : 'a) = [x];; l_bind u f = List.concat (List.map f u);; Recall that `List.map` take a function and a list and returns the @@ -117,21 +121,21 @@ And sure enough, But where is the reasoning that led us to this unit and bind? And what is the type `['a]`? Magic. -So let's take a *completely useless digressing* and see if we can -gain some insight into the details of the List monad. Let's choose -type constructor that we can peer into, using some of the technology -we built up so laboriously during the first half of the course. I'm -going to use type 3 lists, partly because I know they'll give the -result I want, but also because they're my favorite. These were the -lists that made lists look like Church numerals with extra bits -embdded in them: +So let's indulge ourselves in a completely useless digression and see +if we can gain some insight into the details of the List monad. Let's +choose type constructor that we can peer into, using some of the +technology we built up so laboriously during the first half of the +course. We're going to use type 3 lists, partly because we know +they'll give the result we want, but also because they're the coolest. +These were the lists that made lists look like Church numerals with +extra bits embdded in them: empty list: fun f z -> z list with one element: fun f z -> f 1 z list with two elements: fun f z -> f 2 (f 1 z) list with three elements: fun f z -> f 3 (f 2 (f 1 z)) -and so on. To save time, we'll let the Ocaml interpreter infer the +and so on. To save time, we'll let the OCaml interpreter infer the principle types of these functions (rather than deducing what the types should be): @@ -149,7 +153,7 @@ types should be): Finally, we're getting consistent principle types, so we can stop. These types should remind you of the simply-typed lambda calculus types for Church numerals (`(o -> o) -> o -> o`) with one extra bit -thrown in (in this case, and int). +thrown in (in this case, an int). So here's our type constructor for our hand-rolled lists: @@ -162,22 +166,22 @@ ints), we have So an `('a, 'b) list'` is a list containing elements of type `'a`, where `'b` is the type of some part of the plumbing. This is more -general than an ordinary Ocaml list, but we'll see how to map them -into Ocaml lists soon. We don't need to grasp the role of the `'b`'s +general than an ordinary OCaml list, but we'll see how to map them +into OCaml lists soon. We don't need to fully grasp the role of the `'b`'s in order to proceed to build a monad: - l'_unit (x:'a):(('a, 'b) list) = fun x -> fun f z -> f x z + l'_unit (x : 'a) : ('a, 'b) list = fun x -> fun f z -> f x z No problem. Arriving at bind is a little more complicated, but exactly the same principles apply, you just have to be careful and systematic about it. - l'_bind (u:('a,'b) list') (f:'a -> ('c, 'd) list'): ('c, 'd) list' = ... + l'_bind (u : ('a,'b) list') (f : 'a -> ('c, 'd) list') : ('c, 'd) list' = ... Unfortunately, we'll need to spell out the types: - l'_bind (u: ('a -> 'b -> 'b) -> 'b -> 'b) - (f: 'a -> ('c -> 'd -> 'd) -> 'd -> 'd) + l'_bind (u : ('a -> 'b -> 'b) -> 'b -> 'b) + (f : 'a -> ('c -> 'd -> 'd) -> 'd -> 'd) : ('c -> 'd -> 'd) -> 'd -> 'd = ... It's a rookie mistake to quail before complicated types. You should @@ -190,7 +194,7 @@ This time, `u` will only deliver up its contents if we give `u` as an argument a function expecting an `'a`. Once that argument is applied to an object of type `'a`, we'll have what we need. Thus: - .... u (fun (x:'a) -> ... (f a) ... ) ... + .... u (fun (a : 'a) -> ... (f a) ... ) ... In order for `u` to have the kind of argument it needs, we have to adjust `(f a)` (which has type `('c -> 'd -> 'd) -> 'd -> 'd`) in @@ -198,21 +202,21 @@ order to deliver something of type `'b -> 'b`. The easiest way is to alias `'d` to `'b`, and provide `(f a)` with an argument of type `'c -> 'b -> 'b`. Thus: - l'_bind (u: ('a -> 'b -> 'b) -> 'b -> 'b) - (f: 'a -> ('c -> 'b -> 'b) -> 'b -> 'b) + l'_bind (u : ('a -> 'b -> 'b) -> 'b -> 'b) + (f : 'a -> ('c -> 'b -> 'b) -> 'b -> 'b) : ('c -> 'b -> 'b) -> 'b -> 'b = - .... u (fun (x:'a) -> f a k) ... + .... u (fun (a : 'a) -> f a k) ... -[Excercise: can you arrive at a fully general bind for this type +[Exercise: can you arrive at a fully general bind for this type constructor, one that does not collapse `'d`'s with `'b`'s?] As usual, we have to abstract over `k`, but this time, no further adjustments are needed: - l'_bind (u: ('a -> 'b -> 'b) -> 'b -> 'b) - (f: 'a -> ('c -> 'b -> 'b) -> 'b -> 'b) + l'_bind (u : ('a -> 'b -> 'b) -> 'b -> 'b) + (f : 'a -> ('c -> 'b -> 'b) -> 'b -> 'b) : ('c -> 'b -> 'b) -> 'b -> 'b = - fun (k:'c -> 'b -> 'b) -> u (fun (x:'a) -> f a k) + fun (k : 'c -> 'b -> 'b) -> u (fun (a : 'a) -> f a k) You should carefully check to make sure that this term is consistent with the typing. @@ -226,25 +230,377 @@ replicating the behavior of the standard List monad. Let's test: l'_bind (fun f z -> f 1 (f 2 z)) (fun i -> fun f z -> f i (f (i+1) z)) ~~> -Sigh. Ocaml won't show us our own list. So we have to choose an `f` -and a `z` that will turn our hand-crafted lists into standard Ocaml +Sigh. OCaml won't show us our own list. So we have to choose an `f` +and a `z` that will turn our hand-crafted lists into standard OCaml lists, so that they will print out. -# let cons h t = h :: t;; (* Ocaml is stupid about :: *) -# l'_bind (fun f z -> f 1 (f 2 z)) - (fun i -> fun f z -> f i (f (i+1) z)) cons [];; -- : int list = [1; 2; 2; 3] + # let cons h t = h :: t;; (* OCaml is stupid about :: *) + # l'_bind (fun f z -> f 1 (f 2 z)) + (fun i -> fun f z -> f i (f (i+1) z)) cons [];; + - : int list = [1; 2; 2; 3] Ta da! -Just for mnemonic purposes (sneaking in an instance of eta reduction -to the definition of unit), we can summarize the result as follows: +To bad this digression, though it ties together various +elements of the course, has *no relevance whatsoever* to the topic of +continuations... + +Montague's PTQ treatment of DPs as generalized quantifiers +---------------------------------------------------------- + +We've hinted that Montague's treatment of DPs as generalized +quantifiers embodies the spirit of continuations (see de Groote 2001, +Barker 2002 for lengthy discussion). Let's see why. + +First, we'll need a type constructor. As you probably know, +Montague replaced individual-denoting determiner phrases (with type `e`) +with generalized quantifiers (with [extensional] type `(e -> t) -> t`. +In particular, the denotation of a proper name like *John*, which +might originally denote a object `j` of type `e`, came to denote a +generalized quantifier `fun pred -> pred j` of type `(e -> t) -> t`. +Let's write a general function that will map individuals into their +corresponding generalized quantifier: + + gqize (x : e) = fun (p : e -> t) -> p x + +This function wraps up an individual in a fancy box. That is to say, +we are in the presence of a monad. The type constructor, the unit and +the bind follow naturally. We've done this enough times that we won't +belabor the construction of the bind function, the derivation is +similar to the List monad just given: + + type 'a continuation = ('a -> 'b) -> 'b + c_unit (x : 'a) = fun (p : 'a -> 'b) -> p x + c_bind (u : ('a -> 'b) -> 'b) (f : 'a -> ('c -> 'd) -> 'd) : ('c -> 'd) -> 'd = + fun (k : 'a -> 'b) -> u (fun (x : 'a) -> f x k) + +How similar is it to the List monad? Let's examine the type +constructor and the terms from the list monad derived above: type ('a, 'b) list' = ('a -> 'b -> 'b) -> 'b -> 'b - l'_unit x = fun f -> f x + l'_unit x = fun f -> f x l'_bind u f = fun k -> u (fun x -> f x k) -To bad this digression, though it ties together various -elements of the course, has *no relevance whatsoever* to the topic of -continuations. +(We performed a sneaky but valid eta reduction in the unit term.) + +The unit and the bind for the Montague continuation monad and the +homemade List monad are the same terms! In other words, the behavior +of the List monad and the behavior of the continuations monad are +parallel in a deep sense. To emphasize the parallel, we can +instantiate the type of the list' monad using the OCaml list type: + + type 'a c_list = ('a -> 'a list) -> 'a list + +Have we really discovered that lists are secretly continuations? Or +have we merely found a way of simulating lists using list +continuations? Both perspectives are valid, and we can use our +intuitions about the list monad to understand continuations, and vice +versa (not to mention our intuitions about primitive recursion in +Church numerals too). The connections will be expecially relevant +when we consider indefinites and Hamblin semantics on the linguistic +side, and non-determinism on the list monad side. + +Refunctionalizing zippers +------------------------- + +Manipulating trees with monads +------------------------------ + +This thread develops an idea based on a detailed suggestion of Ken +Shan's. We'll build a series of functions that operate on trees, +doing various things, including replacing leaves, counting nodes, and +converting a tree to a list of leaves. The end result will be an +application for continuations. + +From an engineering standpoint, we'll build a tree transformer that +deals in monads. We can modify the behavior of the system by swapping +one monad for another. (We've already seen how adding a monad can add +a layer of funtionality without disturbing the underlying system, for +instance, in the way that the reader monad allowed us to add a layer +of intensionality to an extensional grammar, but we have not yet seen +the utility of replacing one monad with other.) + +First, we'll be needing a lot of trees during the remainder of the +course. Here's a type constructor for binary trees: + + type 'a tree = Leaf of 'a | Node of ('a tree * 'a tree) + +These are trees in which the internal nodes do not have labels. [How +would you adjust the type constructor to allow for labels on the +internal nodes?] + +We'll be using trees where the nodes are integers, e.g., + + +
+let t1 = Node ((Node ((Leaf 2), (Leaf 3))),
+               (Node ((Leaf 5),(Node ((Leaf 7),
+                                      (Leaf 11))))))
+
+    .
+ ___|___
+ |     |
+ .     .
+_|__  _|__
+|  |  |  |
+2  3  5  .
+        _|__
+        |  |
+        7  11
+
+ +Our first task will be to replace each leaf with its double: + +
+let rec treemap (newleaf:'a -> 'b) (t:'a tree):('b tree) =
+  match t with Leaf x -> Leaf (newleaf x)
+             | Node (l, r) -> Node ((treemap newleaf l),
+                                    (treemap newleaf r));;
+
+`treemap` takes a function that transforms old leaves into new leaves, +and maps that function over all the leaves in the tree, leaving the +structure of the tree unchanged. For instance: + +
+let double i = i + i;;
+treemap double t1;;
+- : int tree =
+Node (Node (Leaf 4, Leaf 6), Node (Leaf 10, Node (Leaf 14, Leaf 22)))
+
+    .
+ ___|____
+ |      |
+ .      .
+_|__  __|__
+|  |  |   |
+4  6  10  .
+        __|___
+        |    |
+        14   22
+
+ +We could have built the doubling operation right into the `treemap` +code. However, because what to do to each leaf is a parameter, we can +decide to do something else to the leaves without needing to rewrite +`treemap`. For instance, we can easily square each leaf instead by +supplying the appropriate `int -> int` operation in place of `double`: + +
+let square x = x * x;;
+treemap square t1;;
+- : int tree =ppp
+Node (Node (Leaf 4, Leaf 9), Node (Leaf 25, Node (Leaf 49, Leaf 121)))
+
+ +Note that what `treemap` does is take some global, contextual +information---what to do to each leaf---and supplies that information +to each subpart of the computation. In other words, `treemap` has the +behavior of a reader monad. Let's make that explicit. + +In general, we're on a journey of making our treemap function more and +more flexible. So the next step---combining the tree transducer with +a reader monad---is to have the treemap function return a (monadized) +tree that is ready to accept any `int->int` function and produce the +updated tree. + +\tree (. (. (f2) (f3))(. (f5) (.(f7)(f11)))) +
+\f    .
+  ____|____
+  |       |
+  .       .
+__|__   __|__
+|   |   |   |
+f2  f3  f5  .
+          __|___
+          |    |
+          f7  f11
+
+ +That is, we want to transform the ordinary tree `t1` (of type `int +tree`) into a reader object of type `(int->int)-> int tree`: something +that, when you apply it to an `int->int` function returns an `int +tree` in which each leaf `x` has been replaced with `(f x)`. + +With previous readers, we always knew which kind of environment to +expect: either an assignment function (the original calculator +simulation), a world (the intensionality monad), an integer (the +Jacobson-inspired link monad), etc. In this situation, it will be +enough for now to expect that our reader will expect a function of +type `int->int`. + +
+type 'a reader = (int->int) -> 'a;;  (* mnemonic: e for environment *)
+let reader_unit (x:'a): 'a reader = fun _ -> x;;
+let reader_bind (u: 'a reader) (f:'a -> 'c reader):'c reader = fun e -> f (u e) e;;
+
+ +It's easy to figure out how to turn an `int` into an `int reader`: + +
+let int2int_reader (x:'a): 'b reader = fun (op:'a -> 'b) -> op x;;
+int2int_reader 2 (fun i -> i + i);;
+- : int = 4
+
+ +But what do we do when the integers are scattered over the leaves of a +tree? A binary tree is not the kind of thing that we can apply a +function of type `int->int` to. + +
+let rec treemonadizer (f:'a -> 'b reader) (t:'a tree):('b tree) reader =
+  match t with Leaf x -> reader_bind (f x) (fun x' -> reader_unit (Leaf x'))
+             | Node (l, r) -> reader_bind (treemonadizer f l) (fun x ->
+                                reader_bind (treemonadizer f r) (fun y ->
+                                  reader_unit (Node (x, y))));;
+
+ +This function says: give me a function `f` that knows how to turn +something of type `'a` into an `'b reader`, and I'll show you how to +turn an `'a tree` into an `'a tree reader`. In more fanciful terms, +the `treemonadizer` function builds plumbing that connects all of the +leaves of a tree into one connected monadic network; it threads the +monad through the leaves. + +
+# treemonadizer int2int_reader t1 (fun i -> i + i);;
+- : int tree =
+Node (Node (Leaf 4, Leaf 6), Node (Leaf 10, Node (Leaf 14, Leaf 22)))
+
+ +Here, our environment is the doubling function (`fun i -> i + i`). If +we apply the very same `int tree reader` (namely, `treemonadizer +int2int_reader t1`) to a different `int->int` function---say, the +squaring function, `fun i -> i * i`---we get an entirely different +result: + +
+# treemonadizer int2int_reader t1 (fun i -> i * i);;
+- : int tree =
+Node (Node (Leaf 4, Leaf 9), Node (Leaf 25, Node (Leaf 49, Leaf 121)))
+
+ +Now that we have a tree transducer that accepts a monad as a +parameter, we can see what it would take to swap in a different monad. +For instance, we can use a state monad to count the number of nodes in +the tree. + +
+type 'a state = int -> 'a * int;;
+let state_unit x i = (x, i+.5);;
+let state_bind u f i = let (a, i') = u i in f a (i'+.5);;
+
+ +Gratifyingly, we can use the `treemonadizer` function without any +modification whatsoever, except for replacing the (parametric) type +`reader` with `state`: + +
+let rec treemonadizer (f:'a -> 'b state) (t:'a tree):('b tree) state =
+  match t with Leaf x -> state_bind (f x) (fun x' -> state_unit (Leaf x'))
+             | Node (l, r) -> state_bind (treemonadizer f l) (fun x ->
+                                state_bind (treemonadizer f r) (fun y ->
+                                  state_unit (Node (x, y))));;
+
+ +Then we can count the number of nodes in the tree: + +
+# treemonadizer state_unit t1 0;;
+- : int tree * int =
+(Node (Node (Leaf 2, Leaf 3), Node (Leaf 5, Node (Leaf 7, Leaf 11))), 13)
+
+    .
+ ___|___
+ |     |
+ .     .
+_|__  _|__
+|  |  |  |
+2  3  5  .
+        _|__
+        |  |
+        7  11
+
+ +Notice that we've counted each internal node twice---it's a good +excerice to adjust the code to count each node once. + +One more revealing example before getting down to business: replacing +`state` everywhere in `treemonadizer` with `list` gives us + +
+# treemonadizer (fun x -> [[x; square x]]) t1;;
+- : int list tree list =
+[Node
+  (Node (Leaf [2; 4], Leaf [3; 9]),
+   Node (Leaf [5; 25], Node (Leaf [7; 49], Leaf [11; 121])))]
+
+ +Unlike the previous cases, instead of turning a tree into a function +from some input to a result, this transformer replaces each `int` with +a list of `int`'s. + +Now for the main point. What if we wanted to convert a tree to a list +of leaves? + +
+type ('a, 'r) continuation = ('a -> 'r) -> 'r;;
+let continuation_unit x c = c x;;
+let continuation_bind u f c = u (fun a -> f a c);;
+
+let rec treemonadizer (f:'a -> ('b, 'r) continuation) (t:'a tree):(('b tree), 'r) continuation =
+  match t with Leaf x -> continuation_bind (f x) (fun x' -> continuation_unit (Leaf x'))
+             | Node (l, r) -> continuation_bind (treemonadizer f l) (fun x ->
+                                continuation_bind (treemonadizer f r) (fun y ->
+                                  continuation_unit (Node (x, y))));;
+
+ +We use the continuation monad described above, and insert the +`continuation` type in the appropriate place in the `treemonadizer` code. +We then compute: + +
+# treemonadizer (fun a c -> a :: (c a)) t1 (fun t -> []);;
+- : int list = [2; 3; 5; 7; 11]
+
+ +We have found a way of collapsing a tree into a list of its leaves. + +The continuation monad is amazingly flexible; we can use it to +simulate some of the computations performed above. To see how, first +note that an interestingly uninteresting thing happens if we use the +continuation unit as our first argument to `treemonadizer`, and then +apply the result to the identity function: + +
+# treemonadizer continuation_unit t1 (fun x -> x);;
+- : int tree =
+Node (Node (Leaf 2, Leaf 3), Node (Leaf 5, Node (Leaf 7, Leaf 11)))
+
+ +That is, nothing happens. But we can begin to substitute more +interesting functions for the first argument of `treemonadizer`: + +
+(* Simulating the tree reader: distributing a operation over the leaves *)
+# treemonadizer (fun a c -> c (square a)) t1 (fun x -> x);;
+- : int tree =
+Node (Node (Leaf 4, Leaf 9), Node (Leaf 25, Node (Leaf 49, Leaf 121)))
+
+(* Simulating the int list tree list *)
+# treemonadizer (fun a c -> c [a; square a]) t1 (fun x -> x);;
+- : int list tree =
+Node
+ (Node (Leaf [2; 4], Leaf [3; 9]),
+  Node (Leaf [5; 25], Node (Leaf [7; 49], Leaf [11; 121])))
+
+(* Counting leaves *)
+# treemonadizer (fun a c -> 1 + c a) t1 (fun x -> 0);;
+- : int = 5
+
+ +We could simulate the tree state example too, but it would require +generalizing the type of the continuation monad to + + type ('a -> 'b -> 'c) continuation = ('a -> 'b) -> 'c;;