Eliminate non-local references from closure

Question

For a code similarity detection framework I need to eliminate references to non-local variables, for example having the following closure:

{
  var a;
  var b;

  var f = function() {
    a = 5;
    b = 6;
  }

  f();
}

would be converted into something like:

{
  var a;
  var b;

  var f = function() {
    var a = 5;
    var b = 6;
    return tuple(a, b);
  }

  a, b = f();
}

Another approach would be:

{
  var world;
  var a;
  var b;

  var f = function(world) {
    world = update_world(world, "a", 5);
    world = update_world(world, "b", 6);
    return world;
  }

  world = f(world);
  a = get_world(world, "a");
  b = get_world(world, "b");
}

Basically, convert a stateful program to a purely functional one. The goal is then to transform the resulting program into the lambda calculus, normalize and look for isomorphic subtrees.

Is there a formalization of what I'm trying to do?

Answer 1

I think what you want is essentially conversion to static single assignment (SSA) form, followed by closure conversion, followed by conversion to continuation passing style.

Static single assignment form guarantees that each variable is written exactly once in the program text. That this is the key step in converting imperative to functional programs is argued by

Andrew W. Appel. SSA is functional programming. SIGPLAN Notices, 33(4), 1998.

Richard A. Kelsey. A correspondence between continuation passing style and static single assignment form. Proc ACM SIGPLAN Workshop on Intermediate Representations, January 1995.

Peter J. Landin. The mechanical evaluation of expressions. Computer Journal, 6(4):308–320, 1964.

This starts to get really interesting for programs with loops. You need to convert the loops into recursive calls, and make a unique copy for the variables written in each iteration of the loop. In the traditional loop optimization community this technique is called scalar expansion or (sometimes) dynamic renaming.

David J. Kuck, R. H. Kuhn, David A. Padua, B. Leasure, and Michael Wolfe. Dependence graphs and compiler optimizations. In Conference Record of the Eighth Annual ACM Symposium on Principles of Programming Languages, pages 207–218, Williamsburg, VA, January 1981.

Ron Cytron and Jeanne Ferrante. What’s in a name? The value of renaming for parallelism detection and storage allocation. In Proceedings of the 16th Annual International Conference on Parallel Processing, pages 19–27, St. Charles, IL, August 1987.

In the functional programming community this involves making closures and continuations explicit:

Andrew W. Appel and Trevor Jim. Continuation passing, closure-passing style. Proc. Symp. on Principles of Programming Languages (POPL), 1989.

Guy Lewis Steele. RABBIT: A compiler for Scheme. Technical Report AITR-474, MIT Artificial Intelligence Laboratory, May 1978.

I needed to do these conversions, for the automatic parallelization techniques described in my dissertation. Chapters 3, and 9.1.

Matthew Frank, SUDS: Automatic Parallelization for Raw Processors, Ph.D. thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, May 23, 2003.

Answer 2

Denotational Semantics was an answer almost intended for your question

There is a huge collection of techniques that may be applicable, since the proposed task is basically a compilation process, possibly partial / incremental compilation. The target language is domain extended λ-calculus, which imply some standard transformations such as continuations or passing state as parameter and (part of) result of functions. So it can be described by piecemeal compiler and functional programming bibliography.

But the systematic view of compiling programming languages to λ-calculus is precisely one of the purposes of defining denotational semantics of programming languages, though it is a consequence of the original aim: defining a compositional semantics. Compositionality is precisely what is naturally given by λ-calculus. This is illustrated by the fact that the first environment for building systematically the denotational semantics of a language, P. Mosses'SIS, was also considered the first complete compiler-compiler.

Denotational semantics offers a systematic framework for the stated goal, with a strong and fairly unified theoretical basis.

Basically, convert a stateful program to a purely functional one. The goal is then to transform the resulting program into the lambda calculus, normalize and look for isomorphic subtrees.

Is there a formalization of what I'm trying to do?

Yes, there is a formalization of what you are trying to do, and it has been existing for 35 years, though some techniques may have evolved, It was for example applied to the very imperative language Ada around 1980.

What you want is very precisely and formally what is done by using denotational semantics: transform your stateful program, or program fragment, into a lambda expression. It does use many compiling and functional programming techniques, such as continuations and other higher order functions.

Indeed the structure of the last example in the question, introducing a memory environment called world is typical of denotational semantics. In denotational semantics world is a "state mapping" $S:\mathrm{State}$ representing the memory state. This state is passed as argument to all functions, so that functions can access global memory (by whatever means), and returned by function to be available in the calling environment. Of course the $\mathrm{State}$ value returned may differ from the one received as parameter, exactly as the world value returned by function f is not the one it received.

For some simple examples, you may look at the question: Denotational semantics of expressions with side effects.

If you have a reference denotational semantics for the language you are considering, then you are sure that no language feature is forgotten in the translation process that could invalidate what you are trying to do.

If you do it on an ad hoc basis for a specific program fragment, you get no garantee that you are taking all you should into consideration.

If you do not have a denotational semantics definitions for your language, you may get ideas from looking at how it handles specific features for other existing languages., or look at the denotational semantics literature in general.

I know denotational semantics does not seem very popular these days. But it was developed and used for that purpose, in addition to more theoretical and abstract aims.

The idea of thus using denotational semantics was pionneered by Peter Mosses with his Semantics Implementation System (SIS)¹ developed in the late 1970s. It was doing just that: transform (compile) programs into big lambda expressions according to the denotational semantics of the programming language, which had first to be entered in the system.

There is considerable literature and books on denotational semantics, which you can easily find by searching the web.

Not surprisingly, many of the techniques used are compiler writing techniques, and the system SIS is considered as the first compiler-compiler. Hence much of the literature on compilers can be relevant too, as shown by the important bibliography in the previous answer by Wandering Logic.

The fact is that a denotational semantics specification of a programming language can be read as high-level code for a compiler of the language into lambda calculus, up to some predefined domains of values.

¹ _{Peter Mosses, "SIS: A Compiler-Generator System Using Denotational Semantics," Report 78-4-3, Dept. of Computer Science, University of Aarhus, Denmark, June 1978}