The Next Step in the Evolution of SCION, a.k.a. Scion-ng, a.k.a. SCION-core, a.k.a. SCION 1.0.0

February 22, 2013

Table of Contents

tl;dr: The next version of SCION is going to have real stack traces for SCXML, and an API by which js devs can specify the Statecharts model as a simple, declarative JavaScript object.

I have started this blog for SCION news and updates because I wanted to provide a fuller picture of the major architectural changes that are occurring in SCION, and the motivation for making these changes. I’ll refer to this next major iteration of SCION as “SCION-ng” (for “next generation”), although the branch name on GitHub is “core”, and the semantic versioning number has been rev’d to 1.0.0, so it could be referred to by any of these attributes.

Deficiences in SCION

So far, I think the SCION project has been successful in reaching its initial goals of developing a fast, robust, portable, embeddable SCXML interpreter. SCION is being used in production and provides an effective solution for many developers working with SCXML.

However, in my mind SCION has had two major deficiencies. The first is that SCION lacks traceability, which is to say, if an exception is thrown during execution of an SCXML document originating from some JavaScript action code within that document, SCION will not provide a meaningful stack trace with line and column numbers tracing back to the original source SCXML document. Instead, it provides the line and column number of the JavaScript code generated by SCION, which is something that is private and internal to SCION and not meant to be user-visible, essentially making the resulting stack trace a useless garbage value. The reason for this problem is that SCION relies on the XML DOM API for document parsing, and DOM does not report the original line and column numbers of the parsed elements. So, the first goal of SCION-ng is to fix this problem and bring real stack traces to SCION.

The second major deficiency in SCION is its reliance on SCXML, and subsequently, an XML syntax for specifying Statecharts models. One of the original goals of SCION was that it would be useful to Web front-end developers for managing complex UI state (this subject is actually the second half of my soon-to-be-published Master thesis). However, based on my conversations with many front-end developers, I found that relying on an XML syntax was virtually a non-starter. Note that I work as a Web front-end developer professionally, and I happen to enjoy working with various forms of executable XML (just look at my Google Summer of Code 2010 project, scxml-js, to see how much I like XSLT!), but I recognize that I am very much in the minority here. Front-end developers would prefer to work with either a pure-JSON syntax, or, alternatively, a simple JavaScript object. So, to make SCION more useful and accessible to Web front-end developers, the second goal of SCION-ng is to allow developers to specify the Statecharts model as a simple, declarative JavaScript object literal; entry, exit, and transition actions will simply be JavaScript functions, or arrays of JavaScript functions attached to onEntry/onExit/onTransition properties of the object; and the SCXML datamodel will simply be local variables kept in a closure using the JavaScript module pattern. These data structures and these concepts are a front-end developer’s bread and butter, and it makes perfect sense to create a lean Statecharts interpreter that maximally reuses these concepts.

Solution and Benefits

In order to accomplish these goals, then, I have rearchitected SCION into two separate modules: SCION-core and scxml.js. SCION-core will be the core state machine engine, which accepts a simple JavaScript object as the Statecharts model and then interprets it according to SCION semantics. scxml.js, on the other hand, will be a lightweight compiler which accepts an SCXML document as input, then parses it using the excellent sax-js XML parser, preserving line and column numbers in the process, and generates nicely formatted JavaScript modules as output, with line and column numbers embedded in the generated function name, so that if an exception occurs during execution, it will give you a meaningful stack trace. The generated modules return a Statecharts model object which can be fed into SCION-core. scxml.js will depend on SCION-core, but not vice-versa.

scxml.js will be able to perform this SCXML-to-JavaScript compilation either ahead-of-time, or on the fly at runtime in a way that is transparent to the developer. In fact, scxml.js will be mostly backwards-compatible with the current version of SCION. I say “mostly”, because I’m planning to change or remove some features which were originally useful for research purposes, but were never used much and are now a source of additional complexity. I’ll write in more detail about these features in a future post. The SCION API should remain unchanged.

There will be additional benefits to this refactoring effort as well:

  • It will be possible to entirely eliminate SCION’s somewhat brittle DOM compatibility layer, which should mean greater reliability across environments.
  • Stripping SCION-core down to just the state machine interpreter made it feasible to combine everything into one UMD module (it’s under 1000 LOC, including comments, which I think is quite reasonable for a single module). This means it will be trivial to install and use SCION from CommonJS, AMD, or Vanilla JS - no build step or compatibility shims required! It also means that Google Closure compiler can optimize things really well, leading to the next point…
  • SCION-core is super-small - only 2.3kb after being minified and gzipped! This will give Web developers a lot of expressive power in a very tiny package.
  • Finally, being able to specify a Statecharts model using a simple JavaScript object has a lot of exciting prospects regarding dynamic generation of Statecharts models at runtime. I think this is a capability with many potential applications, and this is something I’ll write more about in a future post.

Examples

So, that’s a lot of talk about architecture, let’s look at some code. All of these examples have been auto-generated by scxml.js from the scxml-test-framework, so they are not the most user-friendly, but should give you a sense of how to specify Statecharts models for SCION-core, and the compiled code output of scxml.js.

Here’s an example based on test basic/basic1.scxml:

{
    "states": [
        {
            "id": "a",
            "transitions": [
                {
                    "target": "b",
                    "event": "t"
                }
            ]
        },
        {
            "id": "b"
        }
    ]
}

As you can see, it’s just a simple JSON file. This can be passed directly into the SCION constructor like so:

var model = require('./basic1.sc.json');
var sc = new scion.Statechart(model);
sc.start();

There is a limited subset of SCXML that can be described by as pure JSON and executed by SCION. If there is a datamodel or action code, then a pure-json representation is not sufficient, and so a regular JavaScript module is used instead. The naming conventions for JSON and JavaScript files containing Statechart models is *.sc.json and *.sc.js.

Here’s an example compiled from test script/test0.scxml that shows how to execute actions and manipulate data:

//Generated on Thursday, February 21, 2013 19:56:28 by the SCION SCXML compiler

var x;

function $script_line_27_column_20(_event, In, _sessionId, _name, _ioprocessors, _x){
    x = 100;
}

function $cond_line_34_column_59(_event, In, _sessionId, _name, _ioprocessors, _x){
    return x === 100;
}

module.exports = {
    "states": [
        {
            "id": "intitial1",
            "transitions": [
                {
                    "target": "a",
                    "onTransition": $script_line_27_column_20
                }
            ]
        },
        {
            "id": "a",
            "transitions": [
                {
                    "target": "b",
                    "event": "t",
                    "cond": $cond_line_34_column_59
                },
                {
                    "target": "f",
                    "event": "t"
                }
            ]
        },
        {
            "id": "b"
        },
        {
            "id": "f"
        }
    ]
};

You could use this model as follows:

var model = require('./test0.sc');
var sc = new scion.Statechart(model);
sc.start();

More examples can be found in the tests directory of the “core” branch of SCION.

Implementation Status

Documentation still needs to be written for SCION-core, but it passes all the tests from the scxml-test-framework, and so I feel it is ready to be used. It exists as a single UMD module here, in the “core” branch of SCION.

scxml.js still needs some more work: it currently only supports ahead-of-time compilation using node.js, so I need to make it portable and support on-the-fly compilation. The code lives here.

I’m planning to continue working in the “core” branch of SCION until scxml.js is fully compatible with the current SCION API. When that’s done, I’ll publish a final release of the SCION 0.0.* branch. I’ll then merge the SCION “core” branch back into “master”, and publish a 1.0.0 release of SCION-ng.

I welcome your questions and comments.