The Evolution of the
StringTemplate
Engine
Presented to the UC Berkeley Harmonia
group
15 December 2004
Terence Parr
University of San Francisco
[email protected]
Overview

Introduction
what is StringTemplate?
 why did I build it?
 the nature of text generation





Design goals
The evolution; 4 phases
Experience
Lessons
What is StringTemplate

Template engine designed with Tom Burns
(CEO, jGuru.com) while building commercial
sites over many years in response to JSP




small: 170k uncompressed binary (w/o test rig)
Evolved from simple “document with holes” into
a functional language capable of generating
large class of languages
Well suited to generative programming as well
as dynamic page generation; being used in new
ANTLR parser generator for code generation
Distinguishing characteristic: strictly enforces
separation of model and view
Canonical Operations

Attribute reference:
<type>

Template references (possibly recursive):
<statementList()>

Apply template to multi-valued attribute:
<decls:decl()> or
<decls:{<it.type> <it.name>;}>
<items:red(), blue(), green()>

Conditional include:
<if(superClass)>extends <superClass><endif>
<if(name)>Name: <name><else>Guest<endif>
Example*

Translated “Program Manipulation via
Interactive Transformations” example to
actual StringTemplate template definition:
generateMemSize(node) ::= <<
int mem_size() {
int result = 0;
<node.fields:{result += <it.name>.mem_size();}>
return result;
>>

Integration with existing code clean, easy
*shameless StringTemplate plug
Semantics







Side-effect free expressions
No “state”
No defined order of execution
Lazy evaluation
Template inheritance group to subgroup
Dynamically scoped; values inherited
Recursive template instantiation
Design Goals


Optimized for enforcement of separation not for
Turing completeness nor expressive “one-liners”
Conceptual integrity





Simple as possible



single-minded fanaticism against entanglement
stick to a few concepts: attributes and templates
consistency; no special cases; driven by design not
implementation details
proper language formalisms, semantics, sane syntax
useful to nonprogrammers
makes it fast, easy to build/maintain
Powerful as possible
Phase I, 1999-2001




“Document with holes”; only tag refs
Could nest templates by setting a tag value to
template; dynamic scoping (“value inheritance”)
Not intended for full page generation
File of template definitions loaded into hash
table; code manually created template instances
<template name="db.topic.insert">
INSERT INTO Topics VALUES (
<TID>,
'<topicname>'
);
</template>
Phase II, 2001-2003




Rewrite of jGuru.com; learned entanglement
allowed by JSP is horrible!
Decided to enhance StringTemplate, letting
needs dictate feature set; would not be Turingcomplete!
Moved to single template per file format with
$$…$$ indicating attribute ref
Needed “include”


added template reference rather than #include, macro
subconciously supported recursion
Phase II, 2001-2003 (Cont’d)

Needed to walk multi-valued attributes; added
“apply” syntax:



Allowed anonymous templates


$names:bold(item=names[i])$
Needed multiple array walk for relational DB


$names:”<b>$names$<b>”$
Iterated value naming convention evolved


$names:bold(item=names)$
$names:bold(item=names, separator=“, “)$
$a,b:foo(x=a[i], y=b[i])$
Finally, needed conditional
inclusion
$if(userName)$
Name: $userName$
$endif(userName)$
Phase III, 2003-Summer 2004

Wanted to say $names:bold()$ w/o arg




Cleaned up IF: $if(foo)$…$endif$
Nested anonymous blocks: $names:{…}$
Round-robin template application




defined default attribute attr
$users: rowOdd(), rowEven()$
Properties: $user.name$ calls user.getName()
Indirect template ref: $(tname)()$; added in
context of “immediate evaluation”
Template inheritance via StringTemplateGroup
Phase IV, 2004

Added group file format: mutuallyreferential templates with formal
arguments
method(type,name,args,body) ::= <<
public <type> <name>( <args:arg(); separator=“,”> ) {
<body>
}
>>
assign(lhs,expr) ::= “<lhs> = <expr>;”
…
Future



Named args for anonymous blocks ala Smalltalk

$list:{x | <b>$x$</b>}$

<node.fields:{f | result += <f.name>.mem_size();}>

iterator could also choose single argument from
named templates
Add parallel array walking back in
Syntax for super group?


group sub : super;
Lots of little clean up; e.g., whitespace, …
Experience (ANTLR v3.0)



Tree walker (controller) collects data from AST
(model), pushes data into templates (view)
Decouples order of computation from order of
output (this is huge)
Enforced separation guarantees easy
retargeting, no code duplication, …




no code in template
no output strings in code generator
Previous code generator hopelessly entangled
Group file format (output grammar) is great!
“Executable documentation”
Practical Lessons

Separate pattern matching from templates!






Recursion required for nested structures
Attributes



don’t make templates also analyze model
lazy evaluation essential; decouple order of
computation from order of output; build in any order
Isolates templates (“user code”) from model changes,
which will surely evolve
Templates may be reusable w/o embedded logic
dynamic scoping really handy
push don’t pull attributes; attributes are inert
Simple language is fast, easy to learn, easy to
use, promotes retargetability
Philosophical Lessons


Say what you mean! e.g., no FOR-loops
Focus on principles, conceptual integrity





focus on what not how
other tools either doc with holes or Turing complete
Difficult to stick to your principles and still create
usable tool; grey areas appear
Language design is hard;
implementation is relatively easy
Selection pressure results in StringTemplate


output conforms to a language, hence, a grammar
strict enforcement of separation leads to grammar
Demo

Generating Java / XML with same model
Descargar

The Evolution of the StringTemplate Engine