LuaXP stands for Lua eXpression Parser. It is written entirely in Lua, and self-contained in a single Lua module with no dependencies beyond the standard Lua runtime libraries. It offers a very simple use model, with a single function call able to evaluate an arbitrarily complex expression.
Why LuaXP?
I had a project (SiteSensor) in which I needed the ability to parse simple user-provided expressions (e.g. temp*9/5+32). A short way of addressing the requirement might have been to simply pass the expression to Lua's loadstring() function, but this presented some issues, not the least of which were potential security concerns, given that loadstring() will run any Lua it is given. It seemed like an interesting challenge to write a simple expression parser in Lua itself, to handle exactly what I needed for the project, and in a limited context, so I went that direction.
I originally wanted the L in its name to stand for lightweight, but as the demands of the SiteSensor project grew, so did LuaXP. I think it's still pretty crisp at about 1400 lines of code, but its original incarnation was about half that. On the plus side, the latest code offers great features and utility, as well as consistent, deterministic rules and good error management. That's worth a lot.
LuaXP's expression syntax is fairly straight-forward. It also includes a small library of functions for common tasks or calculations.
LuaXP is extensible, in that custom functions can be provided. LuaXP also allows data from outside the expression environment to passed in and used in the evaluation of an expression (I refer to this as its context).
Make a donation to support this project.
Using LuaXP
To use LuaXP, one just needs to download the latest release from the Github repository and include it in their project (i.e. place it in an accessible directory and use the Lua require() function to load the module).
LuaXP = require("LuaXP")
result, err = LuaXP.evaluate( expression [, context ] )
The result of the evaluation is stored in the first return value returned (result in the above example). If nil, an error occurred, and the second return value (err in this example) is either a string or a Lua table containing the error data. Please see Error Results for further explanation.
The evaluate() function performs both compilation and evaluation of the passed expression. Outside data may be passed in, if needed, through the context argument, which is expected to be a Lua table.
This function is provided as a shortcut for calling compile() and run() sequentially. However, it would be inefficient to use it for evaluation of same expression repeatedly, because it compiles the expression each time. In that case, compile() should be used, followed by run() as often as needed.
ehandle, err = LuaXP.compile( expression [, context] )
The compile() function compiles an expression and returns the tokenized result in a Lua table. The first return value returns the tokenized form of the expression, or nil if an error occurred. The second value contains error data (or nil if no error occurred); see Error Results below for further explanation.
The tokenized expression (ehandle in the above) can then be passed to run() to perform evaluation of the tokenized expression.
result, err = LuaXP.run( ehandle [, context ] )
The run() function takes a previously tokenized expression (the result of the compile() function) and evaluates it using the optional provided context. The first return value is the result of evaluation, or nil if an error occurred, in which case the second return value contains either a string or Lua table describing the error.
Handling Context
LuaXP's context argument is a way for it to receive data from the outside world that is useful to the expressions being evaluated. LuaXP would have very limited utility if it could not use any external data.
The context is just a Lua table containing various key/value pairs. LuaXP expressions can refer to values in the context in the same manner as one would through Lua directly. Let's make a simple sample context:
local context = {}
context.version = 4
context.weather = {}
context.weather.icon = 103
context.weather.temperature = 17
context.weather.units = "C"
Using Lua, we would access the version number by referring to context.version. Likewise, we could access the current temperature by referring to context.weather.temperature. If this context table were passed to LuaXP, then expressions being evaluated would simply need to use version and weather.temperature, respectively (since LuaXP has no variables of its own, all references are assumed to be relative to the passed context).
LuaXP = require("LuaXP")
print( LuaXP.evaluate(' "The current temperature is " + weather.temperature + weather.units', context ))
-- Prints: The current temperature is 17C
Of course, the context does not need to be created manually. It could be created in any way that Lua can populate a table. My SiteSensor plugin for Vera Home Automation Controllers, for example, populates the context by fetching and parsing JSON data from a remote RESTful API.
Subkey references that cannot be resolved evaluate to null. For example, using the above example context, the expression weather.notthere evaluates to null. This is useful in some cases, where the absence of data in the context can be detected and handled rather than throwing an error. For example, if our example weather data did not always include wind data as a subkey of weather, the following expression would handle it gracefully: if( weather.wind==null, "No wind data", weather.wind.speed + "m/s" ). This example would evaluate to the wind speed (with units "m/s" appended) if wind data is available, or "No wind data" in the absence of the wind subkey). Note that dereferencing through null is still an error: weather.wind.speed on its own (without the if() construction in the prior example) would throw a runtime error if wind data is not present (i.e. weather.wind evaluates to null).
The context also allows the caller to create user-defined functions, expanding on LuaXP's built-in library of functions. See "User-defined Functions," below.
The context is also used to store variables created by LuaXP when the expression makes an assignment. The __lvars key is created and populated if assignments are made.
When evaluating multiple expressions, the same context can be provided to each. The context is not bound to a single compiled expression. It may be reused with any number of expressions. Remember, however, that assignments made in expressions are stored in the context. This creates the opportunity for one expression to pass data to another: the first defines a value, and the second may refer to it. However, it may also create the opportunity for one expression to overwrite the data of another, so caution should be used.
User-defined Functions
While LuaXP has a library of pre-defined functions, it seemed obvious to me (and simple to allow) that a calling application could define its own functions, giving LuaXP even more potential utility. Such user-defined functions can be written by creating them as named or anonymous functions, and setting them by name in a __functions subtable in the passed context.
Here's a trivial example. Let's say we need a function to return a passed string concatenated to itself. We simply define the function and bind it to the context in __functions, like this:
local function myStringDoubler( args )
local s = unpack( args ) -- unpack array into separate values
return s .. s
end
-- Create a context
local context = {}
context.__functions = {}
context.__functions.sdouble = myStringDoubler
-- Evaluate an expression using our function
LuaXP = require("LuaXP")
print( LuaXP.evaluate( "sdouble('hello')", context ) ) -- will print "hellohello"
The example above shows that whatever name the function reference is given in the __functions subtable, becomes the name available to expressions.
The above function and context declarations could also be written more succinctly in this way:
local context = {
__functions = {
sdouble = function( args ) return args[1] .. args[1] end
}
}
User-defined functions should check their arguments and do appropriate type conversions. The LuaXP.coerce( val, type ) function may be used to coerce a value to a specific Lua type (an evaluation error will be thrown if the conversion cannot be made). The function LuaXP.isNull( val ) can be used to check for a null value, while the constant LuaXP.NULL can be used to return a null value. Functions may return any of the types LuaXP knows (see Types in Lua Expressions). If the user-defined function needs to throw its own error for some reason, use the LuaXP.evalerror( message ) function rather than Lua error().
Error Results
The core LuaXP implementation functions all return an error result value as their second argument. If this value is nil, then no error occurred and the first return value is valid.
If the error result is a string, a Lua error occurred in compiling or evaluating the expression. This usually indicates a bug, as I go to great lengths to sanity-check and throw "controlled" errors (below) when things aren't as they should be. Please report any such errors to me--see the Reporting Bugs section below.
If the error result is a Lua table, then the type key contains the type of error (either "compile" or "evaluation"), the message key contains the error message itself, and the location key contains the string position in the source expression at which the error was detected. The location value may be absent if the error could not be pinpointed to a specific spot in the expression readily.
Reporting Bugs
If you have a bug report or feature request, here's what I'd prefer you do when reporting it:
- First, make sure you are using the latest version from GitHub. If not, please take the time to download and install it, and check if the bug has already been fixed or the feature implemented.
- If and only if you feel like running with the big dogs, download the current development version from GitHub and try it.
- If at this point you've got something you can't resolve, feel free to open an issue in the GitHub repository for the project. This is the (strongly) preferred method for reporting problems. I get tons of email and can easily miss your issue if you try to email me problems or questions.
License
LuaXP is licensed under GPL 3.0. Please see the license file for details.