Important News: 26/10/2016 - Call for Presentations to Lua Devroom at FOSDEM 2017
Important News: 04/05/2016 - Community news #2
Important News: 11/12/2015 - Blog opening and contribution guide

Why we rewrote Lua in JS

By Benoit Giannangeli Jul 25 2017 08:00 Webdev Reblogged Comments

There is a latent desire out there to use something else than JS in the browser. Whether it is warranted or not is another topic in itself. But you don't have to search long to find all kinds of projects aiming to bring another language in the browser.

The arrival of Web Assembly reignited that desire for many of us. But wasm is a long way from being usable in production and still does not allow full interaction with all web apis.

Lua is a good fit for the browser

Lua is a simple language with very few concepts to understand and a clear and readable syntax. You can be proficient with it in a matter of hours. Yet it provides very useful features. To name a few:

  • First class functions and closures
  • A versatile data structure: the table
  • Vararg expression
  • Lexical scoping
  • Iterators
  • Coroutines (see below)

It has a sane coercion system (I'm looking at you JS!). Nobody's adding X new concepts to it each version (still looking at you JS), which makes it a stable and reliable language.

Lua is already successfully used server-side with the awesome OpenResty which powers sites like itch.io. There's even some great web frameworks like lapis.

Lua can be found in a number of widely different contexts because of how easy it is to embed it. You can write a whole program with it or simply use it as a glue language thanks to the Lua C api.

Coroutines

One of its main selling point for the browser are coroutines. Coroutines address the issue of writing asynchronous code beautifully. No more promises, generators, etc. You can just write asynchronous code as easily as regular code.

Here's a simple example (fully functional version here):

local bird1 = fetch("http://some.api.com/raven") -- fetch being a random xhr call
local bird2 = fetch("http://some.api.com/dove")

print(bird1.name, bird2.name)

A similar version of that using async/await could be:

let asynchronousFn = async function() {
    let bird1 = await fetch("http://some.api.com/raven");
    let bird2 = await fetch("http://some.api.com/dove");

    console.log(bird1.name, bird2.name);
}

These are close, but notice how, in the Lua version, you don't need to be in a async function. In JS you would have to constantly make the conscious choice of tagging a function async or not. In Lua, any function can be interrupted as long as it's running in a coroutine. Bob Nystrom wrote a great post about it here.

When you're writing asynchronous code in JS, you're in fact piling up function calls. Those callbacks all retain their upvalues which can lead to high memory usage. With coroutines your function can actually be suspended and resumed without having to descend from a pyramid of closures because they run in separate lua states.

Existing ways of using Lua in the browser

There's already a few projects out there to use Lua in the browser in some extent:

  • Moonshine is a reimplementation of Lua in JS. Sadly, it's not being actively developed anymore.
  • Starlight by the same author is a Lua to JS transpiler. Unfortunately coroutines can't be implemented effectively with this approach.
  • lua.vm.js a port of the original Lua implementation to JS using Emscripten. It's fast and works well but its interoperability with JS is compromised by the fact that we end up with two garbage collectors trying to manage the same memory.

Fengari

Fengari (moon in greek) is the Lua VM written in Javascript with the browser as its primary target.

Fengari aims to be 100% compliant with the latest Lua semantics. It stays as close as possible to Lua's codebase and if you're familiar with it, you'll understand Fengari rather easily. 99% of Lua's scope is currently covered so you should be able to run any Lua project with minor adjustments.

With the C API (rather JS API), you can decide to write everything in Lua or to use it as a higher level language that calls your JS codebase to do the heavy lifting. This also means that you can segregate your code in separate insulated Lua states.

You can also interact with the JS side directly from your Lua code effortlessly with the fengari-interop module. It ensures that manipulating JS objects or functions always behave the way you would expect it to:

local global = js.global
local document = global.document

global:alert("Hello from Fengari")

document:getElementById("my-div").textContent = "Hi there !"

The REPL you see on fengari.io is itself written in Lua and JS is only used to create the Lua state and run the main script.

Fengari is still in development and any feedback is welcome !


OAuth2 Authentication with Lua

By Israel Sotomayor Jan 14 2016 14:50 Webdev Reblogged Comments

Just to clarify, this won't be a detailed technical guide about how you can build your own authentication layer using OpenResty + Lua, rather it is a document explaining the process behind the solution.

This is a real example of how moltin's API has come to rely on OpenResty + Lua to handle our oauth2 authentication for all users.

The logic used to authenticate a user was originally embedded into moltin's API, built using the PHP Framework Laravel. This means a lot of code had to be booted before we could authenticate, reject or validate a user request resulting in a high latency.

I'm not going to give details about how much time a PHP Framework could take to give a basic response, but if we compare it to other languages/frameworks, you can probably understand.

This is roughly how it looked at that time:

...
public function filter($route, $request) {
    try {
        // Initiate the Request handler
        $this->request = new OAuthRequest;
        // Initiate the auth server with the models
        $this->server  = new OAuthResource(new OAuthSession);
        // Is it a valid token?   
        if ($this->accessTokenValid() == false) {
            throw new InvalidAccessTokenException('Unable to validate access token');
        }
...

So we decided to move all logic one layer up to OpenResty + Lua which achieved the following:

  • Decoupled responsibilities from our Monolitic API.
  • Improved authentication times and generated access/refresh tokens.
  • Improved times for rejecting invalid access tokens or authentication credentials.
  • Improved times when validating an access token and redirecting the request to the API.

We wanted, and needed, to have more control on each request before hitting the actual API and so we decided to use something fast enough to allow us to pre-process each request and flexible enough to be integrated into our actual system. Doing this led us to use OpenResty, a modified version of Nginx, that allowed us to use Lua, the language used to pre-process those requests. Why? Because it's robust and fast enough to use for these purposes and it's a highly recognised scripting language used daily by many large companies.

We followed the concept behind Kong, who use OpenResty + Lua, to offer several micro-services that could be plugged into your API project. However we found that Kong is in a very early stage and is actually trying to offer more than what we needed therefore we decided to implement our own Auth layer to allow us to have more control over it.

Infrastructure

Below is how moltin's infrastructure currently looks:

  • OpenResty (Nginx)
  • Lua scripts
  • Caching Layer (Redis)

OpenResty

This is the bit that rules them all.

We have routing in place to process each of the different user's requests as you can see below:

nginx.conf

location ~/oauth/access_token {
    ...
}
location /v1 {
    ...
}

So for each of those endpoints we have to:

  • check the authentication access token
  • get the authentication access token
...
location ~/oauth/access_token {
    content_by_lua_file "/opt/openresty/nginx/conf/oauth/get_oauth_access.lua";
    ...
}

location /v1 {
    access_by_lua_file "/opt/openresty/nginx/conf/oauth/check_oauth_access.lua";
   ...
}
...

We make use of the OpenResty directives content_by_lua_file and access_by_lua_file.

Lua Scripts

This is where all the magic happens. We have two scripts to do this:

get_oauth_access.lua

...
ngx.req.read_body()
args, err = ngx.req.get_post_args()

-- If we don't get any post data fail with a bad request
if not args then
    return api:respondBadRequest()
end

-- Check the grant type and pass off to the correct function
-- Or fail with a bad request
for key, val in pairs(args) do
    if key == "grant_type" then
        if val == "client_credentials" then
            ClientCredentials.new(args)
        elseif val == "password" then
            Password.new(args)
        elseif val == "implicit" then
            Implicit.new(args)
        elseif val == "refresh_token" then
            RefreshToken.new(args)
        else
            return api:respondForbidden()
        end
    end
end

return api:respondOk()
...

check_oauth_access.lua

...
local authorization, err = ngx.req.get_headers()["authorization"]

-- If we have no access token forbid the beasts
if not authorization then
    return api:respondUnauthorized()
end

-- Check for the access token
local result = oauth2.getStoredAccessToken(token)

if result == false then
    return api:respondUnauthorized()
end
...

Caching Layer

This is where the created access tokens are stored. We can remove, expire or refresh them as we please. We use Redis as a storage layer and we use openresty/lua-resty-redis to connect Lua to Redis.

Resources

Here are some interesting resources on Lua that we used when creating our authentication layer.

Lua


The Best Lua Web Frameworks

By Etiene Dalcol Dec 16 2015 15:49 Webdev Comments

Why use Lua in web development

Lua is an easy and elegant programming language that is recorded as the fastest interpreted language on many benchmarks and proven success in other domains of development such as games and embedded systems. It has good language semantics, awesome documentation, it is very readable and has very powerful mechanisms such as metatables, proper tail calls and many other features that are worth taking a look. It's a great technical candidate for being a PHP replacement. Lua is being used in production for web development for a long time with success by websites such as TaoBao, a chinese online shopping website that ranks 11 globally on Alexa with over 760 million product listings, Cloudflare, Rackspace, itch.io, mail.ru, Mashape/Kong, Shopify and others. Wikipedia uses Lua for its template system. This blog itself is also running on Lua.

A common complaint of using Lua, though, is the ecosystem, which is exactly why PHP is so popular. PHP is pervasive and there are many tools and tutorials written for it, so the development becomes easy also for reasons that are not technical, specially due to a large & friendly community.

However, the landscape for Lua is changing and now the ecosystem is growing rapidly (a feat I partially attritube to the merge of LuaRocks and MoonRocks). We have been able to write in Lua for the web for years and now we can find a large number of tools available. You can write in Lua for major webservers such as Apache and Nginx/OpenResty (top 2 web servers used), and also others such as Lighttpd and pure Lua stand-alone servers like Xavante or Pegasus. Highlights go to Nginx server, which allows to develop blazing fast non-blocking asynchronous apps written in a sequential fashion keeping the event-driven logic hidden inside Nginx (no callback hell).

There are also many frameworks available, which this post aims to compare. I am myself the lead developer of one of them (Sailor) and I haven't developed using all the options I'm listing, but I hope this is a fairly decent comparison. You can make a pull request to this article to make it better.

Something important to note, there is one advantage that is not listed because it applies to all of them, which is the good performance and small footprint. This is even more enhanced on tools that support LuaJIT.

Web Frameworks Comparison

Micro frameworks

Cousins in other languages: Flask (Python), Sinatra (Ruby)

Lapis

Website: leafo.net/lapis

Source: github.com/leafo/lapis

License: MIT

Lapis is a framework for OpenResty developed by the same creator of the MoonScript language and itch.io

Pros:

  • LuaRocks install
  • Excellent documentation
  • Supports OpenResty
  • Nice templating system
  • Stable and well-tested, being used in production by a number of projects
  • Active development
  • Supports MoonScript and LuaJIT
  • Popular with a growing community

Cons:

  • Does not support a big variety of webservers and databases (OpenResty with Postgres and MySQL only, but that should be enough for most projects)
  • Does not support Lua >= 5.2 (As it supports LuaJIT it might support Lua5.1, though)

MVC frameworks

Cousins in other languages: Zend (PHP), Yii (PHP), Rails (Ruby), Django (Python)

Sailor

Website: sailorproject.org

Source: github.com/Etiene/sailor

License: MIT

Sailor is a fairly new framework that began as an independent project by the same maintainer of this blog and has been mentored under Google Summer of Code

Pros:

  • LuaRocks install
  • Works with a variety of databases through the LuaSQL library and native OpenResty MySQL api (for non-blocking operations)
  • Works on a big variety of webservers, including Nginx/OpenResty
  • Integrated with Lua->JS VMs allowing to use Lua for the front-end as well
  • Extensive and well-put documentation
  • Well tested and in active development
  • Compatible with Lua 5.1, 5.2 and LuaJIT
  • The maintainer constantly tries to reach communities outside of the Lua bubble

Cons:

  • Small community
  • It's still in alpha version, things change fast, it's being used in production only by a small number of projects
  • Single person project with not many active contributors

Orbit

Website: keplerproject.github.io/orbit

Source: github.com/keplerproject/orbit

License: GPL

Orbit is maybe the oldest and most stable framework written for Lua developed by a group of researchers during the Kepler project

Pros:

  • LuaRocks install
  • Stable
  • Works with a variety of databases through the LuaSQL library
  • It's being used in production by a number of projects
  • Clear documentation
  • Well supported by the Lua community, questions about it on the Lua list will most likely be answered

Cons:

  • Does not run on a big variety of web servers
  • The development seems fairly abandoned with no recent updates
  • Compatible with Lua 5.1 only

Event-driven frameworks

Cousins in other languages: Node.js (Javascript)

Luvit

Website: luvit.io

Source:github.com/luvit/luvit

License: Apache2.0

Luvit is a port of node.js to Lua that claims to be 2-4 times faster and save up to 20 times memory

Pros:

  • Popular
  • Stable and very well tested
  • Very active development with a fair number of contributors
  • Active community with chats, mail list and a blog
  • Asynchronous I/O
  • Has a number of packages to extend it

Cons:

  • No LuaRocks install
  • Awful documentation, a quick look can't tell how it really works, or which versions of Lua and which databases it supports
  • Not friendly to developers who aren't already familiarized with Node.js

TurboLua

Website: turbolua.org

Source: github.com/kernelsauce/turbo

License: Apache 2.0

Turbo is a framework for building event-driven, non-blocking RESTful web applications built on the top of Tornado web server

Pros:

  • LuaRocks install
  • Stable and well-tested
  • Active development
  • Excellent and extensive documentation
  • Nice templating system

Cons:

  • Does not support a variety of webservers
  • Supports only LuaJIT
  • I couldn't find information on community
  • I couldn't find information on databases supported

CMS, Wikis & others

Ophal

Website: ophal.org

Source: github.com/ophal

License: AGPL 3.0

Ophal is a is a highly scalable web platform and content management system

Pros:

  • Active development
  • Big number of packages to extend it
  • Runs on major webservers
  • Supports a big variety of databases through LuaSQL
  • Maintainer puts a lot of effort into it

Cons:

  • No LuaRocks install
  • Supports only Lua5.1 and LuaJIT
  • Badly documented
  • Does not seem to be stable, it's currently on alpha v0.1 and I couldn't find tests
  • No/small community
  • Single-person project with no other contributors

LuaPress

Website:luapress.org

Source: github.com/Fizzadar/Luapress

License: MIT

LuaPress is a static blog generator

Pros:

  • LuaRocks install
  • Modern, fresh and in active development
  • Stable and well tested version
  • Nice template system, supports mustache and markdown
  • Reasonable documentation
  • Seems to be compatible with all Lua verions >= 5.1

Cons:

  • Not being used in production on many websites
  • Single person project with no other contributors and little to no community
  • Could be better documented

Sputnik

Website: spu.tnik.org

Source: github.com/yuri/sputnik/

License: GPL

Sputnik is an extensible Wiki

Pros:

  • LuaRocks install
  • Stable and used on production on a number of projects
  • Good documentation

Cons:

  • Compatible only with Lua5.1
  • Very old abandoned project, it's no longer maintained

Others

A very simple rank of some tools based on github stars and LuaRocks downloads as of Dec 16, 2015

Stars LR Downloads
Luvit 2068★ Lapis 55060↓
Lapis 1043★ Lusty 1406↓
Sailor 491★ Turbolua 582↓
Turbolua 270★ Sailor 485↓
Tir 250★ Orbit 481↓
Vanilla 122★ LuaPress 108↓
Orbit 83★ Vanilla 38↓
Lusty 56★ Sputnik 34↓

Extra

My talk on web development in Lua and a Sailor introduction during CodingSerbia 2015: Link


RSS

Subscribe to Lua.Space by Email