Coroutine Oriented Portable Asynchronous Services for Lua

Copas

• Home
  • Status
  • Download
  • Dependencies
  • History
  • Credits
• Manual
• Reference
• Project
  • Bug Tracker
• License

Installing

You can install Copas using LuaRocks:

luarocks install copas

Note: LuaSec is not automatically installed as a dependency. If you want to use ssl with Copas, you need to manually install LuaSec as well.

Runtime

Copas can either be used as a regular Lua library, or as a runtime. A command line script that acts as a runtime engine is included. When using the runtime, the library is available as a global (copas), and the scheduler will automatically be started. For example:

  #!/usr/bin/env copas

  local count = 0
  copas.timer.new {
    delay = 1,
    recurring = true,
    callback = function(self)
      count = count + 1
      print('hello world ' .. count)
      if count >= 5 then
        self:cancel()
      end
    end
  }

Introduction to Copas

Copas is a dispatcher that can help a lot in the creation of servers based on LuaSocket. Here we present a quick introduction to Copas and how to implement a server with it.

Assuming you know how to implement the desired server protocol, the first thing you have to do in order to create a Copas based server is create a server socket to receive the client connections. To do this you have to bind a host and a port using LuaSocket:

server = socket.bind(host, port)

Then you have to create a handler function that implements the server protocol. The handler function will be called with a socket for each client connection and you can use copas.send() and copas.receive() on that socket to exchange data with the client.

For example, a simple echo handler would be:

function echoHandler(skt)
  while true do
    local data = copas.receive(skt)
    if data == "quit" then
      break
    end
    copas.send(skt, data)
  end
end

You may alternatively use copas.wrap() to let your code more close to a standard LuaSocket use:

function echoHandler(skt)
  skt = copas.wrap(skt)
  while true do
    local data = skt:receive()
    if data == "quit" then
      break
    end
    skt:send(data)
  end
end

To register the server socket with Copas and associate it with the corresponding handler we do:

copas.addserver(server, echoHandler)

Finally, to start Copas and all the registered servers we just call:

copas()

As long as every handler uses Copas's send and receive, simultaneous connections will be handled transparently by Copas for every registered server.

Since Copas is coroutine based, using it within a Lua pcall or xpcall context does not work with Lua 5.1 yielding. If you need to use any of those functions in your handler we strongly suggest using coxpcall, a coroutine safe version of the Lua 5.1 protected calls. For an example of this usage please check Xavante.

Why use Copas?

For those who already have a server implemented, here is an explanation of why and how to migrate to Copas. In a typical LuaSocket server usually there is a dispatcher loop like the one below:

server = socket.bind(host, port)
while true do
  skt = server:accept()
  handle(skt)
end

Here handle is a function that implements the server protocol using LuaSocket's socket functions:

function handle(skt)
  ...
  -- gets some data from the client - "the request"
  reqdata = skt:receive(pattern)
  ...
  -- sends some data to the client - "the response"
  skt:send(respdata)
  ...
end

The problem with that approach is that the dispatcher loop is doing a busy wait and can handle just one connection at a time. To solve the busy waiting we can use LuaSocket's socket.select(), like in:

server = socket.bind(host, port)
reading = {server}
while true do
  input = socket.select(reading)
  skt = input:accept()
  handle(skt)
end

While this helps our CPU usage, the server is still accepting only one client connection at a time. To handle more than one client the server must be able to multitask, and the solution usually involves some kind of threads.

The dispatcher loop then becomes something like:

server = socket.bind(host, port)
reading = {server}
while true do
  input = socket.select(reading)
  skt = input:accept()
  newthread(handle(skt))
end

where newthread is able to create a new thread that executes independently the handler function.

The use of threads in the new loop solves the multitasking problem but may create another. Some platforms does not offer multithreading or maybe you don't want to use threads at all.

If that is the case, using Lua's coroutines may help a lot, and that's exactly what Copas does. Copas implements the dispatcher loop using coroutines so the handlers can multitask without the use of threads.

Using Copas with an existing server

If you already have a running server using some dispatcher like the previous example, migrating to Copas is quite simple, usually consisting of just three steps.

First each server socket and its corresponding handler function have to be registered with Copas:

server = socket.bind(host, port)
copas.addserver(server, handle)

Secondly the server handler has to be adapted to use Copas. One solution is to use Copas send and receive functions to receive and send data to the client:

function handle(skt)
  ...
  -- gets some data from the client - "the request"
  reqdata = copas.receive(skt, pattern)
  ...
  -- sends some data to the client - "the response"
  copas.send(skt, respdata)
   ...
end

The other alternative is to wrap the socket in a Copas socket. This allows your handler code to remain basically the same:

function handle(skt)
  -- this line may suffice for your handler to work with Copas
  skt = copas.wrap(skt)   -- or... skip this line and wrap `handle` using copas.handler()
  -- now skt behaves like a LuaSocket socket but uses Copas'
  ...
  -- gets some data from the client - "the request"
  reqdata = skt:receive(pattern)
  ...
  -- sends some data to the client - "the response"
  skt:send(respdata)
   ...
end

Note that by default Copas might return different timeout errors than the traditional Lua libraries. Checkout copas.useSocketTimeoutErrors() for more information.

Finally, to run the dispatcher loop you just call:

copas()

During the loop Copas' dispatcher accepts connections from clients and automatically calls the corresponding handler functions.

Using UDP servers

Copas may also be used for UDP servers. Here is an example;

local port = 51034
local server = socket.udp()
server:setsockname("*",port)

function handler(skt)
  skt = copas.wrap(skt)
  print("UDP connection handler")

  while true do
    local s, err
    print("receiving...")
    s, err = skt:receive(2048)
    if not s then
      print("Receive error: ", err)
      return
    end
    print("Received data, bytes:" , #s)
  end
end

copas.addserver(server, handler, 1)
copas()

For UDP sockets the receivefrom() and sendto() methods are available, both for copas and when the socket is wrapped. These methods cannot be used on TCP sockets.

IMPORTANT: UDP sockets do not have the notion of master and client sockets, so where a handler function can close the client socket for a TCP connection, a handler should never close a UDP socket, because the socket is the same as the server socket, hence closing it destroys the server.

NOTE: When using the copas.receive([size]) method on a UDP socket, the size parameter is NOT optional as with regular luasocket UDP sockets. This limitation is removed when the socket is wrapped (it then defaults to 8192, the max UDP datagram size luasocket supports).

Adding tasks

Additional threads may be added to the scheduler, as long as they use the Copas send, receive or sleep methods. Below an example of a thread being added to create an outgoing TCP connection using Copas;

local socket = require("socket")
local copas = require("copas")

local host = "127.0.0.1"
local port = 10000

local skt = socket.connect(host, port)
skt:settimeout(0)  -- important: make it non-blocking

copas.addthread(function()
   while true do
      print("receiving...")
      local resp = copas.receive(skt, 6)
      print("received:", resp or "nil")
      if resp and resp:sub(1,4) == "quit" then
         skt:close()
         break
      end
   end
end)

copas()

The example connects, echoes whatever it receives and exits upon receiving 'quit'. For an example passing arguments to a task, see the async http example below.

Creating timers

Timers can be created using the copas.timer module. Below an example of a timer;

local copas = require("copas")

copas(function()
   copas.timer.new({
     delay = 1,                        -- delay in seconds
     recurring = true,                 -- make the timer repeat
     params = "hello world",
     callback = function(timer_obj, params)
       print(params)                   -- prints "hello world"
       timer_obj:cancel()              -- cancel the timer after 1 occurence
     end
   })
end)

The example simply prints a message once every second, but gets cancelled right after the first one.

Synchronization primitives

Since Copas allows to asynchroneously schedule tasks, synchronization might be required to protect resources from concurrent access. In this case the copas.lock and copas.semaphore classes can be used. The lock/semaphore will ensure that the coroutine running will be yielded until the protected resource becomes available, without blocking other threads.

local copas = require("copas")

local lock = copas.lock.new()

local function some_func()
  local ok, err, wait = lock:get()
  if not ok then
    return nil, "we got error '" .. err .. "' after " .. wait .. " seconds"
  end

  print("doing something on my own")
  copas.pause()  -- allow to yield, while inside the lock
  print("after " .. ok .. " seconds waiting")

  lock:release()
end

The some_func function may now be called and the 2 lines will be printed together because of the lock.

Ssl support

LuaSec is transparently integrated in the Copas scheduler (though must be installed separately when using LuaRocks).

Here's an example for an incoming connection in a server scenario;

function handler(skt)
  skt = copas.wrap(skt):dohandshake(sslparams)
  -- skt = copas.wrap(skt, sslparams):dohandshake()  -- would be identical

  while true do
    -- perform the regular reading/writing ops on skt
  end
end

A simpler handler would wrap the handler function to do the wrapping and handshake before the handler gets called;

function handle(skt)
  -- by now `skt` is copas wrapped, and has its handshake already completed

  while true do
    -- perform the regular reading/writing ops on skt
  end
end
handle = copas.handler(handle, sslparams)  -- wraps the handler to auto wrap and handshake

Here's an example for an outgoing request;

copas.addthread(function()
  local skt = copas.wrap(socket.tcp(), sslparams)
  skt:connect(host, port)  -- connecting will also perform the handshake on a wrapped socket

  while true do

    -- perform the regular reading/writing ops on skt

  end
end

High level requests

For creating high level requests; http(s), ftp or smtp versions of the methods are available that handle them async. As opposed to the original LuaSocket and LuaSec implementations.

Below an example that schedules a number of http requests, then starts the Copas loop to execute them. The loop exits when it's done.

local copas = require("copas")
local asynchttp = require("copas.http").request

local list = {
  "http://www.google.com",
  "http://www.microsoft.com",
  "http://www.apple.com",
  "http://www.facebook.com",
  "http://www.yahoo.com",
}

local handler = function(host)
  res, err = asynchttp(host)
  print("Host done: "..host)
end

for _, host in ipairs(list) do copas.addthread(handler, host) end
copas()

Controlling Copas

If you do not want copas to simply enter an infinite loop (maybe you have to respond to events from other sources, such as an user interface), you should have your own loop and just call copas.step() at each iteration of the loop:

while condition do
  copas.step()
  -- processing for other events from your system here
end

When using your own main loop, you should consider manually setting the copas.running flag.