best practices for application development

With the introduction of the kernel and sockets facilities in Companion 8.5 and 8.6, application development for civilian controller systems became much more API-intensive. The changes in these new versions are mostly about enabling and facilitating hibernation, so that controllers use only the script time they need, and do not waste the limited region resources to handle linked messages that only one or two scripts actually process. We are also taking this opportunity to explain how to best use the linked messages macro system, which was first used in system code in Companion 8.3.

Terminology: a module is any script in user or system memory. A library is a module that normally hibernates. A system module is one that is part of the operating system, regardless of whether it resides in user memory or system memory. For example, _arabesque (ScriptEngine) is a system library module that resides in user memory.

developing with macros

It is not impossible to develop a compliant application for Companion without using the Firestorm LSL preprocessor, but doing so would be a remarkable feat and is unnecessarily difficult. Standard headers for the latest version of the operating system can be found at develop.nanite-systems.com/includes/, with the latest versions of system.lsl and application.lsl available from here and here, respectively.

To enable the LSL preprocessor in Firestorm, press Ctrl-P to open the Preferences window, and enable all five of the checkboxes at the bottom of the Firestorm > Build 1 tab. You will need to set an includes directory, and place the files from develop.ns/includes inside of it. (Make sure you also update system.lsl and application.lsl to match the auto-generated versions available at the links above.

Includes in LSL are implemented using the Boost::Wave preprocessor, which is a standards-compliant C/C++ preprocessor. To include a file, use the syntax

#include <system.lsl>

at the start of the file. Subdirectories may be used.

The macros in these header files are effectively like constants and functions, and make your code more readable. For example, instead of typing:

llMessageLinked(LINK_ROOT, 24, "Test.", "")

to make the unit speak, you can now use the much clearer

llMessageLinked(LINK_ROOT, SPEAK, "Test.", "")

This also ensures that your code will continue to work properly (after a re-compile) if the meaning of the numeric constants ever change.

This is particularly useful when writing link_message() events.

Important: In previous versions, applications received modified link messages. Values were subtracted from -1000 (e.g. 14 POWER became -1014 APP_POWER) when received by applications, but retained the normal form when sent to system modules. This was confusing and was removed in Companion 8.5 milestone 3. For interim compatibility, the compat system command (renamed to file compat in 8.6.3) can be used to force the generation of these old-style APP_ messages, but as many system modules will hibernate by default in 8.5 and later, developers are strongly recommended to update all old software to the new standard.

module hibernation

Starting in Companion 8.5, some system modules hibernate. Your script, unless it is doing something that requires an always-on state like a fast timer or listening to events on a chat channel, will be expected to hibernate as well. Hibernating means that the script's running state is set to 0 when it is not processing. By sending a MODULE_INFO message to the root prim in the state_entry() event, you can keep the firmware informed about when it should wake your code up to accomplish tasks. It will then be expected to notify the system when it is finished processing those tasks. If you need to keep your module running for something short-term (such as reading a notecard), then you can create a thread that will keep it from being suspended.

Module hibernation is based on the concept of tasks. As long as a module has one or more running tasks, it will not be suspended by the kernel. Tasks are created automatically in response to appropriate linked messages and system commands, or can be forced with the instantiation of threads (see below.) A module marked as a service will never be suspended.

interacting with other modules

The start_task() macro takes all of the guesswork out of sending messages formatted in the correct way. This wraps the standard LSL llMessageLinked() function call with a directive to the task scheduler that wakes up all modules that can handle the indicated message before executing it. For example, to query a subsystem's state the old way, you might type:

llMessageLinked(LINK_ROOT, 170, "linked 4", "");

which is now:

start_task(QUERY_SUBSYSTEM, "linked 4", "");

The parameters to the task_start() macro correspond to the number, string, and key parameters of the original llMessageLinked() call.

sending MODULE_INFO

Macros are provided to make managing the module identification string easy; use a set_identity() call in the state_entry() method, followed by a send_identity() call. For example, the _ambiance system library in Companion has:

	list messages = [
		LINK_ID,
		QUERY_SXDWM_INFO,
		POWER_STATE,
		MODEL,
		SERIAL,
		EFFECT_CHAT,
		EFFECT_DENY,
		EFFECT_PIP,
		EFFECT_TP,
		COLOR,
		GENDER_P,
		GENDER_M,
		CORTEX_COMMAND,
		EFFECT_FAILURE,
		PROBE_MODULES,
		WAKE_CHANGED_OWNER
	];
			
	list commands = ["volume", "scheme"];
		
	set_identity(MT_LIBRARY, "sound and effects library",
		messages,
		commands
	);

	send_identity();

The first list describes all of the linked messages (by number) that the module handles. It is not necessary to include 0 (EXECUTE) or 1 (COMMAND). The second list describes all of the commands it supports for EXECUTE or COMMAND calls. Note that since the Boost::Wave preprocessor is intended for C, not LSL, it does not support list literals with square brackets as macro arguments, so lists must be stored as variables in the local scope and referenced by name. Attempting to pass a list with more than one argument directly to a macro will result in a 'too many macro arguments' error.

Note that the command cache used from 8.3.0 to 8.4.5 is not mentioned; it is now redundant, so code for registering commands with messages 181 through 184 (QUERY_COMMAND, REPORT_COMMAND_PUBLIC, REPORT_COMMAND_SECURE, DELETE_COMMAND) can be safely removed. When a module is deleted from the system, the kernel automatically de-lists it.

The MT_ constants (MT_LIBRARY, MT_SERVICE, or MT_SECSERVICE) determine whether the module can be hibernated (i.e., it is a library) or not. MT_SECSERVICE indicates that owner-level clearance is required to reset the module, as it is important for security reasons.

The other parameter of the set_identity() message is a short description that will be displayed to the user along with the @module command.

If you are writing a library that does not immediately need to start a thread (see below), then it is appropriate to end the state_entry method with a call to the hibernate() macro. This stops the module from running, regardless of whether or not it has on-going tasks.

The link_message event handler should include a case for if(num == PROBE_MODULES) that triggers send_identity(). It is not necessary to include this in the list of messages the module handles, although it is good practice to do so anway.

basic hibernation

At the end of the link_message event, include a call to the end_task() macro. This takes the same parameters as the start_task() macro: the integer, string, and key values provided at the start of the event. Unhandled messages should not generate end_task() calls, e.g.

	link_message(integer sender_num, integer num, string str, key id) {
		if(num == PROBE_MODULES) {
			send_identity();
		} else if(num == EXECUTE || num == COMMAND) {
			list argv = llParseString2List(str, [" "], []);
			integer argc = llGetListLength(argv);
			string cmd = llList2String(argv, 0);
			
			if(cmd == "scheme") {
				/* command implementation here */
			} else {
				return; // no command handled
			}
		} else {
			return; // no message handled
		}
		
		end_task(num, str, id);
	}

This structure prevents sending of redundant accounting messages that would slow down the system, and minimizes the risk of premature termination in case multiple tasks have the same signature but should not be handled (e.g. back-to-back AUTH messages intended for different libraries.)

threading

The start_thread() and end_thread() macros can be used to keep a library active as long as needed to complete an event-driven task, such as reading a notecard. start_thread(string, key) macro begins a thread, and an end_thread(string, key) call with the same parameters will terminate the matching thread. For example, _foundation (BootService) uses start_thread("OEM", "") and end_thread("OEM", "") while loading OEM chip data. It also uses a thread called "boot" to prevent suspension during the many llSleep() calls that are built into the start-up process.

WAKE events

Some events are rare enough that it makes more sense to implement an external event monitor with a library instead of keeping a service around that only listens for them. Companion supports management of changed, on_rez, and longer timer events through the kernel. For example, _ambiance hooks the WAKE_CHANGED_OWNER message, allowing it to update the light bus channel in the event of an owner change. Supported WAKE event messages are:

2044
WAKE_ALL
HV2
2043
2039
2040
2041
WAKE_ON_REZ
mixed
2036
2038
WAKE_TIMER
AM, HV

timer emulation

The SET_WAKE_TIMER message instructs the kernel to create an event and wake the process when the event occurs. The kernel tracks the exact Unix timestamp (to the second) when this should happen, so there is some protection against latency and system powerdowns. SET_WAKE_TIMER can be used conveniently through the set_timer() macro:

set_timer(event_name, time, expiry, permanent, id)

where:

  • event_name is an arbitrary text mnemonic for use by your code,
  • time is either a number in seconds from the present or a 24-hour hh:nn:ss string in SLT at which the event should occur,
  • expiry is how many seconds late the event may be triggered and still considered valid (0 to disable),
  • permanent is a boolean indicating whether (TRUE) or not (FALSE) the task should be repeated indefinitely,
  • and id is an arbitrary (and optional) key mnemonic for use by your code.

To remove a timer, use a permanent value of -1. If it exists, the timer matching the same combination of event_name and id for the module will be deleted. Only one timer with a given combination of event_name and id may exist at a time per module; repeat calls to set_timer() with these parameters will result in changes to the timer's other settings (time, expiry, and permanent values).

Permanent timers specified with a time of day will recur daily.

settings management

Module suspension is not perfect; if a script's running state is 0 when the object containing it is de-rezzed (including when the avatar wearing an attachment logs out), the script will lose its memory and be reset when it is next loaded. To mitigate this, Companion 8.5 introduced a new system module called the configuration manager (_balance or SettingsService), which stores all of the settings used by libraries. The following messages are used to manipulate settings stored by the configuration manager:

125
123
122
124
121
120
SETTING_VALUE
libraries

Settings management is a hierarchical key-value store, similar to the Windows registry and many other configuration systems. Variables are named using a dot-based object hierarchy starting with the module name, e.g. _obedience.autolock, and branches of the tree can retrieved or deleted by referencing an ancestor with the appropriate numeric message.

An additional header file called conf.lsl is available, containing convenience functions for many tasks related to the configuration manager. An example of usage is below:

#include <system.lsl>
#include <utils.lsl>
// tasks.lsl is not used in this example

key conf_key; // used to filter out irrelevant SETTING_VALUE messages
string tp_sound; // value we want to fill, stored in the setting entry "effects.tp"

default {
	state_entry() {
		conf_set("effects.tp f82958c4-e8c6-e992-5a24-4e66e9c1113e"); // sends SETTING_SET

		// retrieve all settings with 'effects' as their first name segment:
		conf_get("effects", conf_key = llGenerateKey());
		// by generating a conf_key we can filter out messages meant for other modules
	}
	
	link_message(integer src, integer num, string message, key id) {
		if(num == SETTING_VALUE) {
			if(id == "" || id == NULL_KEY || id == conf_key) {
				list lines = split(message, "\n");
				integer i = count(lines);
				while(i--) {
					string s = gets(lines, i);
					integer j = strpos(s, " ");
					string name = substr(s, 0, j - 1);
					string value = substr(s, j + 1, -1);
					
					if(!~j) value = "";
					
					list name_parts = split(name, ".");
					string hive = gets(name_parts, 0);
					name = gets(name_parts, 1);
					
					if(hive == "effects") {
						if(name == "tp") {
							tp_sound = value;
						} // else if other "effects.[name]" variables here
					} // else if other "effects" sections here
				}
			}
		}
	}
}

compatibility with 8.4 and earlier applications

As mentioned above, compat or file compat must be enabled for older applications to receive system messages in the format they expect. As long as this is enabled, most older applications will still work with the system, but calls that address libraries may not receive a response—and may cause unpredictable behavior in the unlikely event they interrupt a thread. If your code sends a linked message which is handled by a module marked with one or more asterisks on the module overview page, then it will need to be updated for full compatibility, and we strongly recommend switching away from APP_ messages in the process. This does not affect the APPLICATION_ messages (in the -9000 range) that are used by the system applications menu.

simple recipes

Below are some basic skeletons of example programs to get started.

Non-hibernating, single-command user program

#include <system.lsl>
#include <utils.lsl>
#include <tasks.lsl>

default {
	state_entry() {
		list messages = []; // linked message numbers your script handles - this can be empty in a non-hibernating script
		list commands = ["mycommand"]; // commands your program handles
		set_identity(MT_SERVICE, "example app", messages, commands); // this will emit a memory count in chat; use set_identity_quiet() to avoid
		send_identity();
	}

	link_message(integer src, integer num, string message, key id) {
		if(num == PROBE_MODULES) {
			send_identity();
		} else if(num == COMMAND || num == EXECUTE) {
			// COMMAND: user typed it; EXECUTE: something else triggered it

			list argv = split(message, " "); // same as llParseString2List(message,  , );
			string cmd = gets(argv, 0); // same as llList2String(argv, 0);

			if(cmd == "mycommand") {
				string who;
				if(id) // id is the person who activated the command; may be NULL_KEY or blank in some situations
					who = (string)id;
				else
					who = "nobody";
				
				if(num == EXECUTE)
					echo("Command executed by " + who); // echo() is llOwnerSay()
				else
					echo("Command commanded by " + who);
			}
		}
	}
}

Hibernating, single-command user program

#include <system.lsl>
#include <utils.lsl>
#include <tasks.lsl>

default {
	state_entry() {
		list messages = []; // linked message numbers your script handles - this is only empty because we don't handle anything special
		list commands = ["mycommand"]; // commands your program handles
		set_identity(MT_LIBRARY, "example app", messages, commands); // this will emit a memory count in chat; use set_identity_quiet() to avoid
		send_identity();
		hibernate(); // suspend execution without using kernel: llSetScriptState(llGetScriptName(), FALSE);
	}

	link_message(integer src, integer num, string message, key id) {
		if(num == PROBE_MODULES) {
			send_identity();
		} else if(num == COMMAND || num == EXECUTE) {
			// COMMAND: user typed it; EXECUTE: something else triggered it

			list argv = split(message, " "); // same as llParseString2List(message,  , );
			string cmd = gets(argv, 0); // same as llList2String(argv, 0);

			if(cmd == "mycommand") {
				string who;
				if(id) // id is the person who activated the command; may be NULL_KEY or blank in some situations
					who = (string)id;
				else
					who = "nobody";
				
				if(num == EXECUTE)
					echo("Command executed by " + who); // echo() is llOwnerSay()
				else
					echo("Command commanded by " + who);
			} else {
				return; // guard code to stop premature hibernation - this will be reached if another command is called while this script is awake
			}
		} else {
			return; // guard code to stop premature hibernation - this will be reached if another linked message is called while this script is awake
		}

@bail;
		end_task(num, message, id); // tell kernel we finished a task; when no more tasks are tracked by the kernel, it will put us to sleep
		// to make sure this line is executed, use "jump bail" instead of "return" inside link_message()
	}
}