A Knowbot Program written in Python consists of a main module and any number of additional modules. When the KP is executed, either on its initial submission or after a clone() or migrate() operation, the main module is loaded and executed with the module name (the variable __main__) set to '__main__'. Additional modules must be imported by the main module (or by some other module that is imported, directly or indirectly) in order to be available. All this is just like it is for normal Python programs.
The main module should define a class called "KP" which will be instantiated and invoked automatically by the KP supervisor after the main module has been loaded. (There's a way to choose a different name for this class but the KP launching tools don't support it yet.) The KP class should define a __main__() method taking one argument. If it has a constructor (__init__() method), it should take no arguments.
The KP class is instantiated differently depending on whether this is the initial submission or the result of a clone() or migrate() operation:
Once the KP instance has been created, its __main__() method is invoked with a single argument, the ``KOS bastion''. The KOS bastion defines a number of methods that enable communication with the KP supervisor and other KPs, including the clone() and migrate() operations. Its interface is documented in the KOS Bastion Methods section.
When the __main__() method returns, the KP terminates and its submittor (strictly speaking, the ``Reporting Station'' responsible for it) is notified of its termination. Destructors for objects that are still alive at this point may or may not be invoked (so don't count on them either way).
It is permitted to store the KOS bastion as an instance variable; after migration (or cloning) this instance variable will automatically refer to the KOS bastion for the current KP instance's supervisor.
Python KPs can catch exceptions using the try-except statement. Unhandled exceptions are caught by the KP supervisor and cause termination of the KP. The SystemExit exception is handled specially in that is it handled like normal termination. This is analogous to its treatment in ordinary (non-KP) Python programs. All other unhandled exceptions cause the termination to be treated as "abnormal". For abnormal termination, a stack trace of the program is normally printed by the submittor; for normal termination, nothing is printed.
A KP is executed in Python's restricted execution mode. This mode restricts the damage that a KP can to to its host environment. For this purposes, restrictions on which built-in (or dynamically loaded) modules and functions can be used exist, and some language features are disabled. Two examples of the latter: the function object attribute func_globals and the class and instance object attribute __dict__ are unavailable. The following sections discuss the restrictions on built-in functions and modules.
For full documentation on the functions and modules discussed here, see the Python Library Reference.
The following built-in functions are unavailable or restricted:
The following built-in modules are available without restrictions:
The following built-in modules are available with some restrictions:
The following variables are available with the same meaning as in normal Python programs:
Only a small number of functions are available:
A very minimal stub look-alike of the ilu module is available. It only contants the standard ILU exceptions, so KPs can catch them:
No other built-in modules are available.
Note that even though there are no explicit restrictions on modules implemented in Python, the absence of built-in modules like socket means that any standard library modules that use it are also effectively unusable.
Without turning into a tutorial, we discuss some common operations that a KP might involve in. This complements the KOS API documentation.
A typical KOS service is implemented as a KOS Plugin Module. Many services are structured as a ``factory'' object shared by all clients. Clients interact with the factory only to create a personalized ``interface'' object with which they carry out all further interaction. Thus, there are the following steps:
factory = kos.lookup_service(type, name
Note the .Open() call tacked on the end. The kos.lookup_service() call itself returns a descriptor for the service which is not directly usable. The Open() method on the descriptor returns an interface stub that can be used to interact with the service.
interface = factory.create_interface
Note that both the factory and interface objects are ILU surrogate objects. It is often the case that the interface object refers to a different server process than the factory process, though.
This is generally done in the same way as interaction with plugin services, except that usually the KP service won't use an interface factory. So there are only two steps:
interface = kos.lookup_service(type, name
In the current KOS version, you can only provide a service if its ILU ISL file is present in the koe/interfaces directory and stubs for it have been made in the koe/interfaces/stubs directory. (We will eventually provide a way whereby the interface definition can be carried around in the suitcase of the KP instead.) Given an ISL file and an interface in it, here's how you create a service implementing the interface:
Choose an interface. If you decide to create a new one, you have to make sure that its ISL file and stubs are present on every target KSS where you want to run your KP. It's easier to use an existing interface. Fortunately, there's a ``generic'' interface that lets you pass arbitrary lists of strings between client and server. See the source code for the GenericImpl module.
Your KP should define a class that implements the interface. The class needn't inherit from anything, and you can pick the class name freely. The class should have methods implementing each of the methods of the chosen interface (the method names should correspond -- you cannot choose them freely). The arguments and return value of these methods should correpond to the arguments and return value of the corresponding method specified in the ISL file (this is the same as for ILU true servers, except those need to inherit from the __skel class; see the ILU documentation for details of the mapping, in particular concerning arguments of type OUT or IN OUT, and some more esoteric types such as ILU records).
Pick a name for your service. Some requirements on the name are:
Create an instance of your service class.
Bind your instance to a service name and type. This is done using a statement like
desc = kos.bind_service(name, instance, type)where `kos' refers to the KOS bastion, and the arguments name, instance, type are the service name you chose, the instance you created, and the type name you chose (of the form "module.interface"), respectively. The return value (`desc') is a descriptor for the service, to be saved for later to unbind the service.
Activate your service. This done using the statement
This call starts the ILU main loop. It will not return control until the ILU main loop is stopped again by the statement
When a client uses your service, the ILU main loop will call your instance's methods. (The kos.stop() call could be executed by some method of your service, e.g., a call signalling that no client is in need of your service any more.) Note that an exception occurring in a method called from ILU does not exit the ILU main loop. ILU simply prints a traceback and continues servicing requests; the client that made the failing request receives an IluGeneralError exception.
Unbind your service. In general, it is not necessary to unbind your service if your KP is going to exit. You may explicitly unbind your service using the statement
kos.unbind_service(desc)where `desc' is the descriptor for the service, the saved return value of the kos.bind_service() call.
class Service1: ... class Service2: ... class Service3: ... class KP: def __main__(self, kos): inst1 = Service1() inst2 = Service2() inst3 = Service3() desc1 = kos.bind_service("gvr-service1", inst1, "module1.interface1") desc2 = kos.bind_service("gvr-service2", inst2, "module2.interface2") desc3 = kos.bind_service("gvr-service3", inst3, "module3.interface3") kos.run()
Of course, you could also create several instances of the same service, with somewhat different characteristics.
Every KP carries a ``suitcase'' along. This is a form of ``external'' or ``bulk'' storage which is stuctured and accessed roughly the same way as a file system. The suitcase is moved along with the KP on migration and copied into (but not shared between!) clones. It does not automatically persist when a KP terminates, but KP submission tools have a way to retrieve selected parts from the suitcase on termination. You can think of the suitcase as an individual, portable file system with a limited persistency.
The standard idioms for accessing the suitcase looks something like this:
class KP: def __main__(self, kos): # Access the suitcase filesystem sfs = kos.get_suitcase() # Create a directory sfs.mkdir("/directory") # Write a file f = sfs.open("/directory/filename", "w") f.write("something\n") f.write("something else\n") f.close() # List a directory names = sfs.listdir("/directory") # Change current directory sfs.chdir("/directory") # Read a file f = sfs.open("filename", "r") while 1: line = f.readline() if not line: break f.close() # Remove a file sfs.remove("filename") # Remove a directory sfs.rmdir("/directory")
Complete documentation of the suitface filesystem interface can be found in the documentation for the Suitcase module.
While migration is straightforward (see the tutorial), a program using clones generally needs to implement some form of communication between clones, or between clones and the original. There are a number of different communication patterns:
In all cases, a problem is that you can only use kos.lookup_service() to request a service on your own KSS.
In case 1, the simplest solution may be to migrate back to the KSS where the master resides (assuming the master doesn't move), and initiate communication there. The clone can make the kos.lookup_service() call upon arrival at the original KSS.
In case 2, there's no point in migrating to do the communication, so the clone has to connect with the master which is residing on a different KSS. In this case, there are two solutions. Either the clone is initially sent to the master's KSS, where it makes the kos.lookup_service() call, stores the result as an instance variable on the KP instance, and then it migrates to its destination. Alternatively, it migrates directly to its destination, and uses the approach for case 3 to look up the master's service.
Case 3 really has to deal with the fact that you can only use kos.lookup_service() for local services. Fortunately, kos.lookup_service() mostly just does a namespace lookup, and you can do that yourself if you know the name of the KSS where the service resides. For example,
import nstools obj = nstools.WorldOpen( "/kos/kssname/services/servicename", "type")
Here, kssname is the name of the KSS where the service is running, servicename is the name of the service, and type is the service's type. The result is the interface object which allows interaction with the service (no separate .Open() call is needed when you use nstools.WorldOpen()).
Note that the nstools.WorldOpen()
call can raise a nstools.BadPathError exception. This can
happen when two clones travel with different speed to their destinations,
and the client clone attempts to communicate with the server clone before
the server clone has registered its service. It is best to write a loop
with a delay (e.g.,
time.sleep(1)) that retries the operation
if it fails. A timeout (e.g. a limit of 60 cycles through the loop, after
which you give up) is also useful, in case the server clone was somehow
Case 4 can be solved the same way as case 3, but here the master looks up the clone's service. Because the travel time for clones is indeterminate, the loop-retry-timeout approach should be used.
Case 5 can be solved by combining the above techniques.
There's a caveat for all cases: a KP cannot simultaneously do useful work and operate a service. This is because in order to operate a service, kos.run() must be invoked, which means that the program is suspended, waiting for incoming service requests. The best solution is to design the communication patterns to avoid this -- e.g., some KP's can be servers and others can do useful work and be clients. It is possible to do some useful work first and then start operating a service, perhaps waiting for the cows, ehm, clones to come home. It is also possible to interrupt the work frequently to make a dummy RPC call.
Trigger variables provide an alternative means for communication between KPs on the same KSS. In the current system they are helpful because they provide a way for KPs to communicate with each other without registering a service. The trigger interface is not particularly useful, and its implementation in the kernel is unnecessary. Trigger variables are likely to be eliminated in a future version, in favor of a generic event interface that KPs can use.
Catching exceptions raised by service invocations in a KP requires some additional effort, because the ILU stub module that defines the exception names cannot be imported from a KP (and exceptions must be referenced by their object identity, not by their string value). The KOS bastion provides an interface to access these exceptions. There are two versions: kos.exception("mod.name") returns the exception `name' defined in module `mod'; kos.all_exceptions("mod") returns a dummy module object containing all exceptions defined in module `mod'.
It is customary to store these exceptions in global variables at the start of the KP's __main__() method, so that the rest of the KP's code can use the familiar style of referencing exceptions. For example:
class KP: def __main__(self, kos): global mod mod = kos.all_exceptions("mod") ...other stuff... def some_other_method(self): try: some_remote_operation() except mod.SomeException: ...handle the exception...
The same procedure to access exceptions can be used by a KP that provides a service and needs to raise an exception defined in the service's interface definition.
In order to catch ILU exceptions, KPs can import the module ilu. which defines (only) the ILU exceptions IluGeneralError, IluProtocolError, and IluUnimplementedMethodError.