Difference between revisions of "PyOGP Client Library"

From Second Life Wiki
Jump to navigation Jump to search
(Formatting repair)
 
(46 intermediate revisions by 5 users not shown)
Line 2: Line 2:
== Intro ==
== Intro ==


PyOGP is a young open source python client library which provides an interface to an agent (client) of a Second Life grid. Conceived as a mechanism for testing OGP grid changes, it carries on with a charter of enabling automated testing of Second Life grids, enabling exploration into the client/server relationship and related protocols of Second Life, and enabling prototyping of various client applications.
PyOGP is a young, open source, python client library which provides an interface to an agent (client) of a Second Life grid. The code aims to enable automated testing of Second Life grids, to encourage and enable exploration into the client/server relationship and related protocols of Second Life, and to make possible simple prototyping of various client applications.


Hosted on svn.secondlife.com, it does require a contributor's agreement for commit access, and currently has a few contributors from the Second Life community.
Hosted on svn.secondlife.com, it does require a contributor's agreement for commit access, and currently has a few contributors from the Second Life open source community.


P.S. We'll likely be moving to hg relatively soon...
P.S. We'll likely be moving to hg relatively soon...
Line 21: Line 21:
Having this library available also allows us to test potential changes before we have finalized design and are ready to submit to QA. Not sure how something will play out? Try it, and test it with PyOGP....
Having this library available also allows us to test potential changes before we have finalized design and are ready to submit to QA. Not sure how something will play out? Try it, and test it with PyOGP....


== Structure ==
=== A Brief History ===


PyOGP is comprised of two python packages.
PyOGP was originally created as a tool for testing [[OGP]] related changes to the Second Life grid. By the end of the summer in 2008, the '''pyogp.lib.base''' package provided a skeleton messaging system and was able to test Linden Lab's first implementation of an agent domain and the related changes in an agent's region handoff and teleport. As the development effort around OGP waned, we started to extend pyogp by adding higher level client related functionality. Recently, this functionality was split out into a separate python package ('''pyogp.lib.client'''), and sample scripts (and future apps) were moved into '''pyogp.apps'''.
 
== Architecture Overview==
 
=== Structure ===
 
PyOGP is comprised of three python packages. The library consists of pyogp.lib.base and pyogp.lib.client, while sample scripts, and in time applications, live in pyogp.apps.  
:'''pyogp.lib.base''' - consists of basic networking, messaging systems (UDP and event queue) and capabilities, custom datatypes, and a low level message related event system
:'''pyogp.lib.base''' - consists of basic networking, messaging systems (UDP and event queue) and capabilities, custom datatypes, and a low level message related event system
:'''pyogp.lib.client''' - consists of 'convenience' classes mapping methods and handlers to specific messages (e.g. Agent().say(msg) tells the client to send the ChatFromViewer message to the host region). Raises application oriented events based on processing of messages (these are currently sparsely implemented)
:'''pyogp.lib.client''' - consists of 'convenience' classes mapping methods and handlers to specific messages (e.g. Agent().say(msg) tells the client to send the ChatFromViewer message to the host region). Raises application oriented events based on processing of messages (these are currently sparsely implemented)
:'''pyogp.apps''' - sample scripts and works in progress, the scripts here generally illustrate simple usage of classes as related to in world interactions by an agent of a Second Life grid
:'''pyogp.apps''' - sample scripts and works in progress, the scripts here generally illustrate simple usage of classes as related to in world interactions by an agent of a Second Life grid


== Dependencies ==
=== Dependencies ===
 
==== Platform / Python version compatibility ====
 
PyOGP aims to be compatible across platforms, though there are known problems with various environments. We'll be focusing on ensuring better compatibility soon.
 
'''Known good configurations'''
 
:Windows XP + Python 2.5
:Mac + Python 2.5, 2.6
:Linux + Python 2.4, 2.5, 2.6 (Linden hosts fall into this group)
 
'''Known bad configurations'''
:Windows Vista + Python 2.6
:Windows 7 + Python 2.6
 
There have been challenges in ensuring compatibility between the various dependencies, largely due to greenlet, eventlet, and pyopenssl. Please report bugs on [http://jira.secondlife.com/browse/PYO pJira]
 
==== Python module dependencies ====


The packages that make up PyOGP have some dependencies on python modules not included in a standard install, or sometimes not available on an older Python distribution.
The packages that make up PyOGP have some dependencies on python modules not included in a standard install, or sometimes not available on an older Python distribution.
PyOGP is known to work well in Python 2.4.x and 2.5.x. Known issues are preventing successful use on Python 2.6.


'''pyogp.lib.base dependencies:'''
'''pyogp.lib.base dependencies:'''


  from setup.py
<syntaxhighlight lang="python">  #from setup.py
    
    
     install_requires=[
     install_requires=[
Line 43: Line 65:
         'uuid',
         'uuid',
         'elementtree',
         'elementtree',
         'indra.base',
         'llbase',
         'WebOb',
         'WebOb',
         'wsgiref',
         'wsgiref',
         'eventlet==0.8.14'
         'eventlet==0.8.14',
         ]
        'pyOpenssl'
         ]</syntaxhighlight>


'''pyogp.lib.client dependencies:'''
'''pyogp.lib.client dependencies:'''


    install_requires=[
<syntaxhighlight lang="python">    install_requires=[
         'setuptools',
         'setuptools',
         # -*- Extra requirements: -*-
         # -*- Extra requirements: -*-
         'pyogp.lib.base'
         'pyogp.lib.base'
         ]  
         ] </syntaxhighlight>


'''pyogp.apps dependencies:'''
'''pyogp.apps dependencies:'''


    install_requires=[
<syntaxhighlight lang="python">    install_requires=[
         'setuptools',
         'setuptools',
         'pyogp.lib.client'
         'pyogp.lib.client'
         ]
         ]</syntaxhighlight>
 
=== How to install ===
 
''Lindens can see internal documentation for more specific guidance. https://wiki.lindenlab.com/wiki/Pyogp#How_to_Install''
 
==== Standalone dev environment using buildout ====
 
Buildout is a type of Python development environment, organizing and configuring various components and their dependencies. On a desktop, one may checkout such an environment for working with PyOGP. One may optionally use a virtualenv Python environment to isolate the development code and it's runtime environment on one's host.
 
Dependencies: buildout takes care of everything, grabbing needed modules etc.
 
Wiki instructions: https://wiki.secondlife.com/wiki/Pyogp/Client_Lib/The_Development_Sandbox
 
==== Installing the PyOGP packages ====
 
Each of the PyOGP package may be installed to one's Python install or to a [http://pypi.python.org/pypi/virtualenv virtualenv]. '''Buyer beware''' if installing into your system's install: you'll want to be able to uninstall manually, as we haven't hooked up the uninstall. PyOGP is still coming up to speed with respect to distutils and pypi and the like, but it's relatively close now.
 
To install a package, simply run 'python setup.py install' in a package's root.
 
==== Referencing PyOGP packages via the PATH ====


PyOGP aims to be compatible across platforms, though there are known problems with various environments. PyOGP runs well on Windows XP with Python 2.5.x, on Mac with Python 2.5.x, and on Linden hosts running Python 2.4.4. Problems have been observed on Windows Vista and 7 (due to Python 2.6 and or pyopenssl related problems). We'll be focusing on ensuring better compatibility soon.
Source code can be referenced directly if one simply ensures that a package, and it's dependencies, are available in the PYTHONPATH environmental variable.


== Current functional coverage ==
== Current functional coverage ==
Line 105: Line 148:
:* LSL script uploading
:* LSL script uploading


== How to install and use ==
=== Sample Scripts ===


=== Standalone dev environment using buildout ===
There are a variety of scripted examples that have been used to exercise and test functionality as it is added to the library. These persist as coded documentation.


On a desktop, one can checkout a dev environment for working with pyogp. Currently, buildout is used to configure the environment. One may optionally use a virtualenv python env.
The source code is available in https://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.apps/trunk/pyogp/apps/examples/.


Dependencies: buildout takes care of everything, grabbing needed modules etc.
The following refers to a buildout context. If one installs pyogp.apps vi setup.py, these scripts will exist in the python environment's bin/ directory. In the buildout context, these scripts are available in {buildout root}/bin.
 
The scripts are derived from a package's 'setup.py' via the entry_points parameter, and essentially build executable Python scripts configured to run in the correct environment with the proper dependencies added the the path used by the script. These scripts are currently just simple illustrations of some uses of the PyOGP codebase.
 
<syntaxhighlight lang="python">  #the current entry_points in setup.py og pyogp.apps:
 
    entry_points={
        'console_scripts': [
            'AIS_inventory_handling = pyogp.apps.examples.AIS_inventory_handling:main',
            'agent_login = pyogp.apps.examples.agent_login:main',
            'agent_manager = pyogp.apps.examples.agent_manager:main',
            'appearance_management = pyogp.apps.examples.appearance_management:main',
            'chat_and_instant_messaging = pyogp.apps.examples.chat_and_instant_messaging:main',
            'group_chat = pyogp.apps.examples.group_chat:main',
            'group_creation = pyogp.apps.examples.group_creation:main',
            'inventory_handling = pyogp.apps.examples.inventory_handling:main',
            'inventory_transfer = pyogp.apps.examples.inventory_transfer:main',
            'inventory_transfer_specify_agent = pyogp.apps.examples.inventory_transfer_specify_agent:main',
            'login = pyogp.apps.examples.login:main',
            'multi_region_connect = pyogp.apps.examples.multi_region_connect:main',
            'object_create_edit = pyogp.apps.examples.object_create_edit:main',
            'object_create_permissions = pyogp.apps.examples.object_create_permissions:main',
            'object_create_rez_script = pyogp.apps.examples.object_create_rez_script:main',
            'object_creation = pyogp.apps.examples.object_creation:main',
            'object_properties = pyogp.apps.examples.object_properties:main',
            'object_tracking = pyogp.apps.examples.object_tracking:main',
            'parcel_management = pyogp.apps.examples.parcel_management:main',
            'parse_packets = pyogp.apps.examples.parse_packets:main',
            'region_connect = pyogp.apps.examples.region_connect:main',
            'smoke_test = pyogp.apps.examples.smoke_test:main',
            'chat = pyogp.apps.examples.chat_interface:main',
            ]
        }</syntaxhighlight>
 
== How it Works (High Level) ==
 
=== Eventlet ===
 
PyOGP use [[Eventlet]] to run coroutines to handle multiple 'concurrent' processes, rather than threads or multiple processes. Each client agent instance will spawn a handful of coroutines to handles e.g. the UDP pipe, the Event Queue, various monitors, while yielding time to the parent process which should ensure it yields to the other routines as well.


Wiki instructions: https://wiki.secondlife.com/wiki/Pyogp/Client_Lib/The_Development_Sandbox
PyOGP uses eventlet in very elementary ways at this point, but will perhaps start to use blocking queues in some cases, so that the coroutine only is allocated processing time if there is work for it to do.


=== pyogp.lib.base ===


=== Using only pyogp.lib.base (aka the Linden context) ===
This package handles the protocols used when communicating with a Second Life grid. A high level perspective on the package reveals a MessageManager() (still in development) which provides an interface to the UDP and Event Queue connections, as well as basic networking with enables login and capabilities interactions. The base package also has a low level even system through which all messages are passed and sent to subscribers.


Updated info available via Enus as needed, will be updating docs asap.
Any subcomponent is available for direct interaction at any time, the MessageManager() and the MessageHandler() are the simple access points.


<!--
==== Events & Callbacks ====


All of the below dependencies are available in a standard python path on a Linden host when running tests or when PYTHONPATH points to checkouts/deploys. To use only pyogp.lib.base then, one just needs the package:
The event implementation in pyogp follows the observer pattern, where observers subscribe to and are notified when an event occurs. Data is passed throughout the client instance via events.  


:svn co https://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.lib/base/trunk/pyogp
* MessageManager - is an attribute of a Region and every packet received/sent is filtered through here. subscriptions are by message name
** MessageHandler - is an attribute of MessageManager, and every categorized message received from the event queue or udp dispatcher is filtered through here. (message as defined in message_template.msg, or one of ['ChatterBoxInvitation', 'ChatterBoxSessionEventReply', 'ChatterBoxSessionAgentListUpdates', 'ChatterBoxSessionStartReply', 'EstablishAgentCommunication']. There may be unhandled messages, I just haven't seen em yet :))


and ensure that the 'pyogp' directory is in the PATH, e.g. bringing this repo into indra/lib/base as an external.
===== Message Events =====


Dependencies:
In the most fundamental implementation of event usage, all packets are passed through a MessageManager() instance for evaluation. Observers may register to receive udp packets serialized into the form of Message() instances. The MessageHandler() is a consolidation point for subscribing to messages keyed by message name, and created on demand via subscription.


:eventlet
See pyogp.lib.base.message.message_handler.MessageHandler() for more details.
:greenlet
:pyopenssl
:indra.base
:indra.util
:indra.ipc
:uuid (in the case of an older python instance)
:elementtree


-->
The pyogp agent's Region() instances each monitor their stream of packets (e.g. the host region: agent.region.message_manager.message_handler). (Perhaps this should be changed to a generalized Network() class where all packets (coupled to their originating regions) are evaluated.


== Sample Scripts ==
Event firing passes data on to a callback handler defined in the subscription, in the form of (handler, *args, **kwargs).


There are a variety of scripted examples that have been used to exercise functionality as it is added to the library. These persist as coded documentation.
The Agent class monitors the ImprovedInstantMessage packet:
<syntaxhighlight lang="python">        onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
        onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)


'''Note''':  
    def onImprovedInstantMessage(self, packet):
        """ handles the many cases of data being passed in this message """


Sample scripts are brought into the buildout as an external package, pyogp.apps, and are executable in buildout's bin/ directory. The source code is available in https://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.apps/trunk/pyogp/apps/examples/ .
        {code} # parse and handle the data...</syntaxhighlight>


"python {script} -h" will display {meager} usage for each script.
The messaging system then fires the event when an ImprovedInstantMessage message is received, which calls onImprovedInstantMessage method above to handle the message contents. Multiple subscribers may be listening for any message related event, and each would be notified of the same Message() instance.


:bin/AIS_inventory_handling
Unsubscribing from an event:
:bin/agent_login
<syntaxhighlight lang="python">        onImprovedInstantMessage_received.unsubscribe(self.onImprovedInstantMessage)</syntaxhighlight>
:bin/agent_manager
:bin/appearance_management
:bin/chat_and_instant_messaging
:bin/group_chat
:bin/group_creation
:bin/inventory_handling
:bin/inventory_transfer
:bin/inventory_transfer_specify_agent
:bin/login
:bin/multi_region_connect
:bin/object_create_edit
:bin/object_create_permissions
:bin/example/object_creation
:bin/example/object_properties
:bin/example/object_tracking
:bin/example/region_connect
:etc


== Agent Login ==
There are various event and callback implementations viewable in pyogp, poke around and help consolidate things if you like.


=== Single Agent Login ===
=== pyogp.lib.client ===


non-eventlet context: many test cases may be written this way. the main script process is blocking and will terminate the client when completed.
The client package generally provides a convenient interface to initiate or interpret interactions with host region (or neighboring regions). By listening to the messaging related event system in pyogp.lib.base, the client package interprets the messages that come in off the wire, and executes business logic in building responses. pyogp.lib.client also provides simple methods to enable the ending of messages to a grid.


<python>from pyogp.lib.client.agent import Agent
==== Events ====


client = Agent()
* EventsHandler - an attribute of an Agent, also able to be passed in, that is intended as the primary interface of a pyogp application into the internal state and data events within the lib. This system uses the same base classes as used by the MessageHandler() in the base package, and the descriptions about events and callbacks above apply here as well. The api for subscribing to these events is similar to the MessageHandler(), with an additional timeout parameter passed in the _register() method. When the specified timeout expires, the subscription returns None and expires the subscription.


client.login(options.loginuri, args[0], args[1], password, start_location = options.region</python>
==== Agent Login (examples) ====


Eventlet context: spawn a client in a co-routine, allowing persistent presence until forcefully terminated.
===== Single Agent Login & Chat=====


<python>from eventlet import api
Spawn a client in a co-routine, allowing persistent presence until forcefully terminated.
 
<syntaxhighlight lang="python">from eventlet import api


from pyogp.lib.client.agent import Agent
from pyogp.lib.client.agent import Agent
Line 205: Line 269:
while client.region.connected == False:
while client.region.connected == False:
     api.sleep(0)
     api.sleep(0)
client.say("Hello World!")


# once connected, live until someone kills me
# once connected, live until someone kills me
while client.running:
while client.running:
     api.sleep(0)</python>
     api.sleep(0)</syntaxhighlight>


=== Multiple Agent Login ===
===== Multiple Agent Login =====


Eventlet and non-eventlet spawning methods apply to the agent manager in the same way as they apply to single agent usage. Each agent instance in logged in in a separate coroutine.
Each agent instance in logged in in a separate coroutine.


<python>from pyogp.lib.client.agent import Agent
<syntaxhighlight lang="python">from pyogp.lib.client.agent import Agent
from pyogp.lib.client.agentmanager import AgentManager
from pyogp.lib.client.agentmanager import AgentManager
from pyogp.lib.client.settings import Settings
settings = Settings()


params = [['agent1', 'lastname', 'password'], ['agent2', 'lastname', 'password']]
credentials= [('agent1', 'lastname', 'password'), ('agent2', 'lastname', 'password')]
agents = []


# prime the Agent instances
# prime the Agent instances
for params in clients:
agents = [Agent(settings, firstname, lastname, password)
 
          for firstname, lastname, password in credentials]
    agents.append(Agent(settings, params[0], params[1], params[2]))


agentmanager = AgentManager()
agentmanager = AgentManager()
Line 237: Line 298:
# while they are connected, stay alive
# while they are connected, stay alive
while agentmanager.has_agents_running():
while agentmanager.has_agents_running():
     api.sleep(0)</python>
     api.sleep(0)</syntaxhighlight>


== Events & Callbacks ==
== Extending Functionality ==


The event implementation in pyogp follows the observer pattern, where observers subscribe to and are notified when an event occurs. Data is passed throughout the client instance via events.  
While the implementations and structures in pyogp.lib.base can (and are in the process of) being refactored to improve performance or usability, it is a fairly complete package.
 
The functional coverage PyOGP provides on the other hand is not complete, and there are a variety of needs to complete the implementation in pyogp.lib.base. We need to improve coverage of message handling (dealing with messages sent to the client), add more wrappers for sending various messages and performing multistep tasks (to simplify the initiation of interactions with the region), and we need to raise more application level events in the client package so that applications have easy access to incoming data.
 
=== Sending Messages ===
 
PyOGP, like the Viewer, communicates with the Second Life simulator by sending messages over UDP.
 
In order to extend PyOGP, you'll build a representation of a new UDP message, and send it through the pyogp.lib.base modules for serialization and wire handling.
 
=== Example: Sending an IM ===
 
To send an IM to the simulator, send an ImprovedInstantMessage packet. The base class for message packets is defined in <code>base/message/message.py</code>


There are 2 types of internal event systems:
Packets are assembled using a Message() instance which has the message name and Block() instances passed in through its constructor.  Similarly, Blocks are assembled by passing in the Block name and the value name and values for each of the Block values. (The ability to build Message() instances via an llsd payload is expected to be introduced soon.)


* MessageManager - is an attribute of a Region and every packet received/sent is filtered through here. subscriptions are by message name
Note: It is important that the message name, block name, and value names and types should match what is specified in the message template. (It is also possible to manipulate the representation of the stored template, or to use a custom message template.)
** MessageHandler - is an attribute of MessageManager, and every categorized message received from the event queue or udp dispatcher is filtered through here. (message as defined in message_template.msg, or one of ['ChatterBoxInvitation', 'ChatterBoxSessionEventReply', 'ChatterBoxSessionAgentListUpdates', 'ChatterBoxSessionStartReply', 'EstablishAgentCommunication']. There may be unhandled messages, I just haven't seen em yet :))
* EventsHandler - an attribute of an Agent, also able to be passed in, that is intended as the primary interface of a pyogp application into the internal state and data events within the lib.
Example:
<syntaxhighlight lang="python">    def send_ImprovedInstantMessage(self, AgentID = None, SessionID = None,
                                FromGroup = None, ToAgentID = None,
                                ParentEstateID = None, RegionID = None,
                                Position = None, Offline = None,  
                                Dialog = None, _ID = None, Timestamp = None,  
                                FromAgentName = None, _Message = None,  
                                BinaryBucket = None):
        """
        sends an instant message packet to ToAgentID. this is a  
        multi-purpose message for inventory offer handling, im, group chat,
        and more
        """


        packet = Message('ImprovedInstantMessage',
                        Block('AgentData',
                              AgentID = AgentID,
                              SessionID = SessionID),
                        Block('MessageBlock',
                              FromGroup = FromGroup,
                              ToAgentID = ToAgentID,
                              ParentEstateID = ParentEstateID,
                              RegionID = RegionID,
                              Position = Position,
                              Offline = Offline,
                              Dialog = Dialog,
                              ID = UUID(str(_ID)),
                              Timestamp = Timestamp,
                              FromAgentName = FromAgentName,
                              Message = _Message,
                              BinaryBucket = BinaryBucket))


=== Message Events ===
        # Send the message:
        self.region.enqueue_message(packet, True)</syntaxhighlight>


In the most fundamental implementation of event usage, all packets are passed through a MessageManager() instance for evaluation. Observers may register to receive udp packets serialized into the form of Message() instances. The MessageHandler() is a consolidation point for subscribing to messages keyed by message name, and created on demand via subscription.
=== Handling Incoming Messages and Raising an Event ===


See pyogp.lib.base.message.message_handler.MessageHandler() for more details.
To listen for when messages of a particular type are sent to the client instance, subscribe to the MessageHandler() on the host region's MessageManager(), like the example that follows:


The pyogp agent's Region() instances each monitor their stream of packets (e.g. the host region: agent.region.message_manager.message_handler). (Perhaps this should be changed to a generalized Network() class where all packets (coupled to their originating regions) are evaluated.
<syntaxhighlight lang="python">        onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
        onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)</syntaxhighlight>


Event firing passes data on to a callback handler defined in the subscription, in the form of (handler, *args, **kwargs).
When the event is fired upon receipt of the message matching the name, the specified callback handles the data passed along, and in this case raises an event notifying observers of the 'InstantMessageReceived' event in pyogp.lib.client of the important data.


The Agent class monitors the ImprovedInstantMessage packet:
<syntaxhighlight lang="python">   def onImprovedInstantMessage(self, packet):
<python>...
        """ callback handler for received ImprovedInstantMessage messages. much is passed in this message, and handling the data is only partially implemented """
onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)
...


def onImprovedInstantMessage(self, packet):
        Dialog = packet.blocks['MessageBlock'][0].get_variable('Dialog').data
    """ handles the many cases of data being passed in this message """
        FromAgentID = packet.blocks['AgentData'][0].get_variable('AgentID').data


    {code} # parse and handle the data...
        if Dialog == ImprovedIMDialogue.InventoryOffered:
</python>


The messaging system then fires the event when an ImprovedInstantMessage message is received which calls onImprovedInstantMessage method above to handle the message
            self.inventory.handle_inventory_offer(packet)


Unsubscribing from an event:
        # ...
<python>
        # some of the Dialogue types this message can contain are handled, we are showing 2
onImprovedInstantMessage_received.unsubscribe(self.onImprovedInstantMessage)
        # ...
</python>


=== Event System Events ===
        elif Dialog == ImprovedIMDialogue.FromAgent:


New! And not widely used yet...
            # ... code parses the data from the Message() instance ...


The application level event system is a separate implementation in pyogp.lib.base.event_system.EventsHandler(). This allows for a timeout to be specified for the subscription to a particular event.
            message = AppEvent('InstantMessageReceived', FromAgentID = FromAgentID, RegionID = RegionID, Position = Position, ID = ID, FromAgentName = FromAgentName, Message = _Message)


The api for subscribing to these events is similar to the MessageHandler() examples above, with an additional timeout parameter passed in the _register() method. When the specified timeout expires, the subscription returns None and expires the subscription.
            logger.info("Received instant message from %s: %s" % (FromAgentName, _Message))


The first use of this can be seen with the InstantMessageReceived() implementation as handled in Agent().onImprovedInstantMessage(), and usage of such in the sample script in the dev buildout bin/chat_and_instant_messaging.
            self.events_handler.handle(message)</syntaxhighlight>


== Logging ==
== Logging ==
Line 295: Line 394:
Hooking logging into a new module:
Hooking logging into a new module:


<python>from logging import getLogger
<syntaxhighlight lang="python">from logging import getLogger


# initialize logging
# initialize logging
logger = getLogger('pyogp.lib.base.agent')
logger = getLogger('pyogp.lib.client.agent')


class Agent(object):
class Agent(object):
Line 307: Line 406:
         self.params = params
         self.params = params


         logger.debug("Initializing agent with params: %s" % (params))</python>
         logger.debug("Initializing agent with params: %s" % (params))</syntaxhighlight>


An application can then set up the logging output as follows (or any other way it pleases):
An application can then set up the logging output as follows (or any other way it pleases):


<python>console = logging.StreamHandler()
<syntaxhighlight lang="python">console = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)-30s%(name)-30s: %(levelname)-8s %(message)s')
formatter = logging.Formatter('%(asctime)-30s%(name)-30s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
logging.getLogger('').addHandler(console)
logging.getLogger('').setLevel(logging.DEBUG)</python>
logging.getLogger('').setLevel(logging.DEBUG)</syntaxhighlight>


The output to console is then:
The output to console is then:


<bash>2009-04-21 22:08:58,681      pyogp.lib.base.agent          : DEBUG    agent with params: params</bash>
<code>2009-04-21 22:08:58,681      pyogp.lib.base.agent          : DEBUG    agent with params: params</code>


== Pyogp Unit Tests ==
== Pyogp Unit Tests ==
See [[PyOGP_Package_Unittests]].
See [[PyOGP_Package_Unittests]].
== Writing Test Cases ==
Tests can be written using standard unittest. The tests in pyogp.interop cover some ogp and a couple of legacy cases, these need to be updated to work.
Testing only the call to login.cgi is unique, we don't need to spawn the client in a coroutine, nor do we need to keep the client alive, we just need to post to the login endpoint and evaluate the response.
=== pyogp.interop.tests.test_legacy_login ===
<python>import unittest, doctest
import ConfigParser
from pkg_resources import resource_stream
import time
import uuid
import pprint
from pyogp.lib.base.agent import Agent
from pyogp.lib.base.datatypes import UUID
from pyogp.lib.base.exc import LoginError
from pyogp.lib.base.settings import Settings
import helpers
class AuthLegacyLoginTest(unittest.TestCase):
 
    def setUp(self):
       
        # initialize the config
        self.config = ConfigParser.ConfigParser()
        self.config.readfp(resource_stream(__name__, 'testconfig.cfg'))
               
        self.test_setup_config_name = 'test_interop_account'
       
        self.firstname = self.config.get(self.test_setup_config_name, 'firstname')
        self.lastname = self.config.get(self.test_setup_config_name, 'lastname')
        self.password = self.config.get(self.test_setup_config_name, 'password')
        self.agent_id = self.config.get(self.test_setup_config_name, 'agent_id')
        self.login_uri = self.config.get(self.test_setup_config_name, 'login_uri')
        self.region = self.config.get('test_interop_regions', 'start_region_uri')
        self.successful_login_reponse_params = ['last_name', 'sim_ip', 'inventory-lib-root',
          'start_location', 'inventory-lib-owner', 'udp_blacklist', 'home', 'message',
          'agent_access_max', 'first_name', 'agent_region_access', 'circuit_code', 'sim_port',
          'seconds_since_epoch', 'secure_session_id', 'look_at',  'ao_transition', 'agent_id',
          'inventory_host', 'region_y', 'region_x', 'seed_capability', 'agent_access', 'session_id',
          'login']
        self.settings = Settings()
        self.settings.MULTIPLE_SIM_CONNECTIONS = False
        self.client = Agent(self.settings, self.firstname, self.lastname, self.password)
    def tearDown(self):
       
        if self.client.connected:
            self.client.logout()
       
    def test_base_login(self):
        """ login with an account which should just work """
        self.client.settings.ENABLE_INVENTORY_MANAGEMENT = False
        self.client.login(loginuri = self.login_uri, start_location = self.region, connect_region = False)
        # Make sure that the login response attributes propagate properly,
        # Make sure the login against a grid has worked
        self.assertEquals(self.client.grid_type,
                          'Legacy',
                          'Storing the wrong grid type based on a \'legacy\' login request')
        self.assertEquals(self.client.firstname, self.firstname)
        self.assertEquals(self.client.lastname, self.lastname)
        self.assertEquals(self.client.lastname, self.lastname)
        self.assertEquals(self.client.name, self.firstname + ' ' + self.lastname)
        self.assertEquals(self.client.connected, True)
        self.assertNotEquals(self.client.agent_id, None)
        self.assertEquals(str(self.client.agent_id), self.agent_id)
        self.assertNotEquals(self.client.session_id, None)
        self.assertNotEquals(self.client.secure_session_id, None)
        self.assertEquals(self.client.login_response['last_name'], self.lastname)
        self.assertEquals(self.client.login_response['first_name'], '"' + self.firstname + '"')
        self.assertEquals(self.client.login_response['login'], 'true')
        self.assertEquals(self.client.login_response['secure_session_id'], str(self.client.secure_session_id))
        self.assertEquals(self.client.login_response['session_id'], str(self.client.session_id))
        self.assertEquals(self.client.login_response['agent_id'], str(self.client.agent_id))
        self.assertNotEquals(self.client.login_response['seed_capability'], '')
        fail = 0
        fail_extra = 0
        fail_missing = 0
        extra_keys = ''
        missing_keys = ''
       
        for key in self.client.login_response:
            try:
                self.successful_login_reponse_params.index(key) # if the key is in our valid list, sweet
            except:
                fail_extra = 1
                extra_keys = extra_keys + ' ' + key
        for key in self.client.login_response:
            try:
                self.successful_login_reponse_params.index(key) # if the key is in our valid list, sweet
            except:
                fail_missing = 1
                missing_keys = missing_keys + ' ' + key
     
        self.assertEquals(fail_extra, 0, 'login response has additional keys: ' + extra_keys)
        self.assertEquals(fail_missing, 0, 'login response is missing keys: ' + missing_keys)
    def test_login_with_bad_password(self):
        self.client.settings.ENABLE_INVENTORY_MANAGEMENT = False
        self.assertRaises(LoginError,
                          self.client.login,
                          loginuri = self.login_uri,
                          password = 'BadPassword',
                          start_location = self.region,
                          connect_region = False)
def test_suite():
    from unittest import TestSuite, makeSuite
    suite = TestSuite()
    suite.addTest(makeSuite(AuthLegacyLoginTest))
    return suite</python>
== Adding Functionality ==
=== Wrapping Packets ===
PyOGP, like the Viewer, communicates with the Second Life simulator by sending messages over UDP.
In order to extend PyOGP, you'll enable a new UDP message.
=== Example: Sending an IM ===
To send an IM to the simulator, send an ImprovedInstantMessage packet.  The base class for message packets is defined in <code>base/message/message.py</code>
Packets are assembled using a Message() instance which has the message name and Block() instances passed in through its constructor.  Similarly, Blocks are assembled by passing in the Block name and the value name and values for each of the Block values.  Note: It is important that the message name, block name, and value names and types should match what is specified in the message template.
Example:
<python>
def send_ImprovedInstantMessage(self, AgentID = None, SessionID = None,
                                FromGroup = None, ToAgentID = None,
                                ParentEstateID = None, RegionID = None,
                                Position = None, Offline = None,
                                Dialog = None, _ID = None, Timestamp = None,
                                FromAgentName = None, _Message = None,
                                BinaryBucket = None):
        """
        sends an instant message packet to ToAgentID. this is a
        multi-purpose message for inventory offer handling, im, group chat,
        and more
        """</python>
Assemble the message:
<python>
        packet = Message('ImprovedInstantMessage',
                        Block('AgentData',
                              AgentID = AgentID,
                              SessionID = SessionID),
                        Block('MessageBlock',
                              FromGroup = FromGroup,
                              ToAgentID = ToAgentID,
                              ParentEstateID = ParentEstateID,
                              RegionID = RegionID,
                              Position = Position,
                              Offline = Offline,
                              Dialog = Dialog,
                              ID = UUID(str(_ID)),
                              Timestamp = Timestamp,
                              FromAgentName = FromAgentName,
                              Message = _Message,
                              BinaryBucket = BinaryBucket))</python>
Send the message:
<python>
        self.region.enqueue_message(packet, True)</python>


== Sphinx (api docs) ==
== Sphinx (api docs) ==


Api documentation is now available for pyogp packages (well, not for apps yet, but someday....)!
'''Api documentation is now available for PyOGP packages''' (well, not for apps yet, but someday....)!
   
   
In pyogp/docs in each of the pyogp.lib.base and pyogp.lib.client packages, one will find source, last revision, and build files for sphinx based documents.
In pyogp/docs in each of the pyogp.lib.base and pyogp.lib.client packages, one will find source, last revision, and build files for sphinx based documents. Output is available at {package root}/docs/html/index.html.


Sample output is available at {package root}/docs/html/index.html.
We plan on sharing the api documentation on the web soon, and will work to make simple build wrappers work on various platforms, though this is a low priority.


README.txt contains build instructions, as follows:
Ask Enus for updated documentation to be checked in, or, build a better refresh.py.


<pre>
== Roadmap ==
This checkout contains the most recently complied version of the documentation in docs/html/.
 
To rebuild the sphinx doc set:
 
Get sphinx!!!
 
Either use your virtualenv, or your native python install and run:
    easy_install -U Sphinx
 
Then, from the docs dir:


1. python refresh.py
refresh.py stages the sphinx .rst files, and then runs 'sphinx-build -a -c source/configure/' source/ html/
The docs/html/ directory will contain the fully compiled documentation set.
Please check in updated docs if you add functionality.
</pre>
== Roadmap ==


There is so much yet to implement that it is frightening. Here's what's up in the near term for pyogp:
See [[Pyogp/Roadmap]] for details, or ask in irc or on the mailing list. Here's what's up in the near term for pyogp:


* enabling '''parcel testing'''
* better '''packaging''' and '''platform compatibility'''
* '''teleport''' (in OGP and in the 'legacy' context)
* more '''functional coverage''' of message. We are covering 32% of the messages the viewers sends or handles when received (though it's estimated at more like 50% of normal use cases.
** teleport works in OGP, but is not encapsulated in a method yet
* '''permissions system testing''' to save QA from 3-5 day regression passes on perms
* '''permissions system testing'''to save QA from 3-5 day regression passes on perms
* '''appearance''' - this will require enabling upload and download, plus baking. Anyone have some spare time? :)
* '''appearance''' - this will require enabling upload and download, plus baking. Anyone have some spare time? :)



Latest revision as of 08:31, 27 June 2017

Intro

PyOGP is a young, open source, python client library which provides an interface to an agent (client) of a Second Life grid. The code aims to enable automated testing of Second Life grids, to encourage and enable exploration into the client/server relationship and related protocols of Second Life, and to make possible simple prototyping of various client applications.

Hosted on svn.secondlife.com, it does require a contributor's agreement for commit access, and currently has a few contributors from the Second Life open source community.

P.S. We'll likely be moving to hg relatively soon...

Goals

In the very near future, we can have tests available to be run as soon as a deploy is completed that exercise a simulator/grid in the same way we do a smoke test. We will use these as automated tests run at build time, post deploy validation, and regression testing of simulators and backend systems.

This provides early feedback on code quality. QA is then able to dive deeper in testing the changes specific to a branch.

Having this library available also allows us to test potential changes before we have finalized design and are ready to submit to QA. Not sure how something will play out? Try it, and test it with PyOGP....

A Brief History

PyOGP was originally created as a tool for testing OGP related changes to the Second Life grid. By the end of the summer in 2008, the pyogp.lib.base package provided a skeleton messaging system and was able to test Linden Lab's first implementation of an agent domain and the related changes in an agent's region handoff and teleport. As the development effort around OGP waned, we started to extend pyogp by adding higher level client related functionality. Recently, this functionality was split out into a separate python package (pyogp.lib.client), and sample scripts (and future apps) were moved into pyogp.apps.

Architecture Overview

Structure

PyOGP is comprised of three python packages. The library consists of pyogp.lib.base and pyogp.lib.client, while sample scripts, and in time applications, live in pyogp.apps.

pyogp.lib.base - consists of basic networking, messaging systems (UDP and event queue) and capabilities, custom datatypes, and a low level message related event system
pyogp.lib.client - consists of 'convenience' classes mapping methods and handlers to specific messages (e.g. Agent().say(msg) tells the client to send the ChatFromViewer message to the host region). Raises application oriented events based on processing of messages (these are currently sparsely implemented)
pyogp.apps - sample scripts and works in progress, the scripts here generally illustrate simple usage of classes as related to in world interactions by an agent of a Second Life grid

Dependencies

Platform / Python version compatibility

PyOGP aims to be compatible across platforms, though there are known problems with various environments. We'll be focusing on ensuring better compatibility soon.

Known good configurations

Windows XP + Python 2.5
Mac + Python 2.5, 2.6
Linux + Python 2.4, 2.5, 2.6 (Linden hosts fall into this group)

Known bad configurations

Windows Vista + Python 2.6
Windows 7 + Python 2.6

There have been challenges in ensuring compatibility between the various dependencies, largely due to greenlet, eventlet, and pyopenssl. Please report bugs on pJira

Python module dependencies

The packages that make up PyOGP have some dependencies on python modules not included in a standard install, or sometimes not available on an older Python distribution.

pyogp.lib.base dependencies:

  #from setup.py
  
     install_requires=[
         'setuptools',
         # -*- Extra requirements: -*-
         'uuid',
         'elementtree',
         'llbase',
         'WebOb',
         'wsgiref',
         'eventlet==0.8.14',
         'pyOpenssl'
         ]

pyogp.lib.client dependencies:

     install_requires=[
         'setuptools',
         # -*- Extra requirements: -*-
         'pyogp.lib.base'
         ]

pyogp.apps dependencies:

     install_requires=[
         'setuptools',
         'pyogp.lib.client'
         ]

How to install

Lindens can see internal documentation for more specific guidance. https://wiki.lindenlab.com/wiki/Pyogp#How_to_Install

Standalone dev environment using buildout

Buildout is a type of Python development environment, organizing and configuring various components and their dependencies. On a desktop, one may checkout such an environment for working with PyOGP. One may optionally use a virtualenv Python environment to isolate the development code and it's runtime environment on one's host.

Dependencies: buildout takes care of everything, grabbing needed modules etc.

Wiki instructions: https://wiki.secondlife.com/wiki/Pyogp/Client_Lib/The_Development_Sandbox

Installing the PyOGP packages

Each of the PyOGP package may be installed to one's Python install or to a virtualenv. Buyer beware if installing into your system's install: you'll want to be able to uninstall manually, as we haven't hooked up the uninstall. PyOGP is still coming up to speed with respect to distutils and pypi and the like, but it's relatively close now.

To install a package, simply run 'python setup.py install' in a package's root.

Referencing PyOGP packages via the PATH

Source code can be referenced directly if one simply ensures that a package, and it's dependencies, are available in the PYTHONPATH environmental variable.

Current functional coverage

Anything not listed as covered is probably not yet covered.

pyogp.lib.base:

  • base udp messaging system (message.*)
    • UDP serialization/deserialization
    • message_template.msg parsing
  • base event queue messaging system (event_queue.EventQueueClient())
  • capabilities and their methods. Seed capabilities are a special case. (caps.Capability())
  • Message-based events (message.message_handler)

pyogp.lib.client:

  • agents (agent.Agent())
    • L$ balance request, friending, and walk/fly/sit/stand actions...
  • OGP agentdomain (agentdomain.AgentDomain())
  • application level events (event_system.AppEventsHandler())
  • some object handling (objects.*)
    • edit name, description, next-owner permissions and more
    • object creation is possible
  • some inventory handling (inventory.*)
    • login inv skeletons
    • fetching inventory, including AIS (caps based Agent Inventory Services))
    • some creating of new inventory items (LSL scripts, notecards)
  • regions (region.Region())
    • host and neighboring regions are handled slightly differently
    • udp and event queue connections are optionally enabled for each case
    • currently only pulling caps available to the agent via the seed cap (plus using the inventory related caps in the AIS context)
  • some appearance handling (appearance.AppearanceManager),
  • parcels
  • chat
  • some ImprovedInstantMessage handling
    • raises events on received instant messages
    • can deal with inventory offers/accepts/declines
    • other cases in this message are currently only logged
  • groups
  • group chat
  • LSL script uploading

Sample Scripts

There are a variety of scripted examples that have been used to exercise and test functionality as it is added to the library. These persist as coded documentation.

The source code is available in https://svn.secondlife.com/svn/linden/projects/2008/pyogp/pyogp.apps/trunk/pyogp/apps/examples/.

The following refers to a buildout context. If one installs pyogp.apps vi setup.py, these scripts will exist in the python environment's bin/ directory. In the buildout context, these scripts are available in {buildout root}/bin.

The scripts are derived from a package's 'setup.py' via the entry_points parameter, and essentially build executable Python scripts configured to run in the correct environment with the proper dependencies added the the path used by the script. These scripts are currently just simple illustrations of some uses of the PyOGP codebase.

  #the current entry_points in setup.py og pyogp.apps:
  
     entry_points={
         'console_scripts': [
             'AIS_inventory_handling = pyogp.apps.examples.AIS_inventory_handling:main',
             'agent_login = pyogp.apps.examples.agent_login:main',
             'agent_manager = pyogp.apps.examples.agent_manager:main',
             'appearance_management = pyogp.apps.examples.appearance_management:main',
             'chat_and_instant_messaging = pyogp.apps.examples.chat_and_instant_messaging:main',
             'group_chat = pyogp.apps.examples.group_chat:main',
             'group_creation = pyogp.apps.examples.group_creation:main',
             'inventory_handling = pyogp.apps.examples.inventory_handling:main',
             'inventory_transfer = pyogp.apps.examples.inventory_transfer:main',
             'inventory_transfer_specify_agent = pyogp.apps.examples.inventory_transfer_specify_agent:main',
             'login = pyogp.apps.examples.login:main',
             'multi_region_connect = pyogp.apps.examples.multi_region_connect:main',
             'object_create_edit = pyogp.apps.examples.object_create_edit:main',
             'object_create_permissions = pyogp.apps.examples.object_create_permissions:main',
             'object_create_rez_script = pyogp.apps.examples.object_create_rez_script:main',
             'object_creation = pyogp.apps.examples.object_creation:main',
             'object_properties = pyogp.apps.examples.object_properties:main',
             'object_tracking = pyogp.apps.examples.object_tracking:main',
             'parcel_management = pyogp.apps.examples.parcel_management:main',
             'parse_packets = pyogp.apps.examples.parse_packets:main',
             'region_connect = pyogp.apps.examples.region_connect:main',
             'smoke_test = pyogp.apps.examples.smoke_test:main',
             'chat = pyogp.apps.examples.chat_interface:main',
             ]
        }

How it Works (High Level)

Eventlet

PyOGP use Eventlet to run coroutines to handle multiple 'concurrent' processes, rather than threads or multiple processes. Each client agent instance will spawn a handful of coroutines to handles e.g. the UDP pipe, the Event Queue, various monitors, while yielding time to the parent process which should ensure it yields to the other routines as well.

PyOGP uses eventlet in very elementary ways at this point, but will perhaps start to use blocking queues in some cases, so that the coroutine only is allocated processing time if there is work for it to do.

pyogp.lib.base

This package handles the protocols used when communicating with a Second Life grid. A high level perspective on the package reveals a MessageManager() (still in development) which provides an interface to the UDP and Event Queue connections, as well as basic networking with enables login and capabilities interactions. The base package also has a low level even system through which all messages are passed and sent to subscribers.

Any subcomponent is available for direct interaction at any time, the MessageManager() and the MessageHandler() are the simple access points.

Events & Callbacks

The event implementation in pyogp follows the observer pattern, where observers subscribe to and are notified when an event occurs. Data is passed throughout the client instance via events.

  • MessageManager - is an attribute of a Region and every packet received/sent is filtered through here. subscriptions are by message name
    • MessageHandler - is an attribute of MessageManager, and every categorized message received from the event queue or udp dispatcher is filtered through here. (message as defined in message_template.msg, or one of ['ChatterBoxInvitation', 'ChatterBoxSessionEventReply', 'ChatterBoxSessionAgentListUpdates', 'ChatterBoxSessionStartReply', 'EstablishAgentCommunication']. There may be unhandled messages, I just haven't seen em yet :))
Message Events

In the most fundamental implementation of event usage, all packets are passed through a MessageManager() instance for evaluation. Observers may register to receive udp packets serialized into the form of Message() instances. The MessageHandler() is a consolidation point for subscribing to messages keyed by message name, and created on demand via subscription.

See pyogp.lib.base.message.message_handler.MessageHandler() for more details.

The pyogp agent's Region() instances each monitor their stream of packets (e.g. the host region: agent.region.message_manager.message_handler). (Perhaps this should be changed to a generalized Network() class where all packets (coupled to their originating regions) are evaluated.

Event firing passes data on to a callback handler defined in the subscription, in the form of (handler, *args, **kwargs).

The Agent class monitors the ImprovedInstantMessage packet:

        onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
        onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)

    def onImprovedInstantMessage(self, packet):
        """ handles the many cases of data being passed in this message """

        {code} # parse and handle the data...

The messaging system then fires the event when an ImprovedInstantMessage message is received, which calls onImprovedInstantMessage method above to handle the message contents. Multiple subscribers may be listening for any message related event, and each would be notified of the same Message() instance.

Unsubscribing from an event:

        onImprovedInstantMessage_received.unsubscribe(self.onImprovedInstantMessage)

There are various event and callback implementations viewable in pyogp, poke around and help consolidate things if you like.

pyogp.lib.client

The client package generally provides a convenient interface to initiate or interpret interactions with host region (or neighboring regions). By listening to the messaging related event system in pyogp.lib.base, the client package interprets the messages that come in off the wire, and executes business logic in building responses. pyogp.lib.client also provides simple methods to enable the ending of messages to a grid.

Events

  • EventsHandler - an attribute of an Agent, also able to be passed in, that is intended as the primary interface of a pyogp application into the internal state and data events within the lib. This system uses the same base classes as used by the MessageHandler() in the base package, and the descriptions about events and callbacks above apply here as well. The api for subscribing to these events is similar to the MessageHandler(), with an additional timeout parameter passed in the _register() method. When the specified timeout expires, the subscription returns None and expires the subscription.

Agent Login (examples)

Single Agent Login & Chat

Spawn a client in a co-routine, allowing persistent presence until forcefully terminated.

from eventlet import api

from pyogp.lib.client.agent import Agent
from pyogp.lib.client.settings import Settings

settings = Settings()

settings.ENABLE_INVENTORY_MANAGEMENT = True
settings.MULTIPLE_SIM_CONNECTIONS = False

client = Agent(settings = settings)

api.spawn(client.login, options.loginuri, 'first', 'last', 'password', start_location = options.region)

# wait for the agent to connect to it's region
while client.connected == False:
    api.sleep(0)

while client.region.connected == False:
    api.sleep(0)

client.say("Hello World!")

# once connected, live until someone kills me
while client.running:
    api.sleep(0)
Multiple Agent Login

Each agent instance in logged in in a separate coroutine.

from pyogp.lib.client.agent import Agent
from pyogp.lib.client.agentmanager import AgentManager

credentials= [('agent1', 'lastname', 'password'), ('agent2', 'lastname', 'password')]

# prime the Agent instances
agents = [Agent(settings, firstname, lastname, password)
          for firstname, lastname, password in credentials]

agentmanager = AgentManager()
agentmanager.initialize(agents)

# log them in
for key in agentmanager.agents:
    agentmanager.login(key, options.loginuri, options.region)

# while they are connected, stay alive
while agentmanager.has_agents_running():
    api.sleep(0)

Extending Functionality

While the implementations and structures in pyogp.lib.base can (and are in the process of) being refactored to improve performance or usability, it is a fairly complete package.

The functional coverage PyOGP provides on the other hand is not complete, and there are a variety of needs to complete the implementation in pyogp.lib.base. We need to improve coverage of message handling (dealing with messages sent to the client), add more wrappers for sending various messages and performing multistep tasks (to simplify the initiation of interactions with the region), and we need to raise more application level events in the client package so that applications have easy access to incoming data.

Sending Messages

PyOGP, like the Viewer, communicates with the Second Life simulator by sending messages over UDP.

In order to extend PyOGP, you'll build a representation of a new UDP message, and send it through the pyogp.lib.base modules for serialization and wire handling.

Example: Sending an IM

To send an IM to the simulator, send an ImprovedInstantMessage packet. The base class for message packets is defined in base/message/message.py

Packets are assembled using a Message() instance which has the message name and Block() instances passed in through its constructor. Similarly, Blocks are assembled by passing in the Block name and the value name and values for each of the Block values. (The ability to build Message() instances via an llsd payload is expected to be introduced soon.)

Note: It is important that the message name, block name, and value names and types should match what is specified in the message template. (It is also possible to manipulate the representation of the stored template, or to use a custom message template.)

Example:

    def send_ImprovedInstantMessage(self, AgentID = None, SessionID = None, 
                                FromGroup = None, ToAgentID = None, 
                                ParentEstateID = None, RegionID = None, 
                                Position = None, Offline = None, 
                                Dialog = None, _ID = None, Timestamp = None, 
                                FromAgentName = None, _Message = None, 
                                BinaryBucket = None):
        """ 
        sends an instant message packet to ToAgentID. this is a 
        multi-purpose message for inventory offer handling, im, group chat, 
        and more 
        """

        packet = Message('ImprovedInstantMessage', 
                         Block('AgentData', 
                               AgentID = AgentID, 
                               SessionID = SessionID), 
                         Block('MessageBlock', 
                               FromGroup = FromGroup, 
                               ToAgentID = ToAgentID, 
                               ParentEstateID = ParentEstateID, 
                               RegionID = RegionID, 
                               Position = Position, 
                               Offline = Offline, 
                               Dialog = Dialog, 
                               ID = UUID(str(_ID)), 
                               Timestamp = Timestamp, 
                               FromAgentName = FromAgentName, 
                               Message = _Message, 
                               BinaryBucket = BinaryBucket))

        # Send the message:
        self.region.enqueue_message(packet, True)

Handling Incoming Messages and Raising an Event

To listen for when messages of a particular type are sent to the client instance, subscribe to the MessageHandler() on the host region's MessageManager(), like the example that follows:

        onImprovedInstantMessage_received = self.region.message_handler.register('ImprovedInstantMessage')
        onImprovedInstantMessage_received.subscribe(self.onImprovedInstantMessage)

When the event is fired upon receipt of the message matching the name, the specified callback handles the data passed along, and in this case raises an event notifying observers of the 'InstantMessageReceived' event in pyogp.lib.client of the important data.

    def onImprovedInstantMessage(self, packet):
        """ callback handler for received ImprovedInstantMessage messages. much is passed in this message, and handling the data is only partially implemented """

        Dialog = packet.blocks['MessageBlock'][0].get_variable('Dialog').data
        FromAgentID = packet.blocks['AgentData'][0].get_variable('AgentID').data

        if Dialog == ImprovedIMDialogue.InventoryOffered:

            self.inventory.handle_inventory_offer(packet)

        # ...
        # some of the Dialogue types this message can contain are handled, we are showing 2
        # ...

        elif Dialog == ImprovedIMDialogue.FromAgent:

            # ... code parses the data from the Message() instance ...

            message = AppEvent('InstantMessageReceived', FromAgentID = FromAgentID, RegionID = RegionID, Position = Position, ID = ID, FromAgentName = FromAgentName, Message = _Message)

            logger.info("Received instant message from %s: %s" % (FromAgentName, _Message))

            self.events_handler.handle(message)

Logging

Uses python's standard logging module (http://docs.python.org/library/logging.html). The library defines logging events throughout, it is up to the application/script to determine the output.

Hooking logging into a new module:

from logging import getLogger

# initialize logging
logger = getLogger('pyogp.lib.client.agent')

class Agent(object):
    """ our agent class """

    def __init__(self, params):

        self.params = params

        logger.debug("Initializing agent with params: %s" % (params))

An application can then set up the logging output as follows (or any other way it pleases):

console = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)-30s%(name)-30s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
logging.getLogger('').setLevel(logging.DEBUG)

The output to console is then:

2009-04-21 22:08:58,681 pyogp.lib.base.agent  : DEBUG agent with params: params

Pyogp Unit Tests

See PyOGP_Package_Unittests.

Sphinx (api docs)

Api documentation is now available for PyOGP packages (well, not for apps yet, but someday....)!

In pyogp/docs in each of the pyogp.lib.base and pyogp.lib.client packages, one will find source, last revision, and build files for sphinx based documents. Output is available at {package root}/docs/html/index.html.

We plan on sharing the api documentation on the web soon, and will work to make simple build wrappers work on various platforms, though this is a low priority.

Ask Enus for updated documentation to be checked in, or, build a better refresh.py.

Roadmap

See Pyogp/Roadmap for details, or ask in irc or on the mailing list. Here's what's up in the near term for pyogp:

  • better packaging and platform compatibility
  • more functional coverage of message. We are covering 32% of the messages the viewers sends or handles when received (though it's estimated at more like 50% of normal use cases.
  • permissions system testing to save QA from 3-5 day regression passes on perms
  • appearance - this will require enabling upload and download, plus baking. Anyone have some spare time? :)