Pyogp/Documentation/Testing with Pyogp

From Second Life Wiki

(Redirected from Pyogp/Test Framework)
Jump to: navigation, search

This page is dedicated to the client library aspect of the Pyogp project. The more general page is Pyogp.

Contents

Overview

AWG (in-world discussion group page) and Linden Lab plan to share the effort in writing a test harness which executes tests against implementations of the Open Grid Protocol (OGP). The intent is not to test SL or OpenSim (or gridX) themselves. Rather, the desire is to create a framework where the implementations of the protocols can be tested in any virtual environment implemented per the Open Grid Protocols as documented. We plan to be able to test at the smallest functional level possible, akin to unit testing, and to have the ability to use the framework to sequence steps to enable state specific tests to be run. Daily coders' meetings are held to discuss design and implementation issues. There is an irc channel irc://irc.freenode.com/#pyogp and a mailing list as well.

Objectives

Owing to PyOGP's public status, Linden Lab is planning on using it for more than simple unit tests. Our enhanced objectives for the PyOGP test framework include:

Traditional Unit Testing 
(Some maintain) modern software engineering best practice encourages the production of extensive unit tests prior to the creation of code that implements a feature. Called Test-Driven Development, this mode of development is intended to (among other things) ferret out bugs in requirements before "coding yourself into a corner."
Demonstrating Proper Use of the SLGOGP Interface 
No matter how good your documents are, there always seem to be inconsistencies and omissions. The test cases provided in the PyOGP test framework are also intended to communicate proper use patterns of both the PyOGP library and the SLGOGP interface. In short, if you're a developer and you want to know how to use PyOGP, you should be able to look at the test cases.
Communicating Feature Enhancements and Demonstrating Bugs 
A "stock library" of test cases comes in handy when trying to demonstrate bugs between organizations. Not only does enhancing existing test cases to demonstrate a behavioral deficiency have the advantage of succinctly describing the problem to a technical community; you can then use the test case to see if a remediation is successful.

Thoughts and Concerns

  1. The test framework will not only need to test the protocols and implementations, but also the client library.
  2. The client library will probably use some basic OO principles, being encapsulated and such. This might make testing more difficult if we don't design the client library to be easily testable. For instance, if we have a single Login class that performs all the necessary actions (posting to all needed caps, setting up all sockets) then there must be a way to test each of the actions individually. There are some patterns to do this, such as using a collaborator object, but we have to consider which way we want to implement.
    • Sequences of steps - This might simply be the case of having test sequences, where one test is made up of many smaller individual tests that must pass in sequence, each able to be tested individually. Even still, we either have to use good design principles for the client library, or have everything coded in small non-encapsulated pieces.
  3. State of Components - due to the fact that we are testing network protocols and communication, we have to make sure that each component (agent domain, region domain, simulator, login server) is in the state that we expect. This is something we don't have control over, because they may be non-Linden grids, and so there must be some way to make sure our tests don't fail just because we did things out of order and the state wasn't set how we expected it to be. However, we can test the client library for correctness by having mock objects that make sure the state is how it should be.
    • Test Layers - We might be able use the unit test framework and encapsulate it in some command line tool which then can call the individual suites. A way to pass configuration should also not be hard to be find. In Zope we also use some feature called test layers where every layer can setup some test environment. Some can then be bigger and more complete (but take longer to initialize), some might be more lightweight which is maybe enough to call lowlevel things which do not need so much setup. Not sure we need that here though as it's probably more of a networking based test library anyway.
  4. The tests won't only be interacting with Linden Lab's grids, and there shouldn't be hard-coded agent accounts in the tests, so we need some way of configuring tests so that you can give URLs, usernames/passwords, etc. to the test so that they know what to test. This can be maybe passed by via some .ini or XML-file.
  5. We might want to have separate test suites for separate parts like agent domain/region domain and also smaller modules (if some component is known to not yet implement e.g. inventory then there is no need to test it).
  6. We might need some way to setup certain testing scenarios. If the inventory is empty we probably cannot retrieve something so we need to make sure it's not empty before we test some GET (as an example).
  7. Would like to be able to use this code base to test between an agentd and simulator, without having to login. So, the ability to test not only from a client's point of view, but from an agent d's POV,sim's POV, and from a region's point of view.

Project Status

  • June 26, 2008 - Tess and Leyla have written some tests for the OGP protocol. At this point, they are untestable because the framework isn't sound enough.

Project Tasks

  1. Get the requirements specification
    • May include use-cases
  2. Design the test harness
    • Test framework - unittest, doctest
    • Test data (I put this here, not really as a component of the harness, more as a "i'm not telling you my test account's password and what are we gonna do about it" thing)
    • Determine how to handle test data. I imagine we'll build a template and interface to an implementation of the template, and leave contents up to the developer.
  3. Implement the test harness
    • Distribute the work across the contributors


Links

Please add to the list as you see fit. The more we throw in here, the better off we are in the long run.

Project pages

Licensing

  1. The code written as part of this effort is subject to the Apache v2 license. Read more at http://opensource.org/licenses/apache2.0.php. 
 <excerpt>
 Copyright 2008, Linden Research, Inc.
 

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 

 http://www.apache.org/licenses/LICENSE-2.0
 

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 </excerpt>
Retrieved from "http://wiki.secondlife.com/wiki/Pyogp/Documentation/Testing_with_Pyogp"
Personal tools