Difference between revisions of "SL Cert Guide to Certification Testing"
| Line 51: | Line 51: | ||
* An up-front cost for the development of the tests is paid by the service provider (Linden Lab). This cost reflects the development of the tests and sometimes includes update changes based on a pre-arranged time-scale. | * An up-front cost for the development of the tests is paid by the service provider (Linden Lab). This cost reflects the development of the tests and sometimes includes update changes based on a pre-arranged time-scale. | ||
* There is an administration charge for running each test to the service provider (LL). Most companies pass this charge on to the candidate. What Linden Lab plans to do for | * There is an administration charge for running each test to the service provider (LL). Most companies pass this charge on to the candidate. What Linden Lab plans to do for this is yet to be decided. | ||
* Failure of a test sometimes means that a re-test can be done at a lower rate. | * Failure of a test sometimes means that a re-test can be done at a lower rate. | ||
I'll update this document as new questions arise. | I'll update this document as new questions arise. | ||
Revision as of 12:00, 8 April 2009
3rd Party Certification Guide
Many people have been asking questions about how 3rd party certification companies operate; which are fair questions if you've never dealt with these guys before, as they're not as straight forward as people may at first think. So, here is an overview of the general concepts. Bear in mind that different suppliers vary in their approach, but the general principles here are fairly consistent.
Overview of Certification
Certification is not directly linked to a training program, and neither should it be. Certification is intended to test a person's competence. If a person has a huge knowledge of a topic, they should be able to sit a test without ever having to attend a designated training course. So why involve the schools with certification? Simply because the number of people in the catagory described above will be few and far between. The schools will be an excellent resource for helping people to enhance their knowledge to the point of being able to certify. However, courses should not be written to the tests. There will likely be a large uptake of people requiring training, but the courses should teach the subjects generally, and not be geared to helping people just "pass a test". Providing a list of approved schools to residents hoping to certify will be essential.
How much do 3rd Party Testers know about the topics being tested for Second Life?
Nothing. And that's exactly how it should be. They are not subject matter experts in any of the technologies they provide testing for; for any organisation. Their expertise is:
- Writing tests that comply to prometric standards
- Managing the candidates' applications
- Security with regard to making sure that the person sitting the test is really the person sitting the test
- Global availability. Making sure that the test system is available as widely as possible, taking into consideration geographic locations, language and accessibility issues for those with disabilities.
- Reporting and managing on-going certification requirements over time.
Why can't we run the final tests ourselves?
- If anyone who is affiliated with Second Life - and that includes educators, solution providers, residents, or even Linden Lab itself - administers the tests then questions WILL be asked by people applying to be certified about objectivity. The use of a 3rd party not affiliated with Second Life will help these "impartiality" issues disappear.
- Pro-metric testing is very involved process. In multiple choice questions, how many correct answers are "b"? How long are the correct answers compared to the "decoy" answers? Does the language used in the questions lead people to a certain answer? These are just a few of the issues that the testing agency will have experience in dealing with.
- You will be writing the questions, and expect to write many and have them sent back for rephrasing. For an average 25 question test, expect them to need at least 100 questions to be randomly selected. (I was involved with running the certification process for a company I worked for a few years ago, and the testing agency expected 400 questions to be written for a 50 question test; although this was for a specific purpose, it highlights the point)
- Multi-languge support, although subject matter experts (SME's = you) in world will need to check the phrasing in the local language to ensure that the context of the question hasn't been lost in the translation.
How does the scoring work? Why don't the candidates just "guess" the answers?
Scoring normally applies one point for each correct answer, zero points for a blank answer and minus 0.25 for a wrong answer. This basically eliminates the possibility of someone just choosing option "a" all the way down and "hoping for the best". Usually a review period is given at the end of the test to go back and check answers the person feels they may have got wrong. There is a time limit for each test, which is based on "trial run" tests.
What types of tests can they produce?
This varies depending on provider, but the computer based testing usually follows these formats:
- Single choice answers
- Multiple choice with a point for all correct answers selected
- Multiple choice with a point for each correct answer selected
- Free text answers, using key word inclusion for scoring
- Location mapping, so that a click can indicate an area of the screen. For example, "Click on the incorrect statment in the script in front of you"
Practical testing can also be carried out, but to be honest at this point I haven't the first clue about how these will be done in Second Life. Selecting the correct testing supplier will rely on discussions around such issues.
How much does it cost?
How long is a piece of elestic string? Although, charging usually happens like this:
- An up-front cost for the development of the tests is paid by the service provider (Linden Lab). This cost reflects the development of the tests and sometimes includes update changes based on a pre-arranged time-scale.
- There is an administration charge for running each test to the service provider (LL). Most companies pass this charge on to the candidate. What Linden Lab plans to do for this is yet to be decided.
- Failure of a test sometimes means that a re-test can be done at a lower rate.
I'll update this document as new questions arise.