Difference between revisions of "SL Cert Guide to Certification Testing"

From Second Life Wiki
Jump to navigation Jump to search
m (Configured stable versioning for SL Cert Guide to Certification Testing: [Precedence: Quality, Default: Stable])
(Rewrite to take account of inworld examination.)
Line 1: Line 1:
{{multi-lang}}
{{SLCert Navbox}}
{{SLCert Navbox}}
[[Category:Certification]]
[[Category:Certification]]


=3rd Party Certification Guide=
= Overview of Certification =


Many people have been asking questions about how 3rd party certification companies operate; which are fair questions if you've never dealt with these guys before, as they're not as straight forward as people may at first think. So, here is an overview of the general concepts. Bear in mind that different suppliers vary in their approach, but the general principles here are fairly consistent.
Certification is not directly linked to a training program, and neither should it be. Certification is intended to test a person's competence. If a person has extensive knowledge of a topic they should be able to sit a test without ever having to attend a designated training course.


== Overview of Certification ==
So why involve the schools with certification? Simply because the number of people in the catagory described above will be few and far between. The schools will be an excellent resource for helping people enhance their knowledge to where they can pass an exam. However, courses should not be written to the tests. There will likely be a large uptake of people requiring training, but the courses should teach the subjects generally, and not be geared to helping people just "pass a test." Providing a list of approved schools to residents hoping to certify will be essential.


Certification is not directly linked to a training program, and neither should it be. Certification is intended to test a person's competence. If a person has a huge knowledge of a topic, they should be able to sit a test without ever having to attend a designated training course.
= What issues will need to be dealt with in creating exams? =
So why involve the schools with certification? Simply because the number of people in the catagory described above will be few and far between. The schools will be an excellent resource for helping people to enhance their knowledge to the point of being able to certify. However, courses should not be written to the tests. There will likely be a large uptake of people requiring training, but the courses should teach the subjects generally, and not be geared to helping people just "pass a test". Providing a list of approved schools to residents hoping to certify will be essential.
 
== How much do 3rd Party Testers know about the topics being tested for Second Life? ==
 
Nothing. And that's exactly how it should be. They are not subject matter experts in any of the technologies they provide testing for; for any organisation. Their expertise is:


* Writing tests that comply to prometric standards
* Writing tests that comply to prometric standards
* Managing the candidates' applications
* Managing the candidates' applications
* Security with regard to making sure that the person sitting the test is really the person sitting the test
* Global availability. Making sure that the test system is available as widely as possible, taking into consideration geographic locations, language and accessibility issues for those with disabilities.
* Global availability. Making sure that the test system is available as widely as possible, taking into consideration geographic locations, language and accessibility issues for those with disabilities.
* Reporting and managing on-going certification requirements over time.
* Reporting and managing on-going certification requirements over time.
* Use of pro-metric testing to create the exams.
* Multi-languge support, although subject matter experts (SME's = you) in world will need to check the phrasing in the local language to ensure that the context of the question hasn't been lost in the translation.


== Why can't we run the final tests ourselves? ==
= Who will proctor the exams? =  


* If anyone who is affiliated with Second Life - and that includes educators, solution providers, residents, or even Linden Lab itself - administers the tests then questions WILL be asked by people applying to be certified about objectivity. The use of a 3rd party not affiliated with Second Life will help these "impartiality" issues disappear.
The topic is yet to be answered, but the choices include the school where someone took their classes, a third party testing company in-world, an agency responsible for creating the exams, an agency responsible for maintaining the certification lists, or some other solution.
* Pro-metric testing is very involved process. In multiple choice questions, how many correct answers are "b"? How long are the correct answers compared to the "decoy" answers? Does the language used in the questions lead people to a certain answer? These are just a few of the issues that the testing agency will have experience in dealing with.
* You will be writing the questions, and expect to write many and have them sent back for rephrasing. For an average 25 question test, expect them to need at least 100 questions to be randomly selected. (I was involved with running the certification process for a company I worked for a few years ago, and the testing agency expected 400 questions to be written for a 50 question test; although this was for a specific purpose, it highlights the point)
* Multi-languge support, although subject matter experts (SME's = you) in world will need to check the phrasing in the local language to ensure that the context of the question hasn't been lost in the translation.


== How does the scoring work? Why don't the candidates just "guess" the answers? ==
== How does the scoring work? Why don't the candidates just "guess" the answers? ==
Line 34: Line 28:
There is a time limit for each test, which is based on "trial run" tests.
There is a time limit for each test, which is based on "trial run" tests.


==What types of tests can they produce? ==
= What types of tests can be produced? =


This varies depending on provider, but the computer based testing usually follows these formats:
This varies depending on provider, but the computer based testing usually follows these formats:
Line 44: Line 38:
* Location mapping, so that a click can indicate an area of the screen. For example, "Click on the incorrect statment in the script in front of you"
* Location mapping, so that a click can indicate an area of the screen. For example, "Click on the incorrect statment in the script in front of you"


Practical testing can also be carried out, but to be honest at this point I haven't the first clue about how these will be done in Second Life. Selecting the correct testing supplier will rely on discussions around such issues.
Practical testing can also be carried out in combination with computer-based testing, or in stead of computer-based testing.  Discussions are ongoing about exactly how the tests will be created and given.


== How much does it cost? ==
= How much does it cost? ==


How long is a piece of elestic string? Although, charging usually happens like this:
* There is an administration charge for running each test. Most companies pass this charge on to the candidate. What Linden Lab plans to do for this is yet to be decided.
 
* An up-front cost for the development of the tests is paid by the service provider (Linden Lab). This cost reflects the development of the tests and sometimes includes update changes based on a pre-arranged time-scale.
* There is an administration charge for running each test to the service provider (LL). Most companies pass this charge on to the candidate. What Linden Lab plans to do for this is yet to be decided.
* Failure of a test sometimes means that a re-test can be done at a lower rate.
* Failure of a test sometimes means that a re-test can be done at a lower rate.
 
* Current estimates are US $100.
 
I'll update this document as new questions arise.

Revision as of 20:29, 24 April 2010

Overview of Certification

Certification is not directly linked to a training program, and neither should it be. Certification is intended to test a person's competence. If a person has extensive knowledge of a topic they should be able to sit a test without ever having to attend a designated training course.

So why involve the schools with certification? Simply because the number of people in the catagory described above will be few and far between. The schools will be an excellent resource for helping people enhance their knowledge to where they can pass an exam. However, courses should not be written to the tests. There will likely be a large uptake of people requiring training, but the courses should teach the subjects generally, and not be geared to helping people just "pass a test." Providing a list of approved schools to residents hoping to certify will be essential.

What issues will need to be dealt with in creating exams?

  • Writing tests that comply to prometric standards
  • Managing the candidates' applications
  • Global availability. Making sure that the test system is available as widely as possible, taking into consideration geographic locations, language and accessibility issues for those with disabilities.
  • Reporting and managing on-going certification requirements over time.
  • Use of pro-metric testing to create the exams.
  • Multi-languge support, although subject matter experts (SME's = you) in world will need to check the phrasing in the local language to ensure that the context of the question hasn't been lost in the translation.

Who will proctor the exams?

The topic is yet to be answered, but the choices include the school where someone took their classes, a third party testing company in-world, an agency responsible for creating the exams, an agency responsible for maintaining the certification lists, or some other solution.

How does the scoring work? Why don't the candidates just "guess" the answers?

Scoring normally applies one point for each correct answer, zero points for a blank answer and minus 0.25 for a wrong answer. This basically eliminates the possibility of someone just choosing option "a" all the way down and "hoping for the best". Usually a review period is given at the end of the test to go back and check answers the person feels they may have got wrong. There is a time limit for each test, which is based on "trial run" tests.

What types of tests can be produced?

This varies depending on provider, but the computer based testing usually follows these formats:

  • Single choice answers
  • Multiple choice with a point for all correct answers selected
  • Multiple choice with a point for each correct answer selected
  • Free text answers, using key word inclusion for scoring
  • Location mapping, so that a click can indicate an area of the screen. For example, "Click on the incorrect statment in the script in front of you"

Practical testing can also be carried out in combination with computer-based testing, or in stead of computer-based testing. Discussions are ongoing about exactly how the tests will be created and given.

How much does it cost? =

  • There is an administration charge for running each test. Most companies pass this charge on to the candidate. What Linden Lab plans to do for this is yet to be decided.
  • Failure of a test sometimes means that a re-test can be done at a lower rate.
  • Current estimates are US $100.