User:Mm Alder/SLDev Open Source Viewer

What is the SLDev Open Source Viewer?

According to the announcement from Linden Lab

"Our hope is that we can create a widely-used openly developed version of the Second Life client which is a compelling alternative for a broad set of users, and contains enhancements and development that then rapidly make their way back into the mainstream Second Life version."

Translated, that means it's how you, the open source developer, get your code into the Second Life viewer.

Second Life is Open Source!

In January, 2007 Linden opened the source for the Second Life viewer and many residents began to contribute code to improve or otherwise extend the functionality, reliability, and openness of the viewer. But the code wasn't really open for mutual development between Lindens and residents. Residents got to see the source, and they could create patches to be applied to it, but they needed a Linden to actually apply that patch to the code used to build the Linden public viewer.

In addition there were two systems for reporting bugs, fixes, and features: the public JIRA (PJIRA) and the internal Linden JIRA (just JIRA). Residents read the former and Lindens read the latter. Every now and then a Linden will comment on the PJIRA, but if you add them all up, you'd think Linden was operating out of someone's garage. There was talk of merging the two, but that never happened.

At the same time, there was a clamor from residents to fix the bugs before adding new features that nobody wanted. Like WindLight.

The result was an ever increasing difficulty for open source developers to get code accepted into the published viewer.

The Nicolaz viewer, The Dale Glass viewer, Kirsten's viewer, Imprudence, and more

Eventually, developers became impatient, and decided to publish their own versions of the viewer under the open source license granted by Linden Lab. The result was that residents came to like the alternate viewers more than the official viewer. They were faster, easier to use, had nifty new fietures, and crashed a lot less.

Unfortunately, some of the noble developers who undertook this task, had real lives, and couldn't afford to take the time to figure out and merge in the new features published by Linden every time a new official Linden release came out.

Philip, Benevolent Dictator For Life

But someone at Linden looked around and thought to himself, "If they can do it, why can't we?"

The Open Source Viewer is the brainchild of Philip Linden, the Chairman of the Board at Linden Lab. When Philip decided to give up his role as CEO, he found he had time to do something worthwhile for the company.

Philip realized that developers were becoming discouraged with the difficulty they were having getting their ideas into the official viewer, and if Linden didn't do something, they would lose all interest. So he proposed that Linden follow the trail charted by Nicolaz, Dale Glass, and many others, and have their own Open Source Viewer, but with one decided advantage: he could get the latest Linden changes merged in just by decreeing (oops!) asking. Thus was born the http-texture project! Huh?

http-texture

Philippe Bossut (Merov Linden) had been working on an internal Linden project called http-texture whose goal was to transfer texture assets via HTTP protocol rather than via UDP. Philip decided that Philippe's project would be a good place to start an internal open source project. And so it was.

How do I contribute to it?

Talk to the mailing list

Before you set off coding your great idea for Python versions of the shaders, you better ask around to see if either everyone hates your idea, or Linden is already working on it but hasn't told anyone yet, or someone else has already started a similar project and would welcome your help, or you've managed to design a mousetrap that slices bread and is a dessert topping too!

The first thing you could do is to search the public issue tracker, also known as the PJIRA, to see if there's something similar out there. If there's nothing, or if it's just a request, then take it to the masses.

All of that kind of discussion, and much worse, takes place on the SLDev mailing list. Sure there's a lot of arrogant know-it-alls on the list who will quickly belittle your wonderful idea. No, just kidding! Well, except for Mm Alder, but he's harmless. Just go for it! Everybody has to start somewhere. We've been there too. And there's plenty to do. We can use all the help we can get.

Write your code

OK, everyone loves your mousetrap. Where do you start? Well, first, post your idea to the PJIRA so somebody else interested in mousetraps can find out what you intend to do.

Where do I get the source?

Then, you have to figure out which version of the code to use as your base. You get lots of choices: the source for the official viewer, the source for the Release Candidate, the trunk, or the current SLDev project.

If you're fixing a bug, the trunk is probably your best bet because that gets the latest code fixes. If your project might take months to develop, you should go with the source for the official viewer because it's not going to change for a while. It's a pain to have your code changes break when you do an update from the repository. You ca never tell if it's your fault or somebody else's.

The official viewer changes every few months; the Release Candidate source every few weeks; the trunk and SLDev project source changes constantly. The choice is yours. Before you think about submitting it to the SLDev project, though, you will have to make a patch relative to the SLDev source code.

How do I make sense of this mess?

Indra? Oh God! Newview? Compared to what? And H-E double-hockey-sticks everywhere!

Don't despair. The Viewer Architecture is a good place to start. The Viewer source article gives you some hints about the directory structure. And Doxygen will let you breathe a sigh of relief.

Just watch out for the Hippo.

And if you happen to be wandering around dense murky code that has no documentation on the wiki, then Dr. Livingston, it's your duty to write about it on the wiki for the next intrepid explorer.

How do I build it?

Nowadays people are using CMake to make solution files for the Microsoft Visual Studio integrated development environment (IDE) on Microsoft Windows, or to make project files for XCode on Mac OS X, or to generate makefiles for gcc on Linux. The build process is still being developed and it's getting easier, but it still takes a bit of perseverence.

Are there any style guidelines?

Reading the source code, you may get the impression that there is no standard style used by Linden, but in fact there is a coding standard. The existing code has been developed over many many years and can be excused for violating the standards. You will not be excused. Reading the standards before starting to code could save you a lot of editing later.

Patch? What's a patch?

Once you've written, debugged, documented, and thoroughly tested your code, it's time to publish it to the world. That's done by attaching a patch file to the PJIRA issue that you created way back when you started all of this. If the changes are small, you can describe what you did as a comment on the PJIRA issue. If it's a major change, create a wiki page User:(your avatar name here)/(feature name here) and put a link to it on your PJIRA issue. Then let the SLDev mailing list know about it; stand back; and accept the accolades of your peers. OK, well maybe accolades are a bit scarce these days. Enjoy the satisfaction of a job well done. You knew you weren't getting paid for this anyway.

Get your code reviewed

OK, so I'm completely making this part up, but this is the way I think it should happen.

Every week open source developers and Lindens gather at a miniature Roman Coliseum on Hippotropolis.

Contributors have added their names to a list on the wiki together with a link pointing to the PJIRA issue that describes their contributions and includes a patch file made against the current version of the SLDev open source. One by one the contributors are called before the Linden tribunal. Using voice chat and an in-world display showing the code with highlights on the changes, the contributor describes what that part of the code used to do and how this contribution makes it better. Developers politely ask questions using text chat, and the contributor answers them in turn.

In the end, everyone learns something, and if the code is deemed worthy, the contribution is given a thumbs up approval. If not, lions. Or at least a second chance.

Well that's the way it should work, except for the lions.

Right now the PJIRA issues are added to a list of Pending merge queue proposals and more or less sit there until you bug the mailing list and PJIRA a lot.

Get it committed to the repository

There is a group of committers that do have commit privileges to the SVN repository. The big difference with the "old LL way of doing things" is that this group of committers has Lindens and non-Lindens. If there is community support and a good review of the code from committers, the patch will be taken in and committed by one of the committers. Eventually, if someone does quite a bit of good contributions in line with the project, the committers will propose that that person become a committer themselves. The committers group grows by co-optation only.

If someone objects (and if our community is alive, there will be someone to object...), we debate and try to reach a consensus on the SLDev list. If there's a really contentious decision to be made (split debate), Philip, our BDFL (Benevolent Dictator For Life), will weight in and make the decision.

We won't have SVN wars like Wiki wars since the group of committers will be co-opted. Obviously, the committers will be careful co-opting only people who share their view and philosophy of development. If one committer would all of a sudden commit controversial patches with no respect for due debate or the general direction of the project (as decided by the BDFL), he or she'll see his/her commit rights revoked. The BDFL has full authority in that matter.

Track it through the merge process

The SLDev project is branched off from the trunk, but is kept current with changes to the trunk by Linden developers. In order to get into the official viewer, a patch must make its way from the SLDev branch back to the trunk, which then gets branched off as a Release Candidate, and after beta testing and bug fixes, becomes the official viewer.

Not all contributions will be chosen to get incorporated into the official viewer. That decision is separate from the decision to include it in the SLDev viewer.

If a contribution is selected for inclusion into the official viewer, the contributor is expected to follow the code through the merge process, testing it along the way to make sure it doesn't get broken by other changes.

We do hope that most of the innovations battle tested here will eventually make it to the official viewer codebase but things that are clearly not mainstream enough or not very popular with this community simply won't make it there, and that's OK. People will still be able to get those features in the SLDev viewer download.

Each project (RC, maint, etc...) has its own way of ensuring quality and consistency of its project, mitigating agility vs prudence. As far as the merge back to the official viewer of contributions is concerned, that'll be up to the Linden Lab viewer team to cherry pick and merge patches/commits separately.

This is not easy to do. Making this more manageable is one of the big reason for the push to hg (mercurial) for source version control. It does make merges from various sources much more easier than SVN.