Local data storage methods used in the viewer

This article is a work in progress.

Please do not edit.

This article is a work in progress.

Please do not edit.

Article purpose and design

This page is intended to have two different purposes.

• Document where the VFS is used in the viewer source
• Provide a direction forward for removing/replacing the VFS

The following list of source references is based on viewer source released 1.19.1.4 in the SVN repository, and will need future updating as the viewer source continues to evolve:

• Link to 1.19.1.4 in the SVN repository

This listing duplicates the layout and format of the Viewer Source Files page, providing a direct 1-to-1 correspondence with each section and source file. Although the source list only includes .cpp files, if a matching .h file is found it is listed here below the .cpp for that name. === Table column headings === Each column of the table refers to:

• "VFS" - any reference to this keyward. "VFS" is not used in the code alone, and so is usually referring to comments in the source, or debugging/logging notes.
• "gVFS" - the "general VFS" which is used for temporary storage for assets, in a private RAM-disk held in memory at all times. It is the primary limitation for allowing unlimited local asset caching in the viewer as currently designed. This needs to be removed and replaced with direct filesystem calls to folder-based storage.
• "gStaticVFS" - a read-only VFS which makes up about 75% of the client download. This contains the graphical elements and sounds of the Second Life user interface. It's small but why does it have to be fully loaded in a virtual RAM-disk all the time using limited system memory? May be worthwhile to replace this with direct on-disk storage, perhaps in a ZIP file.
• LFS - the "Local File System", used to directly read and write to the local OS storage. The audio manager for example uses LFS calls to write decoded OGG sounds as WAV files in the cache directory. LFS calls might be usable to replace the VFS as mentioned above.

VFS removal/replacement

As has been discussed in the sl-dev mailing list and in the JIRA, the VFS does not scale up very well at all since it functions as a private RAM-disk for non-texture assets downloaded by the viewer. The on-disk VFS files are used only for storage when the viewer is not running. At startup, the files are parsed for errors and loaded into memory, and continue to occupy memory until the viewer exits and the contents are written back to the disk files. As a RAM disk, increasing the cache size to larger than the available free memory causes the computer to run out of physical RAM and require the use of virtual memory and disk swapping. This is counterproductive when the intent of increasing the VFS size is to cache more of the frequently-used data to speed up the client and reduce network traffic. In order to permit the cache of assets currently stored in the VFS to grow to gigabytes of storage without requiring gigabytes of physical memory to store it, the existing VFS will have to be removed from the viewer and replaced with a traditional file/directory-based disk cache.

Need for lightweight asset obfuscation

In the interests of copyright protection, a directory-based non-texture asset cache should include some form of low-impact data obfuscation, to make it slightly difficult for the copyrighted works of other SL users held in the cache to be casually copied and examined. As has been discussed in SL-Dev, heavy and slow cache encryption would serve no purpose since a decryption tool can be easily built using the client source that would do the cache decrypting. So it's better to just use a simple and fast obfuscation technique which discourages casual asset theft, since more complex and slower protection measures would not offer any real protection anyway.

General - Main and Startup

file	VFS	gVFS	gStaticVFS	LFS
llstartup.cpp	X	X
llappviewer.cpp	X	X	X
llappviewer.h	X	X	X

General - Window Management and Render Pipeline

file	VFS	gVFS	gStaticVFS	LFS
pipeline.cpp
llviewerdisplay.cpp
llviewerprecompiledheaders.cpp
llviewerprecompiledheaders.h	X
llviewerwindow.cpp
llwindebug.cpp

General - Utility and Misc

file	VFS	gVFS	gStaticVFS	LFS
llbbox.cpp
llbox.cpp
llcallbacklist.cpp
llcontroldef.cpp	X
lleventnotifier.cpp
lleventpoll.cpp
llhippo.cpp
llregionposition.cpp
moviemaker.cpp
noise.cpp

Rendering - Asset Management

file	VFS	gVFS	gStaticVFS	LFS
llassetuploadresponders.cpp	X	X
lltexturecache.cpp				X
lltexturefetch.cpp
llviewerassetstorage.cpp	X
llviewerassetstorage.h	X
llwearable.cpp	X	X
llwearablelist.cpp

Rendering - Character and Animation

file	VFS	gVFS	gStaticVFS	LFS
lldriverparam.cpp
llemote.cpp
llviewergesture.cpp
llvoavatar.cpp	X		X
llcharacter(???)

Rendering - Camera

file	VFS	gVFS	gStaticVFS	LFS
llcameraview.cpp
llfollowcam.cpp
llmorphview.cpp
llviewercamera.cpp

Rendering - Effects

file	VFS	gVFS	gStaticVFS	LFS
llhudconnector.cpp
llhudeffect.cpp
llhudeffectbeam.cpp
llhudeffectlookat.cpp
llhudeffectpointat.cpp
llhudeffecttrail.cpp
llhudicon.cpp
llhudmanager.cpp
llhudobject.cpp
llhudrender.cpp
llhudtext.cpp
llhudview.cpp
lltracker.cpp
llvelocitybar.cpp
llvotextbubble.cpp

Rendering - Environment

file	VFS	gVFS	gStaticVFS	LFS
llvlmanager.cpp
llvlcomposition.cpp
llvoclouds.cpp
llvosurfacepatch.cpp
llvoground.cpp
llvosky.cpp
llvostars.cpp
llvowater.cpp
llwind.cpp

Rendering - Viewer Objects: Volumes, Particles, and Trees

file	VFS	gVFS	gStaticVFS	LFS
llvocache.cpp
llvograss.cpp
llvoinventorylistener.cpp
llvopartgroup.cpp
llvotree.cpp
llvovolume.cpp

Rendering - Render Pipeline

file	VFS	gVFS	gStaticVFS	LFS
llcloud.cpp
llcubemap.cpp
llcylinder.cpp
lldrawable.cpp
lldrawpool.cpp
lldrawpoolalpha.cpp
lldrawpoolavatar.cpp
lldrawpoolbump.cpp
lldrawpoolclouds.cpp
lldrawpoolground.cpp
lldrawpoolsimple.cpp
lldrawpoolsky.cpp
lldrawpoolstars.cpp
lldrawpoolterrain.cpp
lldrawpooltree.cpp
lldrawpoolwater.cpp
lldynamictexture.cpp
llface.cpp
llfeaturemanager.cpp
llflexibleobject.cpp
llglsandbox.cpp
llglslshader.cpp
llpatchvertexarray.cpp
llpolymesh.cpp
llpolymorph.cpp
llsky.cpp
llspatialpartition.cpp
llsphere.cpp
llsprite.cpp
llsurface.cpp
llsurfacepatch.cpp
lltexlayer.cpp	X	X
llviewerimage.cpp	X
llviewerimagelist.cpp	X
llviewerjoint.cpp
llviewerjointattachment.cpp
llviewerjointmesh.cpp
llviewerjointshape.cpp
llviewerlayer.cpp
llviewerobject.cpp
llviewerobjectlist.cpp
llviewerpartsim.cpp
llviewerpartsource.cpp
llviewertextureanim.cpp
llviewervisualparam.cpp

Input and Output - Audio

file	VFS	gVFS	gStaticVFS	LFS
fmodwrapper.cpp
llaudiosourcevo.cpp
llaudiostatus.cpp

Input and Output - Input

file	VFS	gVFS	gStaticVFS	LFS
llviewerjoystick.cpp
llviewerkeyboard.cpp

Data and Communication - Agents

file	VFS	gVFS	gStaticVFS	LFS
llagent.cpp
llagentdata.cpp
llagentpilot.cpp

Data and Communication - Region and World

file	VFS	gVFS	gStaticVFS	LFS
llviewerregion.cpp
llworld.cpp

Data and Communication - Network

file	VFS	gVFS	gStaticVFS	LFS
llasynchostbyname.cpp
llcaphttpsender.cpp
llurl.cpp
llurlwhitelist.cpp
lluserauth.cpp
llviewergenericmessage.cpp
llviewermessage.cpp	X
llviewermessage.h	X
llviewernetwork.cpp
llviewerthrottle.cpp
llweb.cpp
llxmlrpctransaction.cpp

User Interface - Alerts

file	VFS	gVFS	gStaticVFS	LFS
llconfirmationmanager.cpp
llfirstuse.cpp
llnotify.cpp

User Interface - Assets

file	VFS	gVFS	gStaticVFS	LFS
llcompilequeue.cpp	X
llcompilequeue.h	X
llfloateranimpreview.cpp	X	X
llfloatergesture.cpp
llfloaterimagepreview.cpp
llfloaterinspect.cpp
llfloateropenobject.cpp
llfloaterproperties.cpp
llfloaterscriptdebug.cpp
llpreview.cpp
llpreviewanim.cpp
llpreviewgesture.cpp	X	X
llpreviewgesture.h	X
llpreviewlandmark.cpp
llpreviewnotecard.cpp	X	X
llpreviewnotecard.h	X
llpreviewscript.cpp	X
llpreviewscript.h	X
llpreviewsound.cpp
llpreviewtexture.cpp

User Interface - Avatar Appearance

file	VFS	gVFS	gStaticVFS	LFS
llfloaterclothing.cpp
llfloatercustomize.cpp
llgenepool.cpp
llgesturemgr.cpp	X
llgesturemgr.h	X
llpanelmorph.cpp

User Interface - Chat, Instant Messaging, Notices, Reports

file	VFS	gVFS	gStaticVFS	LFS
llchatbar.cpp
llfloaterchat.cpp
llfloatermute.cpp
llfloaternewim.cpp
llfloaterpostcard.cpp	X	X
llfloaterreporter.cpp	X	X
llgroupmgr.cpp
llgroupnotify.cpp
llimpanel.cpp
llimview.cpp
lllogchat.cpp
llmutelist.cpp

User Interface - Data

file	VFS	gVFS	gStaticVFS	LFS
llclassifiedinfo.cpp
lleventinfo.cpp
llfloaterabout.cpp
llfloaterauction.cpp	X	X
llfloateravatarinfo.cpp
llfloatergroupinfo.cpp
llfloatergroupinvite.cpp
llfloatergroups.cpp
llfloaterland.cpp
llfloaterlandholdings.cpp
llfloaterregioninfo.cpp	X
llfloaterregioninfo.h	X
llfloatertelehub.cpp
llfloatertos.cpp
llfloatertos.h	X
llpanelavatar.cpp
llpanelclassified.cpp
llpanelevent.cpp
llpanelgroup.cpp
llpanelgroupgeneral.cpp
llpanelgroupinvite.cpp
llpanelgrouplandmoney.cpp
llpanelgroupnotices.cpp
llpanelgrouproles.cpp
llpanelgroupvoting.cpp
llpanelland.cpp
llpanellandobjects.cpp
llpanellandoptions.cpp

User Interface - Data:Inventory

file	VFS	gVFS	gStaticVFS	LFS
llcallingcard.cpp
llfloaterfriends.cpp
llinventoryactions.cpp
llinventorybridge.cpp
llinventoryclipboard.cpp
llinventorymodel.cpp
llinventoryview.cpp
lllandmarklist.cpp	X
lllandmarklist.h	X
llviewerinventory.cpp

User Interface - Data:Money and Land

file	VFS	gVFS	gStaticVFS	LFS
llcurrencyuimanager.cpp
llgivemoney.cpp
llfloaterbuy.cpp
llfloaterbuycontents.cpp
llfloaterbuycurrency.cpp
llfloaterbuyland.cpp
llfloatersellland.cpp

User Interface - Data:Search

file	VFS	gVFS	gStaticVFS	LFS
llfloaterdirectory.cpp
llpaneldirbrowser.cpp
llpaneldirclassified.cpp
llpaneldirevents.cpp
llpaneldirfind.cpp
llpaneldirgroups.cpp
llpaneldirland.cpp
llpaneldirpeople.cpp
llpaneldirplaces.cpp
llpaneldirpopular.cpp
llpanelpick.cpp
llpanelplace.cpp

User Interface - Data:Map

llcompass.cpp

file	VFS	gVFS	gStaticVFS	LFS
llfloatermap.cpp
llfloaterworldmap.cpp
llmapresponders.cpp
llnetmap.cpp
llworldmap.cpp
llworldmapview.cpp

User Interface - Objects and Edit Tools

file	VFS	gVFS	gStaticVFS	LFS
llhoverview.cpp
llmanip.cpp
llmaniprotate.cpp
llmanipscale.cpp
llmaniptranslate.cpp
llfloaterbuildoptions.cpp
llfloaterbump.cpp
llfloatercolorpicker.cpp
llfloaterpermissionsmgr.cpp
llfloatertools.cpp
llpanelcontents.cpp
llpanelface.cpp
llpanelgeneral.cpp
llpanelinventory.cpp
llpanelobject.cpp
llpanelpermissions.cpp
llselectmgr.cpp
lltool.cpp
lltoolbar.cpp
lltoolbrush.cpp
lltoolcomp.cpp
lltooldraganddrop.cpp
lltoolface.cpp
lltoolfocus.cpp
lltoolgrab.cpp
lltoolgun.cpp
lltoolindividual.cpp
lltoolmgr.cpp
lltoolmorph.cpp
lltoolobjpicker.cpp
lltoolpie.cpp
lltoolpipette.cpp
lltoolplacer.cpp				Note 1
lltoolselect.cpp
lltoolselectland.cpp
lltoolselectrect.cpp
lltoolview.cpp
llviewerparcelmgr.cpp
llviewerparceloverlay.cpp

Note 1 - invalid match of text inside something else.

User Interface - HTML

file	VFS	gVFS	gStaticVFS	LFS
llfloaterhtml.cpp
llfloaterhtmlfind.cpp
llfloaterhtmlhelp.cpp

User Interface - Import, Export, Upload

file	VFS	gVFS	gStaticVFS	LFS
llfloaterimport.cpp
llfloatersaveavatar.cpp
llfloatersnapshot.cpp	X	X
lluploaddialog.cpp

User Interface - Menus, Overlays, Login

file	VFS	gVFS	gStaticVFS	LFS
llmediaremotectrl.cpp
llmenucommands.cpp
llmoveview.cpp
lloverlaybar.cpp
llpanellogin.cpp
llprogressview.cpp
llviewermenu.cpp	X			Note 1
llviewermenu.h				Note 1
llviewermenufile.cpp	X	X

Note 1 - invalid match of text inside something else.

User Interface - Preferences

file	VFS	gVFS	gStaticVFS	LFS
llfloaterpreference.cpp
llpanelaudioprefs.cpp
llprefschat.cpp
llprefsim.cpp
llpaneldisplay.cpp
llpanelmsgs.cpp
llpanelnetwork.cpp
llpanelvolume.cpp
llpanelweb.cpp

User Interface - Utility

file	VFS	gVFS	gStaticVFS	LFS
llcolorscheme.cpp
lldirpicker.cpp
llfilepicker.cpp
llfloateravatarpicker.cpp
llfloaternamedesc.cpp
llsavedsettingsglue.cpp
llviewchildren.cpp

User Interface - Widgets

file	VFS	gVFS	gStaticVFS	LFS
llcolorswatch.cpp
llcontainerview.cpp
llfolderview.cpp
llnamebox.cpp
llnameeditor.cpp
llnamelistctrl.cpp
lltexturectrl.cpp
lljoystickbutton.cpp
llviewertexteditor.cpp
llvieweruictrlfactory.cpp
llwebbrowserctrl.cpp

User Interface - Debug

file	VFS	gVFS	gStaticVFS	LFS
llconsole.cpp
lldebugmessagebox.cpp
lldebugview.cpp
llfasttimerview.cpp
llframestats.cpp
llframestatview.cpp
llmemoryview.cpp
llfloateravatartextures.cpp
llfloatereditui.cpp
llfloatergodtools.cpp
llfloatertest.cpp
llfloatertopobjects.cpp
llpaneldebug.cpp
llstatbar.cpp
llstatgraph.cpp
llstatusbar.cpp
llstatview.cpp
lltextureview.cpp				X
llvelocitybar.cpp
llviewercontrol.cpp
llviewerstats.cpp	X
llviewerstats.h	X

Unlisted in Viewer Source Files page

These are either unused, or are background threaded processes.

file	VFS	gVFS	gStaticVFS	LFS
llassetstorage.cpp	X
llassetstorage.h	X
audioengine.cpp	X	X
audioengine.h	X
llaudiodecodemgr.cpp	X	X	X
llaudiodecodemgr.h	X
llextendedstatus.h	X
llhttpassetstorage.cpp	X
llhttpassetstorage.h	X
llhttpclient.cpp	X
lllfsthread.cpp				X
lllfsthread.h				X
llhttpclient.cpp	X	X
llimage.cpp	X
llimage.h	X
llkeyframemotion.cpp	X
llkeyframemotion.h	X
llprocessor.cpp				Note 1
llprocessor.h				Note 1
llstatemachine.cpp				Note 1
llstatemachine.h				Note 1
lltransfersourceasset.cpp	X
lltransfersourceasset.h	X
lltransfertargetvfile.cpp	X
lltransfertargetvfile.h	X
llvfile.cpp	X
llvfile.h	X
llvfs.cpp	X
llvfs.h	X
llvfsthread.cpp	X
llvfsthread.h	X
llxfer_vfile.cpp	X
llxfer_vfile.h	X
llxfermanager.cpp	X
llxfermanager.h	X
vorbisdecode.cpp	X
vorbisdecode.h	X

Note 1 - invalid match of text inside something else.