Difference between revisions of "Local data storage methods used in the viewer"

From Second Life Wiki
Jump to navigation Jump to search
(add new table columns, and clean up completed table columns)
(split off subpages, add table for main and startup, add explanation of table headings to top)
Line 26: Line 26:
This listing duplicates the layout and format of the [[Viewer Source Files]] page, providing a direct 1-to-1 correspondence with each section and source file. Although the source list only includes .cpp files, if a matching .h file is found it is listed here below the .cpp for that name.
This listing duplicates the layout and format of the [[Viewer Source Files]] page, providing a direct 1-to-1 correspondence with each section and source file. Although the source list only includes .cpp files, if a matching .h file is found it is listed here below the .cpp for that name.


 
Each column of the table refers to:
'''NOTE: The function of the LFS is not documented anywhere, but appears to refer to the "Linden File System", a static VFS that is included in the client download which contains the client user interface images and sounds.''' Therefore any references to the LFS are also included since they would need to be replaced with a folder-based storage, along with the regular VFS.
* "VFS" - any reference to this keyward. "VFS" is not used in the code alone, and so is usually referring to comments in the source, or debugging/logging notes.
 
* "gVFS" - the "general VFS" which is used for temporary storage for assets and is the primary problem with the viewer as currently designed. This needs to be removed and replaced with direct filesystem calls to folder-based storage.
* "gStaticVFS" - a read-only VFS which makes up about 75% of the client download. This contains the graphical elements and sounds of the Second Life user interface. It's small but why does it have to be fully loaded in a virtual RAM-disk all the time using limited system memory? May be worthwhile to replace this with direct on-disk storage, perhaps in a ZIP file.
* LFS - the "Local File System", used to directly read and write to the local OS storage. The audio manager for example uses LFS calls to write decoded OGG sounds as WAV files in the cache directory. LFS calls might be usable to replace the VFS as mentioned above.


== General - Main and Startup ==
== General - Main and Startup ==


=== /newview/llstartup.cpp ===
{| cellborder="3" border="2"
 
!file
<php>74 #include "llvfs.h"</php>
!VFS
 
!gVFS
<php>215 // local function declaration
!gStaticVFS
( . . . )
!LFS
233 void asset_callback_nothing(LLVFS*, const LLUUID&, LLAssetType::EType, void*, S32); </php>
|-
 
|llstartup.cpp
<php>298 // Returns FALSE to skip other idle processing. Should only return
|<!-- VFS::YES --> <center><font size="+2">'''X'''</font></center>
299 // TRUE when all initialization done.
|gVFS::no
300 BOOL idle_startup()
|<!-- gStaticVFS::YES --> <center><font size="+2">'''X'''</font></center> (check)
301 {
|LFS::no
( . . . )
|-
504                        // start the xfer system. by default, choke the downloads
|llappviewer.cpp
505                        // a lot...
|<!-- VFS::YES --> <center><font size="+2">'''X'''</font></center>
506                        const S32 VIEWER_MAX_XFER = 3;
|gVFS::no
507                        start_xfer_manager(gVFS);
|<!-- gStaticVFS::YES --> <center><font size="+2">'''X'''</font></center> (check)
508                        gXferManager->setMaxIncomingXfers(VIEWER_MAX_XFER);
|LFS::no
509                        F32 xfer_throttle_bps = gSavedSettings.getF32("XferThrottle");
|-
510                        if (xfer_throttle_bps > 1.f)
|llappviewer.h
511                        {
|<!-- VFS::YES --> <center><font size="+2">'''X'''</font></center>
512                                gXferManager->setUseAckThrottling(TRUE);
|gVFS::no
513                                gXferManager->setAckThrottleBPS(xfer_throttle_bps);
|<!-- gStaticVFS::YES --> <center><font size="+2">'''X'''</font></center> (check)
514                        }
|LFS::no
515                        gAssetStorage = new LLViewerAssetStorage(msg, gXferManager, gVFS);
|}
( . . . )
891                // Display the startup progress bar.
892                gViewerWindow->setShowProgress(TRUE);
893                gViewerWindow->setProgressCancelButtonVisible(TRUE, "Quit");
894 
895                // Poke the VFS, which could potentially block for a while if
896                // Windows XP is acting up
897                set_startup_status(0.07f, "Verifying cache files (can take 60-90 seconds)...", NULL);
898                display_startup();
899 
900                gVFS->pokeFiles(); </php>
 
<php>3032 void init_stat_view()
3033 {
( . . . )
3245        stat_barp = net_statviewp->addStat("VFS Pending Ops", &(gViewerStats->mVFSPendingOperations));
3246        stat_barp->setUnitLabel(" ");
3247        stat_barp->mPerSec = FALSE;
3248        stat_barp->mDisplayBar = FALSE; </php>
 
<php>3495 void asset_callback_nothing(LLVFS*, const LLUUID&, LLAssetType::EType, void*, S32)
3496 {
3497        // nothing
3498 }</php>
 
=== /newview/llappviewer.cpp - Init, Running, and Shutdown ===
 
llappviewer.cpp is the central code for making the Windows client function.
 
http://svn.secondlife.com/trac/linden/browser/branches/Branch_1-19-1-Viewer/indra/newview/llappviewer.cpp?rev=443
 
==== Initialization ====
 
<php>83 #include "lllfsthread.h"
( . . . )
303 // VFS globals - see llappviewer.h
304 LLVFS* gStaticVFS = NULL;
( . . . )
356 // File scope definitons
357 const char *VFS_DATA_FILE_BASE = "data.db2.x.";
358 const char *VFS_INDEX_FILE_BASE = "index.db2.x.";
( . . . )
394 " -nothread                            run vfs in single thread\n"
 
<php>548 int parse_args(int argc, char **argv)
( . . . )
824                else if (!strcmp(argv[j], "-nothread"))
825                {
826                        LLVFile::ALLOW_ASYNC = FALSE;
827                        llinfos << "Running VFS in nothread mode" << llendl;
828                } </php>
 
<php>982 bool LLAppViewer::init()
( . . . )
1208        //
1209        // Initialize the VFS, and gracefully handle initialization errors
1210        //
1211 
1212        if (!initCache())
1213        {
1214                std::ostringstream msg;
1215                msg <<
1216                        gSecondLife << " is unable to access a file that it needs.\n"
1217                        "\n"
1218                        "This can be because you somehow have multiple copies running, "
1219                        "or your system incorrectly thinks a file is open. "
1220                        "If this message persists, restart your computer and try again. "
1221                        "If it continues to persist, you may need to completely uninstall " <<
1222                        gSecondLife << " and reinstall it.";
1223                OSMessageBox(
1224                        msg.str().c_str(),
1225                        NULL,
1226                        OSMB_OK);
1227                return 1;
1228        } </php>
 
<php>1965 bool LLAppViewer::initThreads()
( . . . )
1972        LLVFSThread::initClass(enable_threads && true);
1973        LLLFSThread::initClass(enable_threads && true);
( . . . )
2914        LLSplashScreen::update("Initializing Texture Cache...");
2915         
2916        // Init the texture cache
2917        // Allocate 80% of the cache size for textures
2918        BOOL read_only = mSecondInstance ? true : false;
2919        const S32 MB = 1024*1024;
2920        S64 cache_size = (S64)(gSavedSettings.getU32("CacheSize")) * MB;
2921        const S64 MAX_CACHE_SIZE = 1024*MB;
2922        cache_size = llmin(cache_size, MAX_CACHE_SIZE);
2923        S64 texture_cache_size = ((cache_size * 8)/10);
2924        S64 extra = LLAppViewer::getTextureCache()->initCache(LL_PATH_CACHE, texture_cache_size, read_only);
2925        texture_cache_size -= extra;
2926 
2927        LLSplashScreen::update("Initializing VFS...");
2928         
2929        // Init the VFS
2930        S64 vfs_size = cache_size - texture_cache_size;
2931        const S64 MAX_VFS_SIZE = 1024 * MB; // 1 GB
2932        vfs_size = llmin(vfs_size, MAX_VFS_SIZE);
2933        vfs_size = (vfs_size / MB) * MB; // make sure it is MB aligned
2934        U32 vfs_size_u32 = (U32)vfs_size;
2935        U32 old_vfs_size = gSavedSettings.getU32("VFSOldSize") * MB;
2936        bool resize_vfs = (vfs_size_u32 != old_vfs_size);
2937        if (resize_vfs)
2938        {
2939                gSavedSettings.setU32("VFSOldSize", vfs_size_u32/MB);
2940        }
2941        llinfos << "VFS CACHE SIZE: " << vfs_size/(1024*1024) << " MB" << llendl;
2942         
2943        // This has to happen BEFORE starting the vfs
2944        //time_t        ltime;
2945        srand(time(NULL));              // Flawfinder: ignore
2946        U32 old_salt = gSavedSettings.getU32("VFSSalt");
2947        U32 new_salt;
2948        char old_vfs_data_file[LL_MAX_PATH];            // Flawfinder: ignore
2949        char old_vfs_index_file[LL_MAX_PATH];  // Flawfinder: ignore           
2950        char new_vfs_data_file[LL_MAX_PATH];            // Flawfinder: ignore
2951        char new_vfs_index_file[LL_MAX_PATH];  // Flawfinder: ignore
2952        char static_vfs_index_file[LL_MAX_PATH];        // Flawfinder: ignore
2953        char static_vfs_data_file[LL_MAX_PATH]; // Flawfinder: ignore
2954 
2955        if (gMultipleViewersOK)
2956        {
2957                // don't mess with renaming the VFS in this case
2958                new_salt = old_salt;
2959        }
2960        else
2961        {
2962                do
2963                {
2964                        new_salt = rand();
2965                } while( new_salt == old_salt );
2966        }
2967 
2968        snprintf(old_vfs_data_file,  LL_MAX_PATH, "%s%u",              // Flawfinder: ignore
2969                gDirUtilp->getExpandedFilename(LL_PATH_CACHE,VFS_DATA_FILE_BASE).c_str(),
2970                old_salt);
2971 
2972        // make sure this file exists
2973        llstat s;
2974        S32 stat_result = LLFile::stat(old_vfs_data_file, &s);
2975        if (stat_result)
2976        {
2977                // doesn't exist, look for a data file
2978                std::string mask;
2979                mask = gDirUtilp->getDirDelimiter();
2980                mask += VFS_DATA_FILE_BASE;
2981                mask += "*";
2982 
2983                std::string dir;
2984                dir = gDirUtilp->getExpandedFilename(LL_PATH_CACHE,"");
2985 
2986                std::string found_file;
2987                if (gDirUtilp->getNextFileInDir(dir, mask, found_file, false))
2988                {
2989                        snprintf(old_vfs_data_file, LL_MAX_PATH, "%s%s%s", dir.c_str(),
                                  gDirUtilp->getDirDelimiter().c_str(), found_file.c_str());
                                  // Flawfinder: ignore
2990 
2991                        S32 start_pos;
2992                        S32 length = strlen(found_file.c_str());
                                  /* Flawfinder: ignore*/
2993                        for (start_pos = length - 1; start_pos >= 0; start_pos--)
2994                        {
2995                                if (found_file[start_pos] == '.')
2996                                {
2997                                        start_pos++;
2998                                        break;
2999                                }
3000                        }
3001                        if (start_pos > 0)
3002                        {
3003                                sscanf(found_file.c_str() + start_pos, "%d", &old_salt);
3004                        }
3005                        llinfos << "Default vfs data file not present, found " << old_vfs_data_file << llendl;
3006                        llinfos << "Old salt: " << old_salt << llendl;
3007                }
3008        }
3009
3010        snprintf(old_vfs_index_file, LL_MAX_PATH, "%s%u",              // Flawfinder: ignore
3011                        gDirUtilp->getExpandedFilename(LL_PATH_CACHE,VFS_INDEX_FILE_BASE).c_str(),
3012                        old_salt);
3013
3014        stat_result = LLFile::stat(old_vfs_index_file, &s);
3015        if (stat_result)
3016        {
3017                // We've got a bad/missing index file, nukem!  
3018                llwarns << "Bad or missing vfx index file " << old_vfs_index_file << llendl;
3019                llwarns << "Removing old vfs data file " << old_vfs_data_file << llendl;
3020                LLFile::remove(old_vfs_data_file);
3021                LLFile::remove(old_vfs_index_file);
3022                 
3023                // Just in case, nuke any other old cache files in the directory.
3024                std::string dir;
3025                dir = gDirUtilp->getExpandedFilename(LL_PATH_CACHE,"");
3026 
3027                std::string mask;
3028                mask = gDirUtilp->getDirDelimiter();
3029                mask += VFS_DATA_FILE_BASE;
3030                mask += "*";
3031 
3032                gDirUtilp->deleteFilesInDir(dir, mask);
3033 
3034                mask = gDirUtilp->getDirDelimiter();
3035                mask += VFS_INDEX_FILE_BASE;
3036                mask += "*";
3037 
3038                gDirUtilp->deleteFilesInDir(dir, mask);
3039        }
3040
3041        snprintf(new_vfs_data_file, LL_MAX_PATH, "%s%u",                // Flawfinder: ignore
3042                gDirUtilp->getExpandedFilename(LL_PATH_CACHE,VFS_DATA_FILE_BASE).c_str(),
3043                new_salt);
3044
3045        snprintf(new_vfs_index_file, LL_MAX_PATH, "%s%u",
                gDirUtilp->getExpandedFilename(LL_PATH_CACHE, VFS_INDEX_FILE_BASE).c_str(),
                // Flawfinder: ignore
3046                new_salt);
3047 
3048 
3049        strncpy(static_vfs_data_file, gDirUtilp->getExpandedFilename(LL_PATH_APP_SETTINGS,
                "static_data.db2").c_str(), LL_MAX_PATH -1);          // Flawfinder: ignore
3050        static_vfs_data_file[LL_MAX_PATH -1] = '\0';
3051        strncpy(static_vfs_index_file, gDirUtilp->getExpandedFilename(LL_PATH_APP_SETTINGS,
                "static_index.db2").c_str(), LL_MAX_PATH -1);                // Flawfinder: ignore
3052        static_vfs_index_file[LL_MAX_PATH -1] = '\0';
3053 
3054        if (resize_vfs)
3055        {
3056                llinfos << "Removing old vfs and re-sizing" << llendl;
3057                 
3058                LLFile::remove(old_vfs_data_file);
3059                LLFile::remove(old_vfs_index_file);
3060        }
3061        else if (old_salt != new_salt)
3062        {
3063                // move the vfs files to a new name before opening
3064                llinfos << "Renaming " << old_vfs_data_file << " to " << new_vfs_data_file << llendl;
3065                llinfos << "Renaming " << old_vfs_index_file << " to " << new_vfs_index_file << llendl;
3066                LLFile::rename(old_vfs_data_file, new_vfs_data_file);
3067                LLFile::rename(old_vfs_index_file, new_vfs_index_file);
3068        }
3069 
3070        // Startup the VFS...
3071        gSavedSettings.setU32("VFSSalt", new_salt);
3072 
3073        // Don't remove VFS after viewer crashes.  If user has corrupt data, they can reinstall. JC
3074        gVFS = new LLVFS(new_vfs_index_file, new_vfs_data_file, false, vfs_size_u32, false);
3075        if( VFSVALID_BAD_CORRUPT == gVFS->getValidState() )
3076        {
3077                // Try again with fresh files 
3078                // (The constructor deletes corrupt files when it finds them.)
3079                llwarns << "VFS corrupt, deleted.  Making new VFS." << llendl;
3080                delete gVFS;
3081                gVFS = new LLVFS(new_vfs_index_file, new_vfs_data_file, false, vfs_size_u32, false);
3082        }
3083 
3084        gStaticVFS = new LLVFS(static_vfs_index_file, static_vfs_data_file, true, 0, false);
3085 
3086        BOOL success = gVFS->isValid() && gStaticVFS->isValid();
3087        if( !success )
3088        {
3089                return false;
3090        }
3091        else
3092        {
3093                LLVFile::initClass();
3094                return true;
3095        }
3096 }</php>
 
==== Main Loop ====
 
<php>1344 bool LLAppViewer::mainLoop()
1345 {
1346        //-------------------------------------------
1347        // Run main loop until time to quit
1348        //-------------------------------------------
( . . . )
1497                        while(1)
1498                        {
1499                                S32 work_pending = 0;
1500                                S32 io_pending = 0;
1501                                work_pending += LLAppViewer::getTextureCache()->update(1);
                                          // unpauses the texture cache thread
1502                                work_pending += LLAppViewer::getImageDecodeThread()->update(1);
                                          // unpauses the image thread
1503                                work_pending += LLAppViewer::getTextureFetch()->update(1);
                                          // unpauses the texture fetch thread
1504                                io_pending += LLVFSThread::updateClass(1);
1505                                io_pending += LLLFSThread::updateClass(1);
1506                                if (io_pending > 1000)
1507                                {
1508                                          ms_sleep(llmin(io_pending/100,100));
                                                  // give the vfs some time to catch up
1509                                }
1510 
1511                                F64 frame_time = frameTimer.getElapsedTimeF64();
1512                                F64 idle_time = idleTimer.getElapsedTimeF64();
1513                                if (frame_time >= min_frame_time &&
1514                                        idle_time >= min_idle_time &&
1515                                        (!work_pending || idle_time >= max_idle_time))
1516                                {
1517                                        break;
1518                                }
1519                        }
1520                        frameTimer.reset();
1521 
1522                        // Prevent the worker threads from running while rendering.
1523                        // if (LLThread::processorCount()==1)
                                    //pause() should only be required when on a single processor client...
1524                        if (run_multiple_threads == FALSE)
1525                        {
1526                                LLAppViewer::getTextureCache()->pause();
1527                                LLAppViewer::getImageDecodeThread()->pause();
1528                                // LLAppViewer::getTextureFetch()->pause();
                                          // Don't pause the fetch (IO) thread
1529                        }
1530                        //LLVFSThread::sLocal->pause();
                                  // Prevent the VFS thread from running while rendering.
1531                        //LLLFSThread::sLocal->pause();
                                  // Prevent the LFS thread from running while rendering.</php>
 
==== Shutdown ====
 
<php>1561 bool LLAppViewer::cleanup()
( . . . )
1587        // End TransferManager before deleting systems it depends on (Audio, VFS, AssetStorage)
1588 #if 0 // this seems to get us stuck in an infinite loop...
1589        gTransferManager.cleanup();
1590 #endif
( . . . )
1662        // Wait for any pending VFS IO
1663        while (1)
1664        {
1665                S32 pending = LLVFSThread::updateClass(0);
1666                pending += LLLFSThread::updateClass(0);
1667                if (!pending)
1668                {
1669                        break;
1670                }
1671                llinfos << "Waiting for pending IO to finish: " << pending << llendflush;
1672                ms_sleep(100);
1673        }
( . . . )
1729        //
1730        // Shut down the VFS's AFTER the decode manager cleans up (since it cleans up vfiles).
1731        // Also after viewerwindow is deleted, since it may have image pointers (which have vfiles)
1732        // Also after shutting down the messaging system since it has VFS dependencies
1733        //
1734        LLVFile::cleanupClass();
1735        llinfos << "VFS cleaned up" << llendflush;
( . . . )
1789        while(1)
1790        {
1791                S32 pending = 0;
1792                pending += LLAppViewer::getTextureCache()->update(1); // unpauses the worker thread
1793                pending += LLAppViewer::getImageDecodeThread()->update(1); // unpauses the image thread
1794                pending += LLAppViewer::getTextureFetch()->update(1); // unpauses the texture fetch thread
1795                pending += LLVFSThread::updateClass(0);
1796                pending += LLLFSThread::updateClass(0);
( . . . )
1819        // This should eventually be done in LLAppViewer
1820        LLImageJ2C::closeDSO();
1821        LLImageFormatted::cleanupClass();
1822        LLVFSThread::cleanupClass();
1823        LLLFSThread::cleanupClass();
1824 
1825        llinfos << "VFS Thread finished" << llendflush;
1826 
1827 #ifndef LL_RELEASE_FOR_DOWNLOAD
1828        llinfos << "Auditing VFS" << llendl;
1829        gVFS->audit();
1830 #endif
1831 
1832        // For safety, the LLVFS has to be deleted *after* LLVFSThread. This should be cleaned up.
1833        // (LLVFS doesn't know about LLVFSThread so can't kill pending requests) -Steve
1834        delete gStaticVFS;
1835        gStaticVFS = NULL;
1836        delete gVFS;
1837        gVFS = NULL; </php>
 
=== /newview/llappviewer.h ===
 
<php>287 // VFS globals - gVFS is for general use
288 // gStaticVFS is read-only and is shipped w/ the viewer
289 // it has pre-cache data like the UI .TGAs
290 extern LLVFS    *gStaticVFS; </php>
 


== General - Window Management and Render Pipeline  ==
== General - Window Management and Render Pipeline  ==

Revision as of 18:32, 15 June 2008

This article is a work in progress.


Please do not edit.



This article is a work in progress.


Please do not edit.


This page is intended to have two different purposes.

  • Document where the VFS is used in the viewer source
  • Provide a direction forward for removing/replacing the VFS


The following list of source references is based on viewer source released 1.19.1.4 in the SVN repository, and will need future updating as the viewer source continues to evolve:

This listing duplicates the layout and format of the Viewer Source Files page, providing a direct 1-to-1 correspondence with each section and source file. Although the source list only includes .cpp files, if a matching .h file is found it is listed here below the .cpp for that name.

Each column of the table refers to: 

* "VFS" - any reference to this keyward. "VFS" is not used in the code alone, and so is usually referring to comments in the source, or debugging/logging notes.
* "gVFS" - the "general VFS" which is used for temporary storage for assets and is the primary problem with the viewer as currently designed. This needs to be removed and replaced with direct filesystem calls to folder-based storage.
* "gStaticVFS" - a read-only VFS which makes up about 75% of the client download. This contains the graphical elements and sounds of the Second Life user interface. It's small but why does it have to be fully loaded in a virtual RAM-disk all the time using limited system memory? May be worthwhile to replace this with direct on-disk storage, perhaps in a ZIP file.
* LFS - the "Local File System", used to directly read and write to the local OS storage. The audio manager for example uses LFS calls to write decoded OGG sounds as WAV files in the cache directory. LFS calls might be usable to replace the VFS as mentioned above.

General - Main and Startup

file VFS gVFS gStaticVFS LFS
llstartup.cpp
X
 gVFS::no
X
(check) 		LFS::no
llappviewer.cpp
X
 gVFS::no
X
(check) 		LFS::no
llappviewer.h
X
 gVFS::no
X
(check) 		LFS::no

General - Window Management and Render Pipeline

file VFS gVFS gStaticVFS LFS
pipeline.cpp   gVFS::no   LFS::no
llviewerdisplay.cpp   gVFS::no   LFS::no
llviewerprecompiledheaders.cpp   gVFS::no   LFS::no
llviewerprecompiledheaders.h
X
 gVFS::no   LFS::no
llviewerwindow.cpp   gVFS::no   LFS::no
llwindebug.cpp   gVFS::no   LFS::no

General - Utility and Misc

file VFS gVFS gStaticVFS LFS
llbbox.cpp   gVFS::no   LFS::no
llbox.cpp   gVFS::no   LFS::no
llcallbacklist.cpp   gVFS::no   LFS::no
llcontroldef.cpp
X
 gVFS::no   LFS::no
lleventnotifier.cpp   gVFS::no   LFS::no
lleventpoll.cpp   gVFS::no   LFS::no
llhippo.cpp   gVFS::no   LFS::no
llregionposition.cpp   gVFS::no   LFS::no
moviemaker.cpp   gVFS::no   LFS::no
noise.cpp   gVFS::no   LFS::no

Rendering - Asset Management

file VFS gVFS gStaticVFS LFS
llassetuploadresponders.cpp
X
 gVFS::no   LFS::no
lltexturecache.cpp   gVFS::no   LFS::no
lltexturefetch.cpp   gVFS::no   LFS::no
llviewerassetstorage.cpp
X
 gVFS::no   LFS::no
llwearable.cpp
X
 gVFS::no   LFS::no
llwearablelist.cpp   gVFS::no   LFS::no

Rendering - Character and Animation

file VFS gVFS gStaticVFS LFS
lldriverparam.cpp   gVFS::no   LFS::no
llemote.cpp   gVFS::no   LFS::no
llviewergesture.cpp   gVFS::no   LFS::no
llvoavatar.cpp
X
 gVFS::no
X
 LFS::no
llcharacter(???)

Rendering - Camera

file VFS gVFS gStaticVFS LFS
llcameraview.cpp   gVFS::no   LFS::no
llfollowcam.cpp   gVFS::no   LFS::no
llmorphview.cpp   gVFS::no   LFS::no
llviewercamera.cpp   gVFS::no   LFS::no

Rendering - Effects

file VFS gVFS gStaticVFS LFS
llhudconnector.cpp   gVFS::no   LFS::no
llhudeffect.cpp   gVFS::no   LFS::no
llhudeffectbeam.cpp   gVFS::no   LFS::no
llhudeffectlookat.cpp   gVFS::no   LFS::no
llhudeffectpointat.cpp   gVFS::no   LFS::no
llhudeffecttrail.cpp   gVFS::no   LFS::no
llhudicon.cpp   gVFS::no   LFS::no
llhudmanager.cpp   gVFS::no   LFS::no
llhudobject.cpp   gVFS::no   LFS::no
llhudrender.cpp   gVFS::no   LFS::no
llhudtext.cpp   gVFS::no   LFS::no
llhudview.cpp   gVFS::no   LFS::no
lltracker.cpp   gVFS::no   LFS::no
llvelocitybar.cpp   gVFS::no   LFS::no
llvotextbubble.cpp   gVFS::no   LFS::no

Rendering - Environment

file VFS gVFS gStaticVFS LFS
llvlmanager.cpp   gVFS::no   LFS::no
llvlcomposition.cpp   gVFS::no   LFS::no
llvoclouds.cpp   gVFS::no   LFS::no
llvosurfacepatch.cpp   gVFS::no   LFS::no
llvoground.cpp
X
 gVFS::no   LFS::no
llvosky.cpp   gVFS::no   LFS::no
llvostars.cpp   gVFS::no   LFS::no
llvowater.cpp   gVFS::no   LFS::no
llwind.cpp   gVFS::no   LFS::no

Rendering - Viewer Objects: Volumes, Particles, and Trees

file VFS gVFS gStaticVFS LFS
llvocache.cpp   gVFS::no   LFS::no
llvograss.cpp   gVFS::no   LFS::no
llvoinventorylistener.cpp   gVFS::no   LFS::no
llvopartgroup.cpp   gVFS::no   LFS::no
llvotree.cpp   gVFS::no   LFS::no
llvovolume.cpp   gVFS::no   LFS::no

Rendering - Render Pipeline

file VFS gVFS gStaticVFS LFS
llcloud.cpp   gVFS::no   LFS::no
llcubemap.cpp   gVFS::no   LFS::no
llcylinder.cpp   gVFS::no   LFS::no
lldrawable.cpp   gVFS::no   LFS::no
lldrawpool.cpp   gVFS::no   LFS::no
lldrawpoolalpha.cpp   gVFS::no   LFS::no
lldrawpoolavatar.cpp   gVFS::no   LFS::no
lldrawpoolbump.cpp   gVFS::no   LFS::no
lldrawpoolclouds.cpp   gVFS::no   LFS::no
lldrawpoolground.cpp   gVFS::no   LFS::no
lldrawpoolsimple.cpp   gVFS::no   LFS::no
lldrawpoolsky.cpp   gVFS::no   LFS::no
lldrawpoolstars.cpp   gVFS::no   LFS::no
lldrawpoolterrain.cpp   gVFS::no   LFS::no
lldrawpooltree.cpp   gVFS::no   LFS::no
lldrawpoolwater.cpp   gVFS::no   LFS::no
lldynamictexture.cpp   gVFS::no   LFS::no
llface.cpp   gVFS::no   LFS::no
llfeaturemanager.cpp   gVFS::no   LFS::no
llflexibleobject.cpp   gVFS::no   LFS::no
llglsandbox.cpp   gVFS::no   LFS::no
llglslshader.cpp   gVFS::no   LFS::no
llpatchvertexarray.cpp   gVFS::no   LFS::no
llpolymesh.cpp   gVFS::no   LFS::no
llpolymorph.cpp   gVFS::no   LFS::no
llsky.cpp   gVFS::no   LFS::no
llspatialpartition.cpp   gVFS::no   LFS::no
llsphere.cpp   gVFS::no   LFS::no
llsprite.cpp   gVFS::no   LFS::no
llsurface.cpp   gVFS::no   LFS::no
llsurfacepatch.cpp   gVFS::no   LFS::no
lltexlayer.cpp
X
 gVFS::no   LFS::no
llviewerimage.cpp
X
 gVFS::no   LFS::no
llviewerimagelist.cpp
X
 gVFS::no   LFS::no
llviewerjoint.cpp   gVFS::no   LFS::no
llviewerjointattachment.cpp   gVFS::no   LFS::no
llviewerjointmesh.cpp   gVFS::no   LFS::no
llviewerjointshape.cpp   gVFS::no   LFS::no
llviewerlayer.cpp   gVFS::no   LFS::no
llviewerobject.cpp   gVFS::no   LFS::no
llviewerobjectlist.cpp   gVFS::no   LFS::no
llviewerpartsim.cpp   gVFS::no   LFS::no
llviewerpartsource.cpp   gVFS::no   LFS::no
llviewertextureanim.cpp   gVFS::no   LFS::no
llviewervisualparam.cpp   gVFS::no   LFS::no

Input and Output - Audio

file VFS gVFS gStaticVFS LFS
fmodwrapper.cpp   gVFS::no   LFS::no
llaudiosourcevo.cpp   gVFS::no   LFS::no
llaudiostatus.cpp   gVFS::no   LFS::no

Input and Output - Input

file VFS gVFS gStaticVFS LFS
llviewerjoystick.cpp   gVFS::no   LFS::no
llviewerkeyboard.cpp   gVFS::no   LFS::no

Data and Communication - Agents

file VFS gVFS gStaticVFS LFS
llagent.cpp   gVFS::no   LFS::no
llagentdata.cpp   gVFS::no   LFS::no
llagentpilot.cpp   gVFS::no   LFS::no

Data and Communication - Region and World

file VFS gVFS gStaticVFS LFS
llviewerregion.cpp   gVFS::no   LFS::no
llworld.cpp   gVFS::no   LFS::no

Data and Communication - Network

file VFS gVFS gStaticVFS LFS
llasynchostbyname.cpp   gVFS::no   LFS::no
llcaphttpsender.cpp   gVFS::no   LFS::no
llurl.cpp   gVFS::no   LFS::no
llurlwhitelist.cpp   gVFS::no   LFS::no
lluserauth.cpp   gVFS::no   LFS::no
llviewergenericmessage.cpp   gVFS::no   LFS::no
llviewermessage.cpp
X
 gVFS::no   LFS::no
llviewermessage.h
X
 gVFS::no   LFS::no
llviewernetwork.cpp   gVFS::no   LFS::no
llviewerthrottle.cpp   gVFS::no   LFS::no
llweb.cpp   gVFS::no   LFS::no
llxmlrpctransaction.cpp   gVFS::no   LFS::no

User Interface - Alerts

file VFS gVFS gStaticVFS LFS
llconfirmationmanager.cpp   gVFS::no   LFS::no
llfirstuse.cpp   gVFS::no   LFS::no
llnotify.cpp   gVFS::no   LFS::no

User Interface - Assets

file VFS gVFS gStaticVFS LFS
llcompilequeue.cpp
X
 gVFS::no   LFS::no
llcompilequeue.h
X
 gVFS::no   LFS::no
llfloateranimpreview.cpp
X
 gVFS::no   LFS::no
llfloatergesture.cpp   gVFS::no   LFS::no
llfloaterimagepreview.cpp   gVFS::no   LFS::no
llfloaterinspect.cpp   gVFS::no   LFS::no
llfloateropenobject.cpp   gVFS::no   LFS::no
llfloaterproperties.cpp   gVFS::no   LFS::no
llfloaterscriptdebug.cpp   gVFS::no   LFS::no
llpreview.cpp   gVFS::no   LFS::no
llpreviewanim.cpp   gVFS::no   LFS::no
llpreviewgesture.cpp
X
 gVFS::no   LFS::no
llpreviewgesture.h
X
 gVFS::no   LFS::no
llpreviewlandmark.cpp   gVFS::no   LFS::no
llpreviewnotecard.cpp
X
 gVFS::no   LFS::no
llpreviewnotecard.h
X
 gVFS::no   LFS::no
llpreviewscript.cpp
X
 gVFS::no   LFS::no
llpreviewscript.h
X
 gVFS::no   LFS::no
llpreviewsound.cpp   gVFS::no   LFS::no
llpreviewtexture.cpp   gVFS::no   LFS::no

User Interface - Avatar Appearance

file VFS gVFS gStaticVFS LFS
llfloaterclothing.cpp   gVFS::no   LFS::no
llfloatercustomize.cpp   gVFS::no   LFS::no
llgenepool.cpp   gVFS::no   LFS::no
llgesturemgr.cpp
X
 gVFS::no   LFS::no
llgesturemgr.h
X
 gVFS::no   LFS::no
llpanelmorph.cpp   gVFS::no   LFS::no

User Interface - Chat, Instant Messaging, Notices, Reports

file VFS gVFS gStaticVFS LFS
llchatbar.cpp   gVFS::no   LFS::no
llfloaterchat.cpp   gVFS::no   LFS::no
llfloatermute.cpp   gVFS::no   LFS::no
llfloaternewim.cpp   gVFS::no   LFS::no
llfloaterpostcard.cpp
X
 gVFS::no   LFS::no
llfloaterreporter.cpp
X
 gVFS::no   LFS::no
llgroupmgr.cpp   gVFS::no   LFS::no
llgroupnotify.cpp   gVFS::no   LFS::no
llimpanel.cpp   gVFS::no   LFS::no
llimview.cpp   gVFS::no   LFS::no
lllogchat.cpp   gVFS::no   LFS::no
llmutelist.cpp   gVFS::no   LFS::no

User Interface - Data

file VFS gVFS gStaticVFS LFS
llclassifiedinfo.cpp   gVFS::no   LFS::no
lleventinfo.cpp   gVFS::no   LFS::no
llfloaterabout.cpp   gVFS::no   LFS::no
llfloaterauction.cpp
X
 gVFS::no   LFS::no
llfloateravatarinfo.cpp   gVFS::no   LFS::no
llfloatergroupinfo.cpp   gVFS::no   LFS::no
llfloatergroupinvite.cpp   gVFS::no   LFS::no
llfloatergroups.cpp   gVFS::no   LFS::no
llfloaterland.cpp   gVFS::no   LFS::no
llfloaterlandholdings.cpp   gVFS::no   LFS::no
llfloaterregioninfo.cpp
X
 gVFS::no   LFS::no
llfloaterregioninfo.h
X
 gVFS::no   LFS::no
llfloatertelehub.cpp   gVFS::no   LFS::no
llfloatertos.cpp   gVFS::no   LFS::no
llfloatertos.h
X
 gVFS::no   LFS::no
llpanelavatar.cpp   gVFS::no   LFS::no
llpanelclassified.cpp   gVFS::no   LFS::no
llpanelevent.cpp   gVFS::no   LFS::no
llpanelgroup.cpp   gVFS::no   LFS::no
llpanelgroupgeneral.cpp   gVFS::no   LFS::no
llpanelgroupinvite.cpp   gVFS::no   LFS::no
llpanelgrouplandmoney.cpp   gVFS::no   LFS::no
llpanelgroupnotices.cpp   gVFS::no   LFS::no
llpanelgrouproles.cpp   gVFS::no   LFS::no
llpanelgroupvoting.cpp   gVFS::no   LFS::no
llpanelland.cpp   gVFS::no   LFS::no
llpanellandobjects.cpp   gVFS::no   LFS::no
llpanellandoptions.cpp   gVFS::no   LFS::no

User Interface - Data:Inventory

file VFS gVFS gStaticVFS LFS
llcallingcard.cpp   gVFS::no   LFS::no
llfloaterfriends.cpp   gVFS::no   LFS::no
llinventoryactions.cpp   gVFS::no   LFS::no
llinventorybridge.cpp   gVFS::no   LFS::no
llinventoryclipboard.cpp   gVFS::no   LFS::no
llinventorymodel.cpp   gVFS::no   LFS::no
llinventoryview.cpp   gVFS::no   LFS::no
lllandmarklist.cpp
X
 gVFS::no   LFS::no
lllandmarklist.h
X
 gVFS::no   LFS::no
llviewerinventory.cpp   gVFS::no   LFS::no

User Interface - Data:Money and Land

file VFS gVFS gStaticVFS LFS
llcurrencyuimanager.cpp   gVFS::no   LFS::no
llgivemoney.cpp   gVFS::no   LFS::no
llfloaterbuy.cpp   gVFS::no   LFS::no
llfloaterbuycontents.cpp   gVFS::no   LFS::no
llfloaterbuycurrency.cpp   gVFS::no   LFS::no
llfloaterbuyland.cpp   gVFS::no   LFS::no
llfloatersellland.cpp   gVFS::no   LFS::no

User Interface - Data:Search

file VFS gVFS gStaticVFS LFS
llfloaterdirectory.cpp   gVFS::no   LFS::no
llpaneldirbrowser.cpp   gVFS::no   LFS::no
llpaneldirclassified.cpp   gVFS::no   LFS::no
llpaneldirevents.cpp   gVFS::no   LFS::no
llpaneldirfind.cpp   gVFS::no   LFS::no
llpaneldirgroups.cpp   gVFS::no   LFS::no
llpaneldirland.cpp   gVFS::no   LFS::no
llpaneldirpeople.cpp   gVFS::no   LFS::no
llpaneldirplaces.cpp   gVFS::no   LFS::no
llpaneldirpopular.cpp   gVFS::no   LFS::no
llpanelpick.cpp   gVFS::no   LFS::no
llpanelplace.cpp   gVFS::no   LFS::no

User Interface - Data:Map

llcompass.cpp
file VFS gVFS gStaticVFS LFS
  gVFS::no   LFS::no
llfloatermap.cpp   gVFS::no   LFS::no
llfloaterworldmap.cpp   gVFS::no   LFS::no
llmapresponders.cpp   gVFS::no   LFS::no
llnetmap.cpp   gVFS::no   LFS::no
llworldmap.cpp   gVFS::no   LFS::no
llworldmapview.cpp   gVFS::no   LFS::no

User Interface - Objects and Edit Tools

file VFS gVFS gStaticVFS LFS
llhoverview.cpp   gVFS::no   LFS::no
llmanip.cpp   gVFS::no   LFS::no
llmaniprotate.cpp   gVFS::no   LFS::no
llmanipscale.cpp   gVFS::no   LFS::no
llmaniptranslate.cpp   gVFS::no   LFS::no
llfloaterbuildoptions.cpp   gVFS::no   LFS::no
llfloaterbump.cpp   gVFS::no   LFS::no
llfloatercolorpicker.cpp   gVFS::no   LFS::no
llfloaterpermissionsmgr.cpp   gVFS::no   LFS::no
llfloatertools.cpp   gVFS::no   LFS::no
llpanelcontents.cpp   gVFS::no   LFS::no
llpanelface.cpp   gVFS::no   LFS::no
llpanelgeneral.cpp   gVFS::no   LFS::no
llpanelinventory.cpp   gVFS::no   LFS::no
llpanelobject.cpp   gVFS::no   LFS::no
llpanelpermissions.cpp   gVFS::no   LFS::no
llselectmgr.cpp   gVFS::no   LFS::no
lltool.cpp   gVFS::no   LFS::no
lltoolbar.cpp   gVFS::no   LFS::no
lltoolbrush.cpp   gVFS::no   LFS::no
lltoolcomp.cpp   gVFS::no   LFS::no
lltooldraganddrop.cpp   gVFS::no   LFS::no
lltoolface.cpp   gVFS::no   LFS::no
lltoolfocus.cpp   gVFS::no   LFS::no
lltoolgrab.cpp   gVFS::no   LFS::no
lltoolgun.cpp   gVFS::no   LFS::no
lltoolindividual.cpp   gVFS::no   LFS::no
lltoolmgr.cpp   gVFS::no   LFS::no
lltoolmorph.cpp   gVFS::no   LFS::no
lltoolobjpicker.cpp   gVFS::no   LFS::no
lltoolpie.cpp   gVFS::no   LFS::no
lltoolpipette.cpp   gVFS::no   LFS::no
lltoolplacer.cpp   gVFS::no   LFS::no
lltoolselect.cpp   gVFS::no   LFS::no
lltoolselectland.cpp   gVFS::no   LFS::no
lltoolselectrect.cpp   gVFS::no   LFS::no
lltoolview.cpp   gVFS::no   LFS::no
llviewerparcelmgr.cpp   gVFS::no   LFS::no
llviewerparceloverlay.cpp   gVFS::no   LFS::no

User Interface - HTML

file VFS gVFS gStaticVFS LFS
llfloaterhtml.cpp   gVFS::no   LFS::no
llfloaterhtmlfind.cpp   gVFS::no   LFS::no
llfloaterhtmlhelp.cpp   gVFS::no   LFS::no

User Interface - Import, Export, Upload

file VFS gVFS gStaticVFS LFS
llfloaterimport.cpp   gVFS::no   LFS::no
llfloatersaveavatar.cpp   gVFS::no   LFS::no
llfloatersnapshot.cpp
X
 gVFS::no   LFS::no
lluploaddialog.cpp   gVFS::no   LFS::no

User Interface - Menus, Overlays, Login

file VFS gVFS gStaticVFS LFS
llmediaremotectrl.cpp   gVFS::no   LFS::no
llmenucommands.cpp   gVFS::no   LFS::no
llmoveview.cpp   gVFS::no   LFS::no
lloverlaybar.cpp   gVFS::no   LFS::no
llpanellogin.cpp   gVFS::no   LFS::no
llprogressview.cpp   gVFS::no   LFS::no
llviewermenu.cpp
X
 gVFS::no   LFS::no
llviewermenufile.cpp
X
 gVFS::no   LFS::no

User Interface - Preferences

file VFS gVFS gStaticVFS LFS
llfloaterpreference.cpp   gVFS::no   LFS::no
llpanelaudioprefs.cpp   gVFS::no   LFS::no
llprefschat.cpp   gVFS::no   LFS::no
llprefsim.cpp   gVFS::no   LFS::no
llpaneldisplay.cpp   gVFS::no   LFS::no
llpanelmsgs.cpp   gVFS::no   LFS::no
llpanelnetwork.cpp   gVFS::no   LFS::no
llpanelvolume.cpp   gVFS::no   LFS::no
llpanelweb.cpp   gVFS::no   LFS::no

User Interface - Utility

file VFS gVFS gStaticVFS LFS
llcolorscheme.cpp   gVFS::no   LFS::no
lldirpicker.cpp   gVFS::no   LFS::no
llfilepicker.cpp   gVFS::no   LFS::no
llfloateravatarpicker.cpp   gVFS::no   LFS::no
llfloaternamedesc.cpp   gVFS::no   LFS::no
llsavedsettingsglue.cpp   gVFS::no   LFS::no
llviewchildren.cpp   gVFS::no   LFS::no

User Interface - Widgets

file VFS gVFS gStaticVFS LFS
llcolorswatch.cpp   gVFS::no   LFS::no
llcontainerview.cpp   gVFS::no   LFS::no
llfolderview.cpp   gVFS::no   LFS::no
llnamebox.cpp   gVFS::no   LFS::no
llnameeditor.cpp   gVFS::no   LFS::no
llnamelistctrl.cpp   gVFS::no   LFS::no
lltexturectrl.cpp   gVFS::no   LFS::no
lljoystickbutton.cpp   gVFS::no   LFS::no
llviewertexteditor.cpp   gVFS::no   LFS::no
llvieweruictrlfactory.cpp   gVFS::no   LFS::no
llwebbrowserctrl.cpp   gVFS::no   LFS::no

User Interface - Debug

file VFS gVFS gStaticVFS LFS
llconsole.cpp   gVFS::no   LFS::no
lldebugmessagebox.cpp   gVFS::no   LFS::no
lldebugview.cpp   gVFS::no   LFS::no
llfasttimerview.cpp   gVFS::no   LFS::no
llframestats.cpp   gVFS::no   LFS::no
llframestatview.cpp   gVFS::no   LFS::no
llmemoryview.cpp   gVFS::no   LFS::no
llfloateravatartextures.cpp   gVFS::no   LFS::no
llfloatereditui.cpp   gVFS::no   LFS::no
llfloatergodtools.cpp   gVFS::no   LFS::no
llfloatertest.cpp   gVFS::no   LFS::no
llfloatertopobjects.cpp   gVFS::no   LFS::no
llpaneldebug.cpp   gVFS::no   LFS::no
llstatbar.cpp   gVFS::no   LFS::no
llstatgraph.cpp   gVFS::no   LFS::no
llstatusbar.cpp   gVFS::no   LFS::no
llstatview.cpp   gVFS::no   LFS::no
lltextureview.cpp   gVFS::no   LFS::no
llvelocitybar.cpp   gVFS::no   LFS::no
llviewercontrol.cpp   gVFS::no   LFS::no
llviewerstats.cpp
X
 gVFS::no   LFS::no
llviewerstats.h
X
 gVFS::no   LFS::no


Reasons for VFS removal/replacement

As has been discussed in the sl-dev mailing list and in the JIRA, the VFS does not scale up very well at all since it functions as a private RAM-disk for non-texture assets downloaded by the viewer. The on-disk VFS files are used only for storage when the viewer is not running. At startup, the files are parsed for errors and loaded into memory, and continue to occupy memory until the viewer exits and the contents are written back to the disk files.

As a RAM disk, increasing the cache size to larger than the available free memory causes the computer to run out of physical RAM and require the use of virtual memory and disk swapping. This is counterproductive when the intent of increasing the VFS size is to cache more of the frequently-used data to speed up the client and reduce network traffic.

In order to permit the cache of assets currently stored in the VFS to grow to gigabytes of storage without requiring gigabytes of physical memory to store it, the existing VFS will have to be removed from the viewer and replaced with a traditional file/directory-based disk cache.

In the interests of copyright protection, a directory-based non-texture asset cache should include some form of low-impact data obfuscation, to make it slightly difficult for the copyrighted works of other SL users held in the cache to be casually copied and examined.

Retrieved from "https://wiki.secondlife.com/w/index.php?title=Local_data_storage_methods_used_in_the_viewer&oldid=72239"