Simulator statistics

How do I read the sim performance stats

The 1.7 release of Second Life includes some fairly substantial changes to server operation that should greatly improve overall simulator performance and reduce lag. The following document explains the differences, and how to read the new performance statistics on the simulator when debugging performance and load issues.

- Doug Linden

Overview

There has been a major redesign in how the simulator operates in the 1.7 release. In previous releases, the simulator would run as quickly as it could, and then only run physics at a 45 frames per second. This meant that the simulator would run at very high frame rates (in the thousands) when there was no load, which was not a very efficient mode of operation. In addition, when there was a lot of scripts or many people connected to the simulator, the frame rate would drop and fluctuate as many scripts attempted to run on every frame.

In the following discussion an "agent" is either a user inside of a given simulator region (a "main agent") or a user in a neighboring simulator region (a "child agent"). Any user who can see objects inside a simulator region increases the load on that simulator.

In 1.7, the simulator scheduler has now been redesigned so that it now attempts to run at a fixed frame rate which is the same as the physics frame rate. Instead of the simulator frame rate changing as load changes, the simulator will spend more or less time running LSL scripts each frame. This means that if there are a large number of scripts on the simulator, the frame rate will stay the same, but all of the scripts will run less frequently. Only in situations of particularly heavy physics load or large numbers of agents will the simulator frame rate drop from 45 frames/second.

In addition, there has been a major redesign of the logic which is used to transmit objects from the simulator to the viewers (called the interest list). The resulting improvements reduce the amount of load incurred when there are many agents on the same simulator.

The combination of the above two changes should result in significantly decreased "lag", and improved performance when many agents are on a simulator.

Reading the numbers

Because of the changes to the simulator execution model, most of the statics which were used to evaluate simulator performance have been changed. Below is a detailed summary of what the new simulator performance statistics mean.

All of the numbers below are for simulators running on a single simulator per CPU. On simulators which are not running in this fashion (water simulators, etc), these number will be different.

The following are the different statistics in the Simulator section of the statistics pane (accessed via Ctrl-shift-1 or the Debug menu):

• Time Dilation - The physics simulation rate relative to real-time. 1.0 means that the simulator is running at full speed, 0.5 means that physics is running at half-speed.
• Sim FPS - The simulator frame rate. This should now always be the same as the physics frame rate - 45.0 when things are running well.
• Physics FPS - The frame rate that the physics engine is running at. This should normally be at or near 45.0.
• Agent Updates/Sec - The rate at which agents on this simulator are being updated. Normally 20 updates a second, this will decrease if the simulator has a large number of agents on it.
• Main Agents - The number of agents (users) who are on this simulator.
• Child Agents - The number of agents who are not on this simulator, but can see it.
• Objects - The total number of objects/primitives on the simulator
• Active Objects - The number of moving and/or changing objects on the simulator.
• Active Scripts - The number of running scripts that are currently on the simulator, including scripts attached to agents.
• Script Perf - Number of LSL opcodes being executed a second by the simulator. Note that this is the number of ACTUAL instructions executed in the * last second, not the theoretical maximum opcodes/second. If your simulator is not running very many scripts, this number will be low even if performance is good.
• Packets In - UDP packets being received by the simulator
• Packets Out - UDP packets being sent by the simulator
• Pending Downloads - Number of asset downloads to the simulator that are pending. If this is greater than 1, this means that you may see delays in viewing notecards or scripts, and rezzing objects.
• Pending Uploads - Number of current uploads of asset data pending. If this number is non-zero, this means that there may be performance issues when attempting to teleport.

The following are the different times listed in the Simulator/Time(ms) submenu of the statistics pane:

• Total Frame Time - This measures how much time it takes the simulator to run everything that the simulator is trying to do each frame.
• < 22 ms - the simulator is healthy, everything is running as fast as it can, and more scripts can be added without reducing the performance of individual scripts.
• approx. 22 ms - the simulator is healthy, but there are probably a lot of scripts and agents on the simulator, meaning that script execution is being slowed down in order to maintain the simulator frame rate.
• > 22 ms - the simulator is experiencing severe load, either due to physics or a large number of agents, such that even by slowing down script execution it is impossible to compensate. The simulator frame rate has been reduced as a result.* Net Time - The amount of time spent responding to incoming network data.
• Sim Time (Physics) - The amount of time that frame spent running physics simulation. In general, this should be less than 5 milliseconds.
• Sim Time (Other) - The amount of time that frame spent running other simulation (agent movement, weather simulation, etc.)
• Agent Time - The amount of time spent updating and transmitting object data to the agents.
• Images Time - The amount of time spent updating and transmitting image data to the agents.
• Script Time - The amount of time spent running scripts.