Thank you for your positive feedback.

I guess my description of the flashing (under technical details) could be better. We have a total of four test stages, only three of which are monitored with the lamps (the last via email).

First, the build server runs the unit tests. The foreground lamp will flash while this is happening. It flashes with a set frequency which never changes. (But the flashing turns yellow when the build is running longer than usual, which is a manual setting)

Second, every build is installed in a test environment and we run canned production data over it. The replay happens much faster than production, so we get some indication of how much load the system can handle. This test is not monitored with a lot of automation, just emails from the error logs.

Third, at the end of every iteration, we install a new version in the staging environment. The staging environment receives an asynchronous copy of all requests going into production. The environment is monitored by one of the lamps. We have looked at historical data to find the peak number of requests per 10 minute period. We have determined the fastest practical blink rate and decided that this should correspond to peak. The flash rate of the lamp is the current traffic relative to the peak.

Lastly, we monitor production in the same way as staging.

Was this clearer?

One thing I wanted to ask which wasn't clear from your post – does the frequency of the light flashing, for the one linked to CPU load relative to past measurements – is that the load on the build server or is that flashing load factor for the App while under a load test run?

Automated load/performance testing which can report back with some consistency is something we have often struggled with..