Comments | cloquewerk: Automated Speed Tests!

cloquewerk

Automated Speed Tests!

Jul 12, 2011 10:22

It's hard to find a discussion of the speed of modern browsers that doesn't mention Microsoft's Test Drive speed demos. It's a common occurrence to find hundreds of fish swimming around a graphics developer's monitor. Continuing our mission to make developers' lives easier, the Mozilla A-Team has put together a framework to automatically run a ( Read more... )

mozilla

Comments 8

Maze Solver Test anonymous July 12 2011, 18:59:34 UTC

One test where Firefox is particualrly slow is the "Maze Solver" test (http://ie.microsoft.com/testdrive/Performance/MazeSolver/Default.html). How about including it in the monitored group?

Re: Maze Solver Test cloquewerk July 12 2011, 20:28:57 UTC

Sure, will do!

Java? Seriously? anonymous July 12 2011, 20:37:51 UTC

I get this error for every link in the results page:
java.lang.IllegalStateException: The viewing session is not available or has expired.

Please tell me I'm not expected to install Java just to view some results.

Re: Java? Seriously? cloquewerk July 12 2011, 20:45:34 UTC

No, you shouldn't need Java, though indeed the back end is Java-based (using BIRT). I haven't seen this error before, and I've tried it out on many machines and OSs. I'll do some research and see if I can find out what this error means exactly.

I was also thinking about ditching this reporting software and rolling my own simple JS-based site, but I'm not sure when I'll have time to do so.

Great! anonymous July 12 2011, 21:12:44 UTC

Thanks!!! This is great.

It's a bit hard to read the graphs where the symbols overlap. Maybe using symbols of different shapes would help?

It would be great to add Chrome dev and IE10.

I wonder why we're slower than IE9 on FishIE on your test setup. We're faster on my machine. What hardware are you using?

-- roc

Re: Great! cloquewerk July 12 2011, 21:51:33 UTC

Glad you like it. :)

Yeah the UI is not amazing. I'll see about using different shapes, although I may be redoing it all at some point. Meanwhile, you can actually turn on and off the display for the individual plots by clicking on the browser names in the legend. That should make it easier to see what's there.

I'll look into adding Chrome dev and IE 10. Shouldn't be too hard.

Yeah the results are pretty machine dependent. This machine is an Intel computer with a Clarkdale processor and 4 GB of RAM running 32-bit Windows 7 (I think it's using on-board video). We can always look into setting up other machines with different configurations if you think it would be helpful.

60 fps? anonymous July 14 2011, 21:59:59 UTC

"However some tests report FPSs above 60, which means they must be using some sort of "virtual" frame rate, since no monitor can display that much"

You need better monitors...

Admittedly, a lot of modern LCD monitors derived from glass for TV's is set to 60fps; but I always used to run 85fps - much more pleasant for text scrolling, etc than 60, especially on a CRT. (CRT? Oh, yeah, that's a huge vacuum bottle with phosphors on the front...)

-- Randell Jesup

Re: 60 fps? cloquewerk July 14 2011, 22:12:04 UTC

Robert O'Callahan has discussed the 50-60 fps limit.

Regardless, I think you'd agree that there is some limit to how fast a monitor can refresh, and at least sometimes it is the limiting factor in how fast a sequence is displayed. My point was that some tests seem to measure how many frames are actually displayed, and some seem to measure how many frames could potentially be displayed given an unlimited refresh rate. In the former, once that limit is hit, the comparison between browsers is impossible. One solution is to make these tests more complicated (essentially do more of what they are currently doing).