One of the key challenges in conducting an objective evaluation of software performance across devices that utilize different operating systems lies in accounting for the fundamental differences in the various OSs. While the iPhone 3G and 3GS could potentially run the same app on the same Objective-C-based operating system (making direct comparisons relatively straightforward), Android apps are Java-based, and the Palm Pre runs the entirely new Web OS. Given these divergent OS implementations, is there anything that come close to a standard unit of measure for judging performance of this growing breed of “superphones?”
Finding Common Ground
Medialets ran the SunSpider test suite in the following environments:
- Safari 4.0.1 on a 2.0 GHz Intel Core 2 Duo White MacBook.
The MacBook results were used as a baseline for relative comparisons.
- Mobile Safari on the iPhone 3G with iPhone OS v2.2.1
- Mobile Safari on the iPhone 3G with iPhone OS v3.0
- Mobile Safari on the iPhone 3GS with iPhone OS v3.0
- The “Browser” app on the T-Mobile G1 with Android OS v1.5 (Cupcake)
- The “Web” app on the Palm Pre with Web OS v1.0.2
Do any of these numbers really indicate which phone might be the best choice for a given individual? Absolutely not. At Medialets we use all of these devices, and love each one for many reasons. The fact that these tests can even be performed across this many device/OS combinations is a testament to how far mobile technology has come in such a relatively short time. We are looking forward to seeing an even greater variety of advanced mobile devices and OS revisions enter the market and we’ll keep you posted as we test more devices in our lab. Subscribe to our feed, leave a comment below, or reach out to us directly if you have any questions.