PHP vs .NET Performance part 2
In my first article about .NET vs PHP performance I reported some surprising results.
In a simple benchmark involving a long loop and an incrementing integer, .NET came out around 50 times faster than PHP. Quite a disappointment for PHP fans until I discovered that PHP was 10 times faster than .NET in a similar string manipulation test.
These simple benchmarks produced some interesting results but were not conclusive since real world web application performance is not so simple and relies on many other factors.
To get a better idea of what is going on, I have developed a more realistic benchmark and give my results below.
As before, I have run my benchmark programs on my Core2 Duo 8GB desktop with a 250GB solid state disk. The machine dual boots into Linux, Apache, MySQL, PHP (LAMP) for the PHP test and Windows Vista, IIS, SQL Server, C# (WISC) for the .NET test.
The benchmarks are still quite simple but this time each page executes a loop 100 times, and each time through the loop executes a SQL query that returns 100 rows and outputs the first two rows of those results to a html table.
The code has been specifically developed to be cpu bottlenecked as the idea is to benchmark the processing power of the stack rather than to be bound by other factors such as memory, network bandwith or disk bandwidth.
In fairness, I think the benchmark programs are probably more database intensive than most web applications although 100 sql queries for a given page is far from uncommon, however I still make the caveat that situation in a real world application would be more complex.
I used JMeter to load the respective webservers with simulated requests from 100 concurrent users and measured the throughput and also the average page generation time.
I needed some test data in a sql table for my benchmark so I also created simple php and aspx scripts to load this data into the respective databases and measured how long this took to input 100,000 data rows.
|insert 100,000 rows||36 seconds||67 seconds|
|delete 100,000 rows||0.25 seconds||37 seconds|
Yes, the figures above are correct. I execute the delete from the command line in both cases and it took 0.25 seconds on mySQL and 37 seconds on SQL server. A huge difference !
I ran the import a few times in both environments with or without an alternate index on the table which didn't seem to make any noticeable difference. I also disabled transaction logging (enabled simple recovery mode) in SQL server which also didn't make any significant difference. Strangely, it seems to take 3 seconds longer with transaction logging turned off!
Also, I took my measurement *after* an initial import / delete sequence so as to not include any overhead from the database engine in having to grow it's disk space to accommodate another 100,000 rows.
Dynamic Page Generation
Generating a table from 100 database queries and 200 data rows. 100 concurrent users. All 4 cpu cores maxed and no "500" errors.
Since it turned out that .NET was so *slow*, I actually tried a few different code patterns to see if I could speed things up with some limited success however, .NET / SQL Server still got it's arse kicked ! I've no doubt that I could have increased the speed of the .NET code further still, but my aim was to use code patterns that are typically used by developers in the respective languages.
(requests per minute)
|Average Request Time|
|WISC pattern 1||164||32|
|WISC pattern 2||250||21|
|WISC pattern 3||270||19|
I must say I'm surprised by the result. In this benchmark PHP/LAMP is able to process more than twice as many requests as .NET/WISC and do so in half the time.
As mentioned , a real world web application is significantly more complex, and I suspect as more code / business logic / classes etc are added the balance of power would start to shift more towards .NET's camp.
It's worth noting the performance difference between the different .NET code patterns. Pattern 1 and Pattern 2 represent the most commonly used .NET patterns for turning database rows into html and I think it would be easy for a beginner (or even a seasoned programmer) to write code which created a serious performance problem. PHP on the other hand was easy to code, easy to understand and just delivered great performance.
Also worth noting that my PHP server was not running any kind of op-code cache which would make a significant improvement to performance - especially as the amount of code increased (ie in a real world app)
I still feel I have some more work to do to create a more realistic benchmark but for now I think I have demonstrated that PHP can compete with .NET for building scalable web applications.
If you'd like to take a look at my code or run the benchmarks for yourself, I have attached the code below. Please bear in mind though, my objective was not to solve the problem in the most efficient way possible, but to create a cpu intensive benchmark that used the typical design patterns in use by developers today (or as close as I could get to it).