PHP vs .NET Performance part 2

Introduction

In my first article about .NET vs PHP performance I reported some surprising results.

In a simple benchmark involving a long loop and an incrementing integer, .NET came out around 50 times faster than PHP. Quite a disappointment for PHP fans until I discovered that PHP was 10 times faster than .NET in a similar string manipulation test.

These simple benchmarks produced some interesting results but were not conclusive since real world web application performance is not so simple and relies on many other factors.

To get a better idea of what is going on, I have developed a more realistic benchmark and give my results below.

The Benchmark

As before, I have run my benchmark programs on my Core2 Duo 8GB desktop with a 250GB solid state disk. The machine dual boots into Linux, Apache, MySQL, PHP (LAMP) for the PHP test and Windows Vista, IIS, SQL Server, C# (WISC) for the .NET test.

The benchmarks are still quite simple but this time each page executes a loop 100 times, and each time through the loop executes a SQL query that returns 100 rows and outputs the first two rows of those results to a html table.

The code has been specifically developed to be cpu bottlenecked as the idea is to benchmark the processing power of the stack rather than to be bound by other factors such as memory, network bandwith or disk bandwidth.

In fairness, I think the benchmark programs are probably more database intensive than most web applications although 100 sql queries for a given page is far from uncommon, however I still make the caveat that situation in a real world application would be more complex.

I used JMeter to load the respective webservers with simulated requests from 100 concurrent users and measured the throughput and also the average page generation time.

I needed some test data in a sql table for my benchmark so I also created simple php and aspx scripts to load this data into the respective databases and measured how long this took to input 100,000 data rows.

The Results

Data Import

  LAMP WISC (.NET)
insert 100,000 rows       36 seconds 67 seconds
delete 100,000 rows     0.25 seconds 37 seconds

Yes, the figures above are correct. I execute the delete from the command line in both cases and it took 0.25 seconds on mySQL and 37 seconds on SQL server. A huge difference !

I ran the import a few times in both environments with or without an alternate index on the table which didn't seem to make any noticeable difference. I also disabled transaction logging (enabled simple recovery mode) in SQL server which also didn't make any significant difference. Strangely, it seems to take 3 seconds longer with transaction logging turned off!

Also, I took my measurement *after* an initial import / delete sequence so as to not include any overhead from the database engine in having to grow it's disk space to accommodate another 100,000 rows.

Dynamic Page Generation
Generating a table from 100 database queries and 200 data rows. 100 concurrent users. All 4 cpu cores maxed and no "500" errors.

Since it turned out that .NET was so *slow*, I actually tried a few different code patterns to see if I could speed things up with some limited success however, .NET / SQL Server still got it's arse kicked ! I've no doubt that I could have increased the speed of the .NET code further still, but my aim was to use code patterns that are typically used by developers in the respective languages.

  Throughput
(requests per minute)
Average Request Time
LAMP   550 8
WISC pattern 1   164 32
WISC pattern 2   250 21
WISC pattern 3   270 19

Conclusion
I must say I'm surprised by the result. In this benchmark PHP/LAMP is able to process more than twice as many requests as .NET/WISC and do so in half the time.

As mentioned , a real world web application is significantly more complex, and I suspect as more code / business logic / classes etc are added the balance of power would start to shift more towards .NET's camp.

It's worth noting the performance difference between the different .NET code patterns. Pattern 1 and Pattern 2 represent the most commonly used .NET patterns for turning database rows into html and I think it would be easy for a beginner (or even a seasoned programmer) to write code which created a serious performance problem. PHP on the other hand was easy to code, easy to understand and just delivered great performance.

Also worth noting that my PHP server was not running any kind of op-code cache which would make a significant improvement to performance - especially as the amount of code increased (ie in a real world app)

I still feel I have some more work to do to create a more realistic benchmark but for now I think I have demonstrated that PHP can compete with .NET for building scalable web applications.

The code
If you'd like to take a look at my code or run the benchmarks for yourself, I have attached the code below. Please bear in mind though, my objective was not to solve the problem in the most efficient way possible, but to create a cpu intensive benchmark that used the typical design patterns in use by developers today (or as close as I could get to it).

benchmarks code samples

Categories: 

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
To prevent automated spam submissions leave this field empty.