NBC.COM uses PHP?

PHP is good.

for some things, yes. and some of the problems commonly seen in PHP driven websites aren't due to the language itself but the culture of PHP developers. anyway, the bigger issue is that the script vomits like this when you have too many SQL connections. NBC is part of a huge-ass corporation; i'm sure the developers billed plenty of hours....

I got the same error message this morning; I think that millions of people were wanting to see the same SNL Fey video, and the site was swamped.

I'm a big fan of the LAMP combination (Linux, Apache, PHP, and MYSQL) but it doesn't scale well for massive usage. Works fine for me, though!

I'm a big fan of the LAMP combination (Linux, Apache, PHP, and MYSQL) but it doesn't scale well for massive usage.

my own exp. is that PHP & MYSQL are the ones introducing scalability problems in LAMP. though with MYSQL i don't think that's an issue if the people know what they're doing and are only pulling data via SELECTs. and even with PHP the scalability issues only kick in at a really high level if coded well...but i would say NBC.COM has enough traffic that yes, PHP would not scale well no matter what.

Just like any other language out there, PHP doesn't scale on it's own. But it using the memcached api and one of the many PHP caches avaiable it should scale decently. XCache worked wonders for me.

On scaling MySQL, I found this talk about scaling MySQL for youtube quote interesting.

The P in LAMP doesn't mean just PHP. There are very many sites out there that use PERL or Python as their scripting language.

ok, can i ask readers not to turn this into a language war? remember i'm moderating these comments....

You said...

"I'm a little surprised that such a high traffic corporate site is running with PHP."

I'm surprised you're surprised.

PHP is an extremely popular language for Web Development. It's everywhere!

PHP, ASP, and Java are the big 3 for web development.

Just out of curiosity, what would you expect the site to be written?

-- Charles Iliya Krempeaux
http://changelog.ca/

charles, i've done plenty of development in PHP, so yes, i know. i would expect java or something more "corporate." PHP might be common, but it is more prevalent on the "long tail" of the distribution. lots of lower to mid-level traffic sites.

@razib said...

"my own exp. is that PHP & MYSQL are the ones introducing scalability problems in LAMP. though with MYSQL i don't think that's an issue if the people know what they're doing and are only pulling data via SELECTs. and even with PHP the scalability issues only kick in at a really high level if coded well...but i would say NBC.COM has enough traffic that yes, PHP would not scale well no matter what."

Back when I was the principal engineer of an ad network, I dealt A LOT with scalability issues. (When you have a system that has to handle thousands and thousands of hits per second you have to.)

And we had to deal with all sorts of bottlenecks, and over come them.

Our system was written in PHP (along with Linux, Apache, and MySQL)....

And the language choice was never the problem.

(Nor was our problem the choice of database server or operating system.)

The problems we got were caused by architecture design.[1]

What you design to handle hundreds of hits per second is different than what you design to handle thousands of hits per second... which is different than what you design to handle hundreds of thousands of hits per second.

Some bottlenecks we got were[2]...
- slowness caused by JOINing of very very large tables
- slowness caused by database replication
- slowness caused by (Internet) network latency due to the "distance" between your server and your users.

We'd get various bottlenecks as we increased the level of traffic. And we had to come up with various system architecture changes to handle them.

As you scale to higher and higher levels of traffic, any database accesses becomes too "expensive". (And that's even if you aren't doing any table JOINs.)

As you scale you need to think about caching. (Both on disk and in memory.)

And this is something independent of the language choice. It doesn't matter what language you use... PHP, ASP, or Java... or database server you use... MySQL, Oracle, or MS SQL Server.

Scalability of web systems in a really interesting topic IMO. But this is probably the wrong audience to go into any real detail about it. (Nor do I really have the time to write great amounts about it. I've always been amazed [but very appreciative] at how prolific some writers and bloggers are.)

-- Charles Iliya Krempeaux
http://changelog.ca/

[1] Of course, you don't bother creating a super scalable system from the get go, because that takes time and money. (And you have other things to work on.) You engineer systems to be more and more scalable as you go and as you need it.

[2] You don't get all those bottlenecks at the same time. You encounter them as the amount of traffic you get increases. (Also... these aren't the only bottlenecks. Just some examples that are general enough that I don't have to go into too much detail to explain.)

i am willing to grant that the overwhelming issue in terms of scalability re: PHP vs. java has to do with the differences in programmers and programmer culture, and not the languages themselves (many "PHP programmers" are web design people who picked it up to do some basic things and sometimes get drafted into doing a whole more).

(and i hear you about JOINs!)

PHP is very common in high traffic sites. Flickr uses it, and it is the language in which most yahoo sites (including at least regional portals) are written.

I am always surprised when I remember that many large sites are written in VBScript, but that is a fact.

Well, I've been told by someone who knows that Wikipedia uses PHP...

I've also heard that Wikipedia gets a lot of traffic...though perhaps that's just a rumor or something...

@RKU, you said...

"Well, I've been told by someone who knows that Wikipedia uses PHP"

Yup. Wikipedia's software (called MediaWiki) definitely uses PHP. In fact, you can even download the source code...

http://www.mediawiki.org/wiki/Download

Systems created my many people are always less efficient than systems developed by just one person. It like a lesson out of The Fountainhead.

I doubt that the online dating site I'm developing (www.MatchStation.com) will have these problems.

@Half Sigma, you said...

"Systems created my [sic] many people are always less efficient than systems developed by just one person."

That's a pretty bold claim!

I'll give you the benefit of the doubt, and assume you meant: "systems created by many people tend to be less efficient than systems developed by just one person."

But I'd need some convincing to believe that's true in general.[1]

My experience[2] has been that, it depends on who is creating the system...
- who the developers are
- what their backgrounds are
- what experience they have
- how well do they work together as a team
- etc, etc

Now, don't take that to mean that I'm supporting developing software as a "Democracy". (I.e., "design by committee".)

I think you need someone in charge... even if development happens in an ad hoc, organic fashion (like you see in some shops that use Agile development methodologies).

So... having said all that... it's not to say that I haven't seen bad software development team... but still, I've seen a number of really good software development teams that produce efficient systems.

But this seems correlated to how "good" the team is. (And not the size of the team.) I.e., the efficiency of the software produced seems to be correlated with the "quality" of the people creating the software and not how many people there are.

You just have to hire the right people. (And that's a skill in itself.)

-------------------------

[1] Not that you have to spend your time convincing me of course :-)

[2] And yes, I know, that's just my experience. And it doesn't make my claims any more true.