cURL'd

This is one of those inside baseball posts. On my main page, I have two things that everyone ignores on the sidebar: Word of the Day and Your Random Weather Report. They both use a script to grab data from external sites—Wordsmith.org in the former and the National Weather Service in the latter.

Lately, they’ve been slowing the load time of my site down significantly. It’s taken 13 to 14 seconds for my home page to competely load, which is ridiculous over a cable modem. Sometimes, the sites that I grab the data from have slow downs, but those are temporary and never last more than a day or so, but my home page has been loading slowly for at least two weeks.

I could have just disabled the two scripts, which I eventually did, but I thought that I could figure out the problem, and, stubbornly, let my page load slowly for days.

Since both scripts were slow, I knew that the sites themselves weren’t to blame. Also, since I have the Google Ads, which loaded fine, I knew that it wasn’t my hosting company throttling external connections. It was something with cURL, which is a method for calling external data into a script. If anyone is having a problem with cURL, this is what helped in my case:

If you have a url in the form of a domain name, e.g. wordsmith.org, cURL has to look up the IP address via a lookup table. Sometimes, as in my shared hosting situation, those tables get corrupted or really large or whatever, and it takes cURL some 10 seconds to look up the address. The solution is to give cURL the IP address in the script, e.g. 216.12.219.209. Then there is no delay from resolving the IP address. It’s exceedingly logical, but I didn’t think of it, since I tend to ignore IP addresses. I found it on a discussion site.

This is a really boring post, but I’m hopeful that someone on the great big internets will find it helpful.

Posted by Jonathan at 05:05 PM, 17 May 2008 | Comments (0)