In the course of writing a little python command line RSS engine, I naturally came to a point where I needed to download the RSS feeds to store them and work with them.
My options looked like this:
- Use
urllib2.urlopenwhich would download the feeds serially. When you have hundreds of feeds in your opml file, that takes too long. Besides, what do I have ADSL for? - Use Twisted. But, my application is not a web app and frankly, switching the whole thing over to the Twisted event-driven model is overkill and unnecessarily complicated.
- Use wget with threads. The problem with that would have been that I'd have to jump through hoops to pass the data from the threads back to the main application, possibly with messy kludges like temporary files. No thanks!
- Use libcurl through the pycurl library. Ah, yes, perfect. Or so I thought.
