Including remote files, part 2: what if you're with Movable Type/CGI?

This was one of those ARGH moments where the first reaction is that you want to put a bullet in your head for being so stupid and making an architecture decision that turns out to be incompatible with the rest of your system. But you get over this quickly and start going down the “there must be more than one way to do this” path. If one way fails, there must surely be another one.

So — can’t we really use this method of caching on Movable Type blogs and dynamically built pages? Turns out we can. Quite simply.

Suppose you have a module called “header” which goes like

<?php
echo get_remote_file("http://www.example.com/somefile.html");
?>

Now if you just included this module into your comment preview template, it would be output as-is. And this is not really what you want.

There’s a way out here which may have its downsides but as usual, it just works. The template tag can include a “file” parameter which is just a file in your local filesystem. See where this is going?

So you just create a module “header-static” which can be otherwise similar to “header”, but the content goes just

<$MTInclude file="/wherever/you/cache/the/files/84278972497abfcd..."$>

So… assuming that people don’t see the dynamic page upfront, there’s always a version of the page cached on your system already. So all you need to do in the dynamic page templates is to include the cached version straight away. As long as the URL doesn’t change, the hash is always the same. And you get the value obviously just by looking it up in the cache directory after get_remote_file gets first invoked by any of your “static” pages which are built to contain PHP tags.

The only problem I could see with this is if MTInclude was invoked at the same time when get_remote_file is writing to it, since MTInclude doesn’t obviously know anything about locking. Now the right thing to do would be to build a custom MT tag equivalent of get_remote_file, but until then… there might be an unfortunate user once in a while who accesses your site at an unfortunate time and might get empty contents instead of the included file. I guess sometimes you just have to live with it? Or do the filesystem operations intelligently wait behind each other so that if there’s a write in progress, the read waits until it’s done? Should dig into how that really works one day.