[aur-dev] [PATCH 1/1] add support for etag and conditional get (if-none-match)
Add etag and if-none-match conditional get support. This will allow 'smart client' to save network bandwidth, as they can save the etag hash value for queries and test it later. Still an http request because this patch specifically sets a cache lifetime of zero, and must-revalidate. The benefit here is bandwidth savings. Caching based on expires headers would likely be counter productive, as the api data can change rather quickly...but etag is a nice compromise, and could be quite beneficial for bandwidth recution in some scenarios. --- web/lib/aurjson.class.php | 30 +++++++++++++++++++++++++++++- 1 files changed, 29 insertions(+), 1 deletions(-) diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 50cf6d0..a96cc4b 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -29,6 +29,17 @@ class AurJSON { * @return string The JSON formatted response data. **/ public function handle($http_data) { + // unset global aur headers from aur.inc + // leave expires header to enforce validation + // header_remove('Expires'); + // unset global aur.inc pragma header. We want to allow caching of data + // in proxies, but require validation of data (if-none-match) if + // possible + header_remove('Pragma'); + // overwrite cache-control header set in aur.inc to allow caching, but + // require validation + header('Cache-Control: public, must-revalidate, max-age=0'); + // handle error states if ( !isset($http_data['type']) || !isset($http_data['arg']) ) { return $this->json_error('No request type/data specified.'); @@ -43,6 +54,24 @@ class AurJSON { $json = call_user_func(array(&$this, $http_data['type']), $http_data['arg']); + // calculate etag as an md5 based on the json result + // this could be optimized by calculating the etag on the + // query result object before converting to json (step into + // the above function call) and adding the 'type' to the response, + // but having all this code here is cleaner and 'good enough' + $etag = md5($json); + header("Etag: \"$etag\""); + // make sure to strip a few things off the if-none-match + // header. stripping whitespace may not be required, but + // removing the quote on the incoming header is required + // to make the equality test + $if_none_match = isset($_SERVER['HTTP_IF_NONE_MATCH']) ? + trim($_SERVER['HTTP_IF_NONE_MATCH'], "\t\n\r\" ") : false; + if ($if_none_match && $if_none_match == $etag) { + header('HTTP/1.1 304 Not Modified'); + return; + } + // allow rpc callback for XDomainAjax if ( isset($http_data['callback']) ) { // it is more correct to send text/javascript @@ -179,7 +208,6 @@ class AurJSON { $where_condition = sprintf("Name=\"%s\"", mysql_real_escape_string($pqdata, $this->dbh)); } - return $this->process_query('info', $where_condition); } -- 1.7.2.5
On Fri, May 13, 2011 at 12:55:40PM -0700, elij wrote:
Add etag and if-none-match conditional get support. This will allow 'smart client' to save network bandwidth, as they can save the etag hash value for queries and test it later. Still an http request because this patch specifically sets a cache lifetime of zero, and must-revalidate. The benefit here is bandwidth savings. Caching based on expires headers would likely be counter productive, as the api data can change rather quickly...but etag is a nice compromise, and could be quite beneficial for bandwidth recution in some scenarios. --- web/lib/aurjson.class.php | 30 +++++++++++++++++++++++++++++- 1 files changed, 29 insertions(+), 1 deletions(-)
I kinda like that one although I'm not really sure if this kind of caching is convenient enough here... Having a bit more detailed look at the single query methods: * search, msearch: Probably won't be repeated on a single client very often ("repeated" meaning doing the same search query more than once). * info: Gain will probably be low as the query results are very small. * multiinfo: Query results will change quite often, while the actual diffs will be small in most cases. Some delta based stuff may be more effective here but that will probably be overkill. ETags seem to be a good compromise, yeah :)
On Sun, May 15, 2011 at 12:19 PM, Lukas Fleischer
On Fri, May 13, 2011 at 12:55:40PM -0700, elij wrote:
Add etag and if-none-match conditional get support. This will allow 'smart client' to save network bandwidth, as they can save the etag hash value for queries and test it later. Still an http request because this patch specifically sets a cache lifetime of zero, and must-revalidate. The benefit here is bandwidth savings. Caching based on expires headers would likely be counter productive, as the api data can change rather quickly...but etag is a nice compromise, and could be quite beneficial for bandwidth recution in some scenarios. --- web/lib/aurjson.class.php | 30 +++++++++++++++++++++++++++++- 1 files changed, 29 insertions(+), 1 deletions(-)
I kinda like that one although I'm not really sure if this kind of caching is convenient enough here... Having a bit more detailed look at the single query methods:
* search, msearch: Probably won't be repeated on a single client very often ("repeated" meaning doing the same search query more than once).
* info: Gain will probably be low as the query results are very small.
* multiinfo: Query results will change quite often, while the actual diffs will be small in most cases.
Some delta based stuff may be more effective here but that will probably be overkill. ETags seem to be a good compromise, yeah :)
Yeah. I see a couple of instances that would benefit from it. 1) Direct browser viewing of api urls. Developers or end users entering query strings manually. 2) My live-search implementation (and similar type apps) _should_ work better, as back and forward button (or identical search terms) should be cached client side by the browser. 3) A cli client could be modified to cache data, but I imagine the benefits here would be small, as I don't believe end users would repeat many queries in a short enough timeframe (before some data changes) to get much benefit from caching. Who knows though. heh 4) Could be a benefit for large groups of clients behind a cache proxy. 5) Could be a benefit if the arch server admins ever decided to throw varnish in front of the aur (or just the api).
participants (2)
-
elij
-
Lukas Fleischer