{"id":169,"date":"2016-08-28T03:46:56","date_gmt":"2016-08-28T03:46:56","guid":{"rendered":"http:\/\/federicojacobi.com\/?p=169"},"modified":"2016-08-28T03:46:56","modified_gmt":"2016-08-28T03:46:56","slug":"when-caching-is-not-enough-double-buffered-remote-calls","status":"publish","type":"post","link":"https:\/\/federicojacobi.com\/2016\/08\/28\/when-caching-is-not-enough-double-buffered-remote-calls\/","title":{"rendered":"When caching is not enough: “Double Buffered” Remote Calls"},"content":{"rendered":"

One of the challenges of running WordPress at scale is dealing with API calls to (insert_external_service_here). Using wp_remote_get <\/a>(or curl) is probably your go-to method for API calling and this is a fine function for a low traffic site. On a site that gets millions of pageviews, it is just not going to cut it. You will inevitably run into race conditions.<\/p>\n

In case you don’t know, a race condition is when\u00a0person1 is waiting on the server to finish the api\u00a0call, then person2 makes another call, then person3 … then person X, but person1 is still waiting. If the API server is being slow, there could be a queue of thousands waiting, at that point, your server has crashed for sure. wikipedia<\/a><\/p><\/blockquote>\n

Another reason for not using wp_remote_get on every request\u00a0is API limiting. Some services do not allow more than X calls per second\/minute\/day. If you make a call for every visit, you will surely reach that limit extremely fast!<\/p>\n

Simple Solution: Caching.<\/h4>\n
$result = wp_cache_get( 'my_api_call', 'api_calls' );\n\nif ( false === $result ) {\n\t$result = wp_remote_get( $API_url );\n\twp_cache_set( 'my_api_call', $result, 'api_calls', 300 );  \/\/ cache for 5 minutes\n}\n\n\/\/ do something with $result<\/pre>\n
By caching your call the “traditional” way, you’ve now gone way ahead from where you started. The API will only happen every 5 minutes, and people will not have to wait for the results as you have them stored already! This is just perfect for medium traffic sites and fast response APIs.<\/p>\n
The problem with this approach is that at\u00a0the 5 minute mark you still need to wait for the API to respond. If the response is slow you could run into a race condition again, because cache is invalidated, and it goes like so: person1 triggers cache invalidation (past 5 minutes) and calls API, person2 calls API too because cache is not valid, person3 same ….. person100 same, person1’s call is done and cache is set again for the next 5 minutes, person101 gets a cached result everyone is happy from here on, in the meantime, persons2-100 are still waiting on the response slowly. We have somewhat mitigated the problem, but not completely solved it. If the traffic is really high and the API is really slow, your server could crash.<\/p>\n
If you have that kind of traffic, you are playing with the big boys. Lazy caching is not going to be enough.<\/p>\n
Complex Solution: Double\u00a0Caching<\/h4>\n
Instead of just caching the result, you can double cache it. To do so, we are doing the same thing as above, but twice. And we’ll do it in a way where cache1 lives for 5 minutes and cache2 lives for 10 minutes. When cache1 invalidates, that person makes the api call and sets a switch so everyone else uses cache2. Now only person1 is slow.<\/p>\n
$switch = wp_cache_get( 'switch', 'api_calls');\n\nif ( false === $switch ) {\n\t$result = wp_cache_get( 'my_api_call', 'api_calls' );\n\t\n\tif ( false === $result ) {\n\t\t\n\t\twp_cache_set( 'switch', true, 'api_calls', 300 );\t\t\/\/ Set switch BEFORE making the external call. \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\/\/ Time doesn't matter here, just make sure it is longer than PHP's execution time\n\t\t\n\t\t$result = wp_remote_get( $API_url );\n\t\t\n\t\twp_cache_set( 'my_api_call', $result, 'api_calls', 300 );\t\t\t\/\/ cache for 5 minutes\n\t\twp_cache_set( 'my_api_backup_call', $result, 'api_calls', 600 );\t\/\/ cache longer than 5mins so they don't invalidate at the same time\n\t\t\n\t\twp_cache_delete( 'switch', 'api_calls' );\t\t\t\t\/\/ Call is done, remove the switch\n\t}\n\t\n} else {\n\t$result = wp_cache_get( 'my_api_backup_call', 'api_calls' );\n}\n\n\/\/ do something with $result<\/pre>\n
This is pretty great as is, but we can do better. Let’s say the API starts to malfunction for whatever reason. In that scenario, you’ll have good data for 5 minutes until the next time you do the call … not so awesome. Add a data consistency check (which you SHOULD have), and now we are in business:<\/p>\n
$switch = wp_cache_get( 'switch', 'api_calls');\n\nif ( false === $switch ) {\n\t$result = wp_cache_get( 'my_api_call', 'api_calls' );\n\t\n\tif ( false === $result ) {\n\t\t\n\t\twp_cache_set( 'switch', true, 'api_calls', 300 );\t\t\/\/ Set switch BEFORE making the external call. \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\/\/ Time doesn't matter here, just make sure it is longer than PHP's execution time\n\t\t\n\t\t$result = wp_remote_get( $API_url );\n\t\t\n\t\tif ( is_your_data_valid( $result ) ) {\n\t\t\t\/\/ the result from the call is as expected\n\t\t\t$save_data = $result\n\t\t\t\n\t\t} else {\n\t\t\t\/\/ Use the last good result and set backup\n\t\t\t$save_data = wp_cache_get( 'my_api_backup_call', 'api_calls' );\n\t\t\t\n\t\t\t\/\/ Something went wrong in the api call.... notify somebody? do something?\n\t\t}\n\t\t\n\t\twp_cache_set( 'my_api_call', $save_data, 'api_calls', 300 );\t\t\t\/\/ cache for 5 minutes\n\t\twp_cache_set( 'my_api_backup_call', $save_data, 'api_calls', 2000 );\t\/\/ cache longer than 5mins so they don't invalidate at the same time\n\t\twp_cache_delete( 'switch', 'api_calls' );\t\t\t\t\t\t\t\t\/\/ Call is done, remove the switch\n\t\t\n\t\t$result = $save_data;\n\t}\n\t\n} else {\n\t$result = wp_cache_get( 'my_api_backup_call', 'api_calls' );\n}\n\n\/\/ do something with $result<\/pre>\n
The only caveat with that approach is if your backup cache invalidates. A neat alternative, save the backup in an option:<\/p>\n
“Double Buffer”<\/h4>\n
$switch = wp_cache_get( 'switch', 'api_calls');\n\nif ( false === $switch ) {\n\t$result = wp_cache_get( 'my_api_call', 'api_calls' );\n\t\n\tif ( false === $result ) {\n\t\t\n\t\twp_cache_set( 'switch', true, 'api_calls', 300 );\t\t\/\/ Set switch BEFORE making the external call. \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\/\/ Time doesn't matter here, just make sure it is longer than PHP's execution time\n\t\t\n\t\t$result = wp_remote_get( $API_url );\n\t\t\n\t\tif ( is_your_data_valid( $result ) ) {\n\t\t\t\/\/ the result from the call is as expected, store results\n\t\t\t\n\t\t\twp_cache_set( 'my_api_call', $result, 'api_calls', 300 );\t\/\/ cache for 5 minutes\n\t\t\tupdate_option( 'my_api_backup_call', $result );\t\t\t\t\/\/ options don't invalidate, they just store stuff\n\t\t\t\n\t\t\twp_cache_delete( 'switch', 'api_calls' );\t\t\t\t\t\/\/ Call is done, remove the switch\n\t\t\t\n\t\t} else {\n\t\t\t\/\/ The result from the call is NOT ok deal with it somehow.\n\t\t\t\/\/ In the meantime do not clear the switch OR update the option\n\t\t\t\/\/ Because the switch is on, it'll eventually invalidate allowing another call.\n\t\t\t\/\/ If that other call is good, we return to regular use, if it is bad,\n\t\t\t\/\/ the switch will be set again, using the backup option\n\t\t}\n\t}\n\t\n} else {\n\t$result = get_option( 'my_api_backup_call' );\n}\n\n\/\/ do something with $result<\/pre>\n
Boom! Now even if the API fails or nobody accesses the page and cache dies, your site still has content, all you have to do now is find a way to alert you of the problem.<\/p>\n
Overkill? yes … Works? … Really well!<\/p>\n
Note: wordpress\u00a0options have limited size, if the data you are storing is too big, you may want to consider WP Large Options<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
One of the challenges of running WordPress at scale is dealing with API calls to (insert_external_service_here). Using wp_remote_get (or curl) is probably your go-to method for API calling and this is a fine function for a low traffic site. On a site that gets millions of pageviews, it is just not going to cut it….  Read more »<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11],"tags":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/posts\/169"}],"collection":[{"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/comments?post=169"}],"version-history":[{"count":0,"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/posts\/169\/revisions"}],"wp:attachment":[{"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/media?parent=169"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/categories?post=169"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/federicojacobi.com\/wp-json\/wp\/v2\/tags?post=169"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}