How Do I Store Website Screenshots Locally (caching locally)?
Posted by on 21 April 2011 04:54 PM

We do allow you to cache your requests locally (on your own server); also known as "caching locally". In fact, we strongly recommend it. We recommend giving the user a choice of either "Caching Locally" (Advanced Method) or "Embedded" (Simple Method) but at least using the "Advanced Method", so that you get more data about the request, account status, and any error messages. Otherwise, using the "Embedded Method" limits you to checking for "404 Not Ready" and just getting the image, without any status or error codes.

NOTE: It is best to start with the sample code that parses the "Advanced Method" API XML (or JSON) response. The sample code will not cache errors and queued images, whereas the methods below may inadvertently cache them if the requests are not already captured with a good status. If you must use the "embedded" method, you may check the header response for "404 Not Ready" and ignore until it is ready.

NOTE: The Functions below are only for example. They will inadvertently cache "Thumbnail Queued" and other system messages, so it is not recommended to use these examples as-is.


Always refer to the PagePix documentation, when writing a custom integration or plugin. In short, though, it is best to send a cURL (or fopen()) command to initiate the request. If using the "Advanced Method", we strongly recommend using the "Notify Callback" feature to avoid having to "poll" for status. Polling for status is strongly discouraged now that "Notify Callback" is available and offers quite a bit of functionality and options. Even if you make a request with "Notify Callback" requested, the most efficient and fastest method to get the image onto your server, if the screenshot already exists in our cache, is to just go ahead and download it. This way, you won't have to wait for a notify event (even though it might only take 1s before delivery, it's unnecessary and wastes resources on both sides).

The function that downloads the image, when already cached on our side, is: _downloadRemoteImageToLocalPath()

It is safe to assume that any code you put in that function will only execute if the image is cached on our side and you are downloading it immediately. So, you can put code to update a listing with request specific information, update counters, kick off a custom process, send an email, etc.


Some of the download methods that we've seen others use are:


1. cURL (see STW API Library for latest example)


* More secure because works with allow_url_fopen = OFF
* Allows for more options when making the request


* Requires a few more lines of code to implement


$options = array( CURLOPT_RETURNTRANSFER => true, // return web page
     CURLOPT_HEADER => false, // don't return headers 
     CURLOPT_FOLLOWLOCATION => false, // follow redirects 
     CURLOPT_AUTOREFERER => true, // set referer on redirect 
     CURLOPT_CONNECTTIMEOUT => 5, // timeout on connect 
     CURLOPT_TIMEOUT => 5, // timeout on response 
     CURLOPT_MAXREDIRS => 0, // stop after 10 redirects ); 

$url = $image; 
$ch = curl_init( $url ); 
curl_setopt_array( $ch, $options ); 
$imagedata = curl_exec( $ch ); 
$err = curl_errno( $ch ); // helpful for troubleshooting 
$errmsg = curl_error( $ch ); // helpful for troubleshooting 
curl_close( $ch );

$imagedata = @imagecreatefromstring($imagedata); // to build as image
if($imagedata) {imagejpeg($imagedata,$url,100);} // to cache image locally 

imagejpeg($imagedata, NULL, 100); // to show image directly in browser 
imagedestroy($imagedata); // free up memory (if done with it)

2. PHP Function: imagecreatefromjpeg()


* Easy to implement
* Sample code already provided in STW PHP4 sample functions


* Requires allow_url_fopen = ON (not as safe as using cURL)
* No longer supported by most free/shared web hosting services


$imagedata = @imagecreatefromjpeg($image); 
if($imagedata) {imagejpeg($imagedata,$dom,100);}

3. wget Utility


* Programming language independent; Run from command-line
* Lots of built in error handling


* Some webmasters prefer turn off wget (helps prevent some exploits that gain access to utilities)
* More complex command line usage to output usable images (see example below)

Example (of Advanced Method, step 2 download)*,**:

wget --output-document=/home/[USER]/public_html/testimage.jpg\?STWAccessKeyId=[ACCESS_KEY]\&stwu=[SECRET_KEY]\&stwq=95\&stwxmax=700\&hash=abc123\|def456

*Replace all fields in [brackets] with your information.

**Notice that the image URL must be updated with string literals. In other words, you may need to place a backslash "\" before each "?" or "&" or "|" in the request. If you miss any of these, you will get a file containing the words "Missing Parameters"



Tags: fopen url_fopen url failed to open stream

(5 vote(s))
Not helpful

Contact Us | PagePix Benefits | Learn More | STW Forums | Our Partners | Privacy Policy | Terms of Use

©2016 ShrinkTheWeb. All rights reserved. ShrinkTheWeb is a registered trademark of ShrinkTheWeb.