Fetching Data from a Web Server
ListManager can retrieve data from a remote machine using the httpget mail merge tag. This data must be accessible via http.
IMPORTANT NOTE: Using httpget to fetch content for each recipient can significantly slow your ListManager sending speed. Each ListManager node can use httpget to fetch roughly 20,000 pages per hour. Fetching pages blocks other processes, so sending mail speeds will be significantly slower for concurrent mailings that are not using httpget.
If you are fetching a static page, it is strongly recommended that you use the before; httpget command (below) so ListManager fetches the page only once per mailing try.
If you need dynamic content where each possible page is static (such as ads), you should use the before command to fetch the data into Tcl variables and then programmatically choose between the different pages. Doing so will result in much higher sending speeds than fetching the page for each recipient.
%%httpget%%
The syntax of the httpget command is:
%%httpget "URL"%%
The URL argument can be virtually anything you can pull up with a web browser. The URL may point to a simple text file, some HTML code, or a CGI script.
To use httpget, you must be using ListManager Pro or Enterprise, and Scripting Level must be set to Full.
Example 1
You keep dozens of ads on your web server. You want them to be inserted into your newsletter using your
script. The script's URL is http://yourserver/cgi-bin/ad.pl.
%%httpget "yourserver/cgi-bin/ad.pl"%%
Note that you don't need http:// as part of the URL.
Example 2
You have a script which creates a different page for each user, and would like these unique pages inserted for each email message sent.
%%httpget "localhost/cgi-bin/getdata.pl?id=[merge members_.emailaddr_]"%%
In the above example, ListManager first performs a substitution to merge in the member’s email address,
then retrieves the data at the resulting URL. Note that you must use square brackets and the full mail
merge tag, not the short tag. Let’s assume that bob@example.com is the member that ListManager is currently
sending to. After substitution, the URL looks like:
http://localhost/cgi-bin/getdata.pl?id=bob@example.com
The nature of the script being called ("getdata.pl" in this case) is to generate some data unique to the member with that email address. This script can do whatever you design it to do, whether that means generating an ad or retrieving billing information.
Programmers have the ability to call a function in the middle of messages and insert whatever it returns right into the message at the point from which it was called.
While the httpget merge tag does not require any specific output from the URL, it is highly recommended that you include a standard 'HTTP/1.0 200 OK' status line, then a 'Content-Type: text/plain', or 'Content-Type: text/html' line followed by an empty line and the content of the text to be inserted. For example, a (Perl) script might have:
print "HTTP/1.0 200 OK\r\n";
print "Content-Type: text/plain\r\n";
print "\r\n";
print "Hello! Please buy our products!";
%%httpget_auth%%
If you have http authentication on the page you want to fetch, you can use httpget_auth, which allows specification of a username and password for basic http authentication.
Example
%%httpget_auth "http://www.example.com/private/content.html" "administrator" "mypassword"%%
%%before; httpget%%
Httpget will repeatedly retrieve the data for each individual recipient. If, however, you only need to retrieve the data only once per mailing attempt, you can use the before; httpget tag instead. That is, before the mailing or retry begins, ListManager will pull the URL once, and fill in that same value for all recipients. Since the speed of httpget depends on the speed it can fetch data from your server, using before ; httpget can make your mailing speeds faster.
The syntax of the before httpget tag is:
%%before ; httpget "URL"%%
Note that the httpget will still be run again for each retry.
Example
You keep dozens of ads on your web server. You want them to be inserted into your newsletter, with every recipient getting the same one. The script's URL is http://www.lyris.com/getmailad.pl.
%%before ; httpget "www.lyris.com/getmailad.pl"%%
For more information, see Script Execution Points.
%%after; httpget%%
As of ListManager 8.0, the %%after; httpget%% command has been deprecated.
More
4. Fetching Data from a Web Server
5. Troubleshooting Merge Tag Problems