2010-11-13

Avoiding URL length limits when loading pages with Javascript

Working on a web application that must work with IE, I encountered problems with opening windows (via window.open), or loading content in iframes, that required a large set of parameters to the URL provided. IE has a hard-coded limit on the size of URL strings, so parameter data could get clipped.

Also, it is not pretty to have a large URL string showing in the address bar.

Therefore, the solution was to use an HTTP POST based method.

My first attempt was to use AJAX with a POST request to retrieve the desired page, and then do a document.write() into a blank window. However, I discovered that IE does not process data in a serial manner via document.write(). For example, if the page content contains <script> elements referencing external resources to load, IE will not process those in sequential order, so if you have Javascript code that calls a function that is defined by a previous <script>, you can get a runtime error. IE will load the external javascript resources, but they are done asynchronously, with IE not waiting to execute subsequent Javascript in the page that comes after the <script>.

The solution I came up with is to create a transient page that performs a form post. I.e. In the blank window, I create dynamic HTML page that contains an HTML form with the set of parameter data defined as hidden fields. The HTML page created has an onLoad action to submit the form.

With this approach, the browser is directly fetching the resource, so the document.write() problem described earlier (for IE) is no longer a factor.

What follows is a function for loading a document using a POST-based method:

loadDocument = function(
/*Object*/args
)
{
// summary:
// Load a document for a given document node.
// description:
// This function uses a POST-style method for loading the
// contents of a document node, where the document requested
// takes an arbitrary number of request parameters. This
// function works-around limitations of some browsers where
// there is a limit to the length of query-string parameters
// for GET-based requests.
//
// args: Object
// Object contain the following properties:
//
// doc: Node
// Document node to load content for.
// url: String
// URL to fetch content from.
// params: Object?
// Properties of string values representing parameters
// for the request denoted by url.
// message: String?
// Message text (HTML) to display while content is
// loading.

var doc = args.doc;
doc.open();
doc.write('<html>');
doc.write('<script type="text/javascript">');
doc.write('function submitForm(){document.forms[0].submit();}');
doc.write('</script>');
doc.write('<body style="background-color:#FFF;" onLoad="submitForm()">');
if (args.message) {
doc.write(args.message);
}
doc.write('<form method="post" action="');
doc.write(args.url);
doc.write('">');
if (args.params) {
for (var p in args.params) {
var v = args.params[p];
if (v instanceof Array || typeof v == "array") {
for (var i=0; i < v.length; ++i) {
doc.write('<input type="hidden" name="');
doc.write(p);
doc.write('" value=\'');
doc.write(escapeHTML(v[i]));
doc.write('\'></input>');
}
} else {
doc.write('<input type="hidden" name="');
doc.write(p);
doc.write('" value=\'');
doc.write(escapeHTML(args.params[p]));
doc.write('\'></input>');
}
}
}
doc.write('</form></body></html>');
doc.close();
}

// Helper function to escape HTML specials for use above.
escapeHTML = function(/*String*/s) {
// summary: Convert HTML special characters to entity references.
// s: String to escape.
if (!(typeof s == "string" || s instanceof String)) {
s = s + ""; // force it to a string
}
return s.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#39;");
}


The primary function works on a Document object, allowing it to not just be used for new windows, but for any object that contains a Document object, like frames and iframes.

Simple example using the above:

var w = window.open("");
loadDocument({
doc: w.document,
url: "http://example.com/some/resource/",
params: {
p1: "Hello",
p2: "World!"
}
message: 'Loading..."
});

2010-01-31

HOW-TO: Posting to Gmail servers with non-SSL/TLS-aware MUAs

I'm a long-time user of the nmh mail user agent (MUA), <http://www.nongnu.org/nmh/>, and its predecessor MH before that.

It's an old MUA. It currently does not support submitting emails to mail servers that require SSL/TLS connections. Other venerable MUAs have the same problem. Fortunately, there is an OSS program that provides proxy capabilities that can negotiate the SSL/TLS part, and if necessary, the SMTP AUTH part if the MUA does not support it: DeleGate, <http://www.delegate.org/>.

DeleGate provides proxy capabilities for various Internet protocols (e.g. http, ftp), but for this article, all we care about is its SMTP capabilities.

Nice thing about DeleGate is you do not need root priviledges to run it. You can have it listen to local client requests on any port you choose and forward the request to remote server (which will be smtp.gmail.com in this article).

Best way to show how to use it is with an example:

The following starts delegated on the local system on port 25 (the standard SMTP port):

delegated -Plocalhost:20025 +=$HOME/delegate/conf/gmail.conf


This says to listen for client requests on port 20025 and to read additional configuration parameters from gmail.conf located under delagate/conf of your home directory (you can specify all parameters on command-line, but that gets kind of ugly). In my gmail.conf file, I have something like the following:

SERVER=smtp://smtp.gmail.com:587
STLS=fsv
AUTHORIZER=-list{localuser:MD5:964ba203464913531aa73b9f774b58f7}
MYAUTH=username@gmail.com:pass:smtp


SERVER specifies what remote server the proxy should connect to. In this case, we specify that is the SMTP server smtp.gmail.com on port 587 (TLS port).

STLS setting specifies that SSL/TLS should be used when connecting to remote server (smtp.gmail.com).

The AUTHORIZER parameter is not required, but if specified, client must (SASL) authenticate to the proxy. The username is "localuser" (change to whatever your want). This may be handy so other users on the local system cannot hijack your proxy server and post email thru your Gmail account. If you are the sole user of your system, client authentication may not be needed. Also, if your MUA does not support SASL, you should not set AUTHORIZER.

To determine if your MUA supports SASL, if your MUA provides the ability to specify an username and password when posting a message to a mail server, it probably supports SASL. If no such capability is provided, it probably does not.

The password listed for AUTHORIZER can be provided in plaintext, but DeleGate will show in its log the MD5 hash of it, which you can then insert into the configuration file. A plaintext version of AUTHORIZER is as follows:

AUTHORIZER=-list{localuser:password}


The MYAUTH specifies the username/password to use for authenticating to the remote server (smtp.gmail.com). This should be set to your gmail address and gmail password. Unfortunately, the password cannot be encrypted since DeleGate needs to send the password to the remote server. Therefore, make sure the configuration file is only readable by you and no one else. For Unix-based users, the chmod command can be used:

chmod 600 gmail.conf


For Windows users, right click on the file and select Properties. Then select the Security tab. Make sure you are the only user that has read access to the file.

With DeleGate running, just configure your MUA to post messages to localhost:2005 and you should be good to go.

NOTES:

If you cannot specify an alternate port number for posting messages, you will need to run DeleGate on port 25. If this is the case, you will need to have admin priviledges to start DeleGate. DeleGate does provide an option for it to drop priviledges to a separate user. See documentation for more details.

DeleGate supports multi-user access, but configuration becomes more complex. If you want to support multi-user access, see the documentation.

For nmh user, SASL is supported, so client-based connections can be restricted, especially if running DeleGate as a personal proxy. However, for nmh, if DeleGate is not running on port 25, the whom command at the "What Now?" will not work. The command will also not work if you enable SASL for client connections. For whom to work, nmh code modifications are required. I've committed changes into the nmh project so SASL support is available for the whom command. You'll need to check out the latest source and build if you need the feature.