instantreality 1.0

Http Communication in ECMAScript with XMLHttpRequest

Keywords:
script, ECMAScript, http, xml, XMLHttpRequest, xhr, REST
Author(s): Manuel Olbrich
Date: 2010-06-24

Summary: This tutorial shows you how to communicate with Http servers in script nodes using ECMAScript.

Introduction

Getting information into X3D/VRML-Scene usually requires converting the data into the language's syntax. Also, runtime changes, such as painting on a PaintTexture are lost when the application is closed. A common way to get around this problem involves java based script nodes, which can be allowed to bypass security and write data to disk. The downsides of this method are relatively complex code in an additional programming language and modifications to the java settings. A XMLHttpRequest ECMAScript object allows accessing dynamic data outside of the application, while having a relatively simple setup.

Warning: To prevent foreign scenes from sending personal data to third parties, a XMLHttpRequest object will only communicate with whitelisted servers. If you try to use a non-whitelisted server, a ECMAScript exception will inform you about this problem. You can whitelist a host by adding --ecmaScriptXHRTrustedHosts HOSTNAME to your commandline or by changing this parameter via the SceneGraph web interface (Setup section). HOSTNAME can be a single host name, a comma separated list of host names, or the keyword "any" which would disable this security check.

Connecting to an Http Server

The Http protocol is used in many applications. In this tutorial, we will connect to a simple webserver, to a WebDAV server, and to apaches CouchDB.
  • WebDAV is very similar to a usual webserver, but it also allows writing and deleting documents. All common desktop operating systems can handle WebDAV like a remote file system. Webservers, such as apache and lighttp, support this service, but it is disabled by default, since it can be used to write in a server's file system.
  • CouchDB is a simple and open DBMS that communicates over HTTP. The database entries (called documents in couchdb) are stored in JSON (JavaScript Object Notation) and can therefore be easily used in JavaScript applications.

Code: The following Script node creates an XMLHttpRequest object.

Script{
  url["JavaScript:
    var xhr;
    function initialize()
    {
      xhr=new XMLHttpRequest();
    }
  "]
}
At this point, the server name, localhost in this case, is checked against the whitelist. If it is not in the list a JavaScript exception gets thrown. You can catch this exception, if you like, with JavaScript's try{}catch(err){} construct. If not, the execution gets interrupted at this spot and an exception will be printed to console. All following code sniplets assume that a XMLHttpRequest object called xhr exists.

GETting Data from a Webserver

Code: The following code fetches a file from an Http server.

xhr.open('GET','http://localhost/test.txt');
xhr.send();
print(xhr.responseText);
This example sends a GET request to localhost and prints the results to the console. With the file's content available in the responseText attribute you can interpret it your way with JavaScript.

PUTting Files on a WebDAV Server

Putting files is nearly as simple as getting them. You should check your WebDAV setup with a native client to make sure that your permissions are set correctly and that you can write files.

Code: The following code creates a file on a WebDAV server.

xhr.open('PUT','http://localhost/newfile.txt');
xhr.send('The file content goes here.')

PUTting SFImages as JPEG- or PNG-Files on a WebDAV Server

If you like to save an image that was generated in your application (like an image from a PaintTexture or a frame from a Sensor), you would most likely want to save it in a readable image format rather than using SFImage encoding. XMLHttpRequest allows an second parameter to send() which sets a convertion before the data will be send. Possible values for this parameter are SFImage2jpg and SFImage2png.

Code: The following code creates a several image files on a WebDAV server. img is an SFImage.

xhr.open('PUT','http://localhost/testimage.sfimage');
xhr.send(img); //new file will be the string representation of an SFImage

xhr.open('PUT','http://localhost/testimage.jpg');
xhr.send(img,'SFImage2jpg'); // convertion to JPEG

xhr.open('PUT','http://localhost/testimage.png');
xhr.send(img,'SFImage2png'); // convertion to PNG

DELETE Files from a WebDAV Server

Deleting files works like getting. The following Example shows the deletion of a file.

Code: The following code deletes a file on a WebDAV server.

xhr.open('DELETE','http://localhost/test.txt');
xhr.send();

Asynchronous requests

XMLHttpRequest allows asynchronous requests, which is important since the rendering will be blocked by an synchronous request. The open() method has an third parameter, which determines if the request is async or not. The default Value for this parameter is false. You can set an callback function for XMLHttpRequest with the onreadystatechange attribute. This callback function gets called on every state change. Usually the state we are most interested in is DONE, indicated by the number 4.

Code: Asynchronous version of the GET example.

xhr.open('GET','http://localhost/test.txt',true);
xhr.onreadystatechange = function(){
  if(xhr.readyState==4) //4==DONE
    print(xhr.responseText);
}
xhr.send();

Information about your last Request

An XMLHttpRequest object stores the last reply's header and statusline. The attribute status returns the response status code. It will be 200 if everything went right. The corresponding status phrase is available in the statusText attribute.

Code: The following code fetches a file from an Http server and reads the response status information and headers.

xhr.open('GET','http://localhost/test.txt');
xhr.send();
print('status code: ' + xhr.status + '\n');
print('status phrase: ' + xhr.statusText + '\n');
print('response headers are:\n:' + xhr.getAllResponseHeaders());
status code: 200

status phrase: OK

response headers are:
Accept-Ranges: bytes
Content-Length: 14
Content-Type: text/plain
Date: Tue, 05 Oct 2010 09:46:14 GMT
ETag: "111fc-e-491db7dd4e565"
Last-Modified: Tue, 05 Oct 2010 09:43:21 GMT
Server: Apache/2.2.14 (Ubuntu)
Vary: Accept-Encoding

Working with CouchDB

apache's CouchDB is quite simple and easy to set up and use. However, you should understand the basic concepts before you play with these examples. The CouchDB Wiki is a good starting point. Once you do, here is a neat API-Cheatsheet. You should have understood the following:
  • Every document (database entry) is accessed via its id, which is usually a random generated UUID
  • You have to provide an id when generating a document. You can get a new UUID from couchdb
  • if you want to change or delete an object, you have to provide the current revision (this prevents conflicts when multiple users are working on the same document)
  • Documents can have attachments, just like an email. Adding an attachment is a change, so you need to provide the current revision. Attachments can be directly accessed, which means that you can point a web browser to an attached Image and simply get it. This is quite useful since you can simply provide this url to an ImageTexture
  • A view is a special kind of document, called a design document (with "_design/namehere" as id)
  • All documents (including views) are in one container. use views to separate them
  • CouchDB comes with a web frontend, called Futon, to access the DB. (http://localhost:5984/_utils/)

Parsing JSON

The neat thing about the combination of CouchDB and JavaScript is that the documents (and nearly everything that CouchDB) returns are encoded in JSON (JavaScript Object Notation). That means, you can simply evaluate what CouchDB returns to you have a JavaScript object. The following example shows how to connect to the couchdbms and get its welcome message as a JavaScript object.

Code: Connect to CouchDB, read and eval the welcome msg.

Script{
  url["javascript:
    var xhr;
    function initialize()
    {
      xhr=new XMLHttpRequest();
      xhr.open('GET','http://localhost:5984/');
      xhr.send();
      print(xhr.responseText);
      var myobj = eval('('+xhr.responseText+')');
      print(myobj.couchdb);
      print(myobj.version);
    }
  "]
}
This creates the following output: 
{"couchdb":"Welcome","version":"0.10.0"}

Welcome
0.10.0
The following examples assume that the xhr object already exists.

List All Documents in Your DB

The db in this example is called demodb. You can create yours either with the apicalls or using Futon. If you want to list all DBs in your DBMS, get /_all_dbs from you db.

Code: Listing all documents in your db.

xhr.open('GET','http://localhost:5984/demodb/_all_docs');
xhr.send();
print(xhr.responseText);
This creates the following output: 
{"total_rows":0,"offset":0,"rows":[]}
total_rows shows the number of documents in your db. If you have documents in your db, the array row will contain JSON objects containing each document's id and a value field containing the revision. 
{"id":"e42c296c5201a7506768b60c3074affe","value":{"rev":"1-dfc98c905e3b4899ae2114a63fbdd2c4"}}
The revision contains a number in front of an UUID that indicates how often a document was changed. If you like to have the other data without fetching each document you should have a look at CouchDBs views.

Writing a New Document

Writing a document is like PUTting files to WebDAV, mentioned previously. The document's content in JSON encoding is transferred as the body. The target URI is /databasename/id. As mentioned before, you should use UUIDs as IDs. If you have no convenient way of generating these yourself, simply call

Code: Get UUIDs from CouchDB.

xhr.open('GET','http://localhost:5984/_uuids');
You can call /_uuids?count=someNumber if you need more than one. That explains also why the result has the new UUID in a one element array. 
{"uuids":["7f06ce4956f68193f66b1f172ef5badf"]}
With this id we have all we need to write a new document:

Code: Writing a document to CouchDB

var newDocument = '{\"documentname\":\"myNewDocument\",\"rating\":\"totally awesome\",\"someNumbers\":12345}';
xhr.open('PUT','http://localhost:5984/demodb/7f06ce4956f68193f66b1f172ef5badf')
xhr.send(newDocument);
The xhr.responseText contains next to ok and id the revision string of this document, which is quite useful if you plan to attach files. 
{"ok":true,"id":"7f06ce4956f68193f66b1f172ef5badf","rev":"1-9bfc66754ea484c25293510de7b6ed9f"}

Reading Documents

Reading documents is quite simple. We assume you already have the document's id (either from a view or from a _all_docs request).

Code: Getting a document from CouchDB

xhr.open('GET','http://localhost:5984/demodb/7f06ce4956f68193f66b1f172ef5badf');
xhr.send();
The responseText contains our previously written document with 2 additional fields: _id and _rev. These fields contain the document's id (so you don't have to store this information yourself) and its current revision. You will need this revision to change or delete this document. 
{"_id":"7f06ce4956f68193f66b1f172ef5badf","_rev":"1-9bfc66754ea484c25293510de7b6ed9f","documentname":"myNewDocument","rating":"totally awesome","someNumbers":12345}

Updating a Document

Updating documents is similar to writing. The only difference is that you have to supply the document's current revision to make sure nobody else changed the document since you've last seen it.

Code: Updating a document in CouchDB

var updatedDocument = '{\"_rev\":\"1-9bfc66754ea484c25293510de7b6ed9f\",\"documentname\":\"myUpdatedDocument\",\"rating\":\"still awesome\",\"someNumbers\":42}';
xhr.open('PUT','http://localhost:5984/demodb/7f06ce4956f68193f66b1f172ef5badf');
xhr.send(updatedDocument);
The old version is now replaced with our new document. Note that couchdb doesn't delete an old version immediately. They are still accessible until you decide to "compact" the database. See the CouchDB website for details. 
{"ok":true,"id":"7f06ce4956f68193f66b1f172ef5badf","rev":"2-23892d3ee7cf3b99899dadd3c3ffabe2"}
If you supply a wrong revision identifier (which could mean that somebody has changed this document already) CouchDB refuses to update this document. 
{"error":"conflict","reason":"Document update conflict."}

Adding an Attachment

Adding an attachment to a document is like writing a file to webdav. The documents URI acts as path for the new file. The main difference is that you need to supply the current document revision because you are updating the document.

Code: Adding an attachment to a document.

xhr.open('PUT','http://localhost:5984/demodb/7f06ce4956f68193f66b1f172ef5badf/newpic.jpg?rev=2-23892d3ee7cf3b99899dadd3c3ffabe2');
xhr.send(someSFImageField,'SFImage2jpg');
{"ok":true,"id":"7f06ce4956f68193f66b1f172ef5badf","rev":"3-d00474de6495109df7cdfe4e194ccd9f"}

Deleting a New Document

If you plan to delete a document, you need its id and its current revision string. A revision string is necessary to assure that nobody has changed the document since you have last seen it. Let's delete our updated document:

Code: Deleting a document in CouchDB

xhr.open('DELETE','http://localhost:5984/demodb/7f06ce4956f68193f66b1f172ef5badf?rev=3-d00474de6495109df7cdfe4e194ccd9f');
xhr.send();
Note that couchDB will return a new revision for your document. The old revisions will be available in the db until you decide to "compact" your db. 
{"ok":true,"id":"7f06ce4956f68193f66b1f172ef5badf","rev":"4-e02f8207d71b9ac2c93f11a25be9a770"}
All further attempts to access the current version of this document will return a not_found error with the reason deleted. 
{"error":"not_found","reason":"deleted"}
If you try to delete a document without supplying the right revision identifier, CouchDB refuses to delete it. 
{"error":"conflict","reason":"Document update conflict."}