PALS3004/G308 Web Programming
SPEECH, HEARING & PHONETIC SCIENCES
UCL Division of Psychology and Language Sciences
PALS3004/G308 Web programming for research in psychology and language sciences

8. Server-Side Communications

Learning Objectives

By the end of the session you should:

Topics

  1. Server-side requests
  2. We tend to think of web pages as static HTML-coded information delivered from a web server. However with Javascript, web applications can act as fully-fledged network applications, retrieving data from the server and sending data back to the server while they operate. This communication with the server can be performed without the current page being replaced or reloaded.

    At the heart of communication with the server is a Javascript object with the awkward name of XMLHttpRequest. Methods of this object allow you to request the server to supply some data to your application, or to request the server to take some data and store it at the server end.

    Note: for security, XMLHttpRequest can only communicate with the web server that was the original source of the web application. This also means that you can only use XMLHttpRequest calls when the page has been delivered by a server - it does not work for applications running from local file storage, for example.

    The use of XMLHttpRequest follows these stages:

    1. Create a new XMLHttpRequest object
    2. Send a request for a resource to the server
    3. Wait for the request to be completed
    4. Check that the request was satisfied
    5. Process any data that was returned from the server

    In this example, we show how XMLHttpRequest can be used to obtain a file of plain text from the server and display it in a textarea element:

    function button1click() {
       var req=new XMLHttpRequest();
       req.open("GET","mydata.txt");
       req.onreadystatechange=function () {
          if ((req.readyState==4)&&(req.status==200))
             processData(req.responseText);
       };
       req.send();
    }
    function processData(txt) {
       document.getElementById("response").innerHTML=txt;
    }
    ...
    <button onclick="button1click()">Get Text</button>
    <textarea id="response" rows="25" cols="80">
    </textarea>
    

    In this example you should see how we create a new XMLHttpRequest object, call its open() method with the name of the resource on the server, specify a function which is to be called when the request is processed, and then send the request. As the request is processed, the onreadystatechange function will be called multiple times. When and only when the readyState property has the value 4, and the status property has the value 200 will the request have completed satisfactorily. In this situation we send the response text to our own function to display it.

    In the following example we upload information to the server. Let's assume there is a server application written in PHP that accepts the following pieces of data and stores them:

    NameDescription
    subjectSome identifying code for a subject
    scoreSome number representing the subject's performance

    The way in which such a script would be called from a browser would be to create a URL formatted like this:

    http://host/myapp/storeinfo.php?subject=23&score=75

    Here "storeinfo.php" is the PHP server-side script. The '?' separates the script parameters from its name, and the parameters are supplied as "name=value" pairs separated by '&'.

    We can achieve the same result from within a web application using XMLHttpRequest. We point the request object at the server-side script and encode the parameters in a similar way:

    function saveResult(subject,score) {
       var req=new XMLHttpRequest();
       req.open("POST","storeinfo.php");
       req.setRequestHeader("Content-Type","application/x-www-form-urlencoded");
       var msg="subject="+subject+"&";
       msg += "score="+score;
       req.send(msg);
    }
    

    In this example we use the POST method to send the data values to the server side script. We tell the XMLHttpRequest object that these values will be encoded in the standard format used for web forms. We then create a text string in which the parameter-value pairs are separated by "&". Note that in this example we do not have any function that is called when the process runs, so we cannot check whether the request completed satisfactorily.

    One final note: when sending information by the POST method, you must ensure that illegal characters are "escaped"; for example parameter values may not contain spaces, quotes or the ampersand character. If there is a risk that your parameter values contain illegal characters you should first encode the strings using the JavaScript function encodeURIComponent, for example:

       var msg="subject="+encodeURIComponent(subject)+"&";
       msg += "score="+encodeURIComponent(score);
       req.send(msg);
    

  3. JSON coding
  4. The kind of information we want to exchange with the server may be of various or mixed type. For example we may want to store and retrieve numbers, strings, booleans, arrays or structured objects. However the XMLHttpRequest mechanism works best when everything communicated to the server is coded as strings of characters.

    Fortunately, Javascript provides two functions which will encode an arbitrary Javascript expression into a string value, and decode the resulting string value back to a copy of the original data.

    The encoding and decoding routines are based on a string formatting notation called 'Javascript Object Notation' or JSON.

    To encode a Javascript object or array into a JSON string, call the global function JSON.stringify(). To decode a JSON string back to an object, call JSON.parse(). For example:

    var obj={ x:1, y:[ false, "hello" ] };
    var jstr=JSON.stringify(obj);
    var newobj=JSON.parse(jstr);
    

    Here, jstr would contain "{x:1,y:[false,'hello']}", and newobj would contain a deep copy of obj.

    You can use the JSON functions to convert any complex data into a string for storage server-side, and to convert a string returned from the server back into a Javascript object.

  5. Saving and Retrieving results on the UCL server
  6. For the purpose of constructing web experiments on this course, two server-side scripts have been written to facilitate the saving and retrieving of experimental data. In addition, two JavaScript functions have been written that package the XMLHttpRequest calls needed to send and load data.

    The UCLSave() function has this signature:

    UCLSave(user,expt,subj,cond,stim,resp,callback)

    ParameterDescription
    userA string identifying the student
    exptA string identifying the experiment
    subjA string identifying the subject
    condA string identifying the condition
    stimA string identifying the stimulus
    respA javascript object holding the response
    callbackThe name of a function to be called when the operation completes

    The fields user, expt, subj, cond and stim must be supplied as (non-empty) text strings which identify a particular piece of data. The resp field is an arbitrary JavaScript object, where you can store any information. This will be encoded using JSON.stringify() before it is put in the database.

    Here is an example call:

    <script src="include/uclserver.js"></script>
    <script>
    function report(val) {
      console.log("UCLSave() returns " + val);
    }
    function recordResponse(rtype) {
       var rt=performance.now()-stimtime;
       UCLSave("Mark","Demonstration","1",stimtab[stimpos].condition,
          stimtab[stimpos].stimulus,{ "rtype":rtype, "rt":rt },report);
    }
    </script>
    

    In this example, we assume that the current condition and stimulus are stored in some stimulus table (stimtab) at some position (stimpos). Also that we recorded the value of the performance timer when the stimulus was presented (stimtime). When the subject responds, the recordResponse function is called and stores away the type of response and the reaction time in the database. Here, the callback function is only used to put a debug message on the console.

    The partner function UCLLoad() has this signature:

    UCLLoad(user,expt,subj,cond,stim,callback)

    ParameterDescription
    userA string identifying the student
    exptA string identifying the experiment
    subjA string identifying the subject, or "" for all subjects
    condA string identifying the condition, or "" for all conditions
    stimA string identifying the stimulus, or "" for all stimuli
    callbackThe name of a function to be called with the array of responses

    The function queries the database for the specified experimental results and passes any matching values to the callback function as an array. Each row of the returned array is a structure with fields: id, date, user, expt, subj, cond, stim and resp. The resp property will be the output of JSON.parse applied to the string data retrieved from the database (i.e. it should match the object structure you sent in the resp parameter of UCLSave). Note that you must supply both the user and the expt fields in the query (i.e. you cannot query across users or across experiments).

    Here is an example call:

    <script src="include/uclserver.js"></script>
    <script>
    function process(data) {
       var txt="";
       if (data==null) {
          txt = "Request failed";
       }
       else {
          var sum=0,cnt=0;
          for (var i=0;i<data.length;i++) {
             sum = sum + data[i].resp.rt;
             cnt++;
          }
          txt = "Mean reaction time = " + (sum/cnt).ToFixed(1) + "ms";
       }
       document.getElementById("performance").innerHTML=txt;
    }
    function button2click() {
      UCLLoad("Mark","Demonstration","","1","",process);
    }
    </script>
    

    In this example the UCLLoad() function is called to return all results from all subjects for condition 1 in Mark's Demonstration experiment and to call the function process with the returned data. Within the process function, the values of the rt property of the resp data field are summed and the mean value calculated and displayed.

  7. Simple Statistics
  8. After we have obtained an array of subject responses back from the server we may want to calculate some statistics, such as a mean, standard deviation, inter-quartile range or t-value.

    There is a freely-available library called Simple Statistics that you can use for this purpose.

    Here are some of the methods available in Simple Statistics:

    MethodExampleDescription
    sumss.sum(x)Sum of a single-dimensional array of numbers.
    meanss.mean(x)Mean of a single-dimensional array of numbers.
    modess.mode(x)Returns the number that appears most frequently in a single-dimensional array of numbers. If there are multiple modes, the one that appears last is returned.
    medianss.median(x)Median of a single-dimensional array of numbers.
    minss.min(x)Finds the minimum of a single-dimensional array of numbers.
    maxss.max(x)Finds the maximum of a single-dimensional array of numbers.
    quantiless.quantile(sample, p)Calculates one or more quantiles of a single-dimensional array of numbers. p must be a fraction between 0 and 1 or an array of fractions. If an array is given, an array of results will be returned instead of a single number.
    variancess.variance(x)Variance of a single-dimensional Array of numbers.
    standardDeviationss.standardDeviation(x)Standard deviation of a single-dimensional array of numbers.
    iqrss.iqr(sample)Calculates the Interquartile range of a sample - the difference between the upper and lower quartiles. Useful as a measure of dispersion.
    tTestss.tTest(sample, x)Calculates Students t-value from a sample of numbers and expected value of the mean under the null hypothesis. The output will need to be checked against a table of t-vales to obtain a significance value.
    tTestTwoSampless.tTestTwoSample (samp_x,samp_y)Calculates Students t-value from two independent samples of numbers on the assumption that the expected value of the difference in means is zero under the null hypothesis. The output will need to be checked against a table of t-vales to obtain a significance value.
    sampleCorrelationss.sampleCorrelation(a, b)Produces sample correlation of two single-dimensional arrays of numbers.
    shuffleInPlacess.shuffleInPlace(sample)Given a sample array (with any type of contents), randomly shuffle the order of the array using the Fisher-Yates shuffle algorithm.

Here is some example code:

<script src="include/simple-statistics.js"></script>
<script>
var sample=[];
for (var i=0;i<100;i++) sample.push(10*Math.random());
console.log("Mean="+ss.mean(sample));
console.log("Median="+ss.median(sample));
console.log("Stddev="+ss.standardDeviation(sample));
console.log("IQR="+ss.iqr(sample));
</script>

Try it out here:


Resources

Exercises

Please note that to use the UCL server-side scripts in exercises 8.2 & 8.3, your programs must be served from the departmental web server. You will need to use your departmental web space account for this.

  1. Implement "XMLHttpRequest" demonstration (demo8-1.html). Note that this demonstration will only work when the page is delivered by a web server. You will also need to create the file mydata.txt. Study the code to understand how the request is sent to the server and how the information is sent back.
  2. Implement "Simple Server-side storage" demonstration (demo8-2.html). This example calls the following server-side scripts which you can also test out directly from a web browser:
  3. http://www.phon.ucl.ac.uk/courses/spsci/webprog/storeinfo.php?subject=name&score=100

    http://www.phon.ucl.ac.uk/courses/spsci/webprog/loadinfo.php

    In the processData function, add code that will find and report the subject with the highest score (ex8-2.html). You can use the string methods split("\n") to divide the response string into lines, indexOf to find the ",", and substring to extract the score.

  4. Implement "UCL Server-side storage" demonstration (demo8-3.html). This example demonstrates the course-specific server-side scripts for storing/retrieving experimental data. The experimental task is a simple Lexical Decision task. Modify the code (ex8-3.html) to calculate the mean and standard deviation of the reaction time for each condition.

Homework

  1. Complete the exercises you did not finish in class. Upload your answers to your course web space.
  2. Work out for your web experiment what information you need to store on the server. Then write a simple test program that uploads some sample experimental data using the UCLSave() function. Note that your test program must be run from your departmental web space account to use the UCL server-side scripts. You can test whether the information has been uploaded correctly using the web page:
  3. http://www.phon.ucl.ac.uk/courses/spsci/webprog/UCLTest.html

  4. Next write a simple test program that uses UCLLoad() to download and display all the experimental data you saved with your test program in 2.

Word count: . Last modified: 23:20 26-Nov-2016.