A basic AJAX request and .json data manipulation using .map(), .filter(), .reduce() and arrow functions


Today we are going to have a lot of fun!

As the title suggests, we are going to go through a basic AJAX request and different ways to work with the data from the server response.
There are different programming languages to achieve our goal, but we're going to focus on Javascript here.

The complete example is available on github

The setup

In order to retrieve data from a URL, we need to initialize an XMLHttpRequest. Despite its name, it can retrieve any type of data (not only XML) and it supports protocols other than HTTP such as file (our case in this example) and ftp.

If we are working on a local environment and trying to launch our .html file, the XMLHttpRequest will throw an error you can look up in your browser console log. The message will be something like - file could not be loaded as Cross Origin requests are only supported for HTTP -.
Of course there is a good reason for it (security) but there are quite few solutions to overcome the problem.

My solution is to set up a simple command-line server.
Again, many options here. Mine is the http-server module installed through node.js package manager (NPM).

The AJAX request

Let's start with our AJAX request:

let request = new XMLHttpRequest();
request.open('GET', 'rank.json');
request.onload = function () {
    const data = JSON.parse(this.response);

    // additional code - see later on

}
request.send();

At its core, this is a simple get request to retrieve data from a web server.

I intentionally left out the .onreadystatechange property as we assume that readyState is 4 (request finished and response is ready) and status is 200 (OK).
As our request completed successfully, we can now wrap our function (what to do with our retrieved data) in the .onload event handler.

When receiving data from a web server, the data is always a string. Parse the data with JSON.parse(), and the data becomes a JavaScript object.

Lastly, don't forget to .send the request!

Manipulate the data

We made our server request, we received the response correctly, we parsed the .json file and ... nothing is displayed unless we declare what what to do.

Let's look at our .json file and its structure:

{
  "cities": [
    {
      "name": "Rome",
      "country": "Italy",
      "stats": {
        "area_kmq": 1287.36,
        "population": 2868872,
        "temperature": 25.2
      },
      "visited": true
    },
    {
      "name": "Berlin",
      "country": "Germany",
      "stats": {
        "area_kmq": 891.68,
        "population": 3613495,
        "temperature": 19.5
      },
      "visited": false
    },
    {
      "name": "Paris",
      "country": "France",
      "stats": {
        "area_kmq": 105.4,
        "population": 2206488,
        "temperature": 20.7
      },
      "visited": true
    }
  ]
}

Let's imagine that the above is the server response from an app we use to track our travels.

We have 3 cities with some statistics and our flag whether we already visited them or not.

Different approaches to access the data

After parsing our .json file, let's define our starting point for accessing the data.

const cities = data.cities;

Now, assume we just want the list of city names in our wishlist.
The expected result is an array containing: Rome, Berlin, Paris.

We have many possibilities. Some are faster, some make the code cleaner to read, some have advantages over the others.

Let's start with my favourite one: .map() with arrow functions (ES6)

let method_1a = cities.map(city => city.name);
console.log(method_1a);

// Array(3) ["Rome", "Berlin", "Paris"]

We could also use the traditional .map() method this way:

let method_1b = cities.map(function (city) {
  return city.name
});
console.log(method_1b);

// Array(3) ["Rome", "Berlin", "Paris"]

Then we have our old .forEach():

let method_2 = [];
cities.forEach(function (city) {
  method_2.push(city.name)
});
console.log(method_2);

// Array(3) ["Rome", "Berlin", "Paris"]

Or we could use the for...in loop:

let method_3 = [];
for (let city in cities) {
  method_3.push(cities[city].name)
};
console.log(method_3);

// Array(3) ["Rome", "Berlin", "Paris"]

As well as the for() one:

let method_4 = [];
for (let i = 0; i < cities.length; i++) {
  method_4.push(cities[i].name)
};
console.log(method_4);

// Array(3) ["Rome", "Berlin", "Paris"]

So here it is. 5 ways to query the same data having the same exact result: an array containing the names of our beloved cities.

Which one is better?
Well it depends on the case, but I would still prefer the .map() method for different reasons:

  • you can use it in combination with .filter() and .reduce() to build wonderful things (see next paragraph)
  • you avoid having to manage loops and indexes
  • you avoid having to create empty arrays and push values into them (basically modifying original arrays)

.map(), .filter() and .reduce()

Among these 3 methods, .reduce() is probably the most powerful one. In many cases you could actually avoid using .map() and .filter() and just use .reduce()

Let's see an example.

What if we'd like to know the cities which we haven't visited yet?
Looking at our .json file the expected result is an array with only 1 city: Berlin.

let not_visited = cities
  .filter(city => city.visited === false)
  .map(not_yet => not_yet.name);
console.log(not_visited);

// Array(1) ["Berlin"]

Here is the beauty: we are chaining our methods together and making the code concise with arrow functions.

What the above code does is using the .filter() method to return an array containing the Berlin object and then creating the final array containing the names of the cities (only 1 in this case).
Try to remove the === false (or set it to true) and now the final array will contain Rome and Paris.

Pretty cool huh?

Now let's throw .reduce in the equation and see how we can use it.

We want to know the summed up population of all the cities that we have already visited.

As before we have to first filter the cities which we have already visited (Rome and Paris), then we create an array containing the population of only those cities (2.9 M and 2.2 M) to come to the final total of almost 5.1 M

let total_population = cities
  .filter(city => city.visited)
  .map(visited => visited.stats.population)
  .reduce((acc, population) => acc + population, 0);
console.log(total_population);

// 5075360

While learning to code, I came across to a solution which make use of .reduce method only.
The above code could also be written as:

let reduce_only = cities.reduce((acc, total_population) =>
  total_population.visited ? acc + total_population.stats.population : acc, 0);
console.log(reduce_only.toLocaleString('de-DE');

// 5.075.360 --> prettier formatting with .toLocaleString :)

I prefer the first one as my mind find it easier to follow, but it's totally up to you.

Average and extraction

Let's finish our long post with last 2 examples.

We now look at temperatures!

What if we were to calculate the average temperature for the cities we visited?
As we learned at school, the average (Arithmetic mean) is the sum of a collection of numbers divided by the number of numbers in the collection. So here we go:

let temperatures = cities
  .filter(city => city.visited)
  .map(visited => visited.stats.temperature);

let avg_temperature = temperatures.reduce((acc, temperature) => 
  acc + temperature, 0) / temperatures.length;
  console.log(Math.round(avg_temperature));

// (22.95) --> 23 with Math.round

As in previous examples, we first create the array that filter the cities already visited and stores the temperatures values (let temperatures). Then we sum the values in the previous array with .reduce() and divide it by the length of the array (2 as there are two cities/values).
The result is a 22.95 mean average rounded to the nearest integer with Math.round function.

Last one! Can we also extract selected pieces of information from our .json? Say we want an array which contains only the city names and their temperatures.

let extraction = cities
  .map(i => ({name: i.name, temperature: i.stats.temperature}));
console.log(extraction);

// (3) [{…}, {…}, {…}]
// 0: {name: "Rome", temperature: 25.2}
// 1: {name: "Berlin", temperature: 19.5}
// 2: {name: "Paris", temperature: 20.7}

This wraps it up for today and keep coding!