Building a realtime feed with Node.js and AMP

building-a-realtime-feed-with-node-js-and-amp.jpg

In this tutorial, we will learn how to develop a realtime feed with pagination on AMP using the \`amp-live-list\` component and Node.js.

Introduction

Something we can’t deny about Accelerated Mobile Pages (AMP) is that they are fast.

On mobile devices, web pages can take several seconds to load. However, AMPs are web pages designed to load near instantaneously due to the limited set of allowed functionality they provide and a distribution system built around caching.

AMP also uses custom elements called AMP components to implement features with complex interactions such as video players or embeddable content.

In this tutorial, we’re going to implement a realtime feed in AMP using two of these components:

In the backend, we’ll use a Node.js/Express web server with EJS as the template engine.

This is how the final application will look:

The amp-live-list component will regularly poll the server for updated content. When a new item is added, a button will be shown, which when clicked, will update the page to show the latest videos (with a simple pagination).

You can find the entire source code of the application on this GitHub repository. Also, you need to take into account that realtime features on AMP pages are limited and you can do a lot more using Pusher in your website or app.

Setting up the project

Let’s start by creating a package.json file for our project:

1npm init

Or if you want to accept all the defaults:

1npm init -y

Next, install body-parser, ejs, express, and moment as dependencies:

1npm install body-parser ejs express moment

The project will have the following structure:

1|— views
2| |— index.ejs
3|- package.json
4|- server.js
  • The view directory contains the EJS template for the main page of the app.
  • In the root directory, we can find the package.json file, as well as the file for the Express server (server.js).

Once you have created the necessary files and directory, you’ll be ready to start coding.

Building the Express server

The web server is a standard Express app with an array to hold the information of the videos we’re going to show.

We will have two endpoints:

  • /, with an optional page query parameter, which will render the template with the list of videos (paginated).
  • /new, which will register a new video.

In the server.js file, let’s start by requiring the modules we’re going to use:

1const express = require('express');
2    const bodyParser = require('body-parser');
3    const moment = require('moment');

Next, define the array for the videos and a constant for the number of posts that will be shown on a page:

1const posts = [];
2    const pageSize = 5;

Now let’s configure the Express server to use EJS as the view engine and the JSON parsing:

1const app = express();
2    app.set('view engine', 'ejs');
3    app.use(bodyParser.json());
4    app.use(bodyParser.urlencoded({ extended: true }));

This is the definition of the route that will present the videos:

1app.get('/', (req, res) => {
2      let currentPage = 1;
3      const totalPosts = posts.length;
4      const pageCount = Math.ceil(totalPosts / pageSize);
5
6      if(req.query.page) {
7        currentPage = parseInt(req.query.page, 10);
8      }
9
10      const start = (currentPage - 1) * pageSize;
11      const end = currentPage * pageSize;
12
13      res.render('index', 
14        {
15              posts: posts.slice(start, end),
16              pageSize: pageSize,
17              pageCount: pageCount,
18              currentPage: currentPage,
19        }
20      );
21    });

As you can see, we are implementing simple pagination, calculating the page count, and the start and end elements of the post array that will be shown based on the optional page query parameter.

Then, in the last lines, the index template is rendered, passing an object with the properties we’ll need.

This is the definition of the route to registering a new video:

1app.post('/new', (req, res) => {
2      const post = {
3              id: posts.length + 1,
4              videoID: req.body.videoID,
5              timestamp: moment().unix(),
6              timestampStr: moment().format(),
7      };
8
9      posts.unshift(post);
10
11      res.status(200).json(post);
12    });

We create a new object with an ID based on the length of the array of posts, the ID of the video to show (req.body.videoID), and a timestamp in Unix and ISO 8601 format (for example, 2017-06-09T08:02:17-05:00). Then we add this object at the beginning of the array with unshift and return a success response.

Finally, let’s start the server with:

1app.listen(3000, () => {
2      console.log('Server listening on port 3000');
3    });

Building the live blog template

An AMP HTML page will act as the content of the template. A good starting point is the boilerplate code you can find on the Create Your First AMP Page official tutorial.

For our app, we’ll use a modified version of that code. Open the file views/index.ejs and copy the following code:

1<!doctype html>
2    <html amp lang="en">
3      <head>
4        <meta charset="utf-8">
5        <script async src="https://cdn.ampproject.org/v0.js"></script>
6        <title>AMP Live Video Blog</title>
7        <link rel="canonical" href="http://localhost/" />
8        <meta name="viewport" content="width=device-width,minimum-scale=1,initial-scale=1">
9
10        <link href='https://fonts.googleapis.com/css?family=Roboto' rel='stylesheet'>
11
12        <script async custom-element="amp-youtube" src="https://cdn.ampproject.org/v0/amp-youtube-0.1.js"></script>
13        <script async custom-element="amp-live-list" src="https://cdn.ampproject.org/v0/amp-live-list-0.1.js"></script>
14
15        <style amp-boilerplate>body{-webkit-animation:-amp-start 8s steps(1,end) 0s 1 normal both;-moz-animation:-amp-start 8s steps(1,end) 0s 1 normal both;-ms-animation:-amp-start 8s steps(1,end) 0s 1 normal both;animation:-amp-start 8s steps(1,end) 0s 1 normal both}@-webkit-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-moz-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-ms-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-o-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}</style><noscript><style amp-boilerplate>body{-webkit-animation:none;-moz-animation:none;-ms-animation:none;animation:none}</style></noscript>
16
17      </head>
18      <body>
19
20      <body>
21    </html>

For this example, we’re not going to need the JSON metadata from Schema.org. It’s optional but required to make your content eligible to appear in the Google News carousel.
However, we need to keep the canonical link reference, as it is required for the page to be valid. This link must point to itself (the AMP version of the page), so it’s just changed to:

1<link rel="canonical" href="http://localhost:3000/" />

Refer to this page to know more about both options.

AMP allows the use of custom fonts with some restrictions. This page uses the Roboto font:

1<link href='https://fonts.googleapis.com/css?family=Roboto' rel='stylesheet'>

All AMP components used must be imported in the header. The following are the scripts for the YouTube and live list components we’re going to use:

1<script async custom-element="amp-youtube" src="https://cdn.ampproject.org/v0/amp-youtube-0.1.js"></script>
2    <script async custom-element="amp-live-list" src="https://cdn.ampproject.org/v0/amp-live-list-0.1.js"></script>

In addition to the boilerplate style, we’re going to use some custom CSS styles. Since AMP can’t reference external stylesheets, all styles must live in the head of the document with some restrictions, so let’s add our custom CSS inside a <style amp-custom> tag in the head of the document:

1<!doctype html>
2    <html amp lang="en">
3      <head>
4        ...
5
6        <style amp-custom>
7          body {
8            font-family: 'Roboto', sans-serif;
9            background: #EDEDED;
10          }
11          .header {
12            color: #fff;
13            padding: 20px  20px;
14            top: 0px;
15            width: 100%;
16            background: #E53935;
17            text-align:center;
18          }
19          #amp-live-list-videos {
20            margin-bottom: 50px;
21          }
22          #live-list-update-button {
23            color: #E53935;
24            border: 1px solid #E53935;
25            padding: .7em .8em;
26            cursor: pointer;
27            background-color: #fff;
28            text-transform: uppercase;
29            letter-spacing: .2em;
30          }
31          .post {
32            margin-top: 20px;
33          }
34          #live-list-update-button {
35            position: fixed;
36            top: 10px;
37            right: 16px;
38          }
39          .pagination {
40            display: inline-block;
41            padding: 0;
42          }
43          amp-live-list [pagination] nav {
44            display: flex;
45            align-items: center;
46            justify-content: center;
47          }
48          .pagination li {
49            display: inline;
50            padding: 10px;
51            list-style-type: none;
52            color: #000;
53          }
54        </style>
55
56      </head>
57      ...
58    </html>

Now, in the body section, let’s add the page header:

1<!doctype html>
2    <html amp lang="en">
3      <head>
4        ...
5      </head>
6      <body>
7        <div class="header">
8          <h1>AMP Live Video Blog</h1>
9        </div>
10      </body>
11    </html>

And the amp-live-list component:

1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            id="amp-live-list-videos"
8            data-poll-interval="15000"
9            data-max-items-per-page="<%= pageSize %>"
10            <% if (currentPage > 1) { %>  disabled  <% } %>
11        >
12
13        </amp-live-list>
14      </body>
15    </html>

The id attribute identifies the amp-live-list component since multiple live lists are allowed on a single page.
The data-poll-interval attribute specifies how frequently the component will poll new data (in milliseconds). The minimum interval is fifteen seconds.
The data-max-items-per-page specifies the maximum number of items displayed on the page.
If we’re not on the first page, we’re going to add the disable attribute so the component stops polling for new data. (Since new posts will be added to the first page due to how the pagination mechanism works, there’s no need to poll the server if the user isn’t on that page.)

In this example, the amp-live-list component contains three elements as direct children.

The first element is a button with the event on="tap:[ID_OF_THE_LIVE_LIST_COMPONENT].update" to load the latest changes from the server. This button is required and it will appear when new changes are discovered. These changes won’t be shown to the users until they click the button:

1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            ...
8        >
9          <button
10              id ="live-list-update-button" 
11              update on="tap:amp-live-list-videos.update" >
12            You have updates
13          </button>
14
15        </amp-live-list>
16      </body>
17    </html>

The second element is <div items>, the list of items to show. These items are required to have a unique id and a data-sort-time attribute (a timestamp used for sorting entries. Higher timestamps will be inserted before older entries):

1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            ...
8        >
9          ...
10
11          <div items>
12          <% posts.forEach((post) => { %>
13
14            <div class="post" id="<%= post.id %>" data-sort-time="<%= post.timestamp %>">
15              <amp-youtube
16                width="480"
17                height="270"
18                layout="responsive"
19                data-videoid="<%= post.videoID %>"
20              />
21            </div>
22
23          <% }) %>
24          </div>
25
26        </amp-live-list>
27      </body>
28    </html>

For each post, we’re using an amp-youtube component to render the video. You can find out more about this component here.

Finally, the third element is <div pagination></div> element. It’s our responsibility to insert within this element any markup needed for pagination (which in this case is only shown if there’s more than one page):

1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            ...
8        >
9          ...
10
11          <div pagination>
12            <nav aria-label="amp live list pagination">
13
14              <% if (pageCount > 1) { %>
15              <ul class="pagination">
16
17                <% for (var i = 1; i <= pageCount; i++) { %>
18                  <% if (currentPage == i) { %>
19                    <li><%= i %></li>
20                  <% } else { %>
21                    <li><a href="/?page=<%= i %>"><%= i %></a></li>
22                  <% } %>
23                <% } %>
24
25              </ul>
26              <% } %>  
27            </nav>
28          </div>
29
30        </amp-live-list>
31      </body>
32    </html>

And we’re done. You can find out more about how the amp-live-list component works on this page.

Testing the application

Execute the web server with:

1node server.js

If you go to http://localhost:3000, you’ll see a blank page:

It’s recommended you use Chrome because you can validate an AMP page just by adding #development=1 to the URL, for example, http://localhost:3000/?page=1#development=1 and view the result of the validation in the Chrome DevTools console:

To register videos, you can use a tool like Postman to hit the http://localhost:3000/new endpoint with a JSON payload like the following:

1{
2      "videoID": "[YOUTUBE_VIDEO_ID]"
3    }

Or curl:

1# POST
2    curl -H "Content-Type: application/json" -X POST -d '{"videoID":"[YOUTUBE_VIDEO_ID]"}' http://localhost:3000/new
3
4    # In Windows, change single quotes to quotation marks and escape the ones inside curly brackets
5    curl -H "Content-Type: application/json" -X POST -d "{\"videoID\":\"[YOUTUBE_VIDEO_ID]\"}" http://localhost:3000/new
6
7    # Or use file, for example data.json
8    curl -H "Content-Type: application/json" -X POST --data @data.json http://localhost:3000/new

After a few seconds (fifteen at most), the button to load the new videos should be visible:

And after adding more than five videos, the pagination links will be shown:

In addition, if you look at the network tab of the console, you’ll see that a request to the server is made every fifteen seconds:

Conclusion

You’ve now learned how to implement a realtime feed with pagination on AMP using the amp-live-list component. We have covered insertions, but updates and deletions are also supported by this component. Here’s an example of how to update existing list items.

Remember that you can find the source code of this app in this GitHub repository.