Decoupled Drupal at DrupalCamp Bristol 2017

In July this year, Louis and I were invited to give a presentation at DrupalCamp Bristol on Decoupled (or Headless) Drupal.

We talked about a few case studies, what the goals were, how we achieved those and what we had to take into consideration in doing so. I should point out that Louis did most of the talking – I’m not a developer, so my role was more to structure and organise the presentation.

The Student Room

The Student Room screenshot

The Student Room (TSR) is the world’s largest online student community, with over 1.8m members. The website provides information for students studying, from GCSE to Degree level. There are a large number of information pages managed by the community as well as by editors – so, for example, users of the website might contribute content on revision techniques.

It started out with just forums for students using the vBulletin software, but as the site grew they began to extend the types of content they were creating on the site, and so they began to bend and contort the forum software to achieve what they needed.

Eventually, they wanted to separate the information pages from the forums and so looked at Drupal to provide the CMS functionality. They chose Drupal 8:

  • Mainly for the CMS features
  • It was new software, and given the lifespan of previous Drupal versions it was future-proof
  • Uses object oriented code with Symfony components, which their developers would be comfortable working on
     

So why was this project “headless”? Well, a key part of it was integrating with a system that TSR developed called “Nova”. Nova is a proxy service that provides a number of functions.

At the core, it’s a routing system that serves the site content, but it does more than just that. The key part of the service is that it provides a bridge between new and legacy systems. In this case, Drupal and the vBulletin forum software.

In a typical built page, Nova would provide the header and footer while the Drupal output is displayed within the main content wrapper. In addition, there’s the ability to have non-Drupal content displayed within the Drupal page.

What did we have to understand about Nova?

How are users authenticated in Drupal if they’ve already logged in elsewhere?

  • Authentication through Nova was required as it’s not just in house editors creating and updating content but also members of the public. This meant that we needed a seamless experience when going from a “legacy page” let’s say, to a Drupal page whilst logged in.
  • The way this works is Nova manages the user session and so that’s always sent to Drupal when requesting a page. We replaced the default authentication provider with a custom service that either created a new user, if it’s the first time that they’ve accessed a Drupal page. Or, to log them into an existing account. Also included in the session data is information about the user’s permissions. There are a number of roles setup on the Drupal side, but these needed to map to equivalent permissions schemes from vBulletin side of things.
  • In terms of the administration actions, the admin menu bar only shows on when the administration theme is displayed, obviously only for those with the permission to see it. Otherwise the links to edit and create pages are provided by a number of conditional blocks with simple button call to actions.
     

How do we theme the Drupal content without impacting other parts of the site?

  • We themed the output of the content that Drupal produced but because Nova was going to be defining the core structure of the page this was stripped back a bit. As the header, footer and main content wrapper elements were provided by Nova we were only sending the rendered page template.
  • Also, because of this we had to make sure what we built was completely fluid as the width constraints were defined elsewhere and our content had to fall into place while still looking good and functioning.
  • At the beginning, we worked on the theming before the integration was in place and so to start it wasn’t much different to a normal project.
  • Although we did have to still consider how our markup and CSS could eventually impact the rest of the site and there was the potential for classes and IDs to conflict as well as certain styles being applied to unintended elements.
  • We avoided any conflicts, mainly by luck! The CSS selectors were specific enough on the TSR end that they didn’t conflict with anything that we defined. And we rarely used generic selectors and did the majority of styling based on specific classes.
     

And how do we display promotional blocks with content from other areas of the site?

  • Lastly, there are a number of widgets that are displayed across the site providing blocks of content from other areas of the site, particularly forum specific content. For example, “popular posts about exam preparation” or user submitted ratings for a particular university.
  • In order to use these widgets on our Drupal pages we had to make use of some of the helper services that the Nova system provided or at least some endpoints that Nova exposes.
  • One of these endpoints allowed us to retrieve a number of university widget blocks based on an identifier for that particular university. Examples of these would be: A list of recent forum posts relevant to the Uni; Average student reviews of the Uni; Contact information for the Uni rep, and so on.
  • The widgets were essentially just tokens inserted into the page that Nova replaced during the page build process. We also defined some call to actions in Drupal and the benefit of using tokens for the Nova widget was that we were able to treat them in the same way as the Drupal blocks and have them displayed in and around other content.
     

The main issue with this project was that when we started, Nova was still in development and so it wasn’t clear exactly how certain parts of the integration were going to work until the very end. For example user avatars weren’t available but we still had to provide the theming based on assumptions as to what the content would eventually be. In addition, D8 was still at an early stage, so there were some areas like Workflow which we had to effectively rewrite.

AVERT and [anonymous]

AVERT screenshot

AVERT is a Brighton-based HIV awareness charity with a significant global reach. We initially pitched to rebuild the AVERT website but lost out to an agency who had guaranteed hitting a timeline we had said was unrealistic. Once that timeline had been greatly exceeded, AVERT came back to us to get the website fit for launch. This involved us doing a health check during which we found so many corners which had been cut that it took almost as long to fix it as to build it the first time.

We also did a project which was technically very similar for another client… but we can’t tell you who they were. In this case, we worked with an agency to support them in building a Drupal 7 solution that powers dynamic billboards for their client. The project was to create a content entry interface that outputted a JSON feed to be used by an external system for display .

  • Editors can create an ad schedule for each day of the week based on images, videos and overlayed text.
  • That would build a JSON file describing the content and transition details
  • A separate “player” application (written in C#) reads the JSON and displays the relevant content. In the AVERT example the JSON feed was read and rendered within a front end powered by a JavaScript framework using ChartJS, JS Socials and jQuery Joyride.
  • Player hits Drupal every minute to get the latest schedule and any changes
  • Player ingests the schedule and then builds a preemptive timeline – eg the timeline may have a month’s worth of ad schedules.
     

When producing content for an unknown display layer we had to think about various issues:

Validating content

  • Due to the restrictions in the player application as well as the physical constraints from the billboard screens, validating the uploaded media was of great importance.
  • We had to validate video bitrates, dimensions, verifying file codec and even the frame rate – all this happened on the drupal side.
     

Security

  • Given the public nature of content being added and displayed, there was a large emphasis on the security of the system.
  • Usual practices of IP restriction combined with multi-factor authentication were used to protect the Drupal system.
  • During the project the agency we were working with had penetration testing performed on the site and after a bit of scare of a potential SQL injection vulnerability nothing of any note was raised.
  • The player was put under the same scrutiny, as well as the way that resources were transferred from Drupal to the Player.
     

Development environment

  • It’s best practice to have a staging environment that mirrors the live environment but obviously in this case that wasn't possible.
  • The agency set up a dedicated machine in their offices with a number of monitors plugged in and created a simulator to be able to demo the billboard display.
  • One of the problems we had was around debugging any issues that came up because we didn’t have any way of actually previewing the output. But this wasn’t a huge issue down to all the validation in place and good communication.

Amazon Alexa

Amazon Echo stock image

At miggle we aim to put aside 10% of time each week to be used for R&D. To help inspire this Alick, our MD, brought all team members an Echo Dot for Christmas 2016, and we had a hack day to see what ideas we could come up with.

Alexa is an intelligent personal assistant developed by Amazon, made popular by the Amazon Echo and the Amazon Echo Dot devices. It is capable of voice interaction and can also control several smart devices using itself as a home automation system.

Louis blogged about the results and we made some videos of the demos we did. There’s a lot of interest in what we wrote (it’s one of our most popular blog posts) but commercially there’s been very little interest so far. We’d like to know what other agencies who have looked at this have seen.

We saw the main benefit being its ability to provide succinct snippets of information, similar to Call to Action blocks that would exist on a website. There were a few examples of these sorts of blocks in projects we had worked on.

Amazon has a specialised schema which uses intents, skills, invocations and utterances. We went through these and built examples for our clients PictureBox Films and Ski Safari.

In the end, we thought that the system has exciting potential for content presentation. However, sometimes the skill just would not work for no apparent reason, and we had to start over from scratch.

What we learned

  • Validation plays an important role in ensuring unexpected behaviours from external services.
  • Having a good early understanding of the service that Drupal is integrating with, and what it is capable of or limited by, can help to avoid any major issues.
  • And finally, in cases where there’s no real documentation around the 3rd party system, it’s very important to have good communication and cooperation with the teams involved.

Resources