Magento PWA Studio: What is UPWARD?

Unrelated, please ignore…

For the past few months Magento has been working on PWA Studio which is, in summary, a collection of tools to help build modern Progressive Web Applications on top of Magento 2 stores.

For the most part these tools have been used in production by other projects with great success so it makes sense that PWA Studio is built on top of them, for example:

• Peregrine: collection of React-based UI components to help reuse common functionality like routing, render pages, products, prices, etc.
• PWA Buildpack: build and layout optimisation tooling, it expands webpack to help configure PWA Studio components and sets up local development environment.
• Venia: core storefront built using PWA Studio’s libraries, it uses React, Redux, GraphQL, CSS Modules and many other technologies as part of its stack.

However there’s something that’s not as well known as the rest. UPWARD was built for PWA Studio so it makes sense most people haven’t heard about it but it’s one of its most important pieces, in fact I’d say it’s what glues everything else together.

But before talking about what it is, it’s important to know why it was made…

In the beginning

Earlier versions of the Venia storefront used roughly the same technology stack in the frontend but it was still developed as a regular Magento 2 theme, it wasn’t a common approach specially as similar projects like Vue Storefront or Deity’s Falcon were going for a more headless approach.

It made sense to remove this hard dependency on Magento but something was still needed to handle communication between the two. Of course you could deal with Magento’s API directly but:

• What if you had to use multiple APIs, not only Magento’s, how do you organise all these different endpoints?
• What if you need to extend or improve any of the core API’s? If you need to replace some endpoints, is it easy to do so without many side-effects?
• What about third party API’s that are not specific to Magento (social media, shipping calculations, CMS, etc)? Will you need to create custom extensions just to interact with them?

These are the kind of problems UPWARD helps solve by having a single place where requests are made to and always return data the way you expect it to.

So what is it then?

To put it in simple terms: UPWARD is the server that acts as the middle layer between your PWA and your APIs, it has multiple benefits but for me the main one is that it helps unify all APIs in a single place so you don’t have to keep track of multiple endpoints and how to access them.

Maybe James Zetlen, the mind behind UPWARD, can help explain it better:

What makes UPWARD so unique is that it’s not a server itself, it’s actually a spec that describes how a server should respond when data is requested in different ways. In the same way that GraphQL is a spec and libraries like Apollo and Relay are implementations of this spec, UPWARD has officially been implemented in NodeJS and PHP but there’s no reason why it can’t be implemented in other technologies.

The way that UPWARD defines how the server behaves is with a YAML configuration file using a special syntax. This file is similar to how the .htaccess file is to Apache servers, they don’t work exactly the same but I found the comparison helps wrap your head around the concept.

Even though UPWARD was made for PWA Studio it is platform agnostic so it can be used for anything that requires interaction with a server, it doesn’t even have to be a PWA!

So how does it look like?

As with many things it’s usually easier to understand with examples, I’ll showcase some use cases without using any kind of framework because it makes it simpler to explain and doesn’t require any kind of special tooling but you can adapt them to your JS flavour of choice very easily.

1. Hello World

Let’s start by creating a new node server and installing the required dependencies, I’ll be using yarn for these examples but feel free to use npm if you prefer:

yarn add @magento/upward-js express

UPWARD-JS is built on top of express so we need to install it as a peer dependency.

Now let’s add a single command to our “scripts” block in the package.json file:

"start": "node server.js"

This will start our server using a server.js file that we’ll create now:

This is one of the simplest ways to start an UPWARD-JS server, we only need to call the createUpwardServer function with a few required parameters. Most of them are self-explanatory, all we’re doing here is creating and binding a new server using port 8000, enabling logging for debugging and passing the path to our YAML configuration file.

As you can see the configuration file is fairly easy to read (at least for now…), UPWARD requires that we specify at least 3 properties: the server status code, headers and body. In this case we’re configuring the server to respond to all requests with a 200 response code and the text “Hello World” as HTML in the body.

Notice the “inline” property lets us define values as strings directly in the configuration file, this is known as a resolver in the UPWARD spec and there are more of them we’ll get to know later.

Now if we run yarn run start and visit http://localhost:8000 we’ll be able to see the results of our hard work:

Who needs express uh?

As it is now all URLs will return the same “Hello World” string, efficient but not very useful. Let’s see how we can make something that can actually help us in the real world.

2. Proxying requests

One of the most common use cases in modern web apps is to interact with multiple APIs, instead of having to track multiple endpoints we can setup our UPWARD server to proxy requests to different services so we only need to send requests to it (goodbye CORS issues!).

For this we need to start using resolvers in our YAML configuration, resolvers define the way data is obtained and it’s defined by a property in our YAML file. In the previous example we used the “Inline Resolver” which returns a string but there are many others available for more complex scenarios.

Let’s pretend we want to query two different GraphQL APIs, one for retrieving Pokémon data (gotta catch ’em all!) and another one for countries data. These are two different services but we want to set them up so we can use:

• http://localhost:8000/pokemon/graphql to query one the Pokémon API.
• http://localhost:8000/countries/graphql to query the countries API.

We’ll need to extend our configuration file to look something like this:

Ah things looking more interesting, I added some comments to the file as it’s starting to become more of a flowchart. As you can see the status, headers and body is not hardcoded anymore but instead is retrieved by a response property, this can be named however you want but what that means is that UPWARD will execute this response to get the values that it needs. Let’s follow the request flow in order:

#1 response: this property is using a “Conditional Resolver” which works the same as the PHP switch statement. In this case we’re checking:

1. If the URL matches “/pokemon/graphql” it will use the pokemonGQL resolver,
2. If it matches “/countries/graphql” it will use the countriesGQL resolver,
3. Otherwise use the helloWorld resolver.

#2, #3 pokemonGQL/countriesGQL: these properties are using a “Proxy Resolver”, all they do is set the URL for which requests will be proxied to, in our case we hardcoded the URLs for our third-party services using the “Inline Resolvers” we used before.

#4 helloWorld: if the URL doesn’t match any of the previous we’ll return the same “Hello World” string as before, I just moved the configuration to its own resolver now.

Since nothing else needs to be resolved UPWARD can finally return the response, if everything went right we should have access to our brand new endpoints:

3. Rendering HTML

Displaying a “Hello World” string is super performant but not very useful, so let’s change it up a bit to show more complex HTML markup and explore other types of resolvers.

By default UPWARD supports using the Mustache templating system, it’s very lightweight and basic in terms of features but that’s often a good thing as it prevents us from having too much logic mixed in with our templates. If you want to know more about it feel free to read the official repo.

So let’s use Mustache to display some Pokémon data instead of the “Hello World” string, which will also be a good chance to use a “Service Resolver”:

The first part of the configuration file is the same, but things start to look different in the response property:

#2, #3 pokemonGQL/countriesGQL: Instead of hardcoding the GraphQL endpoints I changed it to use environment variables, they can be set by passing an env parameter to the createUpwardServer function or by using libraries like dotenv. In any case, this can help make our configuration more customisable as you can have different values per environment.

#4 pokemon: We’ll be using the “pokemon” property as the default response now. For this we’ll use a “Template Resolver” to render the body with a Mustache template, this can be set in its own file and we can pass the path to it to keep things more organised. Notice the provide property here as well, this defines the data that will be available in our template which in turn can be a resolver as well, let’s keep going down the rabbit hole.

#5 pokemonData: This will make more sense once we go one step further, for now understand that this is calling another “pokemonResult” property.

#6 pokemonResult: “Service Resolvers” allows us to obtain data from a GraphQL service so we’ll use just that to get the Pokémon information. It takes the url to the endpoint and query as a parameter which we’re hardcoding inline for now to return Pikachu’s stats. If we go back the our previous “pokemonData” value pokemonResult.data.pokemon we can see that this maps to the result we get from the GraphQL query:

GraphQL response data

What this means is that all the data from the pokemon node will be available in our Mustache template now:

This is very useful for Server Side Rendering and the content will be sent from the server once all the resolvers have been executed. Now if we go to any URL in our UPWARD server (except the proxied GraphQL ones) we should see Pikachu’s amazing stats:

Mustache template rendering

4. Using variables

It would be nice to be able to fetch a specific Pokémon stats by its name using the URL so let’s do just that. We’ll clean up the configuration file a bit more to make it easier to read and add a few more resolvers to it:

The configuration is mostly the same with a few tweaks and additions:

#1 response: there’s an additional conditional check here to see if we’re requesting the favicon or a CSS file, for both cases we’ll use a “Static Resolver” (#8) which basically returns the files as-is in the “static” folder (no magic here).

#6 pokemonResult: we moved the GraphQL code to it’s own file and wrapped it with a query so we can pass variables to it, then specify them with a variables node in the property that points to another “Conditional Resolver”.

GraphQL query with variable name

#7 pokemonName: just as in #1 we are checking if the URL matches a specific pattern (anything beginning with forward slash), if so we return it using the special keyword $match.$1, this is the value that will be used for the GraphQL query above.

Now with some sprinkles of CSS we should be able to see the stats for all Pokémons available in the API by passing the name in the URL:

Credits to Bidji for the Pokedex styles

As you can see UPWARD allows us to implement some interesting functionality without writing almost any javascript, however if you are more comfortable with NodeJS or if you already have an express server you can use UPWARD as a middleware as well so it can be used as an add-on to your current server and introduce in a more incremental approach instead of replacing express in one go.

So your server.js could look something like this:

Then requests going to the / route will be served with express and the rest will be outsourced to UPWARD.

In summary

UPWARD can give us a lot of control and customisation over our server without having to interact directly with express or other technologies like this. Even in its early days I’m surprised at how much it allows us to do out of the box so I’m hopeful it will keep getting better and better.

If you want to try it for yourself have a look at the spec’s official Github repo, I noticed it’s not very up to date and in most cases the examples don’t work which makes sense as the project keeps changing quite rapidly so I recommend diving into the UPWARD JS code or the Venia implementation to get the latest and greatest features.

All the examples in the article are available in Github so feel free to play around with it and submit any changes or improvements. 👋🏽

jahvi/upward-examples