🚀 Improving GraphQL - How the PoP API achieves new heights
I had originally started implementing the GraphQL spec using server-side components, providing yet another implementation of GraphQL on PHP (such as those based on the popular graphql-php library), but attempting to also support those features that GraphQL fails at supporting, such as HTTP caching.
I can now claim my attempt was a success: The implementation satisfies the GraphQL spec (except for the syntax... more on this below), adding native support for HTTP caching was straightforward, and I could even add new features that the typical GraphQL implementation out there does not support.
So, I'd say this is a good time to introduce this API to the world, and hope that the world will notice it: Please be introduced to the brand-new PoP API, an iteration and improvement over GraphQL.
Below is the set of its unique, distinctive features (displayed through slides here).
Queries are URL-based permalink
Structure of the request:
/?query=query&variable=value&fragment=fragmentQueryStructure of the query:
/?query=field(args)@alias<directive(args)>This syntax:
- Enables HTTP/Server-side caching
- Simplifies visualization/execution of queries (straight in the browser, without any client)
- GET when it's a GET, POST when it's a POST, pass variables through URL params
This syntax can be expressed in multiple lines:
/?
query=
field(
args
)@alias<
directive(
args
)
> Advantages:
- It is easy to read and write as a URL param (it doesn't make use of
{and}like GraphQL) - Copy/pasting in Firefox works straight!
Example:
/?
query=
posts(
limit: 5
)@posts.
id|
date(format: d/m/Y)|
title<
skip(if: false)
>The syntax has the following elements:
(key:value): Arguments[key:value]or[value]: Array$: Variable@: Alias.: Advance relationship|: Fetch multiple fields<...>: Directive--: Fragment
Example:
/?
query=
posts(
ids: [1, 1499, 1178],
order: $order
)@posts.
id|
date(format: d/m/Y)|
title<
skip(if: false)
>|
--props&
order=title|ASC&
props=
url|
author.
name|
urlDynamic schema permalink
Because it is generated from code, different schemas can be created for different use cases, from a single source of truth. And the schema is natively decentralized or federated, enabling different teams to operate on their own source code.
To visualize it, in addition to the standard introspection field __schema, we can query field fullSchema:
/?query=fullSchemaSkip argument names permalink
Field and directive argument names can be deduced from the schema.
This query...
// Query 1
/?
postId=1&
query=
post($postId).
date(d/m/Y)|
title<
skip(false)
>...is equivalent to this query:
// Query 2
/?
postId=1&
query=
post(id:$postId).
date(format:d/m/Y)|
title<
skip(if:false)
>Operators and Helpers permalink
All operators and functions provided by the language (PHP) can be made available as standard fields, and any custom “helper” functionality can be easily implemented too:
1. /?query=not(true)
2. /?query=or([1,0])
3. /?query=and([1,0])
4. /?query=if(true, Show this text, Hide this text)
5. /?query=equals(first text, second text)
6. /?query=isNull(),isNull(something)
7. /?query=sprintf(%s API is %s, [PoP, cool])
8. /?query=contextComposable fields permalink
The value from a field can be the input to another field, and there is no limit how many levels deep it can be.
In the example below, field post is injected, in its field argument id, the value from field arrayItem applied to field posts:
/?query=
post(
id: arrayItem(
posts(
limit: 1,
order: date|DESC
),
0)
)@latestPost.
id|
title|
dateTo tell if a field argument must be considered a field or a string, if it contains () it is a field, otherwise it is a string (eg: posts() is a field, and posts is a string)
Composable fields with operators and helpers permalink
Operators and helpers are standard fields, so they can be employed for composable fields. This makes available composable elements to the query, which removes the need to implement custom code in the resolvers, or to fetch raw data that is then processed in the application in the client-side. Instead, logic can be provided in the query itself.
/?
format=Y-m-d&
query=
posts.
if (
hasComments(),
sprintf(
"This post has %s comment(s) and title '%s'", [
commentsCount(),
title()
]
),
sprintf(
"This post was created on %s and has no comments", [
date(format: if(not(empty($format)), $format, d/m/Y))
]
)
)@postDescThis solves an issue with GraphQL: That we may need to define a field argument with arbitrary values in order to provide variations of the field's response (which is akin to REST's way of creating multiple endpoints to satisfy different needs, such as /posts-1st-format/ and /posts-2nd-format/).
Composable fields in directive arguments permalink
Through composable fields, the directive can be evaluated against the object, granting it a dynamic behavior.
The example below implements the standard GraphQL skip directive, however it is able to decide if to skip the field or not based on a condition from the object itself:
/?query=
posts.
title|
featuredimage<
skip(if:isNull(featuredimage()))
>.
srcSkip output if null permalink
Exactly the same result above (<skip(if(isNull(...)))>) can be accomplished using the ? operator: Adding it after a field, it skips the output of its value if it is null.
/?query=
posts.
title|
featuredimage?.
srcComposable directives permalink
Directives can be nested, unlimited levels deep, enabling to create complex logic such as iterating over array elements and applying a function on them, changing the context under which a directive must execute, and others.
In the example below, directive <forEach> iterates all the elements from an array, and passes each of them to directive <applyFunction> which executes field arrayJoin on them:
/?query=
echo([
[banana, apple],
[strawberry, grape, melon]
])@fruitJoin<
forEach<
applyFunction(
function: arrayJoin,
addArguments: [
array: %value%,
separator: "---"
]
)
>
>Directive expressions permalink
An expression, defined through symbols %...%, is a variable used by directives to pass values to each other. An expression can be pre-defined by the directive or created on-the-fly in the query itself.
In the example below, an array contains strings to translate and the language to translate the string to. The array element is passed from directive <forEach> to directive <advancePointerInArray> through pre-defined expression %value%, and the language code is passed from directive <advancePointerInArray> to directive <translate> through variable %toLang%, which is defined only in the query:
/?query=
echo([
[
text: Hello my friends,
translateTo: fr
],
[
text: How do you like this software so far?,
translateTo: es
],
])@translated<
forEach<
advancePointerInArray(
path: text,
appendExpressions: [
toLang:extract(%value%,translateTo)
]
)<
translate(
from: en,
to: %toLang%,
oneLanguagePerField: true,
override: true
)
>
>
>HTTP Caching permalink
Cache the response from the query using standard HTTP caching.
The response will contain Cache-Control header with the max-age value set at the time (in seconds) to cache the request, or no-store if the request must not be cached. Each field in the schema can configure its own max-age value, and the response's max-age is calculated as the lowest max-age among all requested fields (including composed fields).
Examples:
//1. Operators have max-age 1 year
/?query=
echo(Hello world!)
//2. Most fields have max-age 1 hour
/?query=
echo(Hello world!)|
posts.
title
//3. Composed fields also supported
/?query=
echo(posts())
//4. "time" field has max-age 0
/?query=
time
//5. To not cache a response:
//a. Add field "time"
/?query=
time|
echo(Hello world!)|
posts.
title
//b. Add <cacheControl(maxAge:0)>
/?query=
echo(Hello world!)|
posts.
title<cacheControl(maxAge:0)>Many resolvers per field permalink
Fields can be satisfied by many resolvers.
In the example below, field excerpt does not normally support field arg length, however a new resolver adds support for this field arg, and it is enabled by passing field arg branch:experimental:
//1. Standard behaviour
/?query=
posts.
excerpt
//2. New feature not yet available
/?query=
posts.
excerpt(length:30)
//3. New feature available under
// experimental branch
/?query=
posts.
excerpt(
length:30,
branch:experimental
)Advantages:
- The data model can be customized for client/project
- Teams become autonoumous, and can avoid the bureaucracy of communicating/planning/coordinating changes to the schema
- Rapid iteration, such as allowing a selected group of testers to try out new features in production
- Quick bug fixing, such as fixing a bug specifically for a client, without worrying about breaking changes for other clients
- Field-based versioning
Validate user state/roles permalink
Fields can be made available only if user is logged-in, or has a specific role. When the validation fails, the schema can be set, by configuration, to either show an error message or hide the field, as to behave in public or private mode, depending on the user.
For instance, the following query will give an error message, since you, dear reader, are not logged-in:
/?query=
me.
nameLinear time complexity to resolve queries (O(n), where n is #types) permalink
The “N+1 problem” is completely avoided already by architectural design. It doesn't matter how many levels deep the graph is, it will resolve fast.
Example of a deeply-nested query:
/?query=
posts.
author.
posts.
comments.
author.
id|
name|
posts.
id|
title|
url|
tags.
id|
slugEfficient directive calls permalink
Directives receive all their affected objects and fields together, for a single execution.
In the examples below, the Google Translate API is called the minimum possible amount of times to execute multiple translations:
// The Google Translate API is called once,
// containing 10 pieces of text to translate:
// 2 fields (title and excerpt) for 5 posts
/?query=
posts(limit:5).
--props|
--props@spanish<
translate(en,es)
>&
props=
title|
excerpt
// Here there are 3 calls to the API, one for
// every language (Spanish, French and German),
// 10 strings each, all calls are concurrent
/?query=
posts(limit:5).
--props|
--props@spanish<
translate(en,es)
>|
--props@french<
translate(en,fr)
>|
--props@german<
translate(en,de)
>&
props=
title|
excerptInteract with APIs from the back-end permalink
Example calling the Google Translate API from the back-end, as coded within directive <translate>:
//1. <translate> calls the Google Translate API
/?query=
posts(limit:5).
title|
title@spanish<
translate(en,es)
>
//2. Translate to Spanish and back to English
/?query=
posts(limit:5).
title|
title@translateAndBack<
translate(en,es),
translate(es,en)
>
//3. Change the provider through arguments
// (link gives error: Azure is not implemented)
/?query=
posts(limit:5).
title|
title@spanish<
translate(en,es,provider:azure)
>Interact with APIs from the client-side permalink
Example accessing an external API from the query itself:
/?query=
echo([
usd: [
bitcoin: extract(
getJSON("https://api.cryptonator.com/api/ticker/btc-usd"),
ticker.price
),
ethereum: extract(
getJSON("https://api.cryptonator.com/api/ticker/eth-usd"),
ticker.price
)
],
euro: [
bitcoin: extract(
getJSON("https://api.cryptonator.com/api/ticker/btc-eur"),
ticker.price
),
ethereum: extract(
getJSON("https://api.cryptonator.com/api/ticker/eth-eur"),
ticker.price
)
]
])@cryptoPricesInteract with APIs, performing all required logic in a single query permalink
The last query from the examples below accesses, extracts and manipulates data from an external API, and then uses this result to accesse yet another external API:
//1. Get data from a REST endpoint
/?query=
getJSON("https://newapi.getpop.org/wp-json/newsletter/v1/subscriptions")@userEmailLangList
//2. Access and manipulate the data
/?query=
extract(
getJSON("https://newapi.getpop.org/wp-json/newsletter/v1/subscriptions"),
email
)@userEmailList
//3. Convert the data into an input to another system
/?query=
getJSON(
sprintf(
"https://newapi.getpop.org/users/api/rest/?query=name|email%26emails[]=%s",
[arrayJoin(
extract(
getJSON("https://newapi.getpop.org/wp-json/newsletter/v1/subscriptions"),
email
),
"%26emails[]="
)]
)
)@userNameEmailListCreate your content or service mesh permalink
The example below defines and accesses a list of all services required by the application:
/?query=
echo([
github: "https://api.github.com/repos/GatoGraphQL/GatoGraphQL",
weather: "https://api.weather.gov/zones/forecast/MOZ028/forecast",
photos: "https://picsum.photos/v2/list"
])@meshServices|
getAsyncJSON(getSelfProp(%self%, meshServices))@meshServiceData|
echo([
weatherForecast: extract(
getSelfProp(%self%, meshServiceData),
weather.periods
),
photoGalleryURLs: extract(
getSelfProp(%self%, meshServiceData),
photos.url
),
githubMeta: echo([
description: extract(
getSelfProp(%self%, meshServiceData),
github.description
),
starCount: extract(
getSelfProp(%self%, meshServiceData),
github.stargazers_count
)
])
])@contentMeshOne-graph ready permalink
Use custom fields to expose your data and create a single, comprehensive, unified graph.
The example below implements the same logic as the case above, however coding the logic through fields (instead of through the query):
// 1. Inspect services
/?query=
meshServices
// 2. Retrieve data
/?query=
meshServiceData
// 3. Process data
/?query=
contentMesh
// 4. Customize data
/?query=
contentMesh(
githubRepo: "getpop/api-graphql",
weatherZone: AKZ017,
photoPage: 3
)@contentMeshPersisted fragments permalink
Query sections of any size and shape can be stored in the server. It is like the persisted queries mechanism provided by GraphQL, but more granular: different persisted fragments can be added to the query, or a single fragment can itself be the query.
The example below demonstrates, once again, the same logic from the example above, but coded and stored as persisted fields:
// 1. Save services
/?query=
--meshServices
// 2. Retrieve data
/?query=
--meshServiceData
// 3. Process data
/?query=
--contentMesh
// 4. Customize data
/?
githubRepo=getpop/api-graphql&
weatherZone=AKZ017&
photoPage=3&
query=
--contentMeshCombine with REST permalink
Get the best from both GraphQL and REST: query resources based on endpoint, with no under/overfetching.
// Query data for a single resource
{single-post-url}/api/rest/?query=
id|
title|
author.
id|
name
// Query data for a set of resources
{post-list-url}/api/rest/?query=
id|
title|
author.
id|
nameOutput in many formats permalink
Replace "/graphql" from the URL to output the data in a different format: XML or as properties, or any custom one (implementation takes very few lines of code).
// Output as XML: Replace /graphql with /xml
/api/xml/?query=
posts.
id|
title|
author.
id|
name
// Output as props: Replace /graphql with /props
/api/props/?query=
posts.
id|
title|
excerptNormalize data for client permalink
Just by removing the "/graphql" bit from the URL, the response is normalized, making its output size greatly reduced when a same field is fetched multiple times.
/api/?query=
posts.
author.
posts.
comments.
author.
id|
name|
posts.
id|
title|
urlCompare the output of the query in PoP native format:
...with the same output in GraphQL format:
Handle issues by severity permalink
Issues are handled differently depending on their severity:
- Informative, such as Deprecated fields and directives: to indicate they must be replaced with a substitute
- Non-blocking issues, such as Schema/Database warnings: when an issue happens on a non-mandatory field
- Blocking issues, such as Query/Schema/Database errors: when they use a wrong syntax, declare non-existing fields or directives, or produce an issues on mandatory arguments
//1. Deprecated fields
/?query=
posts.
title|
published
//2. Schema warning
/?query=
posts(limit:3.5).
title
//3. Database warning
/?query=
users.
posts(limit:name()).
title
//4. Query error
/?query=
posts.
id[book](key:value)
//5. Schema error
/?query=
posts.
non-existant-field|
is-status(
status:non-existant-value
)Type casting/validation permalink
When an argument has its type declared in the schema, its inputs will be casted to the type. If the input and the type are incompatible, it ignores setting the input and throws a warning.
/?query=
posts(limit:3.5).
titleIssues bubble upwards permalink
If a field or directive fails and it is input to another field, this one may also fail.
/?query=
post(divide(a,4)).
titlePath to the issue permalink
Issues contain the path to the composed field or directive were it was produced.
/?query=
echo([hola,chau])<
forEach<
translate(notexisting:prop)
>
>Log information permalink
Any informative piece of information can be logged (enabled/disabled through configuration).
/?
actions[]=show-logs&
postId=1&
query=
post($postId).
title|
date(d/m/Y)