Testing GraphQL for the Beginner Pythonistas
Testing your GraphQL application
Introduction
Previously we took the textbook beginner Flask application and added it some GraphQL capabilities. However an application before being production ready it should have at least both unit and integration tests. There is an abundance of tutorials. But the main idea is never ever release something without both fellas. Or as the official Flask testing tutorial says
Something that is untested is broken.
For the developer, it can uncover bugs in the code very early. For the client it will increase the amount of trust on you (and preference). In this article we focus on the GraphQL testing on your application. In other words we will automate what you would do manually to ensure proper functionality of the GraphQL endpoint. E.g. with Postman or curl. Let’s start.
Scope
This article targets the beginner Pythonista. Just the bare basics but with the ambition to help the beginner blossom. We target the typical Flask / SQLAlchemy duo through the wonderful Graphene library . We will use the Graphene and Pytest to test the application as the Graphene library recomends. But we will not stop there. We will also use Behave in order to show how to do integration testing. To visually explain some concepts, Postman will be used. In order to understand the code, the previous article is strictly a prerequisite!!!
Code
The code here is not always mine but I had to look to StackOverflow or the official documentation libraries for testing hints or use a search engine to solve my questions. Grab you favorite text editor or IDE and lets get started. I used Python 3.9.0 during development with Eclipse but verified that it works with 3.8.6. The code can be found in the repository of the previous article, because code and tests go together.
Testing the Graphene way
Querying
The method we will employ in this section, is to use the official testing recommendation of Graphene in order to create the necessary Pytest.
But what are we going to test anyway? Of course, to keep this artcile short we will not test everything. But the idea is to initialize the database of the application, and then run our query for getting all the posts.
The json response that you will receive if you test with the GraphQL client included with Graphene is
In other words our first step is to test that the initial state of our application is what we expect it to be. Only one post with the above content. Using naively the textbook approach in the following fragment
will fail spectacularly. In order to test it you can use the CLI, go to the top folder of the project and issue
The error is summarized in the traceback clearly if you search
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
It means that everything runs as if Flask does not exist, but behind the scenes, Flask is used heavily. In order to solve it we need to create and use a fixture that during its lifetime it sets up a Flask app context. The bright idea is to use a context manager to be our portal to Flask-land. While our test, the context is up and we ensure it by requiring the fixture. Given the context we can safely execute our test. The context creation is taken for the official Flask testing tutorial which is a treasure trove.
Mutating
Our test string is an easy GET. It is not parametrised like a mutation. Let us also cover this case. We need the concept of variables already introduced in the first part. We will use the GraphQL properties of Postman in order to demonstrate it.
The query under question is :
You can see that we set through JSON a set of variables to parametrize the mutation query in Postman in the right panel. In the bottom we can see the result. Consequently the full test takes the form.
Testing the Behave way
REST to the rescue
This time we will test on a live running server through Behave that implements the BDD testing approach by specifying tests in the high level Gherkin language. The implementation of the test behind the scenes is Python of course. We will start by using REST to test a GraphQL server. I looked up the explanation in a SO post. The main idea is to POST with a suitable JSON body to the GraphQL endpoint of the server. It is not surpising that the POST body carries a serialization of the GraphQL to JSON.
We will use Postman this time as a classical REST client. For the allposts query above the serialization takes the form
and the Postman request (we have added “Content-Type” : “application/json” in headers)
For the post creation, the serialization is done as follows (note the query !!!)
with Postman request
Armed with this knowledge, lets write our Behave tests. This is a very basic introduction and the reader is expected to consult the excellent tutorial of Behave. We first take the role of the customer of the consumer of the api and describe in the Gerkin language our expectations.
In prose, we initialize our database with init-db as previously, we create a post with specific data and finally a new post exists only with data. The limitation, is as the test describes literary, every time we have to initialize the database. We cannot reuse a running instance , because the databased is not initialized and “only one post” is violated. We will see later how this is tackled. For now we will “suffer” temporarily.
This file is created in a features sub-folder where we also create a steps sub-folder to host our python implementation of the statements. In the steps sub-folder create a posting.py skeleton populated with the steps (dummy implementations):
Lets focus on the first one. We have our database initialised and the application running (see documentation in source) and we make sure that the state of the application, for the posts perspective is as it should. In other words it contains only one post with title “Flowers” for user 1. We can make it as detailed as we need. For demo purposes, it suffices. We make a REST call to the running Flask application and assert on the return value. The idea is taken from here.
In this case, we serialized the GraphQL query to a json payload, we setup the headers to signify a JSON payload, we do a POST, and finally we make sure that the response is what we expected. Nothing special. The rest of the steps are implemented this way. How we run the tests? We open a terminal to our folder containing the source and start the application
In an other terminal, go to the top folder and run behave to unleash the magic. In order to re-run the test we must kill and run the shell script again. This is a shortcoming. We will fix it now.
Starting stopping the Flask application programmatically
Believe it or not this was the msot challenging part of the application. We will employ the Environmental Controls of Behave. This was the first part of my challenge, to put things in the right file. To this end create an empty file called environment.py in features (and not steps, it is not like the famous cucumber-Java).
The plan is to start the server in the before_feature section and shut it down in the after_feature section, which means before and after the file containing the scenarios are accessed. I had failed many times to make it work, but every time I learned something. The material comes from this posting for the threaded trick. Please read various documentations for context push and threads to understand what is happening. But the principle is textbook level, start a thread running the server in order not to block test execution (before_feature) and then signal the thread to stop (after_feature). Lets code it assumming we have an object that represents the server itself.
The implementation details can be looked up in the code. Given this definition, we can now describe the server managment process before/after feature execution.
We will need the excellent threading package. We manage server and execution thread independently here. We can do both, but this is basic introduction.
Here we have the typical function that is threaded and where we execute the server. Before the feature is run we create a server, a thread that runs the server and we start the server. In order to cleanup, we stop the server and make sure that thread has finished properly. With a correct implemntation of the MyServer where we initialise the database, we can run the test as many times as we want without any manual intervention.
Conclusion
In this article we showed two ways to test your python Flask GraphQL application. One is through the Graphene client and the other is through Gherkin by doing REST calls. The introduction we gave was very basic but it is a starting point from more complex projects or even for more experimentation. I recommend to take the application apart and implement variou missing bits. E.g. intitalisation from empty database, creation of both users and posts and assertions. Behave has also a lot to keep you occupied, like tags and other Environmental Controls. You journey does not stop here for sure.