In the end of our last post (which was about Securing REST APIs) we mentioned about JWT. I made a promise that in the next post, we would discuss more about JWT and how we can secure our REST APIs using it. However, when I started drafting the post and writing the code, I realized the underlying concepts of JWT themselves deserve a dedicated blog post. So in this blog post, we will focus solely on JWT and how it works.
What is JWT?
We will ignore the text book definitions and try to explain the concepts in our own words. Don’t be afraid of the serious looking acronym, the concepts are rather simple to understand and comprehend. First let’s break down the term – “JSON Web Tokens”, so it has to do something with JSON, the web and of course tokens. Right? Let’s see.
Yes, a JWT mostly concerns with a Token that is actually a hashed / signed form of a JSON payload. The JSON payload is signed using a hashing algorithm along with a secret to produce a single (slightly long) string that works as a token. So a JWT is basically a string / token generated by processing a JSON payload in a certain way.
So how does JWT help? If you followed our last article, you now know why http basic auth is bad. You have to pass your username and password with every request. That is kind of bad, right? The more you send your username and password over the internet, the more likely it is to get compromised, no? Instead, on the first login, we can accept the username and password and return a token back to the client. The client passes that token with every request. We verify that token to see if it’s a logged in user or not. This is the idea behind Token based authentication.
Random Tokens vs JWT
How would you generate such token? You could generate a nice random string and store it in database against that user. Right? This is how cookie based session works too, btw. Now what if your application is scaled across multiple servers and all requests are load balanced? One server will not recognize a token / session generated by another server. Unless of course you also have one central database active all the time, serving all the incoming requests from all the servers. That setup is tricky and difficult, no?
There is another work around using sticky sessions where the requests from one particular user is always directed to the same server by the load balancer. This work around is also not as simple as JWT. Even if all these work nicely, we still have to make database queries to validate the token / session. What if we want to provide single sign on (users from one service wants to access resources on a different service all together)? How does that work? We will need a central auth server and all services will have to talk to it to verify the user token.
The benefit of JWT is that it’s lightweight but at the same time it’s a self contained JSON payload. You can store user identity in the JSON, sign it and send the token to the clients. Since it’s signed we can verify and validate it with just our secret key. No database overhead. No need for sticky sessions. Just share the secret key privately and all your services can read the data stored inside the JWT. Others can’t tamper or forge a new, valid token for an user without that secret key. Single sign on just becomes a breeze and less complicated. Sounds good? Let’s see how JWTs are constructed.
Anatomy of JWT
A JSON Web Token consists of three major parts:
- Header
- Payload
- Signature
These 3 parts are separated by dots (.). So a JWT looks like the following
|
1 |
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ |
If you look closely, there are 3 parts here:
- Header:
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9 - Payload:
eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9 - Signature:
TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ
Okay, so we get the three parts, but what do they really do? Also those strings look like meaningless characters to me. What are they? How are they generated? Well, they are encoded in certain ways as we will see in the following sections.
Header
The header is a simple key value pair (dictionary / hashmap) data structure. It usually has two keys typ and alg short for type and algorithm. The standard is to have the keys at best 3 character long, so the generated token does not get too large.
Example:
|
1 2 3 4 |
{ "alg": "HS256", "typ": "JWT" } |
The typ value is JWT since this is JWT we’re using. The HS256 is the most common and most popular hashing algorithm used with JWT.
Now we just need to base64 encode this part and we get the header string. You were wondering why the strings didn’t make sense. That’s because the data is base64 encoded.
Payload
Here comes our favorite part – the JSON payload. In this part, we put the data we want to store in the JWT. As usual, we should keep the keys and the overall structure as small as possible.
|
1 2 3 4 5 |
{ "sub": "1234567890", "name": "John Doe", "admin": true } |
We can add any data we see fit. These fields / keys are called “claims”. There are some reserved claims – keys which can be interpreted in a certain way by the libraries which decode the JWT. For example, if we pass the exp (expiry) claim with a timestamp, the decoding library will check this value and throw an exception if the time has passed (the token has expired). These can often be helpful in many cases. You can find the common standard fields on Wikipedia.
As usual, we base64 encode the payload to get the payload string.
Signature
The signature part itself is a hashed string. We concatenate the header and the payload strings (base 64 encoded header and payload) with a dot (.) between them. Then we use the hashing algorithm to hash this string with our secret key.
In pseudocode:
|
1 2 |
concatenated_string = base64encode(header) + '.' + base64encode(payload) signature = hmac_sha256(concatenated_string, 'MY_SUPER_SECRET_KEY') |
That would give us the last part of the JWT, the signature.
Glue it all together
As we discussed before, the JWT is the dot separated form of the three components. So the final JWT would be:
|
1 |
jwt = header + "." + payload + "." + signature |
Using a library
Hey! JSON Web Tokens sounded great but looks like there’s a lot of work involved! Well, it would seem that way since we tried to understand how a JSON Web Token is actually constructed. In our day to day use cases, we would just use a suitable library for the language / platform of our choice and be done with it.
If you are wondering what library you can use with your language / platform, here’s a comprehensive list of libraries – JSON Web Token Libraries.
Real Life Example with PyJWT
Enough talk, time to see some codes. Excited? Let’s go!
We will be using Python with the excellent PyJWT package to encode and decode our JSON Web Tokens in this example. Before we can use the library, we have to install it first. Let’s do that using pip.
|
1 |
pip install pyjwt |
Now we can start generating our tokens. Here’s an example code snippet:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
import jwt import datetime payload = { "uid": 23, "name": "masnun", "exp": datetime.datetime.utcnow() + datetime.timedelta(minutes=2) } SECRET_KEY = "N0TV3RY53CR3T" token = jwt.encode(payload=payload, key=SECRET_KEY) print("Generated Token: {}".format(token.decode())) decoded_payload = jwt.decode(jwt=token, key=SECRET_KEY) print(decoded_payload) |
If we run the code, we will see:
|
1 2 3 |
python jwt_test.py Generated Token: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjIzLCJuYW1lIjoibWFzbnVuIiwiZXhwIjoxNDk0NDQ5OTQ0fQ.49okXifPSqc7n_n7wZRc9XVVqekTTeBIBBZdiH0nGJQ {'uid': 23, 'name': 'masnun', 'exp': 1494449944} |
So it worked – we encoded a payload and then decoded it back. All we needed to do is to call jwt.encode and jwt.decode with our secret key and the payload / token. So simple, no? parties.
Bonus Example – Expiry
In the following example, we will set the expiry to only 2 seconds. Then we will wait 10 seconds (so the token expires by then) and try to decode the token.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
import jwt import datetime import time payload = { "uid": 23, "name": "masnun", "exp": datetime.datetime.utcnow() + datetime.timedelta(seconds=2) } SECRET_KEY = "N0TV3RY53CR3T" token = jwt.encode(payload=payload, key=SECRET_KEY) print("Generated Token: {}".format(token.decode())) time.sleep(10) # wait 10 secs so the token expires decoded_payload = jwt.decode(jwt=token, key=SECRET_KEY) print(decoded_payload) |
What happens after we run it? This happens:
|
1 |
jwt.exceptions.ExpiredSignatureError: Signature has expired |
Cool, so we get an error mentioning that the signature has expired by now. This is because we used the standard exp claim and our library knew how to process it. This is how we use the standard claims to ease our job!
Using JWT for REST API Authentication
Now that we’re all convinced of the good sides of JSON Web Tokens, the question comes into mind – how can we use it in our REST APIs?
The idea is simple and straightforward. When the user logs in the first time, we verify his/her credentials and generate a JSON Web Token with necessary details. Then we return this token back to the user/client. The client will now send the token with every request, as part of the authorization header.
The server will decode this token and read the user data. It won’t have to access the database or contact another auth server to verify the user details, it’s all inside the decoded payload. And since the token is signed and the secret key is “secret”, we can trust the payload.
But please make sure the secret key is not compromised. And of course use SSL (https) so that men in the middle can not hijack the token anyway.
What’s next?
JSON Web Token is not only about authentication. You can use it to securely transmit data from one party to another. However, it’s mostly used for authenticating REST APIs. In our next blog post, we shall go through that use case. We will see how we can authenticate our api using JWT.
In the mean time, you can subscribe to the mailing list so you can stay up to date with this blog. If you liked the article and/or learned something new, please don’t forget to share it with your friends.