The Go Programming language (often written as golang) has become quite popular recently. Google is actively backing the project but Golang has seen usage, contribution and success stories from many other popular brands and enterprises across the internet. Go promises a very simple, easy to learn syntax that allows us to build robust, reliable, efficient software. Once we have had invested some time writing production grade code in the language, we would agree that Go delivers on it’s promises. It’s indeed a fantastic language – easy to learn, easy to read, reason about and of course maintain. You get superb performance without sacrificing much on productivity. Don’t just trust my words, give it a Go! (See what I did there?)
A little history
Work on Golang started back in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson at Google. So that kinds of make Golang 10 years old in 2017. Although work started in 2007, the language was however announced in 2009. It reached version 1.0 in 2012.
A little about the creators – Rob Pike was a member of the unix team and he is known for his work on Plan 9. Ken Thompson designed and implemented Unix. He also created the B language (from which C was inspired). He was also involved in the Plan 9 project.
Why did they start working on a new language? Because they were frustrated with the ones that existed. You can choose a dynamic language like Python / Javascript and enjoy the ease of programming in them. Or you can choose something like C/C++ to get performance. But at the same time you lose the ease of programming, productivity reduces and the compilation time can sometimes get too long. More and more developers were choosing dynamic languages in their projects, essentially favouring the ease of use over the safety and performance offered by the statically typed, compiled languages. There wasn’t any popular, easy to use, mainstream language that could ease these problems for the developers. You couldn’t get ease of programming, safety, efficiency, fast compilation – all from a single language.
The creators of Golang saw this problem as an opportunity to create a better language that could solve these problems at hand.
Why did Golang become so popular?
Go came with better offerings, specially with solutions to what many of us (including the Googlers) faced. When I read (or write) Go code, I feel like the following equation makes perfect sense => C + Python = Golang. Go is very very fast. Not just the language but also the compiler. Go compiles fast, runs fast. And you still feel quite productive, much more productive than you would feel in C++ or Java.
The syntax is simple. You won’t have to remember many keywords. The static typing provides safety to a great extent. IDEs can provide better code completion and refactoring assistance. Compiler would help you reduce bugs by catching many potential errors before the program even starts to run (which is applicable to all statically typed languages, nothing Golang specific but with Go’s “light on keywords” design, it’s just more productive and enjoyable).
Go provides a nice, extensive standard library with all the batteries you might need for day to day system or network programming. You want to build an awesome web app? Go standard library has you (mostly) covered.
The major win for Go is perhaps the concurrency primitives. We can create very light weight threads called goroutine which are multiplexed on all available CPU cores. We can easily communicate between the goroutines using channels. I personally found the goroutine and channels based way of writing efficient concurrent programs very easy, elegant and pleasant. Don’t be afraid of writing highly concurrent programs any more!
Golang also compiles everything and generates a single binary that includes everything that you need to run the program. You do not need to have anything installed on your target machine to run the binary. This is a huge win in terms of deployment. Writing and distributing command line tools have never been easier!
How popular is Golang?
It’s quite popular. Golang is currently ranked 16 on the TIOBE Index. Tiobe also declared Go to be the language of the year in 2016. Interestingly they also chose Go as the language of the year back in 2009, the year it was released. It stands 15th on the RedMonk language ratings. If you compare active repositories using Githut, Go would be the 14th most active language on Github. For a new programming language, that has released it’s first major version in 2012, that ranking is quite impressive. If you think about the growth of the language, it’s been getting a lot of traction.
In the latest StackOverflow Survey (2017), Go is 3rd on the most wanted list and 5th on the most loved list. That tells us something, doesn’t it? 😀
Gopher with a Gun!
How’s the job market, you ask? Don’t worry, Go has jobs for you!
Also check out the HackerNews “Who is hiring?” threads. I see a lot of Golang related job postings there. And do not forget, Golang pays well.
I want to learn Golang
I am sure you do. So where can we learn Golang by ourselves? These are my recommendations:
The Official Tour is excellent. I recommend it to all the beginners. You will have a very good basic once you complete the tour. The tour is also available offline in case you want to learn some golang while on vacation or on the go.
Go By Example is another excellent step by step tour. I always refer to this site when I need to lookup a syntax or need a quick reference.
Effective Go is a nice read if you want to learn the best practices and want to write idiomatic Go codes.
Official Documentation is something we all should keep close to us while writing some Go code.
The Go Blog – This is something I really really love. The official Go blog has many interesting and detailed stuff. Trust me, there’s a lot to learn from there. Highly recommended.
If you need some help, please feel free to ask in Gophers Slack or the Go Forum maintained by the GoBridge people.
In our last post about REST APIs, we have learned the basics of how REST APIs function. In this post, we would see how we can develop our own REST APIs. We would use Python and Flask for that. If you are new to Python, we have you covered with our Python: Learning Resources and Guidelines post.
Python / Flask code is pretty simple and easy to read / understand. So if you just want to grasp the best practices of REST API design but lack Python skills, don’t worry, you will understand most of it. However, I would recommend you try out the codes hands on. Writing codes by hand is a very effective learning method. We learn more by doing than we learn by reading or watching.
Installing Flask and Flask-RESTful
We will be using the Flask framework along with Flask-RESTful. Flask-RESTful is an excellent package that makes building REST APIs with Flask both easy and pleasant. Before we can start building our app, we first need to install these packages.
Shell
1
2
pip install flask
pip install flask-restful
Once we have the necessary packages installed, we can start thinking about our API design.
RESTful Mailing List
You see, I just recently started this Polyglot.Ninja() website and I am getting some readers to my site. Some of my readers have shown very keen interest to receive regular updates from this blog. To keep them posted, I have been thinking about building a mailing list where people can subscribe with their email address. These addresses get stored in a database and then when I have new posts to share, I email them. Can we build this mailing list “service” as a REST API?
The way I imagine it – we will have a “subscribers” collection with many subscriber. Each subscriber will provide us with their full name and email address. We should be able to add new subscriber, update them, delete them, list them and get individual data. Sounds simple? Let’s do this!
Choosing a sensible URL
We have decided to build our awesome mailing list REST API. For development and testing purposes, we will run the app on my local machine. So the base URL would be http://localhost:5000. This part will change when we deploy the API on a production server. So we probably don’t need to worry about it.
However, for API, the url path should make sense. It should clearly state it’s intent. A good choice would be something like /api/ as the root url of the API. And then we can add the resources, so for subscribers, it can be /api/subscribers. Please note that it’s both acceptable to have the resource part singular (ie. /api/subscriber) or plural (/api/subscribers). However, most of the people I have talked to and the articles I have read, more people like the plural form.
API Versioning: Header vs URL
We need to think about the future of the API before hand. This is our first iteration. In the future, we might want to introduce newer changes. Some of those changes can be breaking changes. If people are still using some of the older features which you can’t break while pushing new changes, it’s time you thought about versioning your API. It is always best practice to version your API from the beginning.
The first version of the api can be called v1. Now there are two common method of versioning APIs – 1) Passing a header that specifies the desired version of the API 2) Put the version info directly in the URL. There are arguments and counter arguments for both approaches. However, versioning using url is easier and more often seen in common public APIs.
So we accommodate the version info in our url and we make it – /api/v1/subscribers. Like discussed in our previous REST article, we will have two types of resources here – “subscriber collection” (ie. /subscribers) and “individual subscriber” elements (ie. /subscribers/17). With the design decided upon and a bigger picture in our head, let’s get to writing some codes.
RESTful Hello World
Before we start writing our actual logic, let’s first get a hello world app running. This will make sure that we have got everything setup properly. If we head over to the Flask-RESTful Quickstart page, we can easily obtain a hello world code sample from there.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
from flask import Flask
from flask_restful import Resource,Api
app=Flask(__name__)
api=Api(app)
classHelloWorld(Resource):
def get(self):
return{'hello':'world'}
api.add_resource(HelloWorld,'/')
if__name__=='__main__':
app.run(debug=True)
Let’s save this code in a file named main.py and run it like this:
1
python main.py
If the code runs successfully, our app will launch a web server here – http://127.0.0.1:5000/. Let’s break down the code a bit:
We import the necessary modules (Flask and Flask-RESTful stuff).
Then we create a new Flask app and then wrap it in Api.
Afterwards, we declare our HelloWorld resource which extends Resource.
On our resource, we define what the get http verb will do.
Add the resource to our API.
Finally run the app.
What happens here, when we write our Resources, Flask-RESTful generates the routes and the view handlers necessary to represent the resource over RESTful HTTP. Now let’s see, if we visit the url, do we get the message we set?
If we visit the url, we would see the expected response:
1
2
3
{
"hello":"world"
}
Trying out REST APIs
While we develop our api, it is essential that we can try out / test the API to make sure it’s working as expected. We need a way to call our api and inspect the output. If you’re a command line ninja, you would probably love to use curl. Try this on your terminal:
1
2
3
4
5
➜curl-XGET http://localhost:5000
{
"hello":"world"
}
➜
This would send a GET request to the URL and curl would print out the response on the terminal. It is a very versatile tool and can do a lot of amazing things. If you would like to use curl on a regular basis, you may want to dive deeper into the options / features / use cases. These can help you:
However, if you like command line but want a friendlier and easier command line tool, definitely look at httpie.
httpie vs curl command
Now what if you’re not a CLI person? And we can agree that sometimes GUI can be much more productive to use. Don’t worry, Postman is a great app!
If you are developing and testing a REST API, Postman is a must have app!
Our newly created API in Postman
Back to Business
We now have a basic skeleton ready and we know how to test our API. Let’s start writing our mailing list logic. Let’s first layout our resources with some sample data. For this example, we shall not bother about persisting the data to some database. We will store the data in memory. Let’s use a list as our subscriber data source for now.
Note we added a prefix to the Api for versioning reason. All our urls will be prefixed by /api/v1.
We created a list named users to store the subscribers.
We created two resources – SubscriberCollection and Subscriber.
Defined the relevant http method handlers. For now the response just describes the intended purpose of that method.
We add both resources to our api. Note how we added the id parameter to the url. This id is available to all the methods defined on Subscriber.
Fire up the local development server and try out the API. Works fine? Let’s move on!
Parsing Request Data
We have to accept, validate and process user data. In our cases, they would be the subscriber information. Each subscriber would have an email address, a full name and ID. If we used a database, this ID would have been auto generated. Since we are not using a database, we would accept this as part of the incoming request.
For processing request data, the RequestParser can be very helpful. We will use it in our POST calls to /api/subscribers/ to validate incoming data and store the subscriber if the data is valid. Here’s the updated code so far:
We created a new instance of RequestParser and added arguments so it knows which fields to accept and how to validate those.
We added the request parsing code in the post method. If the request is valid, it will return the validated data. If the data is not valid, we don’t have to worry about it, the error message will be sent to the user.
Testing the request parser
If we try to pass invalid data, we will get error messages. For example, if we request without any data, we will get something like this:
1
2
3
4
5
6
7
{
"message":{
"email":"Missing required parameter in the JSON body or the post body or the query string",
"id":"Please enter valid integer as ID",
"name":"Name has to be valid string"
}
}
But if we pass valid data, everything works fine. Here’s an example of valid data:
Cool, now we know how to validate user data 🙂 Please remember – never trust user input. Always validate and sanitize user data to avoid security risks.
Next, we need to implement the user level updates.
Subscriber Views
We went ahead and completed the code for the rest of the methods. The updated code now looks like this:
We added a helper function to find users from the list by it’s id
The update view works – we can update the user data. In our case we’re deleting the data and adding the new data. In real life, we would use UPDATE on the database.
Delete method works fine!
Feel free to go ahead and test the endpoints!
HTTP Status Codes
Our mailing list is functional now. It works! We have made good progress so far. But there’s something very important that we haven’t done yet. Our API doesn’t use proper http status codes. When we send response back to the client, we should also give it a status code. This code would help the client better interpret the results.
Have you ever visited a website and saw “404 Not found” error? Well, 404 is the status code, that means the document / resource you were looking for is not available. Saw any “500 Internal Server Error” lately? Now you know what that 500 means.
So let’s fix our code and start sending appropriate codes. We can return an optional status code from our views. So when we add a new subscriber, we can send 201 Created like this:
We have made decent progress today. We have designed and implemented a very basic API. We chose a sensible url, considered API versioning, did input validation and sent appropriate http status codes. We have done good. But what we have seen here is a very simple implementation. There are a lot of scope of improvements here. For example, our API is still open to public, there is no authentication enabled. So anyone with malicious intentions can flood / spam our mailing list database. We need to secure the API in that regard. We also don’t have a home page that uses HATEOAS to guide the clients. We don’t yet have documentation – always remember, the documentation is very important. We developers often don’t feel like writing documentation but well written documentation helps the consumers of your API consume it better and with ease. So do provide excellent docs!
I don’t know when – but in our next post on REST APIs, we shall explore more into the wonderful world of API development. And may be we shall also talk about some micro services? If you would like to know when I post those contents, do subscribe to the mailing list. You can find a subscription form on the sidebar.
And if you liked the post, do share with your friends 🙂
If you are into web programming you might have come across the terms “REST API” or “RESTful” and you might have felt curious as to what they really are. In this post, we shall take our time to go through some of the concepts / ideas behind REST and it’s applications.
The REST-less Developer
You might have noticed that the technology world is changing so fast. Back in the days, we used to love using Desktop applications. For example, we would probably want to check our emails on Outlook Express or Thunderbird then. Soon the web became popular and web applications started becoming extremely popular for the conveniences they offered. We now love Gmail, don’t we? But then we realized webmails are great and all but that’s not good enough. I want my emails on my phone and tablets. I for example, like to check my emails on my phone – I do that a lot because I can not be in front of a laptop all day. If you think carefully, in a few more years, we would want those emails on our wrist watches.
Now if you were the developer of such a popular email service, how would you serve these different types of devices? For example, the webmail can use HTML fine, mobile phones can browse HTML too but what about desktop and the smart watch apps? Also on phone, people would like native apps more than mobile web apps. So how to feed them data? Don’t be RESTless, some REST always helps! 😉
The RESTful Web
The good old web was working very well for us, for a time being. But the hypertext / HTML is not the native tongue for many devices that we have to accommodate into our services. Also the web is no longer about just “documents”, it is so much more. The REST architecture can shape the modern web in a way that provides uniform access to all our clients (devices). The core idea behind REST is that the client does not need to know anything before hand, it will connect to a server and the server will provide the client with available options via an agreed upon medium. For the web the medium is “HTML”.
For example, when your browser connected to this website, it didn’t know anything beforehand. The server served it a HTML page that has links to various posts and the search form to search content. By reading the source code (HTML), the client (browser) now knows what actions are available. Now, if you click a link or enter a search keyword and perform search, the browser would perform that action. But it has no idea what would happen next. When it performs the action, the server supplies new html telling it what it can do next. So the server is supplying information that the client can use to further interact with the server.
Hypertext or Hypermedia
But hey, not every device can understand HTML, no? Yes, you are absolutely right. That is why the server is no way just confined to HTML. It can provide other responses too, for example XML and JSON are also valid (and two popular) medium of communication. This is why when we describe REST, we usually say “hypermedia” instead of “hypertext”.
The principle that the client does not need to know anything before hand and the server dynamically generates hypermedia responses through which the client interacts with the server – this principle is aptly named “Hypermedia as the engine of application state” aka “HATEOAS“. That is one big name but if you read and think about it, it makes perfect sense. In this principle, the hypermedia generated by the server works as the “engine” of the application’s state. Cool, eh? HATEOAS is a key driving principle of the RESTful web but there’s more. Are you ready to dive in?
Fitting REST into HTTP and APIs
We now understand that in a REST like architecture, there will be a client and a server. The server will provide dynamically generated hypermedia on which the client will act upon. It all makes sense but how do we make our web APIs RESTful?
The idea of communicating over HTTP very often involves Verbs and Resources. Did you notice how very often the same URL can output different responses depending on which http method (GET or POST) we used? The URL can be considered as a resource and the http methods are the verbs. There’s more to just GET and POST. There are PUT, PATCH, DELETE etc.
The purpose/intent of the common http verbs are:
GET: The purpose is to literally get the data.
POST: This method translates to “create“.
PUT / PATCH: We use these methods to update data.
DELETE: Come on, do I even need to explain what this one does? 😀
Now while building our APIs, we can map these verbs to our resources. For example, we have User resources. You can access it on http://api.example.com/user. Now when someone makes a GET request, we can send them a list of available users. But when they send new user data via POST, we create a new user. What if they want to view / update / delete a single user instance?
Resources: Collections vs Elements
We can broadly classify the resources into two categories – “collections” and “elements” and apply the http verbs to them. Now we have two different kinds of resources – “user collection” as a whole and “individual users”. How do we map the different http verbs to them? Wikipedia has a nice chart for us.
For Collections: (/user)
GET – List all users
POST – Create a new user
PUT – Replace all users with these new users
DELETE – Delete all users
For Elements: (/user/123)
GET – Retrieve data about user with ID 123
POST – Generally not used and throws errors but can be used if the resource itself is a nested collection. In that case creates new element within that collection.
PUT – Replace the user data
DELETE – Delete the user
Is this Official?
Everything makes sense and sounds good. So I guess everyone on the web follows this standard? Well, no. Technically, REST is an architecture or architectural style/pattern. It is not a protocol or a standard itself (although it does use other standards like XML or JSON). The sad fact is that nobody has to follow the principles. This is why we often would come across APIs which would not adhere to these principles and design things their own way. And that is kind of alright given REST is not engraved in a holy stone.
But should we follow the principle? Of course we should, we want to be the good citizens of the RESTful web, don’t we?
How can I consume / create REST APIs?
You do have a curious mind, don’t you? What good is our knowledge of REST if we are not using it ourselves? Of course we shall. This blog post just scratches the surface of it. There is so much more to learn about REST and we shall learn those things in time. I have plans to write detailed posts on consuming and creating REST APIs in the future. You just have to stay in touch! If you haven’t yet, it would be a good idea to subscribe to the mailing list and I will let you know when I write the next piece, deal?
If you have any feedback on this post, please feel free to let me know in the comments. I hope you liked reading it. Also don’t forget to tell your friends about the wonderful world of REST, will you? 🙂
