masnun, Author at Abu Ashraf Masnun

There’s a new updated version of this article here: http://masnun.rocks/2016/09/25/introduction-to-django-channels/

Django has long been an excellent web framework. It has helped many developers and numerous businesses succeed over the years. But before the introduction of Channels, Django only supported the http protocol well. With the gradual evolution of the web technologies, standing here in 2016, supporting http only is simply not enough. Today, we are using websockets for real time communications, WebRTC is getting popular for real time collaboration or video calling, HTTP/2 is also being adapted by many. In the current state of the web, any modern web framework needs to be able to support more and more protocols. This is where Django Channels come into play. Channels aim at adding new capabilities to Django, including the support for modern web technologies like websockets or http2.

How does “Channels” work?

The idea behind Channels is quite simple. To understand the concept, let’s first walk through an example scenario, let’s see how Channels would process a request.

A http/websocket request hits the reverse proxy (ie, nginx). This step is not compulsory but we’re conscious developers and always make sure our requests first go through a hardened, battle proven reverse proxy before it hits our application server

Nginx passes the request to an application server. Since we’re dealing with multiple protocols now, instead of application server, let’s call it “Interface Server”. This interface server knows how to handle requests using different protocols. The interface server accepts the request and transforms into a message. It then passes the message on to a channel.

We have to write consumers which will listen on to specific channels. When new messages arrive on those channels, the consumers would process them and if needed, send a response back to a reply/response channel. The interface server listens on to these response channels and when we write back to these channels, the interface server reads the message and transmits it to the outside world (in this case, our user). The consumers are run in background worker processes. We can spawn as many workers as we like to scale up.

So as you can see, the concept is really simple – an interface server accepts requests and queues them as messages on channels. Consumers process these queues and write back responses on response channels. The interface server sends back the responses. Plain, simple yet effective!

There are channels which are already available for us. For example – http.request channel can be listened on if we want to handle incoming http messages. Or websocket.receive can be used to process incoming websocket messages. In reality, we would probably be less interested in handling http.request ourselves and rather let Django handle it. We would be more interested in adding our custom logic for websocket connections or other protocols. Besides the channels which are already available, we can also create our own custom channels for different purposes. Since the project works by passing messages to channels and handling them with background workers, we can actually use it for managing our background tasks too. For example, instead of generating thumbnails on the fly, we can pass the image information as a message to a channel and the worker does the thumbnailing in the background. By default Channels ship with a management command – runworker which can run background workers to listen to the channels. However, till now, there is no retry mechanism if the message delivery somehow fails. In this regard, Celery can be an excellent choice for writing / running / managing the background workers which would process these channels.

Daphne is now the de-facto interface server that works well with Channels. The channels and message passing work through a “channel layer” which support multiple backends. The popular ones are – In Memory, Redis, IPC. As you can guess, these backends and the channel layer is used to abstract away the process of maintaining different channels/queues and allowing workers to listen to those. In Memory backend maintains the channels in memory and is a good fit for local development. While a Redis cluster would be more suitable in a production environment for scaling up.

Let’s Build a WebSocket Echo Server

Enough talk. Let’s build a simple echo server. But before we can do that, we first have to install the package.

pip install channels

1	pip install channels

That should install Django (as it’s a dependency of channels) and channels along with the necessary packages. Start a Django project with django-admin and create an app.

Now add channels to the INSTALLED_APPS list in your settings.py. For local development, we are fine with the in memory channel layer, so we need to put these lines in settings.py to define the default channel layer:

CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "asgiref.inmemory.ChannelLayer",
        "ROUTING": "realtime.routing.channel_routing",
    },
}

CHANNEL_LAYERS = {

"default": {

"BACKEND": "asgiref.inmemory.ChannelLayer",

"ROUTING": "realtime.routing.channel_routing",

}

In the above code, please note the ROUTING key. As the value of this key, we have to pass the path to our channel routing. In my case, I have an app named realtime and there’s a module named routing.py which has the channel routing.

from channels.routing import route
from .consumers import websocket_receive

channel_routing = [
    route("websocket.receive", websocket_receive, path=r"^/chat/"),
]

from channels.routing import route

from .consumers import websocket_receive

channel_routing = [

route("websocket.receive", websocket_receive, path=r"^/chat/"),

]

In the channel routing list, we define our routes which looks very similar to Django’s url patterns. When we receive a message through a websocket connection, the message is passed on to the websocket.receive channel. So we defined a consumer to consume messages from that channel. We also defined a path to indicate that websocket connections to /chat/ should be handled by this particular route. If we omit the path, the clients can connect to any url on the host and we can catch them all! But if we define a path, it helps us namespace things and in another cause which we will see later in this article.

And here’s the consumers.py:

def websocket_receive(message):
    text = message.content.get('text')
    if text:
        message.reply_channel.send({"text": "You said: {}".format(text)})

def websocket_receive(message):

text = message.content.get('text')

if text:

message.reply_channel.send({"text": "You said: {}".format(text)})

The consumer is very basic. It retrieves the text we received via websocket and replies back. Note that the websocket content is available on the content attribute of the message. And the reply_channel is the response channel here (the interface server is listening on to this channel). Whatever we send to this channel is passed back to the websocket connection.

We have defined our channel layer, created our consumer and mapped a route to it. Now we just need to launch the interface server and the background workers (which run the consumers). In local environment, we can just run – python manage.py runserver as usual. Channels will make sure the interface server and the workers are running in the background. (But this should not be used in production, in production we must use Daphne separately and launch the workers individually. See here).

Once our dev server starts up, let’s open up the web app. If you haven’t added any django views, no worries, you should still see the “It Worked!” welcome page of Django and that should be fine for now. We need to test our websocket and we are smart enough to do that from the dev console. Open up your Chrome Devtools (or Firefox | Safari | any other browser’s dev tools) and navigate to the JS console. Paste the following JS code:

socket = new WebSocket("ws://" + window.location.host + "/chat/");
socket.onmessage = function(e) {
    alert(e.data);
}
socket.onopen = function() {
    socket.send("hello world");
}

socket = new WebSocket("ws://" + window.location.host + "/chat/");

socket.onmessage = function(e) {

alert(e.data);

}

socket.onopen = function() {

socket.send("hello world");

}

If everything worked, you should get an alert with the message we sent. Since we defined a path, the websocket connection works only on /chat/. Try modifying the JS code and send a message to some other url to see how they don’t work. Also remove the path from our route and see how you can catch all websocket messages from all the websocket connections regardless of which url they were connected to. Cool, no?

Our websocket example was very short and we just tried to demonstrate how things work in general. But Django Channels provide some really cool features to work with websockets. It integrates with the Django Auth system and authenticates the websocket users for you. Using the Group concept, it is very easy to create group chats or live blogs or any sort of real time communication in groups. Love Django’s generic views? We have generic consumers to help you get started fast. The channels docs is quite nice, I suggest you read through the docs and try the concepts.

Using our own channels

We can create our own channels and add consumers to them. Then we can simply add some messages to those channels by using the channel name. Like this:

Channel("thumbnailer").send({
        "image_id": image.id
    })

Channel("thumbnailer").send({

"image_id": image.id

})

WSGI or ASGI?

Since Daphne and ASGI is still new, some people still prefer to handle their http requests via WSGI. In such cases, we can configure nginx to route the requests to different servers (wsgi / asgi) based on url, domain or upgrade header. In such cases, having the real time end points under particular namespace can help us easily configure nginx to send the requests under that namespace to Daphne while sending all others to wsgi.

This post assumes you already know about Promises, Generators and aims at focusing more on the co library and the generator based workflow it supports. If you are not very familiar with Promises or Generators, I would recommend you study them first.

Promises are nice

Promises can save us from the callback hell and allow us to write easy to understand codes. May be something like this:

function promisedOne() {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve(1)
    }, 3000);
  })
}

promisedOne().then(console.log);

function promisedOne() {

return new Promise((resolve, reject) => {

setTimeout(() => {

resolve(1)

}, 3000);

})

}

promisedOne().then(console.log);

Here we have a promisedOne function that returns a Promise. The promise is resolved after 3 seconds and the value is set to 1. For keeping it simple, we used setTimeout. We can imagine other use cases like network operations where promises can be handy instead of callbacks.

Generator Based Workflow

With the new ES2015 generators and a nice generator based workflow library like co (https://github.com/tj/co) , we can do better. Something like this:

const co = require('co');

function promisedOne() {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve(1)
    }, 3000);
  })
}

function * main() {
  try {
    let promised_value = yield promisedOne();
    console.log(promised_value)
  } catch (e) {
    console.error(e);
  }

}

co(main)

const co = require('co');

function promisedOne() {

return new Promise((resolve, reject) => {

setTimeout(() => {

resolve(1)

}, 3000);

})

}

function * main() {

try {

let promised_value = yield promisedOne();

console.log(promised_value)

} catch (e) {

console.error(e);

}

co(main)

What’s happening here? The co function takes a generator function and executes it. Whenever the generator function yields something, co checks if the object is one of the yieldables that it supports. In our case, it’s a Promise which is supported. So co takes the yielded promise, processes it and returns the results back into the generator. So we can grab the value from the yield expression. If the promise is rejected or any error occurs, it’s thrown back to the generator function as well. So we could catch the error.

co returns a Promise. So if the generator function returns any values for us, we can retrieve those using the promise APIs.

function * returnSomething() {
  return "Done"
}

co(returnSomething).then(console.log);

// Or

co(function * () {
  return "Something else"
}).then(console.log);


// Let's make it a little more complex

co(function *() {
  return yield new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve(1)
    }, 3000);
  })
}).then(console.log);

function * returnSomething() {

return "Done"

}

co(returnSomething).then(console.log);

// Or

co(function * () {

return "Something else"

}).then(console.log);

// Let's make it a little more complex

co(function *() {

return yield new Promise((resolve, reject) => {

setTimeout(() => {

resolve(1)

}, 3000);

})

}).then(console.log);

So basically, we yield from the yieldables, catch any errors and return the values. We don’t have to worry about how the generator is working behind the scene.

The co library also has an useful function – co.wrap. This function takes a generator function but instead of executing it directly, it returns a general function which returns a promise.

const co = require('co');

function promisedOne() {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve(1)
    }, 3000);
  })
}

const main = co.wrap(function *() {
  return yield promisedOne();
});


// main function now returns promise
main().then(console.log);

const co = require('co');

function promisedOne() {

return new Promise((resolve, reject) => {

setTimeout(() => {

resolve(1)

}, 3000);

})

}

const main = co.wrap(function *() {

return yield promisedOne();

});

// main function now returns promise

main().then(console.log);

This can often come handy when we want to use co with other libraries/frameworks which don’t support generator based workflow. For example, here’s a Gist that demonstrates how to use generators with the HapiJS web framework – https://gist.github.com/grabbou/ead3e217a5e445929f14. The route handlers are written using generator function and then adapted using co.wrap for Hapi.

Recent Posts

Recent Comments

Archives

Categories