Author: masnun

Python

Building a Facebook Messenger Bot with Python

Post author By masnun
Post date May 22, 2016
33 Comments on Building a Facebook Messenger Bot with Python

Facebook now has the Messenger Platform which allows us to build bots which can accept messages from users and respond to them. In this tutorial, we shall see how we can build a bot and add it to one of our pages so that the users can interact with the bot by sending messages to the page.

To get started, we have three requirements to fulfill:

We need a Facebook Page
We need a Facebook App
We need a webhook / callback URL to accept incoming messages

I am assuming you already have a Facebook Page. If you don’t, go ahead and create one. It’s very simple.

Creating and Configuring The Facebook App

(1) First, we create a generic facebook app. We need to provide the name, namespace, category, contact email. Simple and straightforward. This is how it looks for me:

Create a New FB App .

(2) Now we have to browse the “Add Product” section and add “Messenger”.

Add Messenger

(3) Generate access token for a Page you manage. A popup will open asking you for permissions. Grant the permission and you will soon see the access token for that page. Please take a note of this token. We shall use it later send messages to the users on behalf of the page.

Next, click the “Webhooks” section.

(4) Before we can setup a webhook, we need to setup an URL which is publicly accessible on the internet. The URL must have SSL (that is it needs to be https). To meet this requirement and set up a local dev environment, we setup a quick flask app on our local machine.

Install Flask from PyPi using pip:

pip install Flask

1	pip install Flask

Facebook will send a GET request to the callback URL we provide. The request will contain a custom secret we can add (while setting up the webhook) and a challenge code from Facebook. They expect us to output the challenge code to verify ourselves. To do so, we write a quick GET handler using Flask.

## Filename: server.py 

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET'])
def handle_verification():
    return request.args['hub.challenge']

if __name__ == '__main__':
    app.run(debug=True)

## Filename: server.py

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET'])

def handle_verification():

return request.args['hub.challenge']

if __name__ == '__main__':

app.run(debug=True)

We run the local server using python server.py. The app will launch at port 5000 by default. Next we use ngrok to expose the server to the internet. ngrok is a fantastic tool and you should seriously give it a try for running and debugging webhooks/callback urls on your local machine.

ngrok http 5000

1	ngrok http 5000

With that command, we will get an address like https://ac433506.ngrok.io. Copy that url and paste it in the Webhook setup popup. Checkmark the events we’re interested in. I check them all. Then we input a secret, which our code doesn’t care about much. So just add anything you like. The popup now looks like this:

Click “Verify and Save”. If the verification succeeds, the popup will close and you will be back to the previous screen.

Select a Page again and click “Subscribe”. Now our app should be added to the page we selected. Please note, if we haven’t generated an access token for that page in the earlier step, the subscription will fail. So make sure we have an access token generated for that page.

Handling Messages

Now every time someone sends a message to the “Masnun” page, Facebook will make a POST request to our callback url. So we need to write a POST handler for that url. We also need respond back to the user using the Graph API. For that we would need to use the awesome requests module.

pip install requests

1	pip install requests

Here’s the code for accepting incoming messages and sending them a reply:

from flask import Flask, request
import requests

app = Flask(__name__)

ACCESS_TOKEN = "EAAP9MMaGh1cBAHS7jZCnuQgm2GWx5grLraIElFlWlIw2r3Afb34m2c2rP0xdkkkKEeiBOykGINAP0tScwmL5NNBJQN9ayPCuq13syvWocmbYZA7BXL86FsZCyZBxTmkgYYp8MDulLc1Tx70FGdU5ebQZAJV28nMkZD"


def reply(user_id, msg):
    data = {
        "recipient": {"id": user_id},
        "message": {"text": msg}
    }
    resp = requests.post("https://graph.facebook.com/v2.6/me/messages?access_token=" + ACCESS_TOKEN, json=data)
    print(resp.content)


@app.route('/', methods=['POST'])
def handle_incoming_messages():
    data = request.json
    sender = data['entry'][0]['messaging'][0]['sender']['id']
    message = data['entry'][0]['messaging'][0]['message']['text']
    reply(sender, message[::-1])

    return "ok"


if __name__ == '__main__':
    app.run(debug=True)

from flask import Flask, request

import requests

app = Flask(__name__)

ACCESS_TOKEN = "EAAP9MMaGh1cBAHS7jZCnuQgm2GWx5grLraIElFlWlIw2r3Afb34m2c2rP0xdkkkKEeiBOykGINAP0tScwmL5NNBJQN9ayPCuq13syvWocmbYZA7BXL86FsZCyZBxTmkgYYp8MDulLc1Tx70FGdU5ebQZAJV28nMkZD"

def reply(user_id, msg):

data = {

"recipient": {"id": user_id},

"message": {"text": msg}

}

resp = requests.post("https://graph.facebook.com/v2.6/me/messages?access_token=" + ACCESS_TOKEN, json=data)

print(resp.content)

@app.route('/', methods=['POST'])

def handle_incoming_messages():

data = request.json

sender = data['entry'][0]['messaging'][0]['sender']['id']

message = data['entry'][0]['messaging'][0]['message']['text']

reply(sender, message[::-1])

return "ok"

if __name__ == '__main__':

app.run(debug=True)

The code here accepts a message, retrieves the user id and the message content. It reverses the message and sends back to the user. For this we use the ACCESS_TOKEN we generated before hand. The incoming request must be responded with a status code 200 to acknowledge the message. Otherwise Facebook will try the message a few more times and then disable the webhook. So sending a http status code 200 is important. We just output “ok” to do so.

You can now send a message to your page and see if it responds correctly. Check out Flask’s and ngrok’s logs to debug any issues you might face.

You can download the sample code from here: https://github.com/masnun/fb-bot

Django Python

Django: Running management commands inside a Docker container

Post author By masnun
Post date April 23, 2016
3 Comments on Django: Running management commands inside a Docker container

Okay, so we have dockerized our django app and we need to run a manage.py command for some task. How do we do that? Simple, we have to locate the container that runs the django app, login and then run the command.

Locate The Container

It’s very likely that our app uses multiple containers to compose the entire system. For exmaple, I have one container running MySQL, one container running Redis and another running the actual Django app. If we want to run manage.py commands, we have to login to the one that runs Django.

While our app is running, we can find the running docker containers using the docker ps command like this:

$ docker ps
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                    NAMES
308f40bba888        crawler_testscript   "/sbin/my_init"          31 hours ago        Up 3 seconds        5000/tcp                 crawler_testscript_1
3a5ccc872215        crawler_web          "bash run_web.sh"        31 hours ago        Up 4 seconds        0.0.0.0:8000-&gt;8000/tcp   crawler_web_1
14f0e260fb2c        redis:latest         "/entrypoint.sh redis"   31 hours ago        Up 4 seconds        0.0.0.0:6379-&gt;6379/tcp   crawler_redis_1
252a7092870d        mysql:latest         "/entrypoint.sh mysql"   31 hours ago        Up 4 seconds        0.0.0.0:3306-&gt;3306/tcp   crawler_mysql_1

$ docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

308f40bba888 crawler_testscript "/sbin/my_init" 31 hours ago Up 3 seconds 5000/tcp crawler_testscript_1

3a5ccc872215 crawler_web "bash run_web.sh" 31 hours ago Up 4 seconds 0.0.0.0:8000->8000/tcp crawler_web_1

14f0e260fb2c redis:latest "/entrypoint.sh redis" 31 hours ago Up 4 seconds 0.0.0.0:6379->6379/tcp crawler_redis_1

252a7092870d mysql:latest "/entrypoint.sh mysql" 31 hours ago Up 4 seconds 0.0.0.0:3306->3306/tcp crawler_mysql_1

In my case, I am using Docker Compose and I know my Django app runs using the crawler_web image. So we note the name of the container. In the above example, that is – crawler_web_1.

Nice, now we know which container we have to login to.

Logging Into The Container

We use the name of the container to login to it, like this:

docker exec -it crawler_web_1 bash

1	docker exec -it crawler_web_1 bash

The command above will connect us to the container and land us on a bash shell. Now we’re ready to run our command.

Running the command

We cd into the directory if necessary and then run the management command.

cd /project
python manage.py <command>

1 2	cd /project python manage.py <command>

Summary

docker ps to list running containers and locate the one
docker exec -it [container_name] bash to login to the bash shell on that container
cd to the django project and run python manage.py [command]

Django Python

Django REST Framework: Remember to disable Web Browsable API in Production

Post author By masnun
Post date April 20, 2016
7 Comments on Django REST Framework: Remember to disable Web Browsable API in Production

So this is what happened – I built an url shortening service at work for internal use. It’s a very basic app – shortens urls and tracks clicks. Two models – URL and URLVisit. URL model contains the full url, slug for the short url, created time etc. URLVisit has information related to the click, like user IP, browser data, click time etc and a ForeignKey to URL as expected.

Two different apps were using this service, one from me, another from a different team. I kept the Web Browsable API so the developers from other teams can try it out easily and they were very happy about it. The only job of this app was url shortening so I didn’t bother building a different home page. When people requested the / page on the domain, I would redirect them directly to /api/.

Things were going really great initially. There was not very heavy load on the service. Roughly 50-100 requests per second. I would call that minimal load. The server also had decent hardware and was running on an EC2 instance from AWS. nginx was on the front while the app was run with uwsgi. Everything was so smooth until it happened. After a month and half, we started noticing very poor performance of the server. Sometimes it was taking up to 40 seconds to respond. I started investigating.

It took me some time to find out what actually happened. By the time it happened, we have shortened more than a million urls. So when someone was visiting /api/url-visit/ – the web browsable api was trying to render the html form. The form allows the user to choose one of the entries from the URL model inside a select (dropdown). Rendering that page was causing usages of 100% cpu and blocking / slowing down other requests. It’s not really DRF’s fault. If I tried to load a million of entries into a select like that, it would crash the app too.

Even worse – remember I added a redirect from the home page, directly to the /api/ url? Search engines (bots) started crawling the urls. As a result the app became extremely slow and often unavailable to nginx. I initially thought, I could stop the search engine crawls by adding some robots.txt or simply by adding authentication to the API. But developers from other teams would still time to time visit the API to try out things and then make the app non responsive. So I did what I had to – I disabled the web browsable API and added a separate documentation demonstrating the use of the API with curl, PHP and Python.

I added the following snippet in my production settings file to only enable JSONRenderer for the API:

REST_FRAMEWORK = {
    'DEFAULT_RENDERER_CLASSES': (
        'rest_framework.renderers.JSONRenderer',
    )
}

REST_FRAMEWORK = {

'DEFAULT_RENDERER_CLASSES': (

'rest_framework.renderers.JSONRenderer',

)

}

Things have become pretty smooth afterwards. I can still enjoy the nice HTML interface locally where there are much fewer items. While on my production servers, there is no web browsable APIs to cause any bottlenecks.

Recent Posts

Recent Comments

Archives

Categories