Year: 2016

Python

Embedding IPython in your application

Post author By masnun
Post date January 15, 2016
2 Comments on Embedding IPython in your application

If you work with Python regularly, you probably know about IPython already. IPython has web based notebooks, QT based GUI consoles and plain old simple Terminal based REPL which is simply fantastic. But that’s not all, we can also embed IPython in our applications too. And this can lead to a number of potential use cases.

Use Cases

A common use case could be to drop into a IPython shell for quick interactive debugging. This can come very handy during prototyping.

Let’s see an example:

import IPython

name = "Masnun"

IPython.embed()

import IPython

name = "Masnun"

IPython.embed()

When we run this code, we will get a nice IPython REPL where we can try out things. In our case, we haven’t done much except defining a variable named name. We can print it out.

In [1]: print(name)
Masnun

1 2	In [1]: print(name) Masnun

I use Iron.io workers/queues/caches at my day to day job. So I often need to check status of the workers or get the size of a queue or even queue a few workers. I also need to check a few records on Mongodb. An interactive prompt can be really helpful for these.

import IPython

from iron_worker import IronWorker
from iron_mq import IronMQ
from iron_worker import Task
from pymongo import MongoClient


worker = IronWorker(project_id="PROJECT_ID", token="TOKEN")
mq = IronMQ(project_id="PROJECT_ID", token="TOKEN")
mongo = MongoClient("MONGOURL")


def launch_workers(name, number=1):
    for x in range(number):
        task = Task(code_name=name)
        worker.queue(task)
    print("Launched {} workers for {}".format(name, number))


def top_buyers():
    customers_collection = mongo.shop.customers
    query = {
        "purchases": {"$gte": 100}

    }

    print(customers_collection.find(query).count())


IPython.embed()

import IPython

from iron_worker import IronWorker

from iron_mq import IronMQ

from iron_worker import Task

from pymongo import MongoClient

worker = IronWorker(project_id="PROJECT_ID", token="TOKEN")

mq = IronMQ(project_id="PROJECT_ID", token="TOKEN")

mongo = MongoClient("MONGOURL")

def launch_workers(name, number=1):

for x in range(number):

task = Task(code_name=name)

worker.queue(task)

print("Launched {} workers for {}".format(name, number))

def top_buyers():

customers_collection = mongo.shop.customers

query = {

"purchases": {"$gte": 100}

}

print(customers_collection.find(query).count())

IPython.embed()

Now I can just do launch_workers("send_emails", 3) to launch 3 worker instances for the “send_emails” worker. Or get the number of buyers with more than 100 purhcases with the top_buyers() function.

Customizing The Prompt

When we embed IPython, it displays it’s common banner when starting.

Python 2.7.8 (default, Nov 15 2014, 03:09:43)
Type "copyright", "credits" or "license" for more information.

IPython 2.3.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

Python 2.7.8 (default, Nov 15 2014, 03:09:43)

Type "copyright", "credits" or "license" for more information.

IPython 2.3.1 -- An enhanced Interactive Python.

? -> Introduction and overview of IPython's features.

%quickref -> Quick reference.

help -> Python's own help system.

object? -> Details about 'object', use 'object??' for extra details.

We can easily disable that. To do so, we need to pass empty string to the banner1 parameter to the embed method.

IPython.embed(banner1="")

1	IPython.embed(banner1="")

Or we can further customize the 2nd banner or the exit message like this:

IPython.embed(
    banner1="Entering Debug Mode", 
    banner2="Here's another helpful message",
    exit_msg="Exiting the debug mode!"
)

IPython.embed(

banner1="Entering Debug Mode",

banner2="Here's another helpful message",

exit_msg="Exiting the debug mode!"

)

NodeJS

Deploying a NodeJS 5 app with nginx on Ubuntu

Post author By masnun
Post date January 1, 2016

In this blog post, we would follow the steps to deploy a NodeJS app using the latest version of Node with nginx on Ubuntu.

Installing Node.js 5

The official repository on Ubuntu doesn’t ship the latest version of NodeJS yet. So we will use a third party source to install it from:

curl -sL https://deb.nodesource.com/setup_5.x | sudo -E bash -
sudo apt-get install -y nodejs

1 2	curl -sL https://deb.nodesource.com/setup_5.x \| sudo -E bash - sudo apt-get install -y nodejs

Using PM2

I love PM2 for keeping my node apps alive. If you didn’t know, PM2 is an awesome tool that launches Node processes and monitors them. If it crashes, it can restart them. PM2 is very easy to setup and use. It’s also quite feature packed.

We would setup PM2, launch our app with it and then generate a launch script so PM2 itself is started on system reboot.

npm install -g pm2 # Install PM2
pm2 start index.js # Start an app
pm2 startup ubuntu # Create a launch script for PM2

npm install -g pm2 # Install PM2

pm2 start index.js # Start an app

pm2 startup ubuntu # Create a launch script for PM2

Nginx Configuration

Now that the app is running, it’s time to setup nginx as our reverse proxy. Here’s the default configurations I use:

server {
    listen 80;


    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

server {

listen 80;

location / {

proxy_pass http://localhost:3000;

proxy_http_version 1.1;

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection 'upgrade';

proxy_set_header Host $host;

proxy_cache_bypass $http_upgrade;

}

Javascript NodeJS

Live Debugging Webhooks with Ngrok

Post author By masnun
Post date January 1, 2016

ngrok is an awesome service – it creates secure tunnels to localhost. With ngrok, you get a url like http://459387bb.ngrok.com which is actually tunnel to a port to your local machine. So any request you make to that url is served by the app that you run on that port.

I know there are many cool services to debug webhooks like Requestbin – but the main benefit of ngrok is the app keeps running on your app, serving live traffic. So you can debug it in real time.

In this blog post, we would use a Node.js server with ngrok to serve Mandrill webhook requests.

Installing ngrok

Downloading and installing ngrok is pretty easy as you can find here — https://ngrok.com/download. However, if you’re on OS X and use Homebrew, you can install it with just one command:

brew install ngrok

1	brew install ngrok

Creating a Node.js App

Here’s a sample Node app that listens on port 3000 and parses the mandrill payload using body-parser package.

"use strict";

//----> Require
var bodyParser = require("body-parser");
var express = require("express");

//----> Bootstrap
var app = express();

//----> Middlewares
app.use(bodyParser.urlencoded({
    extended: false
}));

//----> Routes

// Just a hello world
app.get("/", (req, res) => {
    res.send("Hello world!");
});

// Mandrill webhook
app.post("/webhooks/mandrill", (req, res) => {
    var payload = req.body['mandrill_events'];
    var events = JSON.parse(payload);

    events.forEach((item) => {
        var event = item['event'];
        var email = item['msg']['email'];
        var doc = {
            "event": event,
            "email": email
        };

        var json = JSON.stringify(doc);
        // Do some more processing

    });

    res.send("OK");
});


//----> Run the server
var server = app.listen(3000, function () {
    var host = server.address().address;
    var port = server.address().port;

    console.log('App listening at http://%s:%s', host, port);
});

"use strict";

//----> Require

var bodyParser = require("body-parser");

var express = require("express");

//----> Bootstrap

var app = express();

//----> Middlewares

app.use(bodyParser.urlencoded({

extended: false

}));

//----> Routes

// Just a hello world

app.get("/", (req, res) => {

res.send("Hello world!");

});

// Mandrill webhook

app.post("/webhooks/mandrill", (req, res) => {

var payload = req.body['mandrill_events'];

var events = JSON.parse(payload);

events.forEach((item) => {

var event = item['event'];

var email = item['msg']['email'];

var doc = {

"event": event,

"email": email

};

var json = JSON.stringify(doc);

// Do some more processing

});

res.send("OK");

});

//----> Run the server

var server = app.listen(3000, function () {

var host = server.address().address;

var port = server.address().port;

console.log('App listening at http://%s:%s', host, port);

});

Tunneling Traffic

Once we have the app running on port 3K, we can ask ngrok to create a tunnel for us. For this we just need to pass the port number to the ngrok command:

ngrok 3000

1	ngrok 3000

We would get an url soon afterwards. We can use this url to POST requests. In our case, go to your Mandrill account and create a webhook. Mandrill will send events to this url and it will be served by your app, running locally on your machine. You can make changes to the codes and restart anytime.

Awesome, no?

Recent Posts

Recent Comments

Archives

Categories