Category: Python

Categories
Python

Embedding IPython in your application

If you work with Python regularly, you probably know about IPython already. IPython has web based notebooks, QT based GUI consoles and plain old simple Terminal based REPL which is simply fantastic. But that’s not all, we can also embed IPython in our applications too. And this can lead to a number of potential use cases.

Use Cases

A common use case could be to drop into a IPython shell for quick interactive debugging. This can come very handy during prototyping.

Let’s see an example:

When we run this code, we will get a nice IPython REPL where we can try out things. In our case, we haven’t done much except defining a variable named name. We can print it out.

I use Iron.io workers/queues/caches at my day to day job. So I often need to check status of the workers or get the size of a queue or even queue a few workers. I also need to check a few records on Mongodb. An interactive prompt can be really helpful for these.

Now I can just do launch_workers("send_emails", 3) to launch 3 worker instances for the “send_emails” worker. Or get the number of buyers with more than 100 purhcases with the top_buyers() function.

Customizing The Prompt

When we embed IPython, it displays it’s common banner when starting.

We can easily disable that. To do so, we need to pass empty string to the banner1 parameter to the embed method.

Or we can further customize the 2nd banner or the exit message like this:

Categories
Python

Homebrew & Pyenv: Installing PyQT5 with Python3 on OSX

There was a time back in 2014 and earlier when PyQT5 installation was not straightforward and needed manual compilation. When searching on Google, still those posts come up on top results. But nothing to worry about, things have changed – it’s now quite simple.

Installation

If you are not already using Homebrew, you should start using it. Once Homebrew is installed, let’s install PyQT5 with this single command:

Verifying Installation

Let’s take a sample PyQT5 code as example and run it. For examples, I usually pick one up from the excellent PyQT tutorials on zetcode.com. Here’s one:

Run it using:

If you get a nice looking small window – it worked!

Integrating with Pyenv

I am a big fan of pyenv and use it for running different versions and flavours of Python. If you use pyenv too, chances are you have your own version of Python installed through it. However, the brew formula that installs PyQT5 depends on another formula python3 – homebrew’s own Python 3 installation. When we install PyQT5, this formula is used to install the bindings, so the bindings are only available to this particular Python 3 installation and unavailable to our pyenv versions.

We will discuss two potential solutions to this issue.

Switching to system

One simple work around is to use the Python 3 version installed by Homebrew. We can ask pyenv to switch to the system version whenever we’re doing PyQT5 development.

We can create an alias to quickly switch between Python versions. I have this in my .zshrc:

This way is very quick and simple but we miss the benefits of using pyenv.

Adding Site Packages

Alternatively, we can add the site-packages for this homebrew installed python 3 to our pyenv installation of python 3. Since both installations were built on the same machine and OS, the bindings should work correctly. We would be using .pth files to do this.

Let’s first find out the site-packages for the homebrew installation:

We would notice a message like:

That is the site-packages for this version.

Now let’s find our pyenv python3’s local site directory:

Now create a homebrew.pth file in that directory and put the previously found site packages path there.

Let’s create the file:

And put these contents:

Save and exit. Now you should be able to just use:

Categories
Django Python

Dockerizing a Django Application

I assume you are already familiar with Docker and it’s use cases. If you haven’t yet started using Docker, I strongly recommend you do soon.

I have a Django application that I want to dockerize it for local development. I am also new to Docker, so everything I do in this post might not be suitable for your production environment. So please do check Docker best practices for production apps. This tutorial is meant to be a basic introduction to Docker. In this post, I am going to use Docker Machine and Docker Compose. You can get them by installing the awesome Docker Toolbox.

Components Breakdown

Before we start, we need to break down our requirements so we can individually build the required components. For my particular application, we need these:

  1. Django App Server
  2. MySQL Database Server
  3. Redis Server

We will build images for these separately so we can create individual containers and link them together to compose our ultimate application. We shall build our Django App server and use pre-built images for MySQL and Redis.

Building the Django App Server

Before we begin, let’s talk Dockerfiles. Dockerfiles are scripts to customize our docker builds. It allows us control and flexibility over how we build the images for our applications. We will use our custom Dockerfile to build the Django app server.

To build an image for a Django application we need to go through these following steps:

  • Select a Linux image, we choose Ubuntu
  • Install required packages for the distro.
  • Install Python packages which are required for the app
  • Provide a default command to run and ports to expose

Here’s the Dockerfile we shall use:

So what are we doing here:

  • We’re choosing phusion/baseimage as our base image. It’s a barebone image based on Ubuntu. Ubuntu by default comes with many packages which we don’t need to run inside docker. This base image gets rid of those and provides a very lean and clean image to start with.
  • We just provide a Maintainer name
  • We set DEBIAN_FRONTEND to be non interactive. This will not display any interactive prompts during the build process. Since the docker build process is automated, we really don’t have any way to interact during it. So we disable interaction. And as you might have guessed already ENV sets an environment variable.
  • We install some packages we shall need.
  • We copy our requirements.txt file to /app/src/requirements.txt, change the work directory and install the packages using pip. ADD is used to copy any files or directories to the container while it builds. You might wonder why we didn’t copy over our entire project – that’s because we want to use docker for our development. We will use a nice featire of Docker which would allow us to mount our local directories directly inside the container. Doing this, we would not need to copy files every time they change. More on this will come later.
  • We change directory to /app/src/lisp and run the runall management command. This command runs the Django default server along with some other services my application needs. But usually we would want to just do runserver
  • We EXPOSE port 8000

If you go through the Dockerfile References you will notice – we can do a lot more with Dockerfiles.

Docker Compose and Linking Services

As we mentioned earlier, we shall use pre-built images for MySQL and Redis. We could build them ourselves too but why not take advantage of the well maintained images from the generous folks in the docker community?

We can link multiple docker containers to compose a final application. We can do that using the docker command manually. But Docker Compose is a very nice tool which allows us to define the services we need in a very easy to read syntax. With docker compose, we don’t need to run them manually, we can just use simple commands to do complex docker magic! Here’s our docker-compose.yml file:

In our docker-compose file, we define 3 components:

  • For the web, we pass the path to Dockerfile to build key. We ask to restart always and define volumes to mount. .:/app/src means – mount the current directory on my OS X as /app/src/ on the container. We also define which ports to expose and which containers should be linked with it
  • We also define the mysql and redis components with respective configurations. Note that we define the pre-built image name in the image key. Please make sure the volume paths exist and are accessible.

You can consult the Compose File Reference for more details.

Running The Services

To run the application, we can do:

Please note, the Django server might throw errors if the MySQL / Redis server takes time to initialize. So I usually run them separately:

Database Configuration for Django

Our MySQL server is running on the IP of the Docker Machine. You need to use this IP address in your Django settings file. To get the IP of a docker machine, type in:

Creating Initial Databases

We can pass a MYSQL_DATABASE environment value to the mysql image so the database is created when creating the service. Or we can also connect to the docker machine manually and create our databases.