Receive 10 links you bookmarked on Pinboard every morning using Celery

Written by Yannik Messerli - 8 December 2014

If, like me, you are bookmarking dozen websites each day and forget them after a week, you may want to receive random picks every morning to refresh your memory. I'm always pleased to start my work day by receiving internet gems found years ago. I'm using Pinboard and - like Delicious - it has a nice API. So, I spent my rainy Sunday coding a little script to fetch 10 random links from my account and send it to myself every morning.

The script itself was not really challenging. You can find it here. I had more difficulties to setup a cron task that would repeatly execute it. Something light and simple. Because I'm in a python period (programming languages are like cloth fashion for me), I was looking at Celery. Although Celery is not focused on tasks scheduling - it queues tasks and distributes them, it has a simple built-in system for it.

If you want to setup this script, here is a little help. I'll assume you have an ubuntu or debian server, a gmail account, and some basic knowledge in python. The first step is to get the code from my repository:

git clone https://github.com/yannikmesserli/pinboard-celery
cd pinboard-celery

Then, you'll need to install the required frameworks. I usually use virtualenv and pip to maintain the dependencies. They are all stored in the file reqs.txt:

virtualenv ENV
source ENV/bin/activate
pip install -r reqs.txt

Now comes the tricky part (at least for me). To distribute and sync tasks, Celery relies on different softwares, called brokers. The best known is RabbitMQ that has also proven to have a lot of online documentation. So, I decided to install it:

sudo apt-get install rabbitmq-server

To start the broker, RabbitMQ, as a deamon, just type:

sudo rabbitmq-server -detached

Now we need to be sure it is up and running. A quick check as follows should print a lot of lines:

ps aux | grep rabbit

Then, you are good to go. The broker is ready to accept messages from Celery. Celery's architecture uses workers that will actually execute the scheduled scripts. So, before sending any task we need to start some workers as follows:

nohup celery -A framework.celery.celery worker &

Now we are ready to ask Celery to execute our script every day at 8.00 AM. Celery's component that deals with periodic tasks is called 'beat'. We run it as follows:

nohup celery -A framework.celery.celery beat &

Then, every morning you'll receive a mail like this:


Mail

Email with pinboard links.

Written by

Yannik Messerli

Comments