Project angelhair: Building a queue on cassandra

Edit: this project has since been moved to CassieQ:

A few weeks ago my work had a hack day and I got together with some of my coworker friends and we decided to build a queue on top of Cassandra.

For the impatient, give it a try (docker hub):

docker run -it \
    -e CLUSTER_NAME="" \
    -e KEYSPACE="" \
    -e CONTACT_POINTS="" \
    -e USERNAME="" \
    -e PASSWORD="" \
    -e USE_SSL="" \
    -e DATA_CENTER="" \
    -e METRICS_GRAPHITE "true" \
    -e GRAPHITE_URL=""  \

The core features for what we called Project Angelhair was to handle:

– long term events (so many events that AMQ or RMQ might run out of storage space)
– connectionless – wanted to use http
– invisibility – need messages to disappear when they are processing but be able to come back
– highly scaleable – wanted to distribute a docker … Read more

, , ,

Automating deployments with salt, puppet, jenkins and docker

I know, its a buzzword mouthful. My team has had good first success leveraging jenkins, salt, sensu, puppet, and docker to package and monitor distributed java services with a one click deployment story so I wanted to share how we’ve set things up.

First and foremost, I’ve never been an ops guy. I’ve spent time on build systems like msbuild and fake, but never a full pipeline solution that also had to manage infrastructure, but there’s a first for everything. Most companies I’ve worked at have had all this stuff set up already, and out of the minds of developers, but the place I am at now does not. I actually think it’s been a great opportunity to dive into the full flow of how do you get your damn code out the door. I think if you are developing an application and don’t know how it gets from git … Read more

, , , , ,

Testing puppet with docker and python

In all the past positions I’ve been in I’ve been lucky enough to have a dedicated ops team to handle service deployment, cluster health, and machine managmenent. However, at my new company there is much more of a “self serve” mentality such that each team needs to handle things themselves. On the one hand this is a huge pain in my ass, since really the last thing I want to do is deal with clusters and machines. On the other hand though, because we have the ability to spin up openstack boxes in our data centers at the click of a button, each team has the flexibility to host their own infrastructrure and stack.

For the most part my team and I are deploying our java services using dockerized containers. Our container is a centos7 base image with a logstash forwarder in it and some other minor tooling, and we … Read more

, , ,