Tagged: java

Unit testing DNS failovers

Something that’s come up a few times in my career is the difficulty of validating if and when your code can handle actual DNS changes. A lot of times testing that you have the right JVM settings and that your 3rd party clients can handle it involves mucking with hosts files, nameservers, or stuff like Route53 and waiting around. Then its hard to automate and deterministically reproduce. However, you can hook into the DNS resolution in the JVM to control what gets resolved to what. And this way you can tweak the resolution in a test and see what breaks! I found some info at this blog post and cleaned it up a bit for usage in scala.

The magic sauce to pull this off is to make sure you override the default sun.net.spi.nameservice.NameServiceDescriptor. Internally in the InetAddress class it tries to load an instance of the interface NameServiceDescriptorRead more

A toy generational garbage collector

Had a little downtime today and figured I’d make a toy generational garbage collector, for funsies. A friend of mine was once asked this as an interview question so I thought it might make for some good weekend practice.

For those not familiar, a common way of doing garbage collection in managed languages is to have the concept of multiple generations. All newly created objects go in gen0. New objects are also the most probably to be destroyed, as there is a lot of transient stuff that goes in an application. If an element survives a gc round it gets promoted to gen1. Gen1 doesn’t get GC’d as often. Same with gen2.

A GC cycle usually consists of iterating through application root nodes (so starting at main and traversing down) and checking to see where a reference lays in which generation. If we’re doing a gen1 collection, we’ll also do … Read more

Leadership election with cassandra

Cassandra has a neat feature that lets you expire data in a column. Using this handy little feature, you can create simple leadership election using cassandra. The whole process is described here which talks about leveraging Cassandras consensus and the column expiration to create leadership electors.

The idea is that a user will try and claim a slot for a period of time in a leadership table. If a slot is full, someone else has leadership. While the leader is still active they needs to heartbeat the table faster than the columns TTL to act as a keepalive. If it fails to heartbeat (i.e. it died) then its leadership claim can be relinquished and someone else can claim it. Unlike most leadership algorithms that claim a single “host” as a leader, I needed a way to create leaders sharded by some “group”. I call this a “LeadershipGroup” and we can … Read more

Plugin class loaders are hard

Plugin based systems are really common. Jenkins, Jira, wordpress, whatever. Recently I built a plugin workflow for a system at work and have been mired in the joys of the class loader. For the uninitiated, a class in Java is identified uniquely by the class loader instance it is created from as well as its fully qualified class name. This means that foo.bar class loaded by class loader A is not the same as foo.bar class loaded by class loader B.

There are actually some cool things you can do with this, especially in terms of code isolation. Imagine your plugins are bundled as shaded jars that contain all the internal dependencies. By leveraging class loaders you can isolate potentially conflicting versions of libraries from the host application and the plugin. But, in order to communicate to the host layer, you need a strict set of shared interfaces that the … Read more

Project angelhair: Building a queue on cassandra

Edit: this project has since been moved to CassieQ: https://github.com/paradoxical-io/cassieq

A few weeks ago my work had a hack day and I got together with some of my coworker friends and we decided to build a queue on top of Cassandra.

For the impatient, give it a try (docker hub):

docker run -it \
    -e CLUSTER_NAME="" \
    -e KEYSPACE="" \
    -e CONTACT_POINTS="" \
    -e USERNAME="" \
    -e PASSWORD="" \
    -e USE_SSL="" \
    -e DATA_CENTER="" \
    -e METRICS_GRAPHITE "true" \
    -e GRAPHITE_PREFIX=" \
    -e GRAPHITE_URL=""  \
    onoffswitch/angelhair

The core features for what we called Project Angelhair was to handle:

– long term events (so many events that AMQ or RMQ might run out of storage space)
– connectionless – wanted to use http
– invisibility – need messages to disappear when they are processing but be able to come back
– highly scaleable – wanted to distribute a docker … Read more

Converting akka scala futures to java futures

Back in akka land! I’m using the ask pattern to get results back from actors since I have a requirement to block and get a result (I can’t wait for an actor to push at a later date). Thats fine, but converting from scala futures to java completable futures is a pain. I also, (like mentioned in another post) want to make sure that my async responses capture and set the MDC for proper logging.

My final usage should look something like:

private <Response, Request> Future<Response> askActorForResponseAsync(Request source) {
    final FiniteDuration askTimeout = new FiniteDuration(config.getAskForResultTimeout().toMillis(), TimeUnit.MILLISECONDS);

    final Timeout timeout = new Timeout(askTimeout);

    final scala.concurrent.Future<Object> ask = Patterns.ask(master.getActor(), new PersistableMessageContext(source), timeout);

    return FutureConverter.fromScalaFuture(ask)
                          .executeOn(actorSystem.dispatcher())
                          .thenApply(i -> (Response) i);
}

The idea is that I’m going to translate a scala future with a callback into a completable future java promise.

Next up, the future converter:

public class FutureConverter {
    public static <T> 
Read more
Adding MDC logging to akka

I’ve mentioned before, but I’m working heavily in a project that is leveraging akka. I am really enjoying the message passing model and so far things are great, but tying in an MDC for the SLFJ logging context proved complicated. I had played with the custom executor model described here but hadn’t attempted the akka custom dispatcher.

I was thinking that a custom dispatcher would work great to pass along the MDC since then you’d never have to think about it, but unfortunately I couldn’t get it to work. Akka kept failing to instantiate the dispatcher. I was also worried about configuration data and possible tuning that you might lose giving akka your own dispatcher configurator.

So, given that I wasn’t quite sure what to do. What I ended up with however was a little extra work but turned out well. I went with an augmented dispatcher/subscriber model. Basically for … Read more

Handling subclassed constraints with a DSL in java 8

I really like doing all of my domain modeling with clean DSL’s (domain specific languages). Basically I want my code to read like a sentence, and to hide all the magic behind things. When things read clearly even a non professional can determine if something is wrong. The ideal scenario is to have your code read like pseudocode since nobody really cares what the internals are, what matters is your general solution.

I found myself recently in a scenario where I have some methods that do work, and they all return a subtype of a response root. Something like:

class Response extends ResponseRoot {}

class Worker{
   Response doWork(Request request) { // }
}

And I need to create a dynamic callback given the context of the request. So basically I want

Consumer<ResponseRoot> callback = generateCallback(request);

Response response = doWork(request);

callback.accept(response);

However I am going to have a lot of this … Read more

Simplifying class matching with java 8

I’m knee deep in akka these days and its a great queueing framework, but unfortunately I’m stuck using java and not able to use scala (business decisions, not mine!) so pattern matching on incoming untyped events can be kind of nasty.

You frequently see stuff like this in receive methods:

public void onReceive(Object message){
 if(message instanceof Something){

 }
 else if (message instanceof SomethingElse){

 }
 .. etc
}

And while that technically works, I really hate it because it promotes a monolothic function doing too much work. It also encourages less disciplined devs to put logic into the if block. While this is fine for a few checks, what happens when you need to dispatch 10, or 20 different types? It’s not uncommon in actor based systems to have lots of small message types.

Also, because akka gives you your object as a type erased Object you can’t use normal dispatching … Read more

Conditional injection with scala play and guice

It’s been a crazy year for me. For those who don’t know I moved from the east coast to the west coast to work for a rather large softare company in seattle (I’ll let you figure which one out) and after a few short weeks realized I made a horrible mistake and left the team. I then found a cool job at a smaller .net startup that was based in SF and met some awesome people and learned a lot. But, I’ve been poached by an old coworker and am now going to go work at a place that uses more open source things so I decided to kick into gear and investigate scala and play.

For the most part I’m doing a mental mapping of .NET’s web api framework to the scala play framework, but the more I play in play (pun intended) the more I like it.

On … Read more