Tagged: scala

Unit testing DNS failovers

Something that’s come up a few times in my career is the difficulty of validating if and when your code can handle actual DNS changes. A lot of times testing that you have the right JVM settings and that your 3rd party clients can handle it involves mucking with hosts files, nameservers, or stuff like Route53 and waiting around. Then its hard to automate and deterministically reproduce. However, you can hook into the DNS resolution in the JVM to control what gets resolved to what. And this way you can tweak the resolution in a test and see what breaks! I found some info at this blog post and cleaned it up a bit for usage in scala.

The magic sauce to pull this off is to make sure you override the default sun.net.spi.nameservice.NameServiceDescriptor. Internally in the InetAddress class it tries to load an instance of the interface NameServiceDescriptor… Read more

Consistent hashing for fun

I think consistent hashing is pretty fascinating. It lets you define a ring of machines that shard out data by a hash value. Imagine that your hash space is 0 -Int.Max, and you have 2 machines. Well one machine gets all values hashed from 0 -Int.Max/2 and the other from Int.Max/2 -Int.Max. Clever. This is one of the major algorithms of distributed systems like cassandra and dynamoDB.

For a good visualization, check out this blog post.

The fun stuff happens when you want to add replication and fault tolerance to your hashing. Now you need to have replicants and manage when machines join and add. When someone joins, you need to re-partition the space evenly and re-distribute the values that were previously held.

Something similar when you have a node leave, you need to make sure that whatever it was responsible for in its primray space … Read more

Converting akka scala futures to java futures

Back in akka land! I’m using the ask pattern to get results back from actors since I have a requirement to block and get a result (I can’t wait for an actor to push at a later date). Thats fine, but converting from scala futures to java completable futures is a pain. I also, (like mentioned in another post) want to make sure that my async responses capture and set the MDC for proper logging.

My final usage should look something like:

private <Response, Request> Future<Response> askActorForResponseAsync(Request source) {
    final FiniteDuration askTimeout = new FiniteDuration(config.getAskForResultTimeout().toMillis(), TimeUnit.MILLISECONDS);

    final Timeout timeout = new Timeout(askTimeout);

    final scala.concurrent.Future<Object> ask = Patterns.ask(master.getActor(), new PersistableMessageContext(source), timeout);

    return FutureConverter.fromScalaFuture(ask)
                          .executeOn(actorSystem.dispatcher())
                          .thenApply(i -> (Response) i);
}

The idea is that I’m going to translate a scala future with a callback into a completable future java promise.

Next up, the future converter:

public class FutureConverter {
    public static <T>
… Read more
Auto scaling akka routers

I’m working on a project where I need to multiplex many requests through a finite set of open sockets. For example, I have 200 messages, but I can only have at max 10 sockets open. To accomplish this I’ve wrapped the sockets in akka actors and am using an akka routing mechanism to “share” the 10 open sockets through a roundrobin queue.

This works out great, since now the consumers (who are rabbit mq listeners) just post messages to a facacde on the resource, and akka will route the request and do the appropriate work for me.

However, I wanted to know of a clean way to be able to add more resources (or remove them). Say at runtime I am asked to add 10 more open connections, or that suddenly we need to scale down to 5 connections. I’d like the router to be able to manage that for … Read more

Tiny types scala edition

Previously I wrote about generating value type wrappers on top of C# primitives for better handling of domain level knowledge. This time I decided to try it out in scala as I’m jumping into the JVM world.

With scala we don’t have the value type capability that c# has, but we can sort of get there with implicits and case classes.

The simple gist is to generate stuff like

package com.devshorts.data

case class foo(data : String)
case class bar(data : String)
case class bizBaz(data : Int)
case class Data(data : java.util.UUID)

And the implicit conversions

package com.devshorts.data

object Conversions{
    implicit def convertfoo(i : foo) : String = i.data
    implicit def convertbar(i : bar) : String = i.data
    implicit def convertbizBaz(i : bizBaz) : Int = i.data
    implicit def convertData(i : Data) : java.util.UUID = i.data
}

Now you get a similar feel of primitive wrapping with function level unboxing and … Read more

Simple log context wrapper

I’m still toying around with the scala play! framework and I wanted to check out how I can make logging contextual information easy. In the past with .NET I’ve used and written libraries that wrap the current log provider and give you extra niceties with logging. One of my favorites was being able to do stuff like

var foo = "1";
var bar = "2";
logger.With(new { foo, bar }).Info("data")

Which would output a log line like

data, foo=1; bar=2

The logger’s “with” was even chainable so you could capture a previously built “with” context and re-use it. It was really nice when you want to create a baseline logging context for complex functions.

Java doesn’t have the concept of anonymous classes and I’m not sure you can do optimizations with reflection like you can with .NET (creating reflective based property invokers into delegates).

Either way, this makes for a … Read more

Conditional injection with scala play and guice

It’s been a crazy year for me. For those who don’t know I moved from the east coast to the west coast to work for a rather large softare company in seattle (I’ll let you figure which one out) and after a few short weeks realized I made a horrible mistake and left the team. I then found a cool job at a smaller .net startup that was based in SF and met some awesome people and learned a lot. But, I’ve been poached by an old coworker and am now going to go work at a place that uses more open source things so I decided to kick into gear and investigate scala and play.

For the most part I’m doing a mental mapping of .NET’s web api framework to the scala play framework, but the more I play in play (pun intended) the more I like it.

On … Read more