Strongly typed powershell csv parser

Somehow I missed the powershell boat. I’ve been a .NET developer for years and I trudged through using the boring old cmd terminal, frequently mumbling about how much I missed zsh. But something snapped and I decided to really dive into powershell and learn why those who use it really love the hell out of it. After realizing that the reason everyone loves it is because everything is strongly typed and you can use .NET in your shell I was totally sold.

My first forays into powershell included customizing the shell environment. First I got conemu and made it look nice and pretty. Next was to get an ls highlighting module, since I love that about unix shells.

I set up a few fun aliases in my profile and felt ready to conquer the world! My next experiment was to try and create an actual binary cmdlet. I figured, what … Read more

, , , ,

A simple templating engine

I wanted to talk about templating, since templating is a common thing you run into. Often times you want to cleanly do a string replace on a bunch of text, and sometimes even need minimal language processing to do what you want. For example, Java has a templating engine called Velocity, but lots of languages have libraries that do this kind of work. I thought it’d be fun to create a small templating engine from scratch with F# as an after work exercise.

The goal is to give the templating processor a set of lookup bags that can be resolved by variables. For example, if I use a variable $devshorts.isgreat that should correspond to a bag that is keyed first off of devshorts which returns a new bag, and then a new bag that has a key isgreat which should return a value.

Getting the AST

First, lets parse the … Read more

, , , , ,

Debugging F# NUnit equals for mixed type tuples

Twitter user Richard Dalton asked a great question recently:

Twitter question

And after a bit more digging he then mentioned

Twitter question

Interesting. I downloaded the NUnit source and saw this:

public bool AreEqual(object x, object y, ref Tolerance tolerance)
    this.failurePoints = new List<FailurePoint>();

    if (x == null && y == null)
        return true;

    if (x == null || y == null)
        return false;

    if (object.ReferenceEquals(x, y))
        return true;

    Type xType = x.GetType();
    Type yType = y.GetType();

    EqualityAdapter externalComparer = GetExternalComparer(x, y);
    if (externalComparer != null)
        return externalComparer.AreEqual(x, y);

    if (xType.IsArray && yType.IsArray && !compareAsCollection)
        return ArraysEqual((Array)x, (Array)y, ref tolerance);

    if (x is IDictionary && y is IDictionary)
        return DictionariesEqual((IDictionary)x, (IDictionary)y, ref tolerance);

    //if (x is ICollection && y is ICollection)
    //    return CollectionsEqual((ICollection)x, (ICollection)y, ref tolerance);

    if (x is IEnumerable && y is IEnumerable && !(x is string && y is string))
        return EnumerablesEqual((IEnumerable)x, (IEnumerable)y, ref tolerance);

    if (x is string && 
Read more


Checking if a socket is connected

Testing if a socket is still open isn’t as easy at it sounds. Anyone who has ever dealt with socket programming knows this is hassle. The general pattern is to poll on the socket to see if its still available, usually by sitting in an infinite loop. However, with f# this can be done more elegantly using async and some decoupled functions.

First lets write an async function that monitors a socket and returns true if its connected or false if its not. The polling interval is set to 10 microseconds. This is because if you set the interval to a negative integer (representing to wait indefinitely for a response), it won’t return until data is written to the socket. If, however, you have a very short poll time, you will be able to detect when the socket is closed without having to write data to it.

/// Async worker 
Read more

, ,

F# utilities in haskell

Slowly I am getting more familiar with Haskell, but there are some things that really irk me. For example, a lot of the point free functions are right to left, instead of left to right. Coming from an F# background this drives me nuts. I want to see what happens first first not last.

For example, if we wanted to do (x+2)+3 in f#

let chained = (+) 2 >> (+) 3

Compare to haskell:

chained :: Integer -> Integer
chained = (+3) . (+2) 

In haskell, the +2 is given the argument 3, then that value is given to +3. In f# you work left to right, which I think is more readable.

Anyways, this is an easy problem to solve by defining a cusotm infix operator

(>>>) :: (a -> b) -> (b -> c) -> (a -> c)
(>>>) a b = b . a

Now … Read more

, ,

Reading socket commands

A few weeks ago I was working on a sample application that would simulate a complex state machine. The idea is that there is one control room, and many slave rooms, where each slave room has its own state. The control room can dispatch a state advance or state reverse to any room or collection of rooms, as well as query room states, and other room metadata.

But to do this I need a way to get commands from the control room in order to know what to do. In my application clients were connected via tcp sockets and I wanted commands to be newline seperated. This made it easy to test out via a local telnet (I didn’t need to design any binary protocol).

The socket

You can never assume you’ve read what you want off a socket, since you’re only ever guaranteed 1 or more bytes when a … Read more

, , ,

The Arrow operator

Continuing my journey in functional programming, I decided to try doing the 99 haskell problems to wean my way into haskell. I’ve found this to be a lot of fun since they give you the answers to each problem and, even though I have functional experience, the haskell way is sometimes very different from what I would have expected.

For example, I discovered Arrows via the following problem:

Run-length encoding of a list. Implement the so-called run-length encoding data compression method. Consecutive duplicates of elements are encoded as lists (N E) where N is the number of duplicates of the element E.


* (encode ‘(a a a a b c c a a d e e e e))
((4 A) (1 B) (2 C) (2 A) (1 D)(4 E))
Example in Haskell:

encode “aaaabccaadeeee”

My initial solution I did the way I’d probably write it in F#:

Read more

, ,

Machine Learning with disaster video posted

A few weeks ago we had our second DC F# meetup with speaker Phil Trelford where he led a hands on session introducing decision trees. The goal of meetup was to see how good of a predictor we could make of who would live and die on the titanic. Kaggle has an excellent data set that shows age, sex, ticket price, cabin number, class, and a bunch of other useful features describing Titanic passengers.

Phil followed Mathias‘ format and had an excellent .fsx script that walked everyone through it. I think the best predictor that someone made was close to 84%, though it was surprisingly difficult to exceed that in the short period of time that we had to work on it. I’d implemented my own shannon entropy based ID3 decision tree in C# so this wasn’t my first foray into decision tree’s, but the compactness of the tree … Read more

, ,

Till functions

Just wanted to share a couple little functions that I was playing with since it made my code terse and readable. At first I needed a way to fold a function until a predicate. This way I could stop and didn’t have to continue through the whole list. Then I needed to be able to do the same kind of thing but choosing all elements up until a predicate.


First, folding. I wanted to be able to get all the characters up until white space. For example:

let (++) a b = a.ToString() + b.ToString()

let upToSpaces str = foldTill Char.IsWhiteSpace (++) "" str

Which led me to write the following fold function. Granted it’s not lazy evaluated, but for me that was OK.

let foldTill check predicate seed list= 
    let rec foldTill' acc = function
        | [] -> acc
        | (h::t) -> match check h with 
                        | false 
Read more


Seq.unfold and creating bit masks

In the course of working on ParsecClone I needed some code that could take in an arbitrary byte array and convert it to a corresponding bit array. The idea is if I have an array of

[|byte 0xFF;byte 0x01|]

Then I should get


I’ve done plenty of bit slingin’ in my day, and the trick is just to apply a sequence of bit masks to each byte and collect all the bit arrays. In other languages this is always a little bit of a pain, but in F# the solution was amazingly elegant


As with anything F#, I like to start with the data

type Bit = 
    | One
    | Zero
    override this.ToString() =
        match this with 
            | One -> "1"
            | Zero -> "0"

Make bit masks

Now lets generate the meat and potatoes of this: a sequence of bit masks

let bitMasks = Seq.unfold (fun bitIndex 
Read more

, ,

Previous Posts