GOTO notes: Democratizing Machine Learning

04/29/13

GOTO notes: Democratizing Machine Learning

Democratizing Machine Learning With C# - LINQ goes ML
Erik Meijer

(There are almost certainly transcription errors here; these are entirely my fault. - Bill S.)

Problem: Should I play golf or not?
Streaming, real-time weather information
Want to find Func<Weather, Play>
Once I have this, I can create a real-time LINQ query using IObservable<Weather> as an input.

But I don’t know how to write that function.

What I have is a history of the weather and my calendar history of golf-playing.
I can create a Dictionary<Weather, Play> of examples (test cases).

ML is nothing more than a computer doing test-driven development.

A dictionary is a representation of a finite function.
You want a function that works for all arguments.

People have data and want to write a query to extract data from it, typically.
But how do I group the data? How do I order it? How do I filter it?
They could circle the data points on a whiteboard, but may not be able to write the query.

From example data, you’re generating a function.

If you thought the impedance mismatch between (relational) queries and programs was large, then the mismatch between ML and programming is astronomical.

A query has data and a function, and gives you an answer.
ML has data and answers, and gives you a function.

BigML - super-easy way to do ML (commercial - ka-ching!)
Supports Actional Model Download - gives you the function you’re looking for in your choice of languages.

Writing a decision tree algorithm isn’t that hard.
If you write your algorithms using queries, then you can parallelize them.

Bayes
Mathematicians are extremely sloppy.
Bayes is just monads.
When they talk about vectors, they really mean classes.
Play -> Distribution(Outlook) is the probability monad.
P(Weather|Play) is the way mathematicians represent this.

A monad is a type constructor with some functions.

Probability Monad
Type constructor:
Distribution<T> = List<Pair<T, double>>
Unit:
Return :: T -> Distribution<T>
Return t = [(t,1.0)]
Bind:
Bind :: Dist<S> x (S -> Dist<T>) –> Dist<T>
Bind ds f = [ (t, ps*pt) | (s,ps) <- ds, (t,pt) <- f(s) ]

The trick of functional programming is let the types do the work for you.
I do what the types tell me.

From our data we can compute P(Outlook|Play), P(Temperature|Play), etc.
But we want to know P(Play|Weather).

Bayes’ Theorem gives us a way to invert monadic functions.
It holds for any monad.
There’s a type error in it, though.

Given:
a :: A, f:: A -> B
Want:
g :: A -> AxB
g = a -> (a, f(a))

Using n for intersection, we can type-correct Bayes:
P(BnA|A) = P(B) * P(BnA| B)/P(A)


Your Host: webmaster@truewill.net
Copyright © 2000-2013 by William Sorensen. All rights reserved.