Development Central

Bill Sorensen's software development blog

Notes: Case Study: Algorithmic Trading in a Production Haskell Environment

Notes on a talk by John Lato of Tsuru Capital at Iowa Code Camp.

Coursera - financial modeling, etc. courses are available

Derivatives market is huge. (A quadrillion dollars!)

Create a model for the prices structure and derive a formula from the model.
Example: Black-Scholes Option Model - statistical volatility is the main variable, along with underlying price and time to maturity
Delta - underlying, Vega - volatility, Theta - time (The Greeks) (actually equations)
Wikipedia & Investipedia have articles on this.
Modeling is very math-y.

Algo Trading: Reality

  • Buggy 3rd party code
  • Out of spec implementations
  • Parsing

(Sound familiar?)
Can measure success: Are we making money?

1-2 GB/day for a small subset of the orderbook from the MarketFeed.

Knight Capital SEC article - how software deployment issues can cost your business money.

How Tsuru uses Haskell

  • Type system
  • Concurrency model
  • Foreign interface
  • Meta-programming

The Haskell Kool-Aid: If it compiles, it works! (Mostly true.)

Newtypes are "free," apply liberally. (Similar to units of measure in F#.)
  This way you don't apply the wrong function to a price, etc.
  Way to preserve invariants (can include constraints).

Coming to a strongly-typed functional language from a dynamic language is hard.
You have data, and you have functions. Model your data accurately.
If you have a Maybe Int, you can only use it with things that work with Maybe.
Rather than throwing exceptions, you create types that preserve invariants.

Parametric Polymorphism
Functions that work with, for instance, a list of any type
Can't alter your data, as they have no interface to interact with it
Similar to generics in C#/Java
Example: many :: Parser a -> Parser [a]
  (a is generic)
You can do a lot of this stuff in C# now, but it gets very verbose.
Type inference may infer a type more general than you thought, in which case you get code for free! (In functional languages.)
(In Haskell, it's considered good practice to write type signatures, at least for top-level functions.)

Immutable-by-default and lack of side effects are a big win
Some challenges remain (e.g. no mechanism for thread priority)
In F#, considered good practice to make a method pure unless good reason
Side effects are "Spooky action at a distance"
In Haskell, if something doesn't have IO (monad) in the signature, it doesn't do I/O.
Similar functionality to async/await. In functional languages, typically the syntax is very clean. - lots of code examples
ghc - compiler most people use - offers REPL, includes mem/perf profiler
Cabal - build system

In Haskell, deep debugging can be hard. Traditional stack traces are tricky.

Have both unit tests and functional tests. (HUnit)
QuickCheck - Haskell (and more?) testing tool - generates random input
Checks results - good for catching corner cases
Also use a simulator in place of exchange, playing back feeds, etc.

Notes: Learn Every Programming Language

Notes on a talk by Patrick Delancy at Iowa Code Camp.

If you don't practice it, you lose it.


The inability to learn programming languages could force you to change careers.
The idioms and concepts you learn will help you with your current language.


Start with the underlying principles. Understand the paradigms. The rest is just syntax.

Programming Paradigms (in order of complexity, low to high)

  1. Procedural
  2. Logical
  3. Functional
  4. Object Oriented

That's not the order in which we typically learn them.

Thinking Procedurally

  • Loops
  • Procedures/Functions/Subroutines
  • Global/Static Variables
  • Jump/Goto
  • Lexical Scoping
    • Text in your code defines variable scope
    • Not based on time (such as global after you define it)
    • Block scoping, function scoping, etc.

Thinking Objectively

  • Class/Entity
  • Abstraction
    • Making something simpler.
    • We don't have to understand how it works.
  • Encapsulation
    • An object, if it exists, should never be in an invalid state.
  • Inheritance
  • Polymorphism

Very domain-driven. Easy to translate the business needs.

Thinking Functionally

  • Functions as values
    • Lambdas, anonymous delegates
  • Pattern Matching
    • Regular expressions
    • Define shape of a value or object
    • Compiler can warn you of missing cases
  • Composition
    • Chaining function calls
  • Partial Application
  • Monad/Computation Expression
    • Binding functions together and defining how they behave
    • Helps to solve the problem of side effects (like I/O)
  • Closures
    • A function plus its referencing environment
    • Such as the memory address of a variable
    • Necessary evil (avoids globals)
  • Deconstruction
    • Pull out variables from object (often through pattern matching)
    • Lists are head + tail
  • Recursion
    • Tail recursion avoids running out of stack
  • Cons/Car/Cdr
    • Cons constructs a new value from two inputs
    • Linked lists
  • Etc.

What is the shape of the process?
Focus on behavior, not data.

Thinking Logically

  • Facts
  • Relationships
  • Goal-reduction

Languages don't fit into these buckets anymore.
They steal ideas from other paradigms.

Notes: Agile's Dirty Secret

Notes on a talk by Tim Gifford at Iowa Code Camp. - GitHub community

Agile Manifesto - no processes are listed.
See also the related 12 principles.

SAFe Agile process (new RUP?) - speaker has seen companies turn the PSI's from that into little waterfalls - easy to lose track of what the customer values

Incremental - you don't see the whole picture until everything is done
Iterative - see the picture, but it starts out fuzzy
(Like interleaved graphics downloading)

User Story Maps - read more on this!

Weinberg's Law of Raspberry Jam - the wider you spread it, the thinner it gets (poor understanding)

Testing Boundaries problem
3 code paths with 10 paths each - need 1,000 end-to-end tests
If we get a defect in one code path, 100 tests fail.
Spend all your time fixing broken tests.
10/10/10 unit tests + 200 integration tests + 1 end-to-end = 230 tests
Only 22 or so fail if there's a defect.
(Same combinatorial problem Mark Seemann talks about on Pluralsight.)

Large problems need leadership & courage.

Large Teams
More people => fewer questions => less learning
Interpersonal issues - why not split them up on clique boundaries?
Responsibility is spread out.

Small Feature Teams
Top-to-bottom, delivering customer value.
More opportunities for leadership roles.

He's using Feature Toggles.
He's very much in favor of it, but there's a downside.
You need to make sure you're truly releasing the software, not just delivering with features turned off.

It's important to still have a release plan.

Give pilot teams learning time - an extra point per story, etc.
You don't start out with TDD/BDD/ATDD at full velocity.

You cannot teach craftsmanship. Knowledge does not change behavior.
(There are marriage counselors who are divorced.)

"It's always a people problem" - Weinberg's 2nd Law of Consulting
(Secrets of Consulting is Weinberg's book)

Drowning in Defects
Agile practices can help with new defects - but what about legacy?
Defect age ranged from 2 hours to 2 years on one of his projects.
The variability resulted in angry customers.
What if we do defects in the order in which they were received? FIFO?
What about the old ones? Ignore them - if they're important, they'll be reported again.
Prioritization can be waste.
Capacity needs to align with demand - the business needs to step in if the bug requests are overwhelming the developers (more hires or stop reporting bugs until developers catch up).
Leadership doesn't come from authority. If you see a problem, fix it.
Why do we let defects accumulate?

In order to run fast, you need to run clean.

Don't start coding until you have a Given-When-Then scenario.

Notes: Becoming an Outlier

Notes on a talk by Cory House (@housecor) at Iowa Code Camp.

Career Reboot for the Developer Mind

What would you do if money were not a concern?
Why aren't you doing that?
If you choose money over happiness, you're working to make money so that you can live and work doing something you dislike.

Job, career, or calling?

Get up every day excited about what you're going to do.
Work "with" a company rather than "for" them.
Do it because you're having fun!

How to be an outlier;
Spend your time consistently doing things that people care about.

We're all weird. Some people will always look at you with pity.

Step 1: Be Modal
40% of your day is habitual.
Takes 10,000 hours for expertise (of deliberate practice)
You under-use your free time.

Multithread your life.
Hack the commute. (Podcasts, audiobooks, etc.)
Work from home.
Take public transit.
Read or watch tech videos while doing cardio.
Listen to podcasts while doing yard work.

Rethink giving back.
Contribute to open source, pro bono development, Give Camp, etc.
(Rather than building a house, donate your most valuable skill.)

The Maker's Schedule
Managers live with regular meetings, interruptions, etc.
Developers need uninterrupted thought to get into Flow.

Alter your schedule - go in early, etc.
Don't even open your email for an hour.
Larger, less frequent meetings

Improve your Signal to Noise
The cheaper you can live, the greater your options.

Automate relentlessly.
Direct deposits, auto bill pays, reminders, etc.
RescueTime - software tool

Target media consumption.
Recognize what you're giving up by watching reality TV.
InfoQ, TekPub, Pluralsight, TED talks, DNRTV
Manicure your stream. Slow media is for the focused few.
The less news you consume, the bigger the advantage you have.
Why waste your time on something you cannot control?
Focus on what you can influence.

The more you make per hour, the more time you can buy.
Use delegates. Hire someone to do the yardwork, etc.

Step 2: Manage Your Image
Pick Two - Neil Gaiman graduation speech on YouTube

Purpose built - how do you want people to look at you?
Self-image is your greatest constraint.
Can I see your body of work?
Do it in public - StackOverflow, GitHub, Twitter, blog, etc.
Your words are wasted if you're not blogging.
Measure: GoogleAlerts, TweetBeep, etc.
Shelved credentials - books attract alpha geeks and convey passion.

Step 3: Own Your Trajectory
(Of your career.)
What if every piece of your workday moved you toward independence?
(Example: Pluralsight)

Search for Scale

  1. Work
  2. Lead (talk about work)
  3. Own (products, frameworks, author)

Time = potential knowledge
Beware of becoming an assembly line coder.

Opportunity wedge
Life is characterized by closing horizons and lost opportunities.
You can extend this by enhancing your skills.

Don't work for money.  Work to learn.

Key to success: Learn to teach yourself more efficiently than any institution can.

Luck Surface Area - do what you're passionate about and communicate that to as many people as possible
Speaking, blogging, conferences, user groups, Tweet, never eat alone
Make the Hang - spend time with people you look up to
You are the average of the five people you spend the most time with. - Jim Rohn

Lean into Uncertainty
Find what scares you and run at it.
Speak at a conference.

You Don't Need Permission.

If it's work, we try to do less.
If it's art, we try to do more.

Notes: Pragmatic Architecture in .NET

Notes on a talk by Cory House (@housecor) at Iowa Code Camp.

Clean Code video on Pluralsight.

Religion is the problem in architecture. Always/never.
Look at the context.
It depends.
People who are experts answer questions with more questions.

Over-architecting wastes money.


What if one month late is worthless?

Parkinson's Law: Work expands to fill available time.

Consider Complexity (speaker considers the following complex)

  • Automated testing
  • Coding to an interface
  • SOA
  • Rich Domain Model (DDD)
  • ORM vs. custom DAL
  • Repository Pattern
  • Layered architecture

Consider Simplicity

  • Do the simplest thing that could possibly work
  • Lean/Agile principles (minimize WIP, etc.)
  • 80/20 Rule

Book: Lean Startup - minimal viable product (MVP)
Scalability, maintenance costs, performance, etc. may not matter

We're paid for solutions, not code.

Flexing features (scope) is a lot better way to go than flexing quality.

Is it worth taking on technical debt today?
Hard vs. Soft deadlines

  • Trade show?
  • First to market?

Single loud customer, salesman misspoke, wild-ass guess, MS Project said so - terrible reasons for hard deadline

Layers - logical (separation of concerns)
Tiers - physical (often decided for you)

For a small enough application, methods are layers.

Everything's a trade-off.

Architectural Levels

  1. Simplest thing
  2. Somewhere in between
  3. Every tool in the shed

Active Record pattern mixes Domain and Data. Breaks SRP.
Easy to understand, though. Consider for CRUD apps or simple domain.
Rigid: Domain model = DB
Leads to God object
Hard to test
Converting to Repository Pattern = pain

Eric Evans - DDD book
Great for complicated, long-lived applications
Takes time

Level 2 (somewhere in between)
Focus on the pain

No Free Lunch

At what point does our application complexity grow to the point where the effort to enhance gets too painful?

MVP, junior team, simple domain, tight timeline, throwaway - consider Level 1

Flagship product, senior team, complex domain, long-term, security matters, flexible timeline - consider Level 3

POEAA (Fowler) and Dino Esposito's architecture book inspired this talk.
Microsoft .NET: Architecting Applications for the Enterprise is the latter.

Speaker recommends clean, readable code regardless.