
Within any big tech corporation are many disparate teams. Most of these teams
know very little about the work, makeup or very existence of other teams, yet
they depend on each other's output, directly or indirectly.

Commonly, when one team needs to use another's service or library, meetings are
setup, use cases are discussed, and all kinds of bureaucratic nonsense takes
place. Efficiency Failure.

Open source is need-driven. And while project authors and maintainers like to
know who their users are, it isn't a requirement. Code is built in a way to
solve a general problem. There is no need to "discuss your use case". A README
provides all the information a dev needs to:

- Get the code
- Run it
- Generally understand what it does and how it works

Beyond the README, well-built open source projects will provide docs, mailing
lists, and an IRC channel. Everything you need to quickly ramp up or get help.

Contrast this with typical corporate projects:

- missing or inadequatte README
- no docs or docs that are several versions (or years) out-of-date
- docs probably live on some separate wiki on the internal network and may or
  may not be referenced from project README
- requests for help are met with meeting requests

It's no wonder corporate tech moves so slowly.

A derivatory problem that corps suffer from is re-inventing the wheel over and
over again instead of using existing solutions. That's because it's often easier
to build something from scratch yourself than to try to deal with the awful
inter-team politics and meetings-driven-communication. Over time this leads to a
plethora of abandonware as teams move on and no longer need their
hacked-together solution. Unnecessary LoC growth.

## Solution

Start developing your internal libraries and services as if they were open
source, even when you have 0 users. Write an excellent README and keep it
up-to-date. Write in-repo docs to make it easier to keep them up-to-date, and be
disciplined about maintaining them. Write tests as necessary. Give examples.
Make it so good that when people start using your code, you don't even know it
because they didn't have any questions. But when they do have questions, **don't
schedule a meeting**.


Corporate process should mirror open source

It wasn't until I was well into my twenties that I learned how to really learn.
And it was only by learning many hard concepts related to programming on my own
that I observed and proved what worked. It unlocked a sort of power. Realizing
how to learn nearly any hard topic is an incredibly useful skill. I found that
efficient learning techniques had nothing to do with typical class structures
found in high school or university settings, which had previously drained my
desire to learn. In that regard, the subject of learning, or learning how to
learn is something I find very interesting and think about a fair amount, so
when I found the [Learning How to
Learn](https://class.coursera.org/learning-002) class on Coursera by Dr. Barbara
Oakley, Dr. Terrence Sejnowski I immediately signed up. Here are my notes and
observations gained from the class.

## Modes

There are two primary modes of thought:

0. **Focused**: this mode is engaged when you're doing something you already
 know how to do, quickly accessing the learned connections in your brain. The
 connections are very tightly spaced in the prefrontal cortex.

0. **Diffuse**: this mode is for open minded thought that allows you to make new
 connections in your brain. You can think of this mode as exercising your
 brain muscle and letting your mind wonder. Edison and Dalí are famous
 examples of thinkers who practiced the diffuse mode: they would hold
 something in their hands while day-dreaming until dozing off to sleep at
 which point they'd drop the object and wake themselves back up in order to
 take what they learned and apply it in a focused mode. Diffuse mode relies on
 loosely spaced connections, and is engaged by drifting off to sleep and
 excercise.

## Brain facts

- The brain is the most complex device in the known universe
- Acquiring skills takes years and years of practice (e.g. math, chess)
- We are not consciously aware of how our brains work
- There are 1015 (a million billion) synapses in the brain
- Brain connectivity is dynamic even after it matures
- Our brains develop new synapses while we sleep

## Techniques

- **Pomodoro**: set a 25 minute timer to focus on a specific task or goal and reward
 yourself when you're done (e.g. read the Internet, get a coffee). Use this to
 build the neural patterns you need to learn new concepts. It helps to actually
 write your goal down. There are a few terminal and OS X apps that can help.
 Writing down the goal helps combat initial skatterbrain tendencies, which
 sometimes require continual re-focusing at first.

- **Translation and rewriting**: when I read academic papers, I rarely
 understand the cryptic math syntax. A good way to break through is to
 translate the syntax to something you do understand. For me, this means
 Haskell. Then rewrite the plain text sections in your own words (tip: try
 writing a blog post that sumarizes the paper). You can't help but internalize
 the concepts when you do this.

- **Handwritten notes**: when learning a library of any complexity, simply reading
 the source doesn't always help. You need to run it and observe the values in
 context and the execution path. A debugger can help here. While you're doing
 this, take handwriten notes on important functions and variable values. Sketch
 out an execution tree. You may never refer to these notes again, but the mere
 act of physically writing them makes abstract concepts more concrete.


## Abstract Concepts

Abstract concepts in particular require much more practice to create concrete
neural thought patterns. The initial neural pattern is there but it is very
weak. When you solve a problem again you strengthen that pattern. When you've
done it so many times you have it down, the thought pattern is very strong.

Study and focus intently, then take a break to allow the brain's diffuse mode
to help you out with your conceptual understanding. "Neural mortar has a chance
to dry".

## Memory

**Long term** memory vs **working memory**.

Working memory (prefrontal cortex) is what you're currently focusing and working
on. It can hold about 4 chunks of information. We naturally group things into
chunks (e.g. phone numbers). Working memory is like a blackboard.

Long term memory is like a storage warehouse, spread over different areas of the
brain. To store something in long term memory you need to revisit it several
times. We can store billions of items, but they can bury each other. Spaced
repetition is one technique of storing something in long term memory: repeat
something over a number of days.

## Sleeping

Being awake causes toxic products in your brain. When you sleep your brain cells
shrink, causing an increased space between them, allowing fluid to wash the
toxins out.

It's also an important part of the memory and learning process: it organizes the
things you're learning. It rehearses neural patterns of things you're learning,
strengthening and deepening them.

If you focus on what you're learning right before you take a nap you have an
increased chance of dreaming about it, especially if you want to dream about it.
Dreaming about what you're studying can enhance your ability to understand: it
consolidates your memories into easier-to-grasp chunks.

## Activity

Physical excercise engages the diffuse mode of thinking. This is why it's so
important to excercise. I try to make some form of excercise a part of my daily
routine as a lifestyle rather than a discipline. For example, I used to commute
by bike 5 miles each way to work when I lived in Seattle. This was an excellent
way to start the morning, and end the work day. Now that I work from home, I
mountain bike over lunch during the summer, and walk, run or go to the climbing
gym during the winter. Many times as I've thought about a problem while in the
zen state of singletrack mountain biking, the diffuse mode of thinking has
brought a very different perspective and creativity to problem solving.

## Environment

It's important to work in a space that inspires you to create. If you work from
home, carve out a dedicated space to work, and spend time desigining or
remodeling if necessary to make it suit your needs. For me, this means a modern
space with interesting art and a mix of home made physical computing hackery.

It also helps if you're working near other people are creating and hacking. If
you work from home, try to make time to work from coffee shops, find a coworking
space to occassionally work from, or even meet up with a friend who freelances
to work together.

## Chunking

A chunk is a network of neurons working together, built from practice and
repetition. New concepts often don't make sense in relation to other concepts as
part of the bigger picture. It's important to connect new material to existing
knowledge. Chunks are pieces of information that are bound together. One of
the first steps to learning is creating chunks which are connected to knowledge
through meaning. Once you chunk an idea you don't need to remember all the
underlying details.

### How to form a chunk

Consider learning a new song on the guitar. The entire song could be considered
one large chunk, but can also be broken down into mini-chunks that are easier to
learn through repetition. You can then join the mini-chunks together into larger
chunks. The joining process can be continually repeated to build up larger and
larger chunks. The point of joining small chunks into a larger one is so that
you don't even need to consciously think about connecting the individual neural
networks.

Learning a language is another example. At first you learn pronounciation and
how to string together words to form simple sentences. As you progress, you can
more creatively string together complex phrases to communicate subtleties.

When learning a mental concept, form a chunk by:

0. Focus your undivided attention on the subject
0. Understand the basic idea you're trying to chunk: let focus and diffuse mode
 alternate in order to synthesize the gist. There are two approaches that work
 together. The bottom up approach involves chunking and repetition while top
 down involves understanding the big picture. Both perspectives are vital and
 meet at the context level, which helps you learn how and when a chunk applies
 to the big picture.
0. Practice and repetition

It helps to learn major concepts first, then go back in and fill in the details.


## Further reading

- [Learning How to Learn](https://www.coursera.org/learn/learning-how-to-learn)
 — the class this post references, now available "on dmeand"
- [Brains Sweep Themselves Clean Of Toxins During Sleep](http://www.npr.org/sections/health-shots/2013/10/18/236211811/brains-sweep-themselves-clean-of-toxins-during-sleep)
 — NPR

On learning

The freedom to do anything you wish is rarely a good thing, whether you're an
experienced engineer or a 2-year-old child. Our interests and desires are too
diverse and of varying merit to actually produce anything good without
constraint. That's not to say freedom is intrinsically bad: sometimes we are
able to choose the right constraints ourselves; other times they are chosen for
us. Either way, constraint is what guides and sometimes dictates, removes
decisions and forces creative thinking. Constraint is a framework.

These are some constraints I've embraced over my years as a developer.

## Terminal

The terminal is a beautiful constraint. Because of its spartan nature, tools
built in terminal turn out more consistent in behavior and UX, faster, and less
bug-ridden than their graphical counterparts. You know right away when an app
was built on Java/Swing vs native APIs. Look at all differences between UI junk
in Chrome vs Safari on OS X. Then compare that to the simulated UI junk in Flash
or Silverlight. Then look at Eclipse. It's grotesque and highly inconsistent.

Now compare operating systems. The way in which humans interact with a computer
should be more of a UX science and less of an artifact of personalization, and
especially not the whim of corporations whose interests (profit) don't
necessarily align with advancing human-computer interaction.

You may say terminal is rudimentary. Well I agree, but that's part of what makes
it so great. In my early days, it annoyed me greatly when I had to resort to
using the terminal, but I eventually embraced it, moving all my tools and
workflow into it because it's consistent, fast, reliable, and most importantly
_automation is intrinsic_.

## Keyboard

As many have observed, using a mouse is almost always less efficient than using
a keyboard shortcut or command (there are exceptions, but not many apply to the
type of work done by developers). I've increasing eschewed mouse use and it
so happens that Terminal and vim are a very natural way to do so. The reliance
on keyboard shortcuts is a productivity framework.

## $EDITOR

Consistent with my use of terminal and keyboard shortcuts, I use vim, though
Emacs is also an excellent choice for the precise and thoughtful
automation-and-productivity-minded developer. Many of the points regarding
Terminal's simplicity and consistency also apply to these great editors,
originally built by yesterday's technologists and finely tuned over the decades.
`$EDITOR` only gets better with age. Vim's minimalism and
simplicity give way to an uncluttered mind, allowing you to focus on the problem
at hand rather than the tools—a framework for clear thinking.

<img src="/images/vim_emacs.png" alt="Vim and Emacs vs Atom" />

## Functional Programming

In a similar way that tools constrain us, the functional style of programming
introduces tremendous restrictions on how we write programs. FP is a huge topic
that I won't even attempt to cover here, but we can briefly take a look at some
of its tenets and the benefits that come with.

### Composition

One way to constrain your code is to implement functions purely in terms of core
or library functions, as in this JavaScript example.

```javascript
const mergeValues = _.compose(_.partialRight(_.reduce, _.merge), _.values);
mergeValues({ a: { foo: 1, bar: 2 }, b: { qux: 3 } });
//=> {foo: 1, bar: 2, qux: 3}
```

I'm specifically avoiding the use of `function` in the definition of
`mergeValues`. That is a strong indicator to the reader that _nothing_ fancy is
going on here. It's simply a composition of library functions (lodash.js) which
prevents me from writing any custom code. Instead, I'm forced to think of
existing functions, and compose them in a way that produces intended behavior.

Composition is the foundation of reusability — a worthy pursuit.

### Immutability

The benefits of immutability are quite well-known by now, even among OO
programmers. It prevents all sorts of nonsense like defensive copying and
combinatorial explosion of possible states and aligns well with reality and
intuition. After all, a value is a value. When you say `2 + 3` you are not
mutating `2`; you're producing a new, immutable, unchangeable value, `5`. Why
should `2 :: List(3)` behave any differently?

Immutability enforces a restriction on how we write algorithms. No longer can we
loop over some collection and store computed results on another collection via
mutation. Instead we look to elegant functional constructs like `map` and
`fold`.

See these excellent presentations by Rich Hickey if you'd like to go deeper on
the benefits of immutability. It's a wonderful constraint.

- [The Value of Values](http://www.infoq.com/presentations/Value-Values)
- [Are We There Yet?](http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey)

### Referential transparency

Referential transparency (RT) is an attribute of pure functions, which do not
perform side effects (e.g. IO, mutation), mutate their arguments, nor depend on
outside variables. RT says you can substitute the value for any expression.

```scala
val double = {x: Int => x * 2}
double(4)
// => 8
// double(4) simply represents the expression 4 * 2 so let's substitute it:
4 * 2
// => 8
// 4 * 2 represents 8 so let's substitute it:
8
//=> 8
```

It's obvious how the substitution works in this context. Now imagine the use of
`double` in a slightly wider context:

```scala
double(50) / double(5)
//=> 100 / 10
//=> 10
```

Running this code in your head is very easy because you can safely substitute
`100` and `10`. This concept of substitution scales up as your context gets
wider and wider, as it naturally does in a program of any meaningful size. This
is called equational reasoning. RT gives you the ability to reason locally in a
small context instead of having to worry about who is mutating what or where
unpredictable values are coming from. It also makes functions more testable,
composable, and parallelizable.

## Constraint as freedom

Being intentional about which constraints you adopt and embrace is freeing. No
longer are you lost at sea, being tossed about. Your constraints are fixed, and
you have the freedom to become highly skilled at working with and around them,
using them as a rudder to guide you during the creative problem-solving process.

Maker, choose your constraints wisely.

## Further exploration

- [Constraints Liberate, Liberties Constrain](https://www.youtube.com/watch?v=GqmsQeSzMdw):
 an excellent talk by the always-brilliant [Rúnar Bjarnason](http://blog.higher-order.com/),
 co-author of [Functional Programming in Scala](https://www.manning.com/books/functional-programming-in-scala)

Beautiful constraint

The three Monad laws may seem pretty abstract at first, but they're quite
practical. Let's try to internalize the laws by running through two of Scala's
most popular monads and making sure they adhere. We could use something like
[ScalaCheck](http://scalacheck.org/) to more rigorously check these laws, but
the purpose of this post is to help internalize them, so we'll only be manually
verifying them using our intuition.

Here are the laws, from [Monad laws](https://wiki.haskell.org/Monad_laws) on HaskellWiki.

1. Left identity: `return a >>= f ≡ f a`
1. Right identity: `m >>= return ≡ m`
1. Associativity: `(m >>= f) >>= g ≡ m >>= (\x -> f x >>= g)`

In Haskell, `return` is used to "inject a value into the
monadic type". In Scala, we do this via constructors (unless you're using
Scalaz, in which case you probably already know everything this post has to
offer).

The `>>=` operator in Haskell corresponds to Scala's
`flatMap` method.

## List

The List Monad deals with the context of non-determinism—that is, it represents
multiple values. When we run multiple lists through a sequence comprehension we
end up with the all combinations of values from each list.

```scala
for (x <- List(1, 2, 3); y <- List('a', 'b', 'c')) yield (x, y)
=> List((1,a), (1,b), (1,c), (2,a), (2,b), (2,c), (3,a), (3,b), (3,c))
```

Now the laws. Let's setup two simple functions `f` and `g`, both of type `Int =>
List[Int]`.

```scala
// Let f be a function that takes an Int and produces a List of its
// neighboring Ints along with itself:
val f: (Int => List[Int]) = x => List(x - 1, x, x + 1)

// Let g be a function that takes an Int x
// and produces a List containing +x and -x
val g: (Int => List[Int]) = x => List(x, -x)
```

### Left identity

```scala
val a = 2
val lhs = List(a).flatMap(f)
=> List(1, 2, 3)

val rhs = f(a)
=> List(1, 2, 3)

lhs == rhs
=> true
```

### Right identity

```scala
val m = List(2)

val lhs = m.flatMap(List(_))
=> List(2)

val rhs = m
=> List(2)

lhs == rhs
=> true
```

### Associativity

```scala
val m = List(1, 2)

val lhs = m.flatMap(f).flatMap(g)
=> List(0, 0, 1, -1, 2, -2, 1, -1, 2, -2, 3, -3)
// Sidenote: now do you see what is meant by non-determinism?

val rhs = m.flatMap(x => f(x).flatMap(g))
=> List(0, 0, 1, -1, 2, -2, 1, -1, 2, -2, 3, -3)

lhs == rhs
=> true
```

Looks good to me.

## Option

Let's create new test functions `f` and `g` of type `Int =>
Option[Int]`. Given the type signature, it's natural to think
of `f` and `g` as _partial functions_ that are only defined on certain inputs.

```scala
// If x is not less than 10, return 2x
val f: (Int => Option[Int]) = x => if (x < 10) None else Some(x * 2)

// If x is reater than 50, return x + 1
val g: (Int => Option[Int]) = x => if (x > 50) Some(x + 1) else None
```

For the sake of testing our laws, the implementations of these functions really
don't matter as long as the types line up.

### Left identity

```scala
val a = 30
val lhs = Option(a).flatMap(f)
=> Some(60)

val rhs = f(a)
=> Some(60)

lhs == rhs
=> true
```

### Right identity

```scala
val m = Option(30)

val lhs = m.flatMap(Option(_))
=> Some(30)

val rhs = m
=> Some(30)

lhs == rhs
=> true
```

### Associativity

```scala
val m = Option(30)

val lhs = m.flatMap(f).flatMap(g)
=> Some(61)

val rhs = m.flatMap(x => f(x).flatMap(g))
=> Some(61)

lhs == rhs
=> true
```

## The end

I hope this post helped you internalize the Monad laws. If you need more
practice, continue this exercise in your REPL for the `Try`
and `Either` monads, or better yet: create your own Monad and
verify that it obeys the laws!

### Further reading

- [Monad laws — learning Scalaz](http://eed3si9n.com/learning-scalaz/Monad+laws.html)
- [Monad laws — HaskellWiki](https://wiki.haskell.org/Monad_laws)
- [A Fistful of Monads — Learn You a Haskell](http://learnyouahaskell.com/a-fistful-of-monads#monad-laws)

Monad laws in Scala

Clojure has a useful macro called
[cond->](https://clojuredocs.org/clojure.core/cond-%3E) that conditionally
threads an initial value through a series of predicate/function pairs only
applying each function if its predicate returns true. In this post we're going
to look at a Scala representation, and whether it fits the shape and laws of any
[common algebraic structures](https://en.wikipedia.org/wiki/Outline_of_algebraic_structures#Types_of_algebraic_structures).
We'll look at Functor, Monad, Semigroup, and Monoid.

**TL;DR** — [view the full code listing](https://gist.github.com/devth/308da859e5b584340f93).

Let's start with an example in Clojure. We want to build up a request based on
some arbitrary conditions:

```clojure
;; sample values from user input

(def user-id 1)
(def user-name "devth")
(def user-address nil)
(def accept :json)

;; validation and helper functions

(def accept-map )
(defn valid-accept? [a] (accept-map a))
(defn set-header [req k v] (update-in req [:headers] assoc k v))
(defn set-param [req k v] (update-in req [:params] assoc k v))

;; build up a request map using cond-> to decide which items to add to params
;; and headers maps

(def request
 (cond->
 user-name (set-param :user-name user-name)
 user-address (set-param :user-address user-address)
 (valid-accept? accept) (set-header :accept accept)))

;; request value:

{:target "/users"
 :query-params
 :headers
```

Since Clojure's `->` operator is sometimes referred to as
the "thrush" operator, I'm going to call `cond->`
in Scala `ThrushCond`.

First let's model the `Request` and helpers equivalent to
those we used in the Clojure example:

```scala
case class Request(
 target: String,
 params: Map[String, String] = Map.empty,
 headers: Map[String, String] = Map.empty) {

 // validation and helper functions
 val acceptMap = Map("html" -> "text/html", "json" -> "application/json")
 val isValidAccept: (String => Boolean) = acceptMap.isDefinedAt _

 def addParam(k: String, v: String) = this.copy(params=params.updated(k, v))
 def addHeader(k: String, v: String) = this.copy(headers=headers.updated(k, v))
}

// sample values from user input
val userId: Int = 1
val userName: Option[String] = Some("devth")
val address: Option[String] = None
val accept = "json"
```

Now we'll create the `ThrushCond` class that takes any number
of predicate/function pairs, provides a `guard` function to only run a function
if the predicate passes, a method to flatten the chain of functions via
composition, and finally a `run` method that takes a value and
runs it through the chain.

```scala
type Step[A] = (A => Boolean, A => A)

case class ThrushCond[A](steps: Step[A]*) {
 /** Perform a pipeline step only if the value meets a predicate */
 def guard[A](pred: (A => Boolean), fn: (A => A)): (A => A) =
 (a: A) => if (pred(a)) fn(a) else a
 /** Compose the steps into a single function */
 def comp = Function.chain(steps.map { step => guard(step._1, step._2) })
 /** Run a value through the pipeline */
 def run(a: A) = comp(a)
}
```

Try it out:

```scala
val requestPipeline = ThrushCond[Request](
 ({_ => userName.isDefined}, {_.addParam("userName", userName.get)}),
 ({_ => address.isDefined}, {_.addParam("address", address.get)}),
 ({_.isValidAccept(accept)}, {r => r.addHeader("accept", r.acceptMap(accept))}))

val request = requestPipeline run Request("/users")

//=>
Request(/users,Map(userName -> devth),Map(accept -> application/json))
```

As you can see, it correctly skipped the 2nd step based on the
`address.isDefined` condition and runs the other steps because
their predicates evaluate to `true`.

Will this work as one of the algebraic structures mentioned at the start?

## Functor

Consider Functor's `fmap`:

```scala
def fmap[A, B](f: A => B): F[A] => F[B]
```

In our case, both `A` are the
same type, `Request` produces a
function that fits, but we could easily use that with an existing Functor,
e.g.:

```scala
val step: Request => Request =
 guard({_ => userName.isDefined}, {setParam(_, "userName", userName.get)})
Some(Request("/users")).map(step)
```

The essense of `ThrushCond`
itself so it makes no sense to design a new Functor around it.

## Monad

Likewise, Monad's `flatMap`:

```scala
def flatMap[A, B](f: A => F[B]): F[A] => F[B]
```

We could make `guard`'s
signature, but there's no point in doing so for the same reason it didn't make
sense for Functor: the essense is not how a transformation is applied, it's
_whether_ the transformation is applied, and because of the signature, the
decision whether to perform a transformation must be embedded in the
transformation itself, hence `guard`.

## Semigroup

Let's see if it meets Semigroup's associativity laws:

```scala
case class F(x: Int)
val f = F(10)
val always = Function.const(true) _

val mult2: F => F = guard(always, {f => f.copy(x = f.x * 2)})
val sub4: F => F = guard(always, {f => f.copy(x = f.x - 4)})
val sub6: F => F = guard(always, {f => f.copy(x = f.x - 6)})

val g: (F => F) = (mult2 andThen sub6) andThen sub4
val h: (F => F) = mult2 andThen (sub6 andThen sub4)

g(f)
//=> F(10)
h(f)
//=> F(10)
```

`guard` is associative when composed with itself because
[function composition is associative](https://en.wikipedia.org/wiki/Function_composition#Properties).
Because of this associative binary operation we can provide evidence that
`ThrushCond`'s
`Semigroup` representation:

```scala
import scalaz._, Scalaz._

case object ThrushCond {
 /** Evidence of a Semigroup */
 implicit def thrushCondSemigroup[A]: Semigroup[ThrushCond[A]] =
 new Semigroup[ThrushCond[A]] {
 def append(t1: ThrushCond[A], t2: => ThrushCond[A]): ThrushCond[A] =
 ThrushCond[A]((Function.const(true), t2.comp compose t1.comp))
 }
}
```

We've defined a Semigroup over the set of all
`ThrushCond[A]`s. What does this give us? We can now combine
any number of `ThrushCond`s using Semigroup's
`|+|` operator. A simple example using
`ThrushCond[Int]`:

```scala
import ThrushCond.thrushCondSemigroup

val addPipeline = ThrushCond[Int](
 ((_ > 10), (_ + 2)),
 ((_ < 20), (_ + 20)))

val multPipeline = ThrushCond[Int](
 ((_ == 70), (_ * 10)),
 ((_ > 0), (_ * 7)))

val pipeline = addPipeline |+| multPipeline

// Examples
multPipeline run 70 //=> 70 * 10 * 7 == 4900
pipeline run 2 //=> (2 + 20) * 7 == 154
pipeline run 12 //=> (12 + 2 + 20) * 7 == 238
```

## Monoid via PlusEmpty

Monoids are Semigroups with an identity element.
`ThrushCond`'s identity is simply a
`ThrushCond` arguments.
However, as [@lmm mentioned in the
comments](http://devth.com/2015/thrush-cond-is-not-a-monad/#comment-2082941866):

> it's not ThrushCond itself that forms a Monoid but rather ThrushCond[A] for
> any given A

This is where `PlusEmpty` comes in.
`PlusEmpty` is a ["universally quantified
Monoid"](https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/main/scala/scalaz/PlusEmpty.scala#L3-7)
which means it's like a Monoid but for first-order `* -> *`
types instead of proper `*` types.
`PlusEmpty` itself is a higher-order `(* -> *) -> *` type. A
helpful quote from #scalaz:

> tpolecat: so `String` is a monoid, but
> `List` (which means that
> `List[A]`)

To provide evidence of a `PlusEmpty`, we must be able to implement these two
methods (where `F`):

```scala
def plus[A](a: F[A], b: => F[A]): F[A] // from Plus
def empty[A]: F[A] // from PlusEmpty which extends Plus
```

We already implemented `plus` for `Semigroup`'s `append`, and
`empty` is simply a `ThrushCond` without args.

```scala
case object ThrushCond {
 /** Evidence of a PlusEmpty */
 implicit def thrushCondPlusEmpty: PlusEmpty[ThrushCond] =
 new PlusEmpty[ThrushCond] {
 def plus[A](a: ThrushCond[A], b: => ThrushCond[A]): ThrushCond[A] =
 ThrushCond[A((Function.const(true), b.comp compose a.comp)))

 def empty[A]: ThrushCond[A] = ThrushCond[A]()
 }
 /** Use PlusEmpty to provide evidence of a Monoid[Request] */
 implicit def requestMonoid: Monoid[ThrushCond[Request]] =
 thrushCondPlusEmpty.monoid[Request]
 /** Evidence of a Semigroup */
 implicit def thrushCondSemigroup[A]: Semigroup[ThrushCond[A]] =
 new Semigroup[ThrushCond[A]] {
 def append(t1: ThrushCond[A], t2: => ThrushCond[A]): ThrushCond[A] =
 ThrushCond[A]((Function.const(true), t2.comp compose t1.comp))
 }
}
```

Let's go back to our `Request` example in Clojure and use
PlusEmpty's `<+>` to combine separate transformation
pipelines:

```scala
import ThrushCond._ // evidence

val userPipeline = ThrushCond[Request](
 ({_ => userName.isDefined}, {_.addParam("userName", userName.get)}),
 ({_ => address.isDefined}, {_.addParam("address", address.get)}))

val headerPipeline = ThrushCond[Request](
 ({_.isValidAccept(accept)}, {req =>
 req.addHeader("accept", req.acceptMap(accept))}))

// <+> is an alias for plus
val requestPipeline = userPipeline <+> headerPipeline
// A PlusEmpty[ThrushCond] is implicitly obtained and used to plus the two
// ThrushCond[Request]s

requestPipeline run Request("/users")
//=>
Request(/users,Map(userName -> devth),Map(accept -> application/json))
```

Because `PlusEmpty` can derive a Monoid for a given type, we can combine any
number of `ThrushCond`s from a List. Let's construct one more
`ThrushCond` pipeline that conditionally adds a cache-control
header and try out our Monoid using `Foldable`'s
`suml`:

```scala
import scala.language.postfixOps

val shouldCache = false

val cachePipeline = ThrushCond[Request](
 ({_ => !shouldCache}, {_.addHeader("cache-control", "no-cache")}))

val requestPipeline = List(userPipeline, headerPipeline, cachePipeline) suml
requestPipeline run Request("/users")
//=>
Request(/users,
 Map(userName -> devth),
 Map(accept -> application/json, cache-control -> no-cache))
```

ThrushCond is not a Monad, nor a Functor, **but it is a PlusEmpty from which can
be derived a Monoid**.

[View the full code listing](https://gist.github.com/devth/308da859e5b584340f93).

_Updated July 1, 2015: incorporated lmm's PlusEmpty suggestion._

ThrushCond is not a Monad


Type classes provide a way of achieving [ad hoc
polymorphism](https://en.wikipedia.org/wiki/Ad_hoc_polymorphism). We'll look at
what they are, why they're useful, how they're typically encoded in Scala.
We'll make our own `Simple` type class that has a single
operation called `simplify` returning a
`String`, and provide instances of it for Scala's
`List` and `Int` classes, along with our own
class `C`. Finally, we'll add convenient syntax in the Scalaz
style to make it extremely easy to use the `simplify`
operation. This might be the most hands-on, easy to understand explanation of
type classes you've ever encountered!

## Rationale

A common way to express polymorphism in Scala is inheritance. This is brittle
because the polymorphic relationship between supertype and subtype must be
defined where the subtype itself is defined. For example, if you have an
`Animal` trait, your `Dog` class must
declare its relationship as a subtype of `Animal` as part of
its definition. This is problematic when you do not own potential subtypes (i.e.
they're provided by a library). It also tightly couples the subtype to all of
its implementations of its potential supertypes. A given subtype may already
have all the methods needed to implement the behavior of a given supertype but
doesn't know or care about the supertype at definition time. Though less common,
another problem is a subtype may be able to implement a supertype's operations
in multiple, distinct ways (e.g. multiplication and addition implementations of
a Monoid over `Int`). With inheritance this is impossible.

## Scalaz

Let's start with one of the simplest type classes in Scalaz:
`Equal`. `Equal` describes "a type safe
alternative to universal equality" (if you're wondering why this is useful or
necessary, ask me in the comments). Here's an example of providing an instance
of `Equal` over our own class `C` that uses
value equality to compare two instances.

```scala
class C(val name: String)

implicit val equalC: Equal[C] = new Equal[C] {
  override def equal(c1: C, c2: C) = c1.name == c2.name
}
```

Now let's say we want a `notEqual` method that works on any
`A` provided there is evidence of
`Equal[A]`:

```scala
def notEqual[A: Equal](a1: A, a2: A): Boolean =
  !implicitly[Equal[A]].equal(a1, a2)

val c1 = new C("foo")
val c2 = new C("bar")

notEqual(c1, c2)
//=> true
```

Sidenote for those unfamiliar with _context bounds_: the context bound `[A:
Equal]` is what ensures we have an implicit
`Equal[C]` instance available when running `notEqual(c1,
c2)`. An equivalent and perhaps more clear implementation
without context bounds would look like this:

```scala
def notEqual[A](a1: A, a2: A)(implicit e: Equal[A]) = !e.equal(a1, a2)
notEqual(c1, c2)
//=> true
```

However, there's an even more concise way of writing this using context bounds
and Scalaz' `/==` operator for instances of
`Equal`, or even its unicode `≠` alias:

```scala
def notEqual[A: Equal](a1: A, a2: A): Boolean = a1 /== a2
// or
def notEqual[A: Equal](a1: A, a2: A): Boolean = a1 ≠ a2
```

Since Scalaz already provides these operators, `notEqual` is purely
didactic.

Another common typelcass in Scalaz is `Show`. This corresponds
to Haskell's `Show` type class, and is used to indicate a
type that can be represented in some way as a string, e.g. for logging or
printing in a REPL. Here's an instance for our `C` class
example from before.

```scala
implicit val showC: Show[C] = new Show[C] {
  override def show(c: C) = s"C[name=${c.name}]"
}
```

Scalaz provides syntax helpers that allow us to simply call
`.show` on any type that provides evidence of
`Show`. We'll look at how that works in detail toward the end.

```scala
new C("qux").show
//=> res22: scalaz.Cord = C[name=qux]
```

## Simple

Now that we've gotten a taste for using a few of Scalaz' type classes, let's
build our own, along with some syntax helpers in the Scalaz style.

```scala
trait Simple[F] {
  def simplify(f: F): String
}
```

This is our type class definition. It defines a single operation
`simplify` for a given `F` and returns a
`String`. Before we provide instances, let's define a method
that expects a `Seq` of `Simple` instances
and outputs them separated by newlines.

```scala
def manySimple[A](simples: Seq[A])(implicit s: Simple[A]): String =
  "Many simples:\n\t" + simples.map(s.simplify).mkString("\n\t")
```

We can use this to easily try out instances. Let's start with an instance of
`C`:

```scala
implicit def simpleC: Simple[C] = new Simple[C] {
  override def simplify(c: C) = s"Simplified: ${c.show}"
}
```

We can manually call this by implicitly obtaining a
`Simple[C]` then calling `simplify`:

```scala
implicitly[Simple[C]].simplify(new C("hello"))
```

Or we can try out the `manySimple` method:

```scala
manySimple(Stream(new C("foo"), new C("bar"), new C("qux")))
//=> Many simples:
//=>        Simplified: C[name=foo]
//=>        Simplified: C[name=bar]
//=>        Simplified: C[name=qux]
```

Let's try another one: `Int`.

```scala
implicit val simpleInt: Simple[Int] = new Simple[Int] {
  override def simplify(i: Int) = s"Simplified Int with value of $i"
}
```

This is getting easy, right?

```scala
implicitly[Simple[Int]].simplify(123)
//=> Simplified Int with value of 123
```

We can even provide an instance for `List[A]` but only if
`A` itself has a `Simple` instance:

```scala
// Evidence for Simple[List[A]] given evidence of Simple[A]
implicit def simpleList[A: Simple]: Simple[List[A]] = new Simple[List[A]] {
 override def simplify(l: List[A]) = {
   val simplifyA = implicitly[Simple[A]].simplify _
   s"Simplified list:\n${l.map(simplifyA).mkString("\n")}"
 }
}
```

Try it out:

```scala
implicitly[Simple[List[Int]]].simplify((1 to 5).toList)
//=> Simplified list:
//=> Simplified Int with value of 1
//=> Simplified Int with value of 2
//=> Simplified Int with value of 3
//=> Simplified Int with value of 4
//=> Simplified Int with value of 5
```

Hopefully you have a feel for how this works (ignoring how unuseful our
`Simple` is IRL). Next, let's follow Scalaz lead and
provide some convenient syntax for working with our new type class; typing
`implicitly[Simple[_]]` over and over again is starting to get
old.

## Syntax

Wouldn't it be nice if we could just call `.simplify` on objects which provide
evidence of a `Simple`? Well, we can via some neat implicit
tricks. Check it out:

```scala
final class SimpleOps[F](val self: F)(implicit val F: Simple[F]) {
  final def /^ = F.simplify(self)
  final def ⬈ = F.simplify(self)
}

trait ToSimpleOps {
  implicit def ToSimpleOps[F](v: F)(implicit F0: Simple[F]) =
    new SimpleOps[F](v)
}

object simple extends ToSimpleOps

trait SimpleSyntax[F] {
  def F: Simple[F]
  implicit def ToSimpleOps(v: F): SimpleOps[F] =
    new SimpleOps[F](v)(SimpleSyntax.this.F)
}

// New definition of our Simple typeclass that provides an instance of
// SimpleSyntax
trait Simple[F] { self =>
  def simplify(f: F): String
  val simpleSyntax = new SimpleSyntax[F] { def F = Simple.this }
}
```

Here we've provided two syntax operators as aliases to
`simplify` (because no type class is legit without unicode
operator aliases).

```scala
import simple._

1 ⬈
//=> Simplified Int with value of 1

new C("I am C") ⬈
//=> Simplified: C[name=I am C]

List(1,2,3) ⬈
//=> Simplified Int with value of 1
//=> Simplified Int with value of 2
//=> Simplified Int with value of 3

// boring
new C("bar") /^
//=> Simplified: C[name=bar]
```

## Conclusion

Type classes are a powerful method of adding support for a set of operations on
an existing type in an ad hoc manner. Because Scala doesn't encode type classes
at the language level, there is a bit of boilerplate and implicit trickery to
create your own type classes (compare this with Haskell, where classes are
elegantly supported at the language level) and operator syntax.

[View the full code listing for this post](https://gist.github.com/devth/735ddd34e8f29fc6b872).

### Further reading

- [Introduction to Typeclasses in Scala (2013)](http://tpolecat.github.io/2013/10/12/type class.html)
- [Types and Typeclasses — Learn you a Haskell](http://learnyouahaskell.com/types-and-typeclasses)
- [Simalacrum](https://github.com/mpilquist/simulacrum) — a modern, concise type
  class encoding for Scala using annotations
- [Typeclassopedia](https://wiki.haskell.org/Typeclassopedia) — overview of
  Haskell's type classes, many of which are also represented in Scalaz, along
  with a very useful diagram showing relationships between type classes.


A Simple type class

[Yetibot](http://yetibot.com) is now [on
Docker](https://hub.docker.com/u/yetibot)! This is the fastest way
to get up and running. To demonstrate, let's run it with Docker using the most
minimal configuration possible: an in-memory (non-durable) Datomic configuration
and a single IRC adapter config. (I'm assuming your local Docker is
[installed](https://www.docker.com/docker-toolbox).)

**Update 2024-05-01:** This configuration is outdated and no longer valid.
Please refer to the [officiel Yetibot Ops guide](https://yetibot.com/ops-guide)
for the most up-to-date instructions.

```bash
mkdir -p ~/tmp/config

cat << EOF > ~/tmp/config/config.edn
{:yetibot
 {:db {:datomic-url "datomic:mem://yetibot"}
 :adapters
 [{:name "freenode-irc",
 :type :irc,
 :host "chat.freenode.net",
 :port "6665",
 :username "yetibot-docker"
 :rooms #{"#yetibot"}}]}}
EOF

docker run --name yetibot \
 -d -p 3000:3000 \
 -v ~/tmp/config:/usr/src/app/config \
 devth/yetibot

docker logs -f yetibot
```

**N.B.** I chose `~/tmp/config` because:

> If you are using Docker Machine on Mac or Windows, your Docker daemon has only
> limited access to your OS X or Windows filesystem. Docker Machine tries to
> auto-share your /Users (OS X) or C:\Users (Windows) directory.
> — [Mount a host directory as a data volume](https://docs.docker.com/engine/userguide/dockervolumes/#mount-a-host-directory-as-a-data-volume)

If you're not using Docker Machine feel free to put it wherever you like.

Hop on Freenode and join the #yetibot channel. Once `yetibot-docker` joins, try
it out!

```
!list yetibot on docker | xargs echo ⚡️⚡️⚡️ %s ⚡️⚡️⚡️
```


<img src="/images/yetibot-on-docker-irc.png" />

When you're done, clean up with:

```bash
docker rm -f yetibot
```

To get the most out of Yetibot you'll want to configure some of the optional
services. See the full [config
sample](https://github.com/devth/yetibot/blob/master/config/config-sample.edn)
and check out [yetibot.com](http://yetibot.com) for more info on the cool things
you can do with Yetibot!

where 𝓧 ≈ somewhere between 1 and 3.5 minutes

Yetibot on Docker in 𝓧 minutes or less

Compiled queries in Scala

There are many [too-strong
opinions](https://hn.algolia.com/?query=environment%20variables&sort=byDate&prefix&page=0&dateRange=all&type=story)
on the "right" way to do configuration but they primarily come down to these
options:

1. flat files in some known format like JSON, YAML, properties files, EDN, etc.
1. environment variables
1. CLI arguments at start up
1. config from a data store (which ironically requires separate initial config
 in one of the above methods in order to access the data store)

While #4 is an interesting option in light of modern cluster-based infrastructure
and tools like `consul` and `etcd`, we're going to focus on the first three
mechanisms for this post: flat files, env vars and CLI arguments.

**Systems should not require configuration via only one mechanism, but instead
allow any or all available mechanisms in combination.** This affords the
operator of the system to configure it in the way that's most suitable according
to their infrastructure, preferences, policies, and politics.

When using multiple mechanisms there needs to be a very clear priority in which one
mechanism overrides the other. Typically env vars override flat file config, as
they are more ephemeral and easy to set.

A few modern examples of systems that work this way today include:

- [ElasticSearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html)
- HashiCorp tools like [Consul](https://www.consul.io/docs/agent/options.html)
- TICK stack e.g. [Chronograf](https://docs.influxdata.com/chronograf/v1.6/administration/config-options#chronograf-service-options)

## Symmetry

A problem that immediately comes to light once we say that any individual value
can be configured by any available mechanism is the representational difference
between mechanisms:

- Env vars and CLI args are stringly typed key value pairs
- Config files support arbitrarily-deep tree structures with varying support for
 primitive and collection types like strings, numbers, booleans, arrays and
 maps

Because of these intrinsic significant structural differences we need a
translation layer between KV pair and tree. If we adopt a single constraint the
translation layer becomes simpler to express: **All leafs in the tree are
strings.**

The implication is that our system which consumes the config must parse the
string to obtain its expected format.

## Introducing dec: Deep Environmental Config

[dec](https://github.com/devth/dec) is a tiny library that embraces this
constraint and provides an `explode` function to transform KV pairs into an
expected shape equivalent to a potentially deep EDN structure.

<img
 style={{ width: "100%" }}
 src="https://github.com/devth/dec/raw/master/img/dec-colors.png?raw=true"
 alt="dec logo"
/>

The following configurations are equivalent from the perspective of dec:

```bash
MY_DB_PORT=4567
MY_DB_HOST=database
MY_URL=https://my-system/
```

```clojure
{:my
 {:db {:port "4567" :host "database}
 :url "https://my-system"}}
```

Notice how trees are serialized into KV pairs with a simple `_` delimiter. (The
delimiter is customizable in `dec` but `_` is the default.)

Similarly, arrays can be represented:

```bash
MY_SERVER_0_HOST=serverA
MY_SERVER_1_HOST=serverB
```

```clojure
{:my
 {:server [{:host "serverA"} {:host "serverB"}]}}
```

## Schema

Due to our constraint that all values be strings, it becomes important that
the system consuming the config strings have a valid way to parse and validate
the strings into actual expected type.

Building a configuration schema into config consumers has some other nice
properties, particularly when using clojure.spec:

- Validate expected shape of config at runtime with precise error messaging
 describing the exact location of invalid config and expected values
- Generate config example structures from schema in any format (e.g. the full
 list of supported env vars or EDN tree)

## In the wild

To demonstrate we can look at [Yetibot](https://yetibot.com), an open source
chat bot written in Clojure that embraces the above concepts.

Yetibot can be configured via env, edn, or a combination of both. For example, a
minimal Yetibot config might be:

```bash
YB_ADAPTERS_MYSLACK_TYPE=Slack
YB_ADAPTERS_MYSLACK_TOKEN=xoxb-my-token
YB_DB_URL=postgresql://yetibot:yetibot@postgres:5432/yetibot
```

Yetibot expects all env vars to be either prefixed by `YB` or `YETIBOT`, thus
the following is equivalent:

```bash
YETIBOT_ADAPTERS_MYSLACK_TYPE=Slack
YETIBOT_ADAPTERS_MYSLACK_TOKEN=xoxb-my-token
YETIBOT_DB_URL=postgresql://yetibot:yetibot@postgres:5432/yetibot
```

And since it uses `dec`, an edn file is also equivalent:

```clojure
{:yetibot
 {:adapters {:myslack {:type "Slack" :token "xoxb-my-token"}}
 :db {:url "postgresql://yetibot:yetibot@postgres:5432/yetibot"}}}
```

Note that `edn` isn't a requirement; anything that can be parsed into Clojure
collections would be equivalent.

## Future

It might be useful to create a library that actually slurps the config and munge
multiple mechanisms, using `dec`. Prefix handling and merging is slightly
tricky. This is all currently part of
[yetibot.core](https://github.com/yetibot/yetibot.core/blob/4b607726bae926de31a48bb8a05e7345a8668484/src/yetibot/core/config.clj#L19-L47).
This library should also utilize `clojure.spec` to validate the expected shape
of a config, provide validation, friendly error messages, and config generation.

## Conclusion

There are known pros and cons to both approaches. A system should not perscribe
which sets of constraints to adopt, but instead allow consumers to weigh their
own tradeoffs and allow them to run things however they want.

By adopting a simple constraint and providing a very small library we can
support this.

dec: Deep Environmental Config

The aim of this guide is to demonstrate the fastest way to create a client-side
web app and get it deployed. I'm writing it for myself in preparation for an
upcoming [Startup Weekend
event](http://communities.techstars.com/usa/colorado-springs/startup-weekend/13311).
We'll keep it CLI-heavy and automation-friendly as opposed to point-and-clicking
our way through web dashboards.

## Tech

- [GitLab](https://gitlab.com) - GitLab's unlimited private repos, amazing CI
 and free public pages make it an excellent choice for deploying static sites.
- React - [`create-react-app`](https://github.com/facebook/create-react-app)
 is the quickest way to start hacking on React without messing around with
 Babel and Webpack configs.
- ES6 - we'll use plain ES6 instead of something fancier like TypeScript,
 ReasonML, or Elm because the focus is speed and simplicity.

## Caveats

- We'll eschew typical production concerns like performance, security,
 configuration, and tests since we're aiming to build an MVP in a weekend.
- Even though a web app will likely need some CRUD we won't spend the time and
 effort to use an actual database; instead the prototype will use client-side
 state to simulate how it will work.

## Prerequisites

This guide assumes [Homebrew](https://brew.sh/) but the following packages can
be installed by many other means.

```bash
brew install yarn
brew install jq
brew install ruby
```

## GitLab

Let's create a GitLab group so we can collaborate with our teammates.

```bash
gem install gitlab
export GITLAB_API_ENDPOINT=https://gitlab.com/api/v4

# visit https://gitlab.com/profile/personal_access_tokens and generate a token
echo "Enter your GitLab token"
 read -s gitlab_token
export GITLAB_API_PRIVATE_TOKEN=$gitlab_token

gitlab user # make sure we're authenticated
user_id=`gitlab user --json | jq '.result.id'`
echo $gitlab_user_id # this should be an integer

gitlab help create_group
# pick your own unique group name and group path
# these are global on GitLab
group_json=`gitlab create_group startup-weekend-2019 startup-weekend-2019 --json`
echo $group_json | jq
group_id=`echo $group_json | jq '.result.id'`
echo $group_id # should be an integer

# add your teammates (you'll need their GitLab user IDs)
gitlab help add_group_member
# we'll give everyone owner permissions for this exercise
# see https://docs.gitlab.com/ee/api/access_requests.html for docs
access_level=50
gitlab add_group_member $group_id $user_id $access_level
# the above will fail because we're already a member
# instead perform this for gitlab users on your team that you want to add
# you can do this via the web dashboard if you like
```

Now that our group is setup and members added, let's create the repo under it.

```bash
gitlab help create_project
project_json=`gitlab create_project app "{namespace_id: '$group_id'}" --json`
# confirm it succeeded:
echo $project_json
```

## Node.js React App

Let's generate a fresh React app:

```bash
yarn global add create-react-app
create-react-app startup-weekend
cd startup-weekend
yarn install
```

Now configure the git repo we setup in the previous step and push to it:

```bash
git remote add origin `echo $project_json | jq -r '.result.ssh_url_to_repo'`
git push -u origin master
```

Congratulations, your app source is now safe in GitLab. The next step is to
deploy!

## Deployment on GitLab Pages

First we need to specify the `homepage` property in `package.json` so CRA
computes the correct asset paths when it builds assets.

```bash
group_path=`echo $group_json | jq -r '.result.full_path'`
project_path=`echo $project_json | jq -r '.result.path'`
gitlab_pages_url=`echo https://$group_path.gitlab.io/$project_path`

tmp=$(mktemp)
cat package.json \
 | jq --arg homepage $gitlab_pages_url '. + {homepage: $homepage}' > $tmp \
 && mv $tmp package.json

git commit -am 'Add homepage to package.json'
```

Now we can setup CI. Run this to add a `.gitlab-ci.yml` to your repo specifying
how to build in GitLab CI and deploy to GitLab Pages:

```bash
cat << 'EOF' > .gitlab-ci.yml
cache:
 untracked: true
 key: "$CI_BUILD_REF_NAME"
 paths:
 - node_modules/
 - .yarn-cache

image: node

stages:
 - deps
 - test
 - publish

deps:
 stage: deps
 script:
 - yarn config set cache-folder .yarn-cache
 - yarn install --pure-lockfile

pages: # must be named pages to publish to GitLab pages!
 stage: publish
 script:
 - yarn build
 # replace public contents with build - GitLab requires a `public` dir
 - rm -rf public/*
 - mv build/* public
 - ls -al public/
 artifacts:
 paths:
 - public
EOF
```

With this in place, GitLab CI will build and deploy our app on every `git push`.
Let's try it out:

```bash
git add .gitlab-ci.yml && git commit -m 'Setup GitLab CI deployments'
git push
```

Open up the pipelines dashboard and sit back while your app is built and
deployed:

```bash
open `echo $project_json | jq -r '.result.web_url'`/pipelines
```

Once it's finished, view your app:

```bash
open $gitlab_pages_url
```

At this point you can start developing your app and have it auto deployed any
time someone pushes to `master`.

## Closing thoughts

One of the [Startup Weekend
resources](https://startupweekend.org/attendees/resources) is $3k worth of
credits on Google Cloud Platform. While GCP is an excellent option for real
apps, I opted to keep things as minimal as possible, and it ended up being free.

We could very easily extend this further using the [Firebase Realtime
Database](https://firebase.google.com/docs/database/) and [Firebase
Authentication](https://firebase.google.com/docs/auth/) which gets you Google /
Facebook / Twitter / GitHub login and a schemaless data store with very little
work. These options change nothing from a deployment standpoint as Firebase
usage remains purely client side. And since Firebase is a part of the GCP
offering you can use those credits.

Fast app generation and deployment

These are my notes from the mind-expanding talk ["Unison: a new distributed
programming language" by Paul
Chiusano](https://www.youtube.com/watch?v=gCWtkvDQ2ZI&feature=youtu.be).

> Unison is an open source functional programming language with special support
> for building distributed, elastic systems. It began as an experiment: rethink
> all aspects of the programming experience, including the core language,
> runtime, tooling, as well as code versioning and publishing, and then do
> whatever is necessary to eliminate needless complexity and make building
> software once again delightful, or at the very least, reasonable.
>
> We're used to thinking of a program as a thing that describes what a single OS
> process will do, and then using a separate layer of technologies outside of
> our programming languages to "configure" many separate programs into a single
> distributed, elastic "system". This gets complicated. The core language of
> Unison starts with the premise that no matter how many nodes a computation
> occupies, it should be expressible via a single program, not many separate
> programs. Unison programs can describe their own deployment, elastically scale
> and orchestrate themselves, and deploy themselves in parallel onto any number
> of nodes for execution.
>
> This talk introduces the Unison language and its tooling and shows what it can
> be like to program systems of any size with this model of computing.
>
> Paul Chiusano 
> Unison Computing 
> @pchiusano
>
> Paul Chiusano started the research that led to the Unison language and is a
> cofounder of Unison Computing, a public benefit corp. He has over a decade of
> experience with purely functional programming in Haskell and Scala and
> coauthored the book Functional Programming in Scala. He lives and works in
> Somerville, MA.

Unison is language inspired by Haskell, Erlang and Frank, designed in a way that
lends itself to some amazing characteristics. The issues that Unison addresses
cause tons of headache and lost hours in other languages and build systems.

Unison's goal is to drastically improve the developer experience around writing,
testing and deploying code "by rethinking anything and everything about
programming".

The language is based on a core technical principle: **content-addressed code**.
Definitions are identified by the hash of their content. Names are simply
metadata for the purpose of human consumption but have no affect on code that
depends on that function since they simply reference it by hash.

So what happens when you embrace this principle? You gain a ton of benefits,
including:

- **Function renames never break the rest of the code**, including downstream
 dependencies because all code is content-addressable via the hash of its
 definition. Names are just metadata attached to the hash.
- **No builds**. Once someone adds a function to a codebase, its AST and
 metadata (like name) is built and stored in the code base, so when you
 checkout a code base everything is already built.
- **Cached test results**. Test results can be cached because unit tests are
 pure (aka referentially transparent) and only run again if a function being
 called by the test changes.
- **Codebase is append only**. Definitions are hashed, so if a definition
 changes a new hash is created and appended to the codebase. The cache never
 needs to be invalidated.
- **Code is indexed in interesting ways**, such as by types like `Nat -> [a] ->
[a]` which yields `List.drop` and `List.take`.
- **Solves the diamond dependency problem** (e.g. `D` depends on `B` and `C`,
 but `B` and `C` both depend on `A` - which version of `A` does `D` get?). Libs
 in traditional langs operate on shared/global namespaces but since Unison
 relies on hashes instead of names there are no conflicts. Both "versions" of A
 can co-exist.
- **Typed durable storage**. Typically when we persist data we serialize
 it into some other form like bytes, JSON, or SQL then write a ton of code to
 transfer in and out of that format and validate it back into some type, e.g.
 `Employee`. When you stop referring to things by name, multiple "versions" of
 a type can exist, and you can simply persist and un-persist structures.
- 🤯 **Programs that deploy themselves** + describe whole elastic distributed
 systems 🤯
 - Because code is content-addressable it's easy to distribute definitions
 across a cluster via nodes asking other nodes for the definition on demand
 when that node is asked to evaluate a function
 - Nodes require no setup to run your code other than provisioning Unison
 - Code can provision new nodes to run code at runtime
 - Think of it as "distributed programming as a library"
 - Abilities like `Remote` (which allows code to run on a remote machine) can
 be defined in the type system (this looks like a Type Class).
 - Can define custom ability types and a corresponding interpreter that encodes
 the meaning of that type i.e. what it does at runtime
 - e.g. `Cloud.usEast.run '(dsort hugeDataset)`
 - What about stateful elastic services?
 - Unison is researching this currently. For example: a system with
 high-concurrency that still involves some sort of consensus, like a KV
 store with updates happening concurrently on many nodes

Highly recommend watching the talk.

Learn more at:

- [unisonweb.org](https://unisonweb.org)
- [Paul's Unison blog posts](https://pchiusano.github.io/unison/)

Notes on Unison



Many software engineers are unfamiliar with open source software (OSS) and have
never contributed, but we all consume it, heavily. I suggest all software
engineers have familiarity with OSS and consider contribution as part of the
foundational culture and rhythm of software engineering. It could be authoring a
new project or contributing to other projects. It could be writing tests and
documentation, or helping users in community chat rooms, or blogging about your
experience with various tools and libraries.

## Essentiality

Imagine if we didn't have languages, tools and web frameworks to build on top
of. Or if we had to purchase a license to use some stack (a well known company
already tried this and apparently decided [open source was a superior
strategy](https://opensource.com/business/14/11/microsoft-dot-net-empower-open-source-communities)).

Progress would be significantly slowed. Instead of going from idea to `rails new` and having a proof of concept in a few hours or days you'd be reinventing
the wheel for the nth time or scouring pricing information to figure out which
stack strikes the right balance of time vs cost savings. In other words: not
building.

## It legitimizes your work

Code and systems that have gone through the crucible of public scrutiny and
survived are that much stronger, more legitimate, with vastly more permanence.

It forces the author to create and maintain all the peripheral work around a
solid piece of technology. It pushes the tech through edge cases and unexpected
use cases. It forces one to write documentation, maintain test coverage, publish
artifacts, build community, and consider how much generality is sufficient to
elevate a system from one-off usage to becoming useful in other contexts making
itself resilient to bit-rot.

It forces the author to achieve a level of excellence that probably wouldn't
have otherwise been realized without the extrinsic motivation.

Contrast this with internal code. Whether that code lives or gets extirpated
depends on a ton of variable factors in the company. Reorganizations, rewrites,
politics, strategy, and all kinds of other factors out of your control affect
the longevity of your code. And if it does live on for some amount of time, it
will most likely be just good enough to maybe barely do what it's supposed to,
in a perpetual alpha state. Many Project Managers don't understand the value in
comprehensive technical excellence, and therefore won't prioritize technical
debt cleanup, necessary refactoring, coverage, documentation, or giving talks.

After working X years at Y corp don't you want something to show for yourself?
As for me, every time I solve a problem the first thing I ask is "can this be
open sourced?"

## It sharpens your skills

When you put your work out in the open, you invite criticism. That may seem
scary, but you know makes it more than worth it? Leveling up your skills and
sharpening your thoughts in the crucible of public opinion. Do hard things and
you can't help but grow and become stronger and more mature in the process.

## Social responsibility

Great. You've decided open source is worth pursuing. You want to make the world
a better place. So what is your responsibility to the community when you open
source a project and that project achieves some degree of fame or number of
users? Do you "owe" your users nothing? Or something?

If your thing is worth the attention of the community there is a certain
standard of excellence that should be met and maintained. By achieving some
level of recognition, whether you wanted it or not, you now have a
responsibility to your users.

- make a best effort to ensure the quality and security of your code
- publish a roadmap
- let users know how they can contribute
  - optional: welcome new users and offer to mentor engineers who are making
    an honest effort to learn and grow
- be open to feedback and criticism
- aim for the good of the community
- fix bugs in a reasonable amount of time
- delegate stewardship when appropriate to reduce becoming a single point of
  knowledge
- pass off stewardship and leadership if you personally can't commit to the
  above

These are not easy things to do. Ideally we engineers have the support of our
employers to uphold this commitment to social good and technical excellence.
After all, it's often the tech companies who are directly benefiting financially
from the wealth of open source technology that they build upon.

## Epilogue

We consume millions (billions?) of hours of work put into the technology we use
and built atop every day. Each of us will never come close to giving back as
much as we consume. But please contribute if you can, however great or small,
from a place of joy, curiosity, and sincerity, striving to to advance technology
and shape community while growing yourself. Be an advocate for building culture
that cares about open source at your company.

Raise the floor.


Essentiality of Open Source


Starting a new job is hard. Really hard. It might take anywhere from 6 months to
2 years to really hit one's stride.

Depending on the organization, pockets of legacy tech, turnover, culture of
technical excellence (or not) and a hundred other reasons, the state of tech
could be anywhere on a spectrum of terrible to amazing. And most likely it
leans a little harder toward terrible.

Lately I've been reading the [Staff Eng Book](https://staffeng.com/book) and
thinking about the qualities of Staff+. I'm not going to enumerate these, but
instead consider one particular quality.

There are many things competing for an IC's time:

- Interviewing
- Reviewing PRs
- Meetings
  - Eng allhands
  - Team standups
  - Org standups
  - Company days
  - Demo days
- and on and on and on

Internal projects tend to have a perpetually-alpha quality. I've seen it at
every company I've worked for. It's rare to escape the sticky gravity well of
perma-alpha. (One trick for doing it is to [open
source](/essentiality-of-open-source) your thing, when appropriate).

To combat this, as software engineers we should leave things in a better state
every time we interact with them, even if we don't technically own the thing.

For example, if I need to make a change to an existing codebase, I hope and pray
that I can clone the repo and very quickly find:

1. links to the environments it's running in
1. how to build (this should be 1 command that works on everyone's machine)
1. docs (even some basic pointers)
1. specs for expected shapes of data - hopefully you use a typed language
1. how to run the tests
1. a link to the project's CI/CD
1. etc...

As an engineer, I want to interact with well-defined black boxes. Until I have a
reason to understand the internals and turn the black box into a white box by
consuming available materials, including the code itself, which is the ultimate
source of truth, as well as the other hooks the authors have given me.

A system is a composition of black boxes, each with its own unique
characteristics.

A great engineer should be taking the apps and services they work on beyond
alpha.

A great SRE or platform architect should be applying the same holistic
excellence to the collection of apps they care for. How are you exposing a
consistent interface into the system? Is it a GraphQL BFF? A consistent
adherence to gRPC with well-defined Protobufs?

Leave it better.


Leave it better


Modern web apps are built in a complex style, requiring many tools
and systems to support both their development (e.g. CI/CD, secrets, config,
feature flags, etc.) and production runtime (e.g. databases, caches, CDNs, etc.).

For each of these components engineers must decide where on the spectrum of
fully-managed to custom-built the solution should lie. It can be useful
to think through the lens of _entropy_ and _extropy_.

## Forces of extropy

Discounting pivots and shutdowns, the core product itself is undergoing constant
extropy. Engineers come and go but the product is always on a growth trajectory,
being reconciled from current state into some better state with better utility
and better design and newer underlying technology. It's the focus of everyone
from CEO to legal to customer support to engineers to designers. High degree of
attention from a diverse set of disciplines prevents the product from
defaulting into a state of entropy.

Underneath the product are many layers of supporting infrastructure and
technology. The further away from the core product a component lives the less
often forces of extropy are applied, and the further that component's
implementation should be toward the fully-managed side of the spectrum.

## Fast forward

Ask ChatGPT to write feedback 3 years in the future for what you're building,
written by engineers who will join the company 18 months from now. Which of
these will it most closely resemble?

> Wow whoever wrote this did a great job thinking about the future needs of
> the company and set us up for success

> I understand why they did it this way at the time but now it no longer makes
> sense

> I wish they had integrated this in an orthogonal manner so it'd more of a
> two-way decision

## Focus

It might be fun to custom build a CI/CD system, but unless your core product is
in the CI/CD space, this creates unnecessary, compounding risk and complexity.
By focusing on what unique value your product delivers you can build a better
product, faster.

Beware entropy over time.

## Further reading

- [Extropy by Kevin Kelly](https://kk.org/thetechnium/extropy/)


Will it survive entropy?


Monorepos have become popular in recent years. What problems are they solving,
and what problems are they introducing? Should we switch to monorepo?

## Monorepos introduce problems

Monorepos introduce many technical, organizational and political problems.

Monorepos are typically an alternative to standard build tooling. That means
every feature your favorite, wildly popular, community-supported build tool's
features will likely be missing in your monorepo tooling. You either need to
custom build it, or tell your users "sorry, we don't support that".

## Monorepos increase coupling

In a large company there are people, teams, services, languages, frameworks that
exist without you actually knowing about them. This is a good thing. We need
boundaries and abstractions in order to make the world smaller.

Counterpoint: Google. (Counterpoint: you are not Google).

## Single language vs multi language

There's a very big difference between monorepo tools that support multiple
languages (like Bazel) and single language monorepo tools like Turborepo, Rush,
or Lerna.

Single language tools typically have a much better developer experience and are
less complex because the language gets first class support. All the tooling is
built around that particular language, which means there's likely much less of a
gap between the full ecosystem of tooling for that language and what's available
in the constrained monorepo world.

Contrast that with a tool like Bazel. Bazel is very extensible, so even if your
language isn't already supported you can add support pretty easily. But
"support" just means opting in to the Bazel way of doing things, including
its impressive incremental builds and extreme use of caching. What you don't get
is IDE or standard tooling support. Want to use LSP and NeoVim? Too bad. The
tools are setup in a way that expect common build tooling (e.g. deps in
`node_modules` or compiled code in `target`). Bazel breaks all these
conventions, so the vast ecosystem of tools built upon them just don't work
unless you go to great lengths (and introduce incredible complexity) to bridge
the gap between Bazel-land and established standards.

## Go all in

If you're going to use a monorepo, you need to absolutely go all in. You need a
dedicated team to maintain your monorepo indefinitely. **Monorepos are an
incredibly complex and expensive endeavor** but most people don't talk about
this.

This team should be come an expert in the tooling, whether it's Bazel, Pants,
Buck, Earthly, Lerna, or something else.

The monorepo should be the de facto way of building software at your company. Do
not end up in a state where you still have polyrepos alongside your monorepo, or
worse yet: multiple monorepos. This is the path to destruction.

## Vendors as outsourced experts

Because monorepos are incredibly complex and the tooling is nowhere near on-par
with the standard tooling, whole companies have spun up around the monorepo
ecosystem. You can hire them to be your experts and help fill the missing
tooling gaps.

They are expensive, but they are probably worth it when you compare their fee to
a few annual developer salaries.

**Monorepos are very expensive.** Make sure you're all in and understand the
costs before embarking.

## So what's the point?

It's not all bad. Monorepos exist for a reason, and as long as you can avoid the
many traps, it can provide a great developer experience.

Monorepos give you:

- Atomic code changes that you can be developed, tested, reviewed, and released
  together.
- Managed tooling. The team who supports the monorepo can take care of details
  for you so you no longer need to care about them. Things like security
  patches, dependency upgrades, CI/CD pipelines, and even automatic provisioning
  of infrastructure (although these are all things you can do in a polyrepo
  too).
- A constrained set of standard langauges, frameworks, dependencies, and
  tooling.


2024

You might not want a Monorepo

4/17/2024 Monorepos have become popular in recent years. What problems are they solving, and what problems are they introducing?

2023

Will it survive entropy?

5/3/2023 For every component that makes up a modern web app engineers must decide where on the spectrum of fully-managed to custom-built the solution should lie. It can be useful to think through the lens of entropy and extropy.

2021

Leave it better

4/22/2021 Great engineers improve everything they touch.

2020

Essentiality of Open Source

4/7/2020 Open source is the foundation we build upon.

2019

Notes on Unison

9/17/2019 Notes on the Strange Loop talk 'Unison: a new distributed programming language' by Paul Chiusano. Warning: mind expanding future tech 🤯

Fast app generation and deployment

2/3/2019 The aim of this guide is to demonstrate the fastest way to create a client-side web app and get it deployed.

2018

dec: Deep Environmental Config

7/10/2018 dec is a tiny library that embraces constraints to afford users equivalency between multiple ways of setting environment variables according to their particular requirements.

2017

Compiled queries in Scala

2/26/2017 Compile a data structure representing a query into native code to speed up a query loop using ASM.

2016

Yetibot on Docker in 𝓧 minutes or less

1/6/2016 Yetibot is now on Docker!

2015

A Simple type class

8/3/2015 Type classes provide a way of achieving ad hoc polymorphism

ThrushCond is not a Monad

5/19/2015 Clojure has a useful macro called cond-> — let's explore a Scala equivalent

Monad laws in Scala

3/17/2015 The three Monad laws may seem pretty abstract at first, but they're quite practical. Let's try to internalize the laws by running through two of Scala's most popular monads and making sure they adhere.

Beautiful constraint

1/30/2015 The freedom to do anything you wish is rarely a good thing, whether you're an experienced engineer or a 2-year-old child.

2014

On learning

10/5/2014 It wasn't until I was well into my twenties that I learned how to really learn

Corporate process should mirror open source

9/28/2014 Within any big tech corporation are many disparate teams. Most of these teams know very little about the work, makeup or very existence of other teams, yet they depend on each other's output, directly or indirectly.