Showing posts with label book. Show all posts
Showing posts with label book. Show all posts

Monday, August 4

Why Object-Oriented Programmers Should Understand Category Theory (Part 1) - LINQ, Lists and Optionals

Table of Content

  • Part 1 (this part)
    • Functional Ideas
    • Category theory
    • LINQ, Lists and Optionals
  • Part 2 (coming)
    • Duality and RX Extensions
    • Monads and async await
  • Part 3 (coming)
    • Abstract Data Types
  • Part 4 (coming)
    • Making Invalid States Unrepresentable


Y ou’ve Been Using Functional Ideas All Along—Now Understand Them


Many of the most important and widely adopted programming features of the last two decades have their roots in functional programming—and deeper still, in category theory. These ideas have spread across languages often underpinning popular features without developers realizing their origins. For object-oriented programmers, understanding these ideas can radically reshape how we think about types, structure, and behaviour in software.
 

The Value of Abstraction

Category theory gives us a mathematical language for thinking about types, how they relate, and how they can be transformed. Once you see this, you can't unsee it. Patterns across libraries and languages start to click. Abstractions become more coherent. Features you once struggled with begin to make sense—not because you've gotten smarter, but because you're seeing the bigger picture.

Often I see programmers struggle with new features, when there are older features that have similar properties and behaviour that they have no problem with. Understanding is only a slight shift in perspective away.


My First Steps: LINQ and Composability

My first brush with category theory came with C# 3.0 and the introduction of LINQ. At the time, I was refactoring a codebase and trying to make libraries more reusable. Lists and "optionals" were giving me grief. Iteration, filtering, paging, and threading all felt like they required too much boilerplate. My object-oriented toolset—design patterns, interfaces, inheritance hierarchies—wasn’t helping. 

Then came LINQ, and suddenly, things started to fall into place.


Lists, Optionals, and What They Really Are

Beyond the Collection: Functors, Monads, and the Real Shape of Data

In category theory, structures like List<T> or IEnumerable<T> aren’t just containers—they’re functors and often monads. These might sound like intimidating terms, but they capture consistent, powerful ways to model and transform data across a wide range of scenarios.

Imperative programmers, especially in C#, often get caught up in choosing between T[], List<T>, Dictionary<TKey, TValue>, or HashSet<T>. While these distinctions matter for performance and semantics, it’s more valuable to start with a higher-level question: what is the shape of this data, conceptually? Is it a sequence, a choice, a mapping, a possible absence? This shift in mindset—from implementation to abstraction—pays dividends in clarity and reuse.


The Optional Mess in C#

Calling nullable value types and reference types "optionals" is being generous.  The functionality fell far short of what you would expect from a Maybe or Optional type. 

Nullable value types (int?, DateTime?, etc.) carried the right idea—a value that might be absent—but the ergonomics were awkward. The HasValue/Value pattern was clunky, and the risk of misuse high. Reference types were worse: everything was nullable, with no way in the type system to express a non-null guarantee. That made every object a potential landmine. 

The new extension methods worked on null reference types, however extension methods and null checks helped only slightly. You could filter out null values or use fluent chains to avoid some errors—but this was a teaspoon against the ocean. A better approach was the Null Object Software Design Pattern, replacing null with benign default objects. It worked, but was too burdensome to apply everywhere.

These weren’t uniquely C# issues. They’re legacy decisions, dating back to C, which inherited them from older languages like ALGOL W. Tony Hoare, inventor of the null reference, later called it his “billion dollar mistake.”—an error so deep that it rippled across decades of software development.


Why This Was a Big Deal

The core issue was this: you were asking humans to track where null might sneak in. And humans are terrible at boring, repetitive, unforgiving tasks. Worse, the static type system—the thing that’s supposed to catch errors before they escape into production—was blind to it. NullReferenceException became one of the most common runtime failures in C#, and developers had no way to formally express or enforce "this value should never be null."

When C# 8.0 introduced non-nullable reference types in 2019, it was a welcome (if long-overdue) step forward. But it had to be opt-in, riddled with compiler tricks, and careful not to break existing code—because Microsoft, understandably, feared disrupting the vast ecosystem of legacy code.


LINQ: Category Theory in Disguise

Though designed for querying databases, LINQ also worked on in-memory collections. And to make LINQ composable and consistent, Microsoft had to bake in ideas straight out of functional programming and category theory—whether they admitted it or not.  

If you define two extension methods—Select and SelectMany—with the right signatures, you can make almost anything LINQ-able. That’s the power of monads.

  • Select (a.k.a. map in FP circles) is for transforming values inside a context. Think: turning a List<int> into a List<string>, or a Nullable<string> into a Nullable<bool>.

  • SelectMany (a.k.a. flatMap, bind, or >>=) flattens nested contexts: a List<List<T>> becomes a List<T>, a Nullable<Nullable<T>> becomes a Nullable<T>. This same idea powers async/await and promise chaining in JavaScript. Something similar is happening when a do try catch block intercepts an error from several layers down in the stack (This flattening behaviour is more obvious with languages with checked exceptions like Java or Swift)

These operations are so essential that in C# 6.0, Microsoft introduced the null-conditional operator (?.)—a syntactic sugar for optional objects that provides a limited form of mapping for honest properties and binding for optional properties. On the upside it works for both Nullables and reference types.
 
The elvis operator  (?.) was so popular it made it into many other languages under an assortment of names.

And of course the optional situation was such that one of the first things people tried was getting LINQ to work Nullable types or to create their own Maybe types.

Functional Programming's Influence Grows

Around the same time, other functional staples were entering the mainstream. Hadoop brought map, reduce, and filter into the spotlight. C# had higher-order functions from the start, but lambdas in C# 3.0 finally made them usable without drowning in boilerplate.

LINQ’s use of extension methods also meant you could tack on new functionality to types you didn’t own—a clever trick that helped functional patterns sneak into OOP-heavy ecosystems. Without a coat of paint functional concepts are often resisted as being too procedural and not O-O enough.

Even the type system itself was changing. Parameterized types (aka generics) have roots in abstract algebra and were first seen in functional languages like ML (1973), then Ada (1977), then C++ templates (1991). Category theory takes this further with higher-kinded types—types that take types that take types.

Languages like Haskell and Scala can express these abstractions natively. C# can’t, but you can still use the ideas—just in a less elegant way.


The Big Realization

The real epiphany for me came when I saw that monadic operations acted on containers—like List<T> and Nullable<T>—as whole things. They respected encapsulation. I didn’t need to peek inside and mess with internals. I could compose operations, transform values, and chain computations without writing glue code every time.

Contrast that with traditional imperative code, which often tears through internal structures and violates encapsulation to “get things done.” Functional concepts gave me a cleaner, more composable way to think.


Conclusion

You don’t need to become a Haskell expert or abandon object-oriented programming to benefit from category theory. It might feel foreign at first. But it will:

  • Change how you think about types and data
  • Help you recognize deeper patterns across languages and libraries
  • Reduce boilerplate and improve composability
  • Let you write more robust and expressive code

Gaining familiarity with these concepts can dramatically improve your ability to design clean, reusable, and maintainable code—especially as languages like C# continue to adopt features born in functional programming.

If you're serious about mastering modern software development, understanding the abstractions behind LINQ, monads, and functors is no longer optional—it's essential.


Related Posts

More Information

Videos

2007 Era

2009 Era


Books

Category Theory

Category Theory for Programmers  by Milewski

Friendly but rigorous intro to category theory with direct links to programming. If you’ve ever wondered why LINQ and monads work, this is the bridge.
Category Theory by Awodey

Written for mathematicians, logicians, and computer scientists, aiming to provide both the abstract machinery and its motivations/applications.
Abstract and Concrete Categories - The Joy of Cats by Adámek,  Herrlich, and Strecker
Sharpen your understanding of abstraction vs. implementation, to gain a deeper grasp of type systems and functional programming, and to acquire powerful mental models for software design.

C#
C# in Depth by Jonathan Skeet
Explains LINQ, nullable types, and async/await in detail, with practical, production-ready advice.


Tuesday, July 15

Deliberate Practice (Part 2)

Table of Contents


D

eliberate Practice In Software Development

The Case of the Test First Developer


So how do we deliberately practice in order to improve our programming skills?

I use the same principles that I use to improve my bass playing to improve the software I produce.

Overlooked Opportunities

I find that many developers focus too much on learning language and frameworks features. While a full understanding of the tools you are using is essential, if you simply stop there, you are missing so many opportunities for improvement. Implementation is only one phase in the software lifecycle, you should not neglect the other phases and within implementation itself there are so many dimensions that it is unlikely you will run out of things to improve even if you feel you know the implementation platform back to front.

On-the-job Learning

Within my normal work there are many opportunities to practice. To keep it fresh I will focus on different improvement goals each session. In one session I may focus on composability, in another readability in another testability. In one session I will identify and exact reusable components from existing code in another test whether a particular approach or design pattern will keep the design simple or if an alternative would be better. 

Feedback

Test first design is ideal for maximizing immediate feedback. Pair programming gives lots of potential for feedback if you take advantage of it. Code reviews are a great way of getting feedback, however so often programmers see it as an annoyance. Tightening the feedback loop by getting early feedback from testers or users can be helpful.


Learning by Teaching

Teaching others is helpful in improving your understanding of the material. Whether helping a teammate with a problem or teaching a workshop on a topic or presenting at a meetup, the need to break down what needs to be done into small easily understood steps can help your own understanding and help organize your existing knowledge.


Learning from the Pain Points

I used the need to document an existing internal API to help drive improvement to the API. For each section in the API I would ask how can I change the API so as to make this section of the documentation u6nnecessary. I ended up simplifying the API a great deal making it much easier to use.

Learning using Practice Problems

You can use code katas or code koans to help practice your programming skills and there are many sites that offer problems for you to practice (see below).  


More Information

Articles

Videos

Problems to Practice

  • Project Euler - Solve mathematical problems using programming skills
  • LeetCode - Solve coding challenges, competitions and mock interview
  • HackerRank - Offers a variety of different types of programming challenges
  • TopCoder - Solve algorithmic challenges within a time limit
  • Code Wars - Community submitted challenges and discussions
  • Code Chef - Community site with challenges, competitions and tutorials
  • CoderByte - Coding challenges and tutorials
  • Exercism - Coding challenges with assistance from mentors.
  • Sphere Online Judge - Coding challenges, competitions and discussions
  • Hacker Earth - programming challenges and  interview preparation

Programming Games

  • Codingame - program simple games then play them
  • RoboCode - program tank AIs and pit them against others
  • CodeCombat - write code to solve puzzles

Saturday, February 6

Double Loop Learning

T he plan–do–check–act cycle is a great way of structuring what you are doing to incorporate feedback and learning into your results. Its great and I use it all the time, however it is an example of single loop learning. It is a good starting point, but if you stop at the PDCA cycle you are missing out.

What you are doing and what you are learning does not exist in isolation but as a part of a wider context. Paraphrasing the well worn marketing tip "Our customers do not want drills bits they want holes", that is, smaller goals exist to further larger goals. Periodically zoom out to that wider context and re-assess based on what you have learned.



Does what you have learned during your first attempts mean you need to change

  • The theoretical framework you use to understand the topic

  • The learning techniques you are using to master the topic

  • The skill you are learning e.g. whether learning this topic will still help you meet your wider goals or do you need to change to learning a different skill.


Adjust goals, decision-making rules, strategies and mental models in the light of experience.

If you are having trouble meeting your goals then instead of beating your head against the wall, look at why you are pursuing that goal. It is probably part of a larger goal. If you can not make progress on the original goal perhaps you can make progress in the larger goal by taking a different approach.

The 5 whys may help you identify the larger goal. are you concentrating on the drills or the holes?

The Pivot

This is why the pivot is popular in the startup culture.

If there is a lack of enthusiasm for the product by potential customers or an inability to translate  enthusiasm to  willingness to pay, they will change the product,  the type of product, how they sell the product, who they sell it to or the business model.

This willingness to reassess fundamental assumptions is an example of double loop learning.


Double Loop Learning in My Own Career

The Case of the Endless Iterations

Note:
  • a sprint or iteration is a fixed period or timebox which developers use to help plan their work (it is usually 1 to 4 weeks)
  • a backlog is a todo list of work items used for planning
  • a burndown chart shows how quickly work items are getting done.

One of the teams I was coaching was having three problems

  1. They kept abandoning work items they had committed to during backlog refinement and sprint planning, changing their minds from sprint to sprint about the backlog.
  2. They were accomplishing work at the start and end of sprints but nothing was happening in the middle.
  3. They kept on wanting to extend the sprint, claiming they had not finished the work.

In summary they were bad at starting the sprint, they were bad at ending the sprint and they were bad at maintaining focus during the sprint. Although the team had made improvements in many other areas I had not had any luck directly improving the above mentioned three problem areas, so I decided to tackle the problems indirectly. Instead of helping them play the current game I would change the game.

Inspired by eXtreme Programming's recommendation that if you are bad at something you should do it more frequently I convinced them to change the sprint length from two weeks to one week. That may not seem like a big deal but one week sprints can be problem if
  1. Your work items are too big, The majority of items should take a day or less with a one week sprint, while you can get way with larger items with a longer sprint.
  2. Your sprint ceremonies are inefficient, dragging on, causing you to be bogged down with administrivia
  3. Your testing, integration and delivery pipeline has long delays and onerous manual steps, reducing productivity.
Thankful the team had made dramatic improvements in this second trio of issues paving the way for a smooth transition to one week sprints. There was an almost immediate improvement.
  1. The team had an easier time predicting what they would do in the next 5 days instead of the next two weeks. Abandoned work items dropped by a factor of twenty.
  2. The flat horizonal section in their burndown chart disappeared. They were getting things done throughout the sprint instead of just at the beginning and end.
  3. They stopped asking to extend the sprint, because it felt more acceptable and less of a big deal to just add uncompleted work to next weeks sprint. Sprints were less intimidating.
By changing the context I reduced the impact of these weaknesses of the team so they could focus on their strengths.

Articles

Videos

Books

Teaching Smart People How to Learn by Chris Argyris (
 Summary Video + Book )  

Friday, January 1

Goal Setting

I

t is that time of year where people make New Year's resolutions. 

These resolutions are easy to make, but much harder to keep.

So how can you build habits that actually last?

There has been quite a bit of research on this topic in recent years. I’ve found three methods particularly helpful: MCII, Hope Theory, and Atomic Habits. These approaches all share common themes—optimism, planning, and breaking goals down into smaller steps. They don’t contradict each other; in fact, they complement one another. The authors even refer to each other’s work, showing that they are aware of the connections between their ideas.

Below, I’ll go over each method in turn:

Mental Contrasting with Implementation Intentions

MCII  also known as WOOP , is a practical technique for changing habits and behaviour. It involves four steps:

  1. Wish – Choose a goal you want to achieve.

  2. Outcome – Visualise how it will feel to achieve that goal.

  3. Obstacle – Identify the internal or external obstacles that might get in the way.

  4. Plan – Make if-then plans. For example: If I face obstacle A, then I will do behavior B.

Videos

Books

Rethinking Positive Thinking by G.Oettingen ( Summary Video + Book )



Hope Theory

Prof. Snyder breaks his advice into three components.

  1. Goals – Set goals that are specific, measurable and with a clear deadline.
  2. Pathways – Plan multiple ways to reach your goal. If one path doesn’t work, use double loop learning to adapt.  
  3. Agency – Build confidence and maintain motivation through optimism and agrowth mindset. This helps develop grit  and persistence, which are essential for success. 


Videos 

Books

Psychology of Hope  by C.R. Snyder ( Summary Video + Book )
Handbook of Hope by C.R. Snyder ( Book )


Atomic Habits

James Clear outlines his method through what he calls The Four Laws of Behavior Change. According to him, a habit should be:

  1. Obvious – Link the habit to a specific time, place, or situation so it can be triggered easily.

  2. Easy – Break the habit down into small, manageable steps.

  3. Attractive – Connect the habit with a reward to make it appealing.

  4. Satisfying – Track your progress. Turn the habit into a game to keep it fun and motivating.


These three approaches offer useful strategies for creating lasting habits. By combining them, you can improve your chances of sticking to your resolutions and making meaningful changes.

Book


Tuesday, December 29

Deliberate Practice (Part 1)

Table of Contents



I

ntroduction


Mindless repetition does not count as deliberate practice as without focused attention and narrow specific and detailed goals it is easy to repeat your mistakes instead of improving your performance. Deliberate practice needs to be purposeful and systematic. 

I will describe two examples of deliberate practice, one as a musician and one as a developer. You will see that there are commonalities in strategies and techniques that are independent of  discipline. 


Deliberate Practice In Music

The Case of the Shut In Drummer

I knew a drummer once who took a year off to practice his drumming. He sacrificed his income and drove his poor wife to distraction. After a year he was no better than when he started. 

So what did he do wrong? 

Small vs Big Changes 

Small frequent commitments are generally easier to keep and stay focused during than big one-off commitments. I practice my bass 15 minutes a day plus a couple of longer sessions a week. Maintaining focus for an entire year is unrealistic. If I wanted to increase my commitment to music I would gradually increase the length and frequency of my sessions rather than make radical changes that are hard to maintain. 

Spaced Repetitions


For memorization activities, few repetitions each day for most of a week is more effective than 20 repetitions in a single day (Cramming). Anki is a useful app for enabling spaced repetitive memorization.

Feedback 

Frequent and constant feedback is essential for practice. The only people listening to the drummer were non-musicians, his long suffering family. As well as giving feedback, practice partners can provide social pressure to maintain practice, as well as making practice far more enjoyable. Teaching music to my daughters and playing with them is a major factor in maintaining my bass practice. That and the fact that it is fun and great for stress relief. 

Specific vs General

Narrowing your focus by breaking down what you're doing into the smallest possible parts and aspects, and focusing on each part individually rather than merely having a vague desire to improve is essential. Without this it is easy to simply practice the errors you are making like the unfortunate drummer. Constantly changing the aspect you are trying to improve helps with the next point. 

Variety

Keeping things fresh is essential. If your practice is monotonous then it is almost certain that you will go off on a tangent in order to take a break. In this case, psychology professor and author Angela Duckworth, advises to substitute nuance for novelty. In one practice session I may concentrate on alternate fingering and in another I may concentrate on timing and feel. In one practice session I may concentrate on my fretting hand and in another I may concentrate on my plucking hand. In this way I maintain focus without getting bored as well as making small maintainable improvements in all aspects of my playing. 

Careful and Patient

It is also essential to be slow and meticulous in order to practice the correct way rather than practice making errors. I usually practice with either a metronome or a backing. When learning a new song I will often set the metronome or backing to 0.75x speed then slowly increase the speed over the course of the practice until its 1.5x speed then slowly adjust it back to 1x speed. It is important to practice without errors. It is also important to practice at the edge of your abilities. 


Related Posts


More Information

Articles

Videos

Books

Peak by Anders Ericsson ( Summary Video  + Book )
Deep Work by Cal Newport ( Summary Video + Book )
The Art of Learning by Josh Waitzkin ( Summary Video + Book )
The Talent Code by Daniel Coyle ( Summary Video + Book )
The Compound Effect by Darren Hardy (  Summary Video + Book )
Mastery by Robert Greene (  Summary Video  +  Book )
The Practicing Mind by Thomas M. Sterner ( Summary Video Book )