James Eastham | Just a storytelling technologist thinking about systems, software and people.

An Ode To Building What Matters

James Eastham — Thu, 17 Sep 2026 00:00:00 GMT

What was the original purpose of software? Why did the industry come to exist in the first place?

It was to make things easier for us. To address the things humans aren’t very good at. A bit of yin and yang in the world. Us humans, typically, are great at empathy, at communication, at relationships, at critical thinking, at creativity and at novelty. Empathy as the most important. We have this amazing ability to see someone else’s point of view and understand why we might feel something/take an action if we were in their shoes. Although a lot of the problems in the world today come from people not doing enough of that.

There are things we humans are bad at though. Imagine I asked you to just receive some forms people fill in, validate the data, do a bit of calculation on the input and then send the output on. And that’s all you’re going to do every day, for 8 hours, 5 days a week. Aside from being bored out of your mind, you’d also probably do it wrong. That’s not a slight on you, dear reader, or on the fact that these roles do need to exist, it’s just a reality of human nature. Most of us would get it wrong. Alongside that mundane task, if I also asked you to store in your mind every piece of data you come across whilst processing the inputs, and be able to regurgitate it at a second’s notice, you’d think I was mad. Our brains are great processing machines, and they do a pretty good job at storing information… Recall and working memory let us down.

We can’t run complex algorithms in seconds, and sometimes our emotions affect our decisions. Sometimes, this is a good thing. But herein lies the problem with the current state of the majority of the tech industry. The people in charge, the people leading, the enormous, capitalist, make money at all costs organisations have forgotten this. Software, computers, technology… foremost, they should make people’s lives better.

I’m reading Enshittification by Cory Doctorow at the moment (highly recommended, as is his blog). The case study on Google is fascinating. In court hearings, internal memos came out that showed Google actively pushed to make Google Search results worse in order to make you search more often. You know what happens if you search more often? You see more ads, which means Google makes more money. Similar stories exist for Facebook (leave MySpace and join us, we promise not to spy on you), Amazon (you can buy anything, and we definitely won’t undercut the sellers) or Apple (spying on people is bad, buy an iPhone, but we will also spy on you).

The ‘personal computer’ should be something that is filled with things that are personal to us, that help us. Not subscriptions that we pay for a below-par service that is getting worse. But hey, if the stock price goes up, that must be good, right? I realise I’m saying all of this as someone who has worked for two big tech companies. It’s why a couple of paragraphs ago I used the word majority of tech companies. I’ve seen both sides. I’ve seen the enormous, capitalist, make money at all cost organisations (hey AWS) and seen what a tech company looks like that cares about its people and in the main builds products that make users’ lives easier.

This isn’t a slight on people on the ground at tech companies either. There are thousands of brilliant people at these companies. But a mandate from above to push AI into every single crevice of every single part of every single product doesn’t leave these brilliant people with much choice. Because remember, in 2026, if your product doesn’t have an AI chatbot, it’s worthless (also said as someone who is actively working on an AI chatbot).

None of that is to say the technology itself is the problem though, it’s how, and why, it gets used. There are genuine, brilliant use cases for LLMs and the chatbots built on top of them. The human weakness I mentioned earlier, that recall and working memory let us down? That’s one of the things an LLM can be good at solving. Summarising huge amounts of random information into something a person can actually digest. Used well, a chatbot isn’t there to keep you searching for longer, it’s there to get you to the answer faster so you can go and do something more useful. That’s a good use of the technology, and it’s the bit of this whole thing I still believe in. Although, when said chatbot has a habit of making things up and hallucinating things do get a little less valuable.

There is a paradox with the current use of large language models (LLMs), particularly for software development. It gives a huge number of people the power to take back some element of control and build systems for their personal computers that are personal to them.

I’m a trustee of a local food bank, I manage all of their IT systems. Recently, I’ve built some custom software for them. Nothing fancy, at it’s core, it’s a database with some validation on the frontend that can send automatic text messages. Previously, with a full-time job, I wouldn’t have had time to build it. The food bank would have gone on with a pretty inefficient process, or bought some cloud subscription that kind of gave them the right functionality but not really. AI coding assistants enable me to build something that works for me. It’ll never be sold, we’ll never charge a subscription, we’ll never ask people to pay for it or enshittify it and sell everyone’s data. We’ve just built a tool, that makes everyone’s lives easier.

Whilst I’m not suggesting you go and try to build the next Amazon/Netflix/Google/Uber with your laptop and a Claude subscription. Although it never gets old watching the hilarity of seeing people try to and fail dramatically because, ya’know, distributed systems are hard. But a lot of the problems that come with large scale systems don’t apply when your application is small, has 10 users who access it intermittently.

I’d still consider myself an AI skeptic, but equally, I’m not afraid to go all in on the tools if I think the outputs can add value to people’s live.

The Importance Of Arguing Against Yourself!

James Eastham — Mon, 20 Jul 2026 00:00:00 GMT

I never disagree with someone else’s opinion unless I understand their side better than they do – Charlie Munger

We all hold opinions; it’s part of human nature. We hold these opinions as part of our identity. But other people hold different opinions, and conflict arises when they clash. But what if, instead of conflict, there was curiosity? Instead of shooting people down or convincing them otherwise, what if you sought understanding? People don’t just dream up their opinions; they have reasons for believing what they believe. Your job is to understand it.

Sometimes I run thought experiments in my head about how the world would look if certain nuggets of wisdom were the norm. This one is powerful. Imagine a world where for every opinion someone held, they understood the opposite viewpoint better. Imagine a world where people didn’t disagree until they were sure they had full understanding. It’s an oversimplification of all the world’s problems into this simple act of understanding, but I’m confident it would help.

The ‘normal’ way to tackle differing opinions is to show why you are right. “The other person is stupid because they don’t understand what I’m saying; therefore, I must shout louder”. A successful outcome is “I’m right and you’re wrong” instead of “we both learned something and moved forward”. One of the most demotivating things that can happen is to feel you’re not being listened to, yet this is all this line of reasoning does.

Instead, seek to understand. Approach a differing of opinion with curiosity. Let’s follow this through with a tangible example, and not to get too political with my writing, it’s just a great example. Many of you reading will have a political affiliation. Left, right, center? I don’t care for the specifics. Whichever way you lean, though, ask yourself:

What are the reasons for not supporting them?
Do you understand why you support them?
Do you support them because it’s what you believe, or because you saw an inflammatory, algorithmically vomited video on social media?

Politics is a pretty inflammatory topic, but you can apply this same thinking anywhere. In your relationships, maybe your spouse/parent/sibling wants to do something one way, and you think it would work better in a different way. Don’t defend your path. Try to understand why they think their way will work. It will surprise you how often you are wrong!

It’s a tricky thing to do, but if you can optimise for curiosity and understanding, believe me, your personal relationships will be better for it. And whilst it might not solve all the world’s problems, you can do your bit to make the world a more understanding place.

James

How To Guarantee You Don't Miss Things When The Pressure Is On?

James Eastham — Fri, 17 Jul 2026 00:00:00 GMT

No wise pilot, no matter how great his talent and experience, fails to use his checklist. - Charlie Munger

Pilots deal with a lot of pressure. Imagine having sole responsibility for a few hundred people; one mistake from you and it’s a disaster. Oh, and the thing you’re controlling, that all these people depend on, is a highly complex piece of machinery with lots of moving parts and things that could change. Dealing with a complex system, pilots prioritize checklists to make sure they never miss anything. Pilots do this when things are going well, but also when they are under pressure. Pre-take off, run the checklist. After landing, run a checklist. Something goes wrong in the air, take out the manual and run the checklist. Preparing the checklist when you aren’t under pressure, when you can think rationally, allows you to run it without thinking when things are going badly.

Life is a pretty complex thing. It comes with pressures, challenges, good times, and hard times. You might not be responsible for a few hundred people, but you are certainly responsible for yourself. The lowly checklist can teach you to be more productive, avoid mistakes, and deal with pressure. For many of you, this will sound like a chore, not something that is useful. “I’ve done thing X a hundred times before, why do I need a checklist?”. Doing the same thing repeatedly, eventually you’ll get complacent. You’ll skip steps. You’ll miss things. Sometimes when you miss things, it might be ok, until the one time it isn’t.

In the past, I’ve talked about second-order thinking. It’s dangerous if you make decisions without thinking about the downstream affects. I said in that post that I struggle myself to apply this sometimes. So why not formalise it? That’s what checklists are for. Every time you have a major decision to make, run the checklists.

Have I considered the downstream impact of this decision?
What happens if I get punched in the face?
Have I considered the entire system and the dependencies of this decision?

You have the list ready, and you run it without thinking. Now you don’t have to remember to think about the downstream impact, you take the run book and then just run it.

There is obviously some ceremony involved. In some cases, it’s not worth it. Jeff Bezos’ famous one-way and two-way doors works. If you aren’t familiar, a two-way door is an easily reversible decision. A one-way door is the opposite. For two-way door decisions, speed can matter more than considering all your options. If it doesn’t work, you reverse it. With a one-way doors though, you need to ask these kinds of questions. Run your checklist, ensure you’ve thought about the options.

James

Becoming Antifragile

James Eastham — Thu, 16 Jul 2026 00:00:00 GMT

When the variety or complexity of the environment exceeds the capacity of a system (natural or artificial) the environment will dominate and ultimately destroy that system. - Ross Ashby

Just because something is great today doesn’t mean it will be great tomorrow. It’s not always the best idea that wins, but often the one that can best deal with change. From natural selection to the big companies of old, who over time descend into obscurity. But what is it that allows something or someone to continue to be great? The things that go beyond great are anti-fragile.

Anti-fragility is a concept I first read about in Nassim Taleb’s book of the same name (the same Taleb who wrote Black Swan). In the opening part of the book, Taleb describes anti-fragility as:

Some things benefit from shocks; they thrive and grow when exposed to volatility, randomness, disorder, and stressors, and love adventure, risk, and uncertainty. Yet, in spite of the ubiquity of the phenomenon, there is no word for the exact opposite of fragile. Let us call it anti-fragile. Anti-fragility is beyond resilience or robustness. The resilient resists shocks and stays the same; the anti-fragile gets better.

This is where the opening quote of this post comes in: being able to deal with the complexity of the environment you operate within. Job one is to deal with change. To be resilient. You can’t gain from disorder if you can’t even survive it. Many of you reading this will operate in a complex environment, accepting that the world around you is constantly changing and unpredictable. You can have the best plan in the world, and then you get punched in the face. But how do you actually become anti-fragile?

I believe there are two ways, from two different areas. The first is from Taleb himself, and the idea of the barbell. The barbell strategy pairs extreme safety with extreme risk-taking, evading moderate “middle” positions. Reid Hoffman, of Netflix fame, advocates for this under a different name with his ABZ strategy. Plan A is the core pursuit, and Plan Z is the safe fallback. Applying this to investments, your plan A is a relatively ‘safe’ investment. Something that you’d be confident in, say, bonds. Then the extreme risk-taking might be something like a leveraged ETF or a crypto investment. Something with a potentially huge upside, and because you only invest a small percentage of your wealth, you minimise your risk.

The second is Barry O’Reily’s idea of residuality. Particularly the idea of stressing the system. Barry comes from a software architecture perspective, but you can apply it anywhere. Say you’ve made your investment decision, you can stress that decision against a variety of different factors. What happens if the stock market crashes? What happens if the economy slows down? What happens if aliens descend upon Earth and take over the world? You can apply the stressor and then look at what the potential solution would be. This doesn’t necessarily mean you solve for every solution, but at least you understand the risks.

This is important for risks that are more systematic. The global food system, the global financial system, energy supply, etc. Stress testing your current approach against this can help you understand how you can prepare. Investing in solar power, battery storage, and an electric car might seem like an enormous investment. But in a not impossible future where the climate is unpredictable, rolling blackouts become normal, or a wannabe dictator decides on a whim that affects the global fossil fuel supply, having guaranteed self-sufficiency seems less expensive.

Think about anti-fragility, think about if you’re ready to deal with the complexity and unpredictability of the world around you. You won’t catch all the things that could go wrong, and you might not solve it all. But at least you are aware.

James

What happens to your plans when you are punched in the face?

James Eastham — Wed, 15 Jul 2026 00:00:00 GMT

Everyone has a plan, until they get punched in the face. - Mike Tyson

Golf is known as a game of misses. What separates the best players isn’t necessarily their best shots. It’s the quality of their worst shots. When I hear Rory McIlroy hit a bad shot and shout “I’m so bad at golf” I look at my last round and wish that I had his bad shot. When you’re playing golf, though, often when picking a target you’ll look at what’s my ‘best miss’.

Imagine you’re standing behind a golf ball, needing to hit a shot. For this thought experiment, imagine you can actually strike a golf ball. There is a lake on the left and a couple of deep bunkers in front. The ‘best miss’ here is long and/or right. Obviously, the intention is still to hit the ball to the green. But knowing the best miss will help you select a better target. You might pick a club based on the distance to the back of green rather than the middle (doesn’t matter too much if you go long) and when you line up the shot you’re going to pick a spot somewhere on the right side. Take the bunkers out of play, take the water out of play, and hey you might still get the ball on the green. But if you don’t I’ll still be ok.

I don’t know about all of you, but often when facing a decision or planning my next few months of work, I think about the happy path. When I started playing golf again, I’d always aim for the the flag. Back when I was spending more time designing systems and writing code, rather than teaching people about those things, I’d design a system based on what the expected good state looks like. For anyone who has worked in software, or made plans, this is a terrible idea.

This idea of the best miss is useful for life. That’s not to say you don’t want or expect the happy path, not at all. That is still the goal, still the desired outcome. But spend time thinking about where you want to miss. If you’re designing a software system, your best miss is that your users still have a good experience even if everything behind the scenes is on fire. If you’re budgeting, your best miss is that you have a bit of space every month for if an unexpected expense comes up. If you’re writing up a strategy for your next 6 months of work, your best miss is to consider what happens if things don’t work out as expected and plan for it.

If, in life, you could strive to be like an elite golfer. Where your bad shot is still 10x better than the average Joe, over time you’ve got an excellent chance of reaching your goals. Whatever they may be.

James

What Are The Impacts Of Your Decisions? And Then What?

James Eastham — Tue, 14 Jul 2026 00:00:00 GMT

When a measure becomes a target, it ceases to be a good measure - Goodhart’s Law

I first came across the idea of Goodhart’s Law when I was working at AWS. The story I was told, which I’ve since learned has no proof, is that when the British Empire was occupying India. The local cobra population was getting too big. The British wanted to control the numbers. They offered locals reimbursement for each cobra head the locals brought in.

The Brits, thinking they were smart, assumed this would cause the cobra population to shrink as they encouraged the locals to kill cobras. Some enterprising local Indians then proceeded to breed cobras, knowing they could breed them in large numbers, kill them and make a tonne of money from the stupid British. Ethics aside, what the Brits didn’t consider was the second order impact of their decision.

I’ve missed these impacts several times in my career. Choosing to build microservices for a startup with a development team of 1 (me). Fun initially, 6 months later trying to lockstep releases across 4 different codebases… not so fun. Taking on a project I wasn’t that interested in but knew it would be good for my promotion, not thinking about the second order impact of “oh shit, that’s going to take up basically all of my free time for 3 months”.

It’s a simple thing to challenge. Ask yourself, “and then what?”. If you’re making your decision within a complex system, you still might not catch every possible second order effect. But at least you are questioning it. Some of Barry O’Reily’s work on residuality, particularly around stressors, is relevant here. If you’re thinking about an architecture for a system, think about the possible stressors that could impact it. There are no bad ideas. Throw them all into the pot. What are the things that could happen that will cause stress on the system?

Once you’ve got your list of stressors, you can then apply that to your decision. You might not solve all the problems, you might accept some of the risk. But at least you’ve acknowledged them. You’ve acknowledged that your decision might not perfect, but it’s the best you have right now.

If you layer these second order impacts into a fictional person who goes all in on AI, let’s call them Karl. Karl has spent the last 12 months using AI for everything. He’s basically automated his entire job away. He uses it for coding, for note-taking, for summarising his notes, for sending his emails, managing his inbox and calendar. Karl feels great because he’s got an AI to automate away his entire job. Now he can just play golf and get paid for it. The first order impacts are overwhelmingly positive.

Let’s stress this decision. What if, as many people predict, the per-token cost of using LLM’s skyrockets. AI providers have got to get out of their 32 billion dollar losses somehow. With an X number of times cost. Is it still viable? Karl has offloaded his ability to think to a hallucinogenic bag of words. Maybe the only way he can move forward is to pay the LLM companies. Bad for Karl, great for OpenAI.

An alternative stressor is that his employer realises Karl has just proved AI can automate away his entire job. Great for the employer, they fire Karl and keep his automations. Karl is sad. But at least he has time to play golf now? For the AI company, though, they now need to consider the aforementioned stressor because what if token costs go up X amount because someone needs to find 32 billion dollars somewhere.

Now Karl’s company is wishing they had Karl back.

There are a lot of if’s and but’s in that short narrative. The point doesn’t have to be that you believe something will happen. The point is to go wild and consider all possible outcomes. Maybe Karl decides he is happy with both of those risks and he carries on. Cool. But at least you’ve considered the longer tail of impacts.

Ask yourself, “and then what?”. And start to consider the second order impacts.

James

You're A Human, You Can Do It!

James Eastham — Mon, 13 Jul 2026 00:00:00 GMT

“Do not think that what is hard for you to master is humanly impossible; and if it is humanly possible, consider it to be within your reach.” ― Marcus Aurelius, Meditations

A few years ago, I was running ultra-marathons. 26-mile plus running races, with the furthest I competed in being 50 miles. There were a lot of reasons I started running ultra-marathons, which I won’t go into here. What I didn’t expect was the number of life lessons I took away. One of my favourites comes from Stoic philosophy.

Paraphrasing Marcus Aurelius, “If a human can do it, and you yourself are a human, then you can do it too.”. If there are examples of a human performing a task, given you are also a human, then it is within your grasp. It goes without saying, there are limits to this statement. I’m not going to will myself into being the next Usain Bolt. There are some biological limitations. When you look around at almost any human endeavour there are people of all different backgrounds and abilities. If they can do it, and they are human, then why can’t I?

I find this incredibly motivating, and it feeds the work by Carol Dweck on growth mindset. Simplifying Carol’s work, there are two types of mindsets: fixed and growth. A fixed mindset is one that believes you can’t improve, and a growth mindset is one that believes you can. I can look at Usain Bolt and say a) I’m never going to be as fast as him, why would I try or b) I’m going to train hard and get as fast as I can. If he can do it, why can’t I?

One thing that separates some of the hardest working, talented people I know from some incredible people who don’t seem to move ‘forward’. Do you look at the world and think everyone is better than me why bother trying? Or do you look around and think all these other people are human just like me, I’m going to push and become the best that I can be. This mindset has served me well, but it’s worth noting that not everyone starts from the same place.

It’s important to apply this thinking to things within your control, things outside your control are just that. Of course, societal barriers and discrimination are real, and not everything is within our control. But where you do have agency, this mindset can help push you forward. Remember, if it is humanly possible, it’s within your reach.

James

What Can Swans Teach You About Making Better Decisions?

James Eastham — Sun, 12 Jul 2026 00:00:00 GMT

I have just finished re-reading (I think for the 3rd time now) ‘The Black Swan: The Impact of the Highly Improbable’ by Nassim Taleb. It’s a dense read, but if you need to make decisions in unpredictable arenas, then it’s well worth picking up. Oh, wait… Making decisions in unpredictable arenas? That’s all of you. It’s called life.

A Black Swan is a completely unpredictable event that has outsized consequences. Human history is made up of these big unpredictable events that nobody saw coming. Things like the 2008 financial crisis, 9/11, along with many other events throughout history. It can even be something you predict but under-or overestimate the impact. You launch a startup expecting 1,000 users, and you go viral and get a 1,000,000.

If you are working in a domain that is unpredictable, and where the outcome has big consequences. You have to be aware of the Black Swans. Part of the problem is that we humans aren’t great at being ready for them. We love stories. We look back at an event, assign a narrative, and use that to rationalize why something happened. This gets dangerous when we use that to project forward. We think we are “learning” from history. But if you are dealing with something complex, there is no guarantee the future will be anything like the past. In fact, it’s probably the opposite.

What does this mean for you? First off, beware the expert. If somebody calls themselves an “expert” in a domain that is based on prediction, is complex with variability, then believe nothing they say. You can see that in real time with AI. It seems everyone has become an “expert” in AI and knows what the future is going to be. In reality, nobody knows anything. Without real, hard, empirical facts, all you have is speculation. If the “expert” omits the potential variability in their prediction, trust them even less.

An important aside, you should definitely trust the experts in domains that are more static. Doctors, surgeons, etc… If the domain isn’t unpredictable, trust them. Human brains are pretty static.

Once you become skeptical of experts, you can make more pragmatic decisions about your day-to-day life. You can take the views of ‘experts’, look at the variability in outcomes and use that variability to decide your approach. Look at the positive and negative potential outcomes, and ensure you are prepared both. The Stoic idea of premeditatio malorum feeds into this. What’s the worst thing that could happen, prepare for it.

You might not change, you might even carry on as if the positive outcome is going to happen. Being prepared for the worst makes it less of a shock if it does happen, though.

Expect much more from The Black Swan over the coming weeks.

James

What If Today Was The Last Time You Got To Do Your Favourite Thing?

James Eastham — Sat, 11 Jul 2026 00:00:00 GMT

You could leave life right now. Let that determine what you do and say and think. - Marcus Aurelius, Meditations

Memento mori - Remember you are going to die. Bit of a pessimistic thing to open with on a Saturday, I’ll grant you that. But the idea of memento mori is one of the most misunderstood pieces of Stoic philosophy and is more widely applicable than just making you question your own mortality.

In Meditations, Marcus Aurelius writes, “You could leave life right now. Let that determine what you do and say and think”. Taken at face value, it’s the grim thought that you are going to die. Underneath that is a deep appreciation for the things you do every day. Everything you do in your life, at some point you are going to do it for the last time. Eat pizza, go for a walk, play golf, go out for a run, give your parents/partner/sibling a hug. The cruel part is that you never know when the last time is going to be.

When you apply this every day, though, it is genuinely life-changing (and I don’t use that word lightly). The impermanence of everything around you increases your appreciation of it. This might be the last time you get to eat pizza; savour every single bite. This might be the last time you can hug your partner, squeeze them extra tight. You just never know.

Next time you do something you enjoy, remember it might be the last time you get to do it. Don’t get sad about it, appreciate it. Enjoy it. Savour it.

James

You Must Not Forget About The Entire System

James Eastham — Fri, 10 Jul 2026 00:00:00 GMT

‘You think that because you understand “one”, that you must therefore understand “two” because one and one make two. But you forget, that you must also understand “and”’ - Donella Meadows, Thinking in Systems

How do think? It’s not something you probably think about very often. You’re born, you go to school, you learn some stuff, you go to work, where you hopefully use the things you’ve learned to think about a problem. At least in Western culture, most things we are taught follow a model of linear thinking. The school year has a sequence, every lesson has a sequence. A and then B and then C and then D.

This is great for a lot of things. Passing exams in school, at least with the way most school systems work, requires this style of thinking. With the chaos of the modern world, there’s another type of thinking I’d argue is more important. It starts with the child asking “why”.

If you’ve spent any amount of time around a child, they are constantly asking why. Why is this like this? How does X relate to Y? What caused that to happen? It’s cute from the outside, as someone who’s about to become a father for the first time, ask me if it’s still cute in a few years. The child is trying to make connections. To work out how the different pieces of their understanding and how they relate to each other. The child, quite naturally, is thinking in systems.

Somewhere along the way, this gets lost. Linear thinking is pushed down your throat, learn this, then this, then this. You get a job, perform task X, then task Y, then task Z, and hopefully at the end get a pay rise. When you’re down in the detail this works well, but the world around us isn’t linear and definitely doesn’t follow a straight, easy to understand path.

Imagine for a moment you had to make a decision about the future of the company you work for. For arguments sake assume you work in finance. One way to make the decision is to simply make the best decision that benefits you, right now. The selfish option. And honestly, I’ve seen lots of people over the years who take this approach. But the company is made up of lots of different people, departments, customers, suppliers. There is an entire complex web of relationships that make up the system as a whole. To make something like a sensible decision, you need to have some understanding of the entire system. If you were to apply linear thinking to this task, you might think the best decision (for accounts) is to put up your prices 100x and find a cheaper supplier. $$$ for everyone! The short-term impact - profits go up and you look great for your insightful decision. But what’s the long-term impact of your decision? If you’ve forgotten about the rest of the system (in this case, customer satisfaction) your system is going to die.

Systems thinking is about trying to understand all the nodes, all the relationships, and how they work together to make the system as a whole. Famous system thinker Russel Ackoff said “A system is never the sum of it’s parts - it’s the product of their interaction”. When it comes to making decisions, think about the different pieces but more importantly think about their interactions. If you understand one piece, ask yourself what are the other pieces that I interact with and if I make change X how is that going to impact them. It’s not about slowing down, it’s about truly understanding the long term impacts of your actions.

James

Golf & The Dichotomy of Control

James Eastham — Thu, 09 Jul 2026 00:00:00 GMT

Recently I have started playing golf again. I used to play when I was younger, and after a 15 year hiatus I decided to start swinging a club again. For anyone reading who hasn’t played golf before, what you won’t realise is that golf is a sport in which you spend 50% of your time thinking it’s the greatest game ever, and the other 50% asking why some folks in Scotland decided it was a good idea to try and hit a tiny white ball 400 yards into a tiny little hole.

Practicalities of golf aside, the other thing I love about the sport is that it teaches you SO MANY life lessons. The venn diagram of things that cause problems in golf and things that cause problems in real life has a pretty decent cross over. One of my favourite, is about controlling the controllables and the false level of control that golf, and life, gives you.

In golf, all you can control is your own process. Just this morning, I hit one of my best shots of the round. The feeling off the tee, the flight of the ball through the air. Then it takes a crappy bounce and somehow ends up in a bunker (a big hole filled with sand for the uninitiated). I’d got everything right. The pre-shot, the target, the setup, the swing. And still, the outcome wasn’t the outcome that I hoped for.

I’m sure reading that, many of you can relate that to things that have happened in your life. You got everything right, and the outcome wasn’t the outcome you wanted. This is where, sometimes, the game of golf can start to differ from life though. I’m slowly realising that golf is 80% mental. Yes, you might physically swing a club but it’s what between your ears that has a huge impact on how you play.

In a round of golf, it’s tough shit. You have two options here:

Complain about being in the bunker, and the tricky shot you’ve got out of the sand. Get yourself in a mental tangle and beat yourself up OR
Understand that you got everything right. Your setup, your swing, your strike - everything was perfect. Just not the outcome you wanted.

Replace the world golf and bunker with life and challenge, and you can see that the dichotomy of control applies to life in the same way. Day to day, all you can control is yourself. As much as you might like to think otherwise, the only thing you have real agency over is yourself and your decisions. Everything else is out of your control. So be ready for it.

When things don’t go your way, ask yourself did I do everything in my power? Was I the best I can be? If you consistently answer yes to that question, I promise you the outcome will take care of itself.

James

Best Practices Can Be The Worst Decisions

James Eastham — Tue, 07 Jul 2026 00:00:00 GMT

The world is full of ‘best practices’. Things that are accepted as always being the ‘right’ way to do things across a range of different industries. And yes, whilst there are things that might nudge you in the right direction, blindly following something because everyone else is doing it misses a lot of nuance.

Before I joined AWS I did quite a bit of freelance software development, mostly with startups. At the time, I had mostly been building monolithic .NET applications (and a bit of SQL Server DBA-ing) and had started to hear about this thing called microservices. The startup I was working with at the time had reasonably good domain boundaries, and pretty obvious splits in the functionality. “What an excellent opportunity to take on this current industry best practice” I thought.

It was not!

Because yes, microservices are a good way of building software. Smaller units that run independently and are easier to understand. In principle, it sounds great. What is often missed when talking about microservices though, is the wider system. What is the environment like around the software, and how does that impact your decisions. This is what I missed.

Because at the time, I was in a development team of 1. Me. And the startup was also pivoting as the needs of it’s customers changed. What was a logical split today, caused pain later as two domains ended up needing to be smashed back together.

Ross Ashby was a psychologist and cyberneticist, and he stated “if the complexity of the environment exceeds the systems capacity to respond, the environment will always win and the system will die”. You see this play out in the natural world. The climate is changing rapidly, ecosystems can’t respond, the ecosystem dies.

Initially, you might think microservices allow you to respond to a changing environment more quickly. And yes, in some cases, they might. In many cases, they won’t. Here, the changing environment led to domains completely changing. Pieces of functionality disappearing, new ones appearing, and some joining together. Having independent codebases, independent deployments, and independent functionality hindered the system’s ability to respond.

That’s not to say microservices are bad. Not at all. The important thing to remember is to always think about the second order impacts of your decisions. What changes could happen later and how might that affect my decision today?

There’s probably a lesson for life in there as well.

James

Why I'm Writing More, And Actually Writing

James Eastham — Mon, 06 Jul 2026 00:00:00 GMT

I tried starting a blog for years, somehow it never quite caught on. I’d post intermittently for a month or two before eventually giving up. Starting a YouTube channel and creating video content scratched that itch for a while. Somehow, I just found it easier to hit record and go rather than think about what sitting down and writing.

But skills that aren’t used atrophy. And with the rise of AI to basically do everything, it seemed like a good time to revisit writing. Without AI, without any help. Just me scribbling down my thoughts. For the next 30 days, I’m loosely following the ‘Ship 30 for 30’ approach. Writing short pieces, every day, for the next 30 days.

Nobody might read it, that’s ok. Think of it as skill building, as keeping skills sharp. In much the same way I still write some code, even though and LLM could generate it for me. I for one don’t fancy offloading all my thinking, or my skills, to an AI owned by a US tech company. You shouldn’t either.

Topics may vary, in my head ideas bounce around about systems thinking. Stocism. Philsophy. Software Architecture. Maybe even some lessons I’ve learned from picking up golf again. It might be practical, it might be random musings about something I’ve read.

The point isn’t to get a million subscribers, or become the next big things on substack. The point is to keep skills sharp, to write regularly, and to think.

James

Ergodicity Why Averages Lie and Systems Break

James Eastham — Tue, 21 Apr 2026 14:02:36 GMT

Ergodicity: Why Averages Lie and Systems Break

The migration that shouldn’t have been hard

Your company has always been an Azure shop. Then a decision comes down from way above your pay grade. Your cloud provider of choice has gone out to tender. AWS won over Azure. Now it’s come down from the top: you need to move all your applications to AWS.

You’re not worried. You’ve containerised everything with Kubernetes. Kubernetes runs anywhere. That’s the whole point. You sit there, smugly I might add, knowing the migration will be easy.

It isn’t.

Almost immediately, you realise a monumental problem. The problem isn’t Kubernetes. That part works exactly as you expected. Stateless containers run anywhere. The problem is CosmosDB — Azure’s proprietary database service. CosmosDB doesn’t run on AWS.

It’s embedded across multiple applications. Swapping it out means significant application-level code changes. Not just infrastructure changes. Not the kind of thing you can abstract away with a container orchestrator. The kind of thing that means months of rework you hadn’t budgeted for.

You’d made an assumption: our architecture is portable across cloud providers. Kubernetes gave you confidence in that assumption. But Kubernetes abstracts away the compute layer. It doesn’t abstract away the data layer. Or any other layer.

Nobody had asked the question that would have surfaced this: “What happens if we need to leave Azure?” Not because leaving Azure was likely. Just to understand what the answer would be.

You hadn’t made a technical mistake. You’d made a reasoning mistake — and it has a name.

What ergodicity actually means

Ergodicity. It sounds like something from a statistics textbook you skimmed in university and never thought about again.

A coin flip is the clean example. Send 100 people out to each flip a coin once and you’ll get roughly 50 heads and 50 tails. Have one person flip a coin 100 times and you’ll get roughly the same. The path doesn’t matter. A run of 10 tails in a row doesn’t change the probability of the next flip. The group average and the individual average converge.

As humans, we think in averages. We apply this same reasoning to many real world scenarios — mean time to failure, average returns, expected value. On average, mean time to failure is 2 hours. Great. But what works for the group doesn’t always hold for the individual. The average could be fine, but one massive outlier can destroy you.

Most of the statistical tools we reach for assume ergodicity. Most real-world systems don’t satisfy it.

That distinction — between the group average and the individual path — is what we most often get wrong. Once you see it though, you start noticing it everywhere.

The place it’s easiest to see isn’t in software. It’s in money.

The pattern is everywhere

Your production architecture has more in common with a 1966 retirement portfolio than you might think.

Say you retire and plan to draw down a fixed amount each year. You’ve done the maths. 8% is the average historical return, so you use 5% as your number — conservative, sensible. The numbers work. Excellent.

If you retired in 1966, here’s what actually happened. Near-zero returns from 1966 to 1982. Then roughly 15% per year from 1982 to 1997. The average return across the full period looks fine. But you were drawing down during the flat years, before the growth arrived. By the time the good years came, there might not be enough left in the pot to benefit from them.

The group average looks healthy. It tells you nothing useful about the person who retired in 1966. The path mattered. The sequence mattered. The average didn’t protect you.

Sequence of returns risk is a well-documented phenomenon. You can mitigate it. Diversified investments, asset allocations, emergency funds, regular reviews of your portfolio.

The answer isn’t to predict the future better. It’s to build in a buffer that keeps you in the game long enough for the averages to work in your favour. A cash reserve. Shifting some of your investments into bonds. Keeping a margin of safety. Something that absorbs a bad sequence before it becomes fatal.

Back to software

So back to you, your Azure shop, and CosmosDB.

The ergodic assumption the team had made was reasonable on the surface: because we chose to use Kubernetes, we built portability as a first-class citizen. In an ergodic system that would hold. The path — which cloud you’re on — wouldn’t matter, because the architecture behaves the same regardless.

But the system was non-ergodic. The path had locked you in. CosmosDB was embedded in ways Kubernetes couldn’t abstract away. Nobody thought to ask what a migration would cost until it was already underway.

The question that should have been asked wasn’t “are we likely to leave Azure?” It was “what would it take if we did?” The answer would have surfaced the dependency. It might not have changed the decision to use CosmosDB. But wrapping it behind an interface would have made it a contained change, not a codebase-wide rework.

Migrating from CosmosDB to an AWS-managed database isn’t easy — it definitely isn’t. But at least the changes stay in the codebase.

That’s one side of it. What about when it goes well?

A team builds a system for roughly 100 active users. They choose a serverless architecture — partly for cost, partly for simplicity. The system grows. 10,000 users. Then 20,000. And because of the serverless foundation, it scales elastically.

Here’s the honest version of that story though: the resilience was accidental. The team didn’t sit down and reason explicitly about ergodicity or future states. They got lucky. The early architectural choice happened to be path-independent in exactly the right way.

Whilst you can get the right answer without thinking formally about ergodicity, you’re playing a luck game. Sometimes you might come good. If you were the investor from 1982 to 1997 you’d think you were a genius. You just happened to be in the right place at the right time.

Using ergodicity as a lens gives you a deliberate path to better outcomes. Instead of hoping your early choices happen to age well, you ask the question upfront: am I locking in a path here? What does this system look like if usage grows by 100x? What does it look like if we need to change providers? What does it look like if we discover Godzilla is real, and he comes storming out of the sea and stomps all over us-east-1.

This isn’t about predicting every possible future. It’s about knowing where your path dependencies live before they surface in an emergency.

The intellectual lineage

This isn’t a new idea. It’s been formalised — and that formalisation matters.

Barry O’Reilly’s work in Residuality Theory overlaps closely with this framing. I’d read both of his books — Residuality Theory and The Architect’s Paradox — and understood the ideas. But a specific post O’Reilly made on LinkedIn — explicitly connecting ergodicity to software architecture — made the concept click.

The central move in Residuality Theory is this: map the possible future states of a system, not just the expected state. Ask “what if…” systematically rather than optimising only for the happy path. Instead of asking “will my architecture handle the average case?”, ask “what does my architecture look like across the distribution of possible futures?”

If you want to go deeper, Barry’s work is the place to start.

Which brings it down to a single question you can ask before any design decision.

One question

Have I thought about whether my architecture has hidden dependencies that would make future changes expensive?

That is the question.

The follow-up is just as useful: how will this system evolve over time, and have I thought through the possible future states?

The CosmosDB team didn’t need to predict the contract change. They needed to know that CosmosDB was a path dependency — so that if a migration was ever required, it wasn’t a surprise. The serverless team got lucky. Asking the question deliberately means you don’t have to rely on luck.

The goal isn’t to solve every potential future problem upfront. It’s knowing where the constraints live. Where the assumptions are baked in. Where a path dependency could cost you months when the future diverges from the average. Ask the questions, briefly capture the answers. A paragraph in a decision log, a note in the ADR (architecture decision record) and work from there. You need a record the question was asked and what the answer was at the time.

Most of the time the answer will be “this is fine.” But occasionally you’ll catch something. And when you do, it’s far more valuable than discovering it three months into a full codebase refactor.

Design Thinking and the Question Serverless Should Be Asking

James Eastham — Sun, 12 Apr 2026 10:58:36 GMT

“At the heart of any good story is a central narrative about the way an idea satisfies a need in some powerful way.”

“What’s the story of serverless?” What need does it satisfy?” Not the technical need. The human need. Why does it resonate so well with some people and not so well with others?

I believe that serverless is the #1 way for anyone to get started building something new. Whether you believe the hype or the haters, it is irrefutable that if you have an idea and you want to get it in front of users as quickly as possible, serverless is the way. By using the word serverless here, I am referring to all managed services, including things like Azure Container Apps, Amazon ECS/Fargate, Google Cloud Run, not just functions.

When most people talk about serverless, me included, they lead with the technical stuff. No servers to manage. Pay per invocation. Auto-scaling. All true, all useful. But none of that is a story. The emotional connection just isn’t there.

Maybe the Story Is Wrong?

Design thinking has this concept of always starting with a human-centred question. Not “how do we build a better voting machine,” but “how might we enhance the democratic experience.” The focus shifts from the noun to the verb. From the thing to the feeling.

So what does that look like for serverless?

How might we help someone ship their brand new idea in a way that minimises personal risk?
How might we enable anyone, anywhere, to bring their idea to life?
How might we help developers prototype and pivot without getting bogged down in infrastructure?

Those questions feel very different to “how to run your app in a way that means it will scale automatically, and mean you don’t need to care about servers”. We are all humans talking to other humans, and as humans we love stories.

Homo Sapiens by Yuval Noah Harari is a book I refer to often. And one of the core ideas in that book is the importance of stories. Stories are fundamental to the success of the human species. From the obvious stories our ancestors will have told “don’t go down to the watering hold, there’s a lion that will eat you” to the not so obvious example of things like the modern financial system. The modern financial system, like many things in our lives, are built on a shared story. A shared belief. A shared idea.

It’s stories and shared beliefs that allow us to operate together at massive scales. Whether it’s the financial system, the company you work for, your nationality, your religion, your political leaning or anything in between. Our ability to connect, and to share stories, is what sets us apart as a species.

Yet all too often, not only in tech, we start with the technical details not the human-ness.

The Container Story is Already Written

Looking away from serverless for a moment, containerisation and Kubernetes in particular have a compelling story, even if it’s not always the right one.

Build your application using a well-documented, well-understood web framework. Package it into a container, which everyone knows and understands. Ship it to an orchestrator that everyone in the industry says is the way to run large scale applications.

Nobody got fired for buying IBM, right? And no CTO got fired for doing what everyone else is doing… Which in this case is deploying the app to Kubernetes. The safe choice maps to the existing mental models which makes the story easier to tell.

That’s a genuinely hard thing to compete with when you’re trying to get someone to shift their thinking toward a new paradigm. The serverless story needs to be just as easy to tell. And right now, I’m not sure it always is.

Prototyping as a First-Class Citizen

One of the other ideas from design thinking that really resonated with me was this: “A vibrant design-thinking culture will encourage prototyping — quick, cheap and dirty — as part of the creative process, not as a way of validating finished ideas.”

That’s exactly what serverless should be. There is no better tool available for moving quickly. You have an idea, you write some code that focuses purely on the business logic, you deploy it, and it scales when it needs to. No infrastructure setup beforehand. No cluster to configure. Just your idea, running in the cloud.

That’s a story worth telling. An early career developer who wants to learn fast. A founder with a weekend and an idea. An experienced developer who wants to prototype something without spinning up a bunch of infrastructure first. You’ve just vibe coded something into existence, now what? It certainly isn’t go and spin up an EKS cluster, provision an ingress controller and write a Helm chart.

Every single one of us now has the power to go from idea -> working product -> running in production in an afternoon. The aha moment of having the app created is great, and serverless technologies allow you to productionise it. But also to do that in a way that has low financial risk whilst also being scaleable if you do happen to go viral.

The fundamental starting point of most applications is a web app and an API. What does the customer journey of building and shipping a web app on serverless look like compared to doing the same thing on a container orchestrator? I think if you map those two journeys out honestly, the serverless path is significantly shorter and significantly less painful for the person just trying to get their idea in front of real users.

Granted, if you have an entire platform team around you to build and run the container orchestrator for you then great. The journey to shipping the POC is probably equally as simple in that case. I once spoke with someone at a conference who said “My company uses Kubernetes, but all I do to is push a container image, the rest just happens and it scales automatically. Does that mean I’m serverless?”.

Yes, yes it does my friend. Not everyone is that lucky though.

To be clear, I’m not suggesting that you can’t run proof of concepts on Kubernetes. And equally, I know of hundreds of companies running large scale production workloads purely on AWS Lambda. As I’ve argued before, framing this as ‘serverless vs. containers’ misses the point. The real question is: What’s the fastest path to delivering value to users? https://blog.jameseastham.co.uk/blog/on-factors-of-modern-compute/.

Maybe you’re reading this an you’ve heard all about serverless. And you’ve heard the naysayers. Frankly, some of them are right. Yes, your serverless functions will have cold starts (they probably aren’t as bad as you think). Yes, you may be locked into a specific vendors managed service (thinking about code design prevents this. Kubernetes has lock in too ya’know).

But if you’re trying to get anything you build in front of actual real people, in a way that will scale alongside your business with next to 0 effort on your part, that’s a worthwhile trade off to make.

Shifting the Focus Outward

A lot of the conversation in the platform engineering and infrastructure space is inward-facing. How do we run our systems better? How do we manage our clusters? How do we reduce operational overhead for our internal teams?

All valid questions. But design thinking pushes you to zoom out. To think about the organisation and their customers. To think about the users of the software that teams are building.

What are the benefits to them? Does your user care whether their request was handled by a Lambda function or a container on a Kubernetes node? Absolutely not. They care that it was fast, reliable, and that the product keeps improving to meet their changing needs.

Spending a huge amount of time and money on platform engineering is not innovating. It’s not human-centred. It’s not solving the needs of your users. There’s a version of it that enables teams to move faster, sure. But a lot of the time it’s just undifferentiated heavy lifting dressed up as strategy.

Eventually, maybe Kubernetes is on the horizon. If you start with a serverless function, and 18 months down the line you realise your workloads better suits a long running container then amazing. The answer still isn’t Kubernetes, but maybe you start to head down that road.

At each specific moment in time you made a pragmatic architectural choice about how you want to get your application in front of users in the best possible way.

It’s another reason why I think the ability to adapt your system over time as it’s needs change, is the most important consideration for any software system. Things will change, it’s inevitable. Have you designed your system to be able to deal with that change? The #1 problem I see with AI generated code is that, without careful prompting, evolvability goes out the window.

A Few Practical Things I’ve Taken From This

Design thinking has some really simple habits that I think are worth stealing regardless of what you’re working on. Be it serverless, be it something else in tech, or not in tech at all.

Stop regularly and ask yourself five whys. Why do I do this thing this way? Why has this always been the default? It’s a surprisingly effective way to challenge assumptions that have just been sitting there unchallenged.

Sketch things out. Not as polished diagrams, just rough sketches. Customer journeys, system flows, the path from idea to production. Drawing forces clarity in a way that writing often doesn’t.

And reframe your questions as verbs. Don’t ask how to build a better serverless platform. Ask how you might enhance a developer’s ability to quickly prototype and ship. Don’t ask how to improve adoption metrics. Ask how you might make the getting started experience so good that people just want to rave about it to their colleagues.

That last one feels like the real opportunity to me. If anyone who has a positive experience with serverless can communicate its essential elements in a way that encourages others to try it — that’s how you build momentum. That’s the story worth telling.

Next time you’re talking about serverless, or in fact, any decision. Think about the human-centered side of things. Think less about the technology, and more about how is this going to positively (or negatively) impact another human being. That will inevitable lead you to building better software.

The Internet Gave Us Our Tribe and Took Away Our Neighbours

James Eastham — Sun, 29 Mar 2026 10:58:36 GMT

The internet promised to connect us. Instead, it’s tearing us apart!

The internet is one of the greatest achievements of humankind. A massively distributed and decentralized system with no single ‘owner’. The internet connected people across the world, gave a platform to voices your would never hear otherwise, and made information available to anyone with a phone signal.

I recently read Deep Economy by Bill McKibben. A common theme in the book is to push for engagement with your local community and the people around you. Studies have shown that almost every daily activity is better when engaging in it alongside other people, and what better to engage with than the ones close to you.

You could argue that the internet has made this social interaction easier. With the push of a button I can connect with anyone, basically anywhere in the world. This is a wonderful thing. But it’s also a driver of the increase in division and the eroding of our shared humanity.

Before the internet, if you wanted to socialise, to exchange ideas, to just be around other people, you went to your local pub. Or a market. Or a community hall. And whoever showed up, showed up. Your neighbour with completely different political views. The guy from the other side of town with a totally different background. The older woman who’d lived through things you couldn’t imagine.

The people you grew up around, the circles you were in. They were just there. You had very little control over that. This was a double edged sword. On one hand, you were forced to spend time around people you might not necessarily agree with or even like. Diverse ideas spark better ideas and solutions. On the other hand, your gender/class/race/sexuality might have meant you meant certain circles or ways of thinking were inaccessible to you. This was, and still is, something we all collectively need to push against.

Just think about how you spend your time online now though. I know how I spend mine. I follow people who think roughly like I do. I read news/content that broadly aligns with my interests and worldview. The algorithms quietly learn what keeps me engaged, and also what makes me angry, and serve me more of the same. And slowly, without really noticing, I end up in a little bubble of people who are a lot like me.

Modern media (I’m grouping social and traditional together there) has moved to place that prioiritzes eyeballs/views/clicks over facts.

The False Promise of Connection

I’m not suggesting times before the internet were perfect. In Deep Economy Bill McKibben discusses talk radio. The whole premise of talk radio stations, and by extension all modern media, is that you can go an entire day without hearing a single opinion that you disagree with.

Whilst talk radio stations might have this problem, at least there is an element of human editorial oversight. If you pick any given newspaper/radio station/podcast you can quickly asscertain what the subject matter is going to be. What the hosts views are. Whether it aligns to you. Whether it seems factual, sensationalist, or anything in between.

Social media changes this dynamic completely. Now, there is a black box algorithm owned by a billionaire somewhere determining what is ‘valuable’ to you. You don’t know why you’re seeing this content. You just find yourself infinitely scrolling through a feed of content that someone else has decided that you will. Or, in a lot of cases, something that will make you angry.

Only hearing ideas from people who think like you is self-affirming. Equally only hearing ideas that make you anrgy entrenches you in your beliefs.

When you focus exclusively on these two things. Re-affirming or entrehcing your beliefts, you lose something amazing. The accidental collision of different perspectives. It’s a joyous moment when you hear someone say something you’d never have thought of, from a life you’ve never lived, and it genuinely changes how you see things.

The way you view the world will be defined by your experiences. And your experiences are different to mine. If you encounter someone you disagree with, take a beat to ask yourself if you’d lived that persons exact experiences maybe would you find yourself thinking like them as well?

A difference of opinion shouldn’t be about convincing the other person how right you are, it should be about being interested in each others ideas enough to both come out of the experience ‘better’.

That used to happen almost by default, simply by being around enough people in face to face settings. Now, I’m not so sure. If you disagree with someone on the internet how do you deal with it? Quietely get angry, feel it rise up inside you? Maybe you angrily post on social media indirectly calling them out. Maybe you engage directly, reply to the person in question explaining how stupid they are and why they are wrong. I’ve been guilt of all of those things in the past.

Face to face is a different situation. Remove the keyboard, put people face to face and humanity has a chance of shining through. Eye contact triggers oxytocin, the ‘trust hormone.’ A handshake or a shared laugh disarms defensiveness. Online, we lack these cues—so we default to outrage. It doesn’t always. I’ve seen many a shouting match (and sometimes a physical match) from a differing in opinion. But keyboard warriors are real, and I’m sure you’ve all seen things on the internet and thought I don’t think they’d say that if they were in person.

Aim For 1% better

How many of you actively seek out differing opinions to your own? This doesn’t have to be focused on the bigger challenges of humanity. It could be related to your day job, your relationship. Even what might seem like a micro decision, there is still value in understanding another persons perspective.

Why would you do that? It’s uncomfortable. It’s easier to stay in the feed that feels familiar.

Social media companies, and the media writ large employ hordes of people to grab your attention. Media is built to maximise engagement, and engagement goes up when you show people things that re-affirms their beliefs. Whether that’s through something they agree with, or something that makes them so angry it further entrenches what they believe.

Does this accidental diversity of ideas still happens anywhere? Even if it does, does it happen in a constructive way. Maybe at work, if you’re lucky enough to be around people with genuinely different backgrounds and opinions. Maybe in person, depending on who’s in your social circles. Maybe in the comments section of the internet, although more often than not that descends into a shouting match rather than a genuine conversation.

The internet gave us the ability to find our people, which is wonderful. But what did we give up in return. I challenge you all to ask yourself one simple question, how are you going to find some content that you disagree with and engage with it in a way where you seek to understand? Not where you seek to dismiss.

On Flow, Focus & Recovery

James Eastham — Sun, 15 Mar 2026 14:00:00 GMT

I’ve been thinking a lot lately about the relationship between doing deep, focused work and recovering from it properly. On the surface they seem like opposites. Work hard, then rest. Simple enough. But the more I dig into it, the more I think they’re actually part of the same loop, and getting either one wrong breaks the whole thing.

This has become more important in the age of AI driven development. AI fatigure is real, Siddhant Khare covered it on his blog. We are more productive than ever, shipping faster than ever, yet getting more exhausted. I’m certainly feeling it, I’m sure many of you reading are as well.

The Flow Cycle

Flow is something I’ve been interested in for a long time. Not just the state itself, that feeling of being completely locked in where hours pass and you barely notice, but the actual cycle that surrounds it.

What I’ve come to realise is that flow doesn’t just happen. You can’t sit down, open your laptop and will yourself into it. There’s a process. First, you have to struggle. You need to be working on something that sits right at the edge of your ability. Just frustrating enough that your brain is fully engaged, but not so far beyond you that you shut down completely. That tension is the entry point.

Then comes the release. You step away. Go for a walk, make a coffee, do something completely unrelated. This is where I think a lot of people (myself included) get it wrong. The release feels like giving up, like you’re losing momentum. But that discomfort of stepping away is actually part of the process. Your brain needs that space to work through the problem without you consciously forcing it.

After the release, you deploy whatever triggers help you drop back into focus. Then you’re back in it. And then the part that genuinely surprised me when I started paying attention to it: recovery. Proper, intentional recovery.

This cycle is well studied, Steven Kotler has wrote some fantastic books on the topic. In our current AI driven workflows, you could argue the struggle never happens and you never stop because the “just one more prompt” effect is real.

Recovery Isn’t Passive

Here’s the thing I keep coming back to. Recovery sounds easy. It sounds like doing nothing. But in practice, it’s probably the hardest part of the whole cycle to actually commit to.

When you could just write one more prompt, vibe one more feature, it’s very tricky.

I have a restorative yoga practice I try to do as part of my daily shutdown. It’s maybe 20 minutes. Nothing dramatic. But the number of times I’ve let it slip because I worked late, or because I told myself I’d just finish this one thing first, is embarrassing. And those are exactly the days I need it most.

It’s counterintuitive, but recovery actually requires grit. It requires the same commitment as the work itself. When things get hard, when the day runs over, when you’re tired and just want to sit on the sofa, the recovery protocols are the first things to go. But they’re the things that make the next day’s deep work possible.

If you skip recovery consistently, you’re not just tired. You’re slowly degrading your ability to get into flow at all. The cycle breaks.

Deep Work Is the Point

Underneath all of this is a question I keep asking myself. What am I actually trying to focus on?

For me it comes down to a few things. Refining my craft, which means writing code, writing articles, producing content. Constant learning, reading a lot, actively trying to challenge my own views rather than just reinforcing them. Sharing intentionally. And being genuinely present during the things that matter.

None of those are things I can do whilst half-distracted. You can’t refine a craft while checking Slack every ten minutes. You can’t really learn something deeply if you’re skimming it between meetings. You can’t be a productive meeting participant if you’re vibing with Claude on the side. These things demand focus, and focus is a finite resource that needs protecting.

I’ll be the first to admit I’m guilt of all these things.

My normal working day is full of interruptions. That’s just the reality of working in a team, in a company, with other humans who need things from me. So the deep work, the stuff that actually moves the needle on my own growth, has to happen outside of that. Early mornings, evenings, weekends. Time I carve out specifically because it’s mine to control.

The question I try to keep asking is: is this the most important thing I could be doing right now? It’s a simple filter but it’s ruthless when you actually apply it honestly.

The Loop

What I’ve ended up with is something like this. Deep, focused work requires flow. Flow requires struggle, release and the right triggers. And the whole thing is only sustainable with genuine recovery. Not passive, accidental recovery. Intentional, committed, even-when-you-don’t-want-to recovery.

Grit isn’t just about pushing harder. Sometimes it’s about having the discipline to stop, to rest, to do the 20 minutes of yoga even though you’re tired and it’s late and you just want to be done with the day.

The cycle only works if you respect every part of it.

Why .NET Devs Should Stop Writing Lambda YAML Forever

James Eastham — Tue, 10 Mar 2026 16:01:00 GMT

Infrastructure Shouldn’t Be This Hard

Building out a serverless .NET application on AWS can feel like pushing a piano upstairs. Each step – creating Lambda functions, setting up API Gateway, defining DynamoDB tables, gluing it all together with IAM – can leave even experienced engineers tired. And don’t get me started on uploading zipped project files and hacking together permissions using YAML or console clicks. That isn’t just tedious, it’s a asking for errors and inconsistency.

I’ve felt the sting of “isn’t there a better way?” far too often. There’s a reason we moved from hand-crafting EC2 instances to infrastructure-as-code: serverless should be even easier, right?

Time for a reality check: it doesn’t have to be this painful. It’s 2026 after all, should we really be writing YAML anymore?

A Better Way: CDK for .NET Developers

AWS’s CDK – Cloud Development Kit – is a quiet revolution for anyone serious about system architecture. Forget scribbling YAML or clicking through AWS’s console. With CDK, you define all your infrastructure and wiring in a familiar programming language. For .NET devs? Say hello to C#. Write actual code, leverage your IDE, and get the power of compile-time checks and IntelliSense. Deploying your infrastructure starts to feel like writing your app.

Here’s how that looks in practice:

// Install the CDK CLI first
npm install -g aws-cdk

// Scaffold a new CDK project in C#:
cdk init app --language=csharp

You get a Program.cs (entry point), and a stack class – essentially your blueprint for AWS resources. Everything lives in code, so you can refactor, reuse, and share custom constructs.

Want an SQS queue? Spin up an SNS topic? Just instantiate the relevant class:

var queue = new Amazon.CDK.AWS.SQS.Queue(this, "MyQueue", new QueueProps());
var topic = new Amazon.CDK.AWS.SNS.Topic(this, "MyTopic", new TopicProps());

Everything’s named intuitively. Your editor’s autocomplete all but eliminates time spent hunting documentation.

Compiling .NET Lambda: No More Zip Files

Let’s tackle .NET Lambdas. Traditionally, you’re stuck compiling your app, zipping the output, uploading it manually (or with some awkward CLI script), and then crossing your fingers for a successful deployment. CDK brings sanity back: add Amazon.CDK.AWS.Lambda.NET, and you can point the resource at your project directory. That’s it.

var myFunction = new DotnetFunction(this, "ItemsApiFunction", new DotnetFunctionProps {
    ProjectDir = "../src/ItemsApi",
    // Specify handler, runtime, memory as needed
});

When CDK deploys, it handles compilation, packaging, and uploads automatically. No more manual zip files, no brittle scripts, and no wasted time.

Even better: the Lambda Annotations framework (for .NET) generates helpful serverless.template files so you can easily grab handler strings. It’s not just about ease–it’s about eliminating those subtle, recurring errors that make serverless adoption a headache.

Permissions: Grant, Don’t Guess

IAM is a perennial pain point. Previously, you’d hack together policies, pray you had the right actions, and hope it all worked. CDK lets you grant specific permissions directly in code:

table.GrantReadWrite(myFunction); // DynamoDB table permission
myFunction.GrantInvoke(apiGateway); // Lambda invocation permission
queue.GrantSendMessages(myFunction); // SQS publish permission

This declarative approach is not just concise—it’s robust. The abstraction more closely matches how architects think about relationships between components. No more YAML incantations or black-box IAM troubleshooting.

Wiring Up the API

API Gateway integrations are a notorious rabbit hole: create endpoints, wire up integrations, assign permissions, and then (finally) add routes. With CDK, you stay inside the world of C#, and, crucially, you can refactor and abstract repeated patterns.

For example, to define an HTTP API with Lambda integrations:

var api = new HttpApi(this, "ItemsApi", new HttpApiProps());
var integration = new LambdaProxyIntegration({ Handler = myFunction });
api.AddRoutes(new AddRoutesProps {
    Path = "/items",
    Method = HttpMethod.POST,
    Integration = integration
});

You define API stages, integrations, and set permissions, all as code. It’s all composable, testable, and accessible to your whole team.

Custom Constructs: Codify Best Practices

This is where things get interesting. Infrastructure code is finally just code–so you can leverage inheritance, composition, and abstraction. Imagine custom constructs that centralize repeated patterns: dead-letter queue wiring, API-Lambda integration scaffolding, security hardening. You can create custom classes and libraries, push them to your internal registry, and empower your team to reuse solid foundations.

Say you had to make ten APIs with similar configuration. Previously, you’d copy-paste YAML, risk error, and throw consistency out the window. With CDK:

public class ApiDotnetFunction : DotnetFunction {

public ApiDotnetFunction(ApiProps props) : base(props) {
    // Wiring up integration, routes, permissions centralised here
}

}

Now, deploy a new Lambda with API integration by instantiating a single construct. It’s as DRY (and as powerful) as any application code. Need loops and conditionals? Use them. Need custom logic for config? Go wild. Best practice becomes encapsulated, not bolted-on.

// Lambda Function - CDK builds and packages automatically
var getAllItemsFunction = new ApiDotnetFunction(this, "ItemsFunction", new ApiDotnetFunctionProps()
{
    ProjectDir = "../src/ItemsApi",
    Handler = "ItemsApi::ItemsApi.Functions_GetAllItems_Generated::GetAllItems",
    Runtime = Runtime.DOTNET_10,
    MemorySize = 512,
    Timeout = Duration.Seconds(30),
    Environment = new Dictionary<string, string>
    {
        { "TABLE_NAME", table.TableName }
    },
    Api = api,
    RouteKey = "GET /items",
    Region = this.Region,
    Account = this.Account
});

Nothing Else to Learn

The heart of modern serverless engineering is about minimizing friction. CDK for .NET lets you write both your application and infrastructure code in C#. No switch to YAML, Terraform syntax, or ever more brittle scripting. You can leverage your existing skill set, move fast, and make fewer mistakes–all while unlocking the expressive power of typed languages.

Practical Takeaway

Serverless was supposed to abstract away the pain. Yet until recently, for .NET devs, deploying meaningful apps felt half-baked. The AWS CDK transforms that landscape. By merging infrastructure and application logic into code, you get the harmony you always wanted: deep reuse, expressive abstractions, and a consistent deployment model.

Try out CDK in your next project. Refactor construct patterns. Push your infra classes to your internal registry. Treat infra code as engineering, not bureaucracy. Move your teams up the stairs and keep the music playing.

Curious for hands-on visuals? Check out the video version of this blog post.

You're Crazy To Think You Can Predict Your Next Outage!

James Eastham — Sun, 01 Mar 2026 16:02:00 GMT

You’re probably terrible at predicting the next outage in your system!

Not because you’re a bad engineer, I’m guilty of it too. It’s just how we humans work. You’ll have a mental model of what could go wrong based on what has gone wrong. If a particular type of failure has never hit your system, the odds of you proactively designing against it are pretty slim. You don’t know what you don’t know.

This has some pretty uncomfortable implications for how you might think about reliability. And even more implications as you move further away from writing individual lines of code yourself, into the world of LLM’s software for you.

The Monitoring Trap

The classic response to “how do we know when things go wrong?” is to add more monitoring. Dashboards, alerts, thresholds. CPU over 80%? Alert. Error rate above 1%? Alert. P99 latency over 500ms? Alert.

Imagine you have an outage, your pager goes off at 2am because something’s wrong. You sleepily stumble out of bed, and open up your observability platform of choice. You see that the production RDS instance has ran out of disk space. Annoying, but a relatively easy fix. You bundle more memory into the instance (gotta love the cloud) and then go back to bed.

Hopefully, the next day you run some kind of post-mortem. And one of the probable outcomes of that to create some kind of monitor watching the disk space. If the disk space is over 80%, notify somebody. Problem solved.

This kind of stuff has value. But it’s not observability. There’s a really important distinction between monitoring and observability, and conflating the two is where you’re going to get stuck.

Monitoring tells you when something you already anticipated is happening. You set a threshold because you know that metric matters. You’ve seen it spike before. You know what it means.

Observability is different. Observability is the ability to sit down at any point in time, ask any question about your system, and actually get an answer. Slice the data in ways you hadn’t thought of before. Follow breadcrumbs back through a chain of events to find the actual root cause, not just the symptom.

Logs, metrics and traces (and to some extent profiling) aren’t observability. They’re the raw material. What you do with them, and how flexibly you can query and explore them, that’s where observability actually lives.

The reason this matters so much comes back to that first point. If you can only answer questions you thought to ask in advance, you’re stuck inside your own mental model of how the system can fail. And your mental model has gaps. It always does.

Metrics for Systems, Observability for Software

One thing that’s helped me think about this more clearly is drawing a line between the components my application runs on top of, and the application itself.

Infrastructure has a fixed-ish set of constraints and problems. Redis starts to fall over under certain memory pressure. Kafka consumer lag grows when throughput spikes. EC2 instances hit CPU/memory limits. These failure modes are well understood, well documented, and relatively stable. Metrics work brilliantly here. Set your thresholds, configure your auto-scaling, and you’re largely covered.

If you’re running something like Nginx and want to check the traffic running through it, logs are about your only option. Stream the logs from Nginx and do something with them. The same goes for Amazon CloudTrail logs. Many infrastructure services output logs in some shape or form, and that’s really helpful.

The software your (or Claude) writes is a completely different story. Application code changes constantly. A new feature ships, a refactor lands, a dependency gets bumped. Any metric threshold you defined last month might be completely meaningless today because the code it was measuring looks nothing like it did when you set it.

These changes are also being made by several people. Whether you’re running a lean, vibe-coded startup or part of an enormous development organisation. There are multiple parties contributing to the code base. And all those parties will probably have a slightly different mental model of the system, and a very different set of previous experiences. I use the word parties intentionally, I’d include coding assistants in that bucket as well.

On top of all of that, an individual software component runs as part of a larger system. The complexity of a system emerges from the interactions between the parts, not the individual parts themselves. And when you truly consider the entire system, the software components, the developers, coding assistants, the product managers, executive leadership, executive leaders using LLM’s to make decisions, users of he software, prospects of the software, the cloud provider you use to host everything and everything else.

All of which is changing constantly. Not so simple is it.

This is where the ability to ask arbitrary questions of your system becomes genuinely useful. Instead of asking “is this metric above the threshold I set six weeks ago?”, you can ask “how does this system behave right now compared to before this deployment?”. That kind of comparison, made freely and without having to pre-define everything, is what lets you keep up with software that never stops changing.

Designing for Change in the First Place

Which brings me to something I think gets undervalued in many architectural conversations: the rate of change itself is something worth designing for. I’d argue evolvability is the most important ‘-ility’.

Speed of change is a competitive advantage. If you can ship safely, iterate quickly, and roll back confidently you have an advantage over other teams that are scared of their own codebase. Some of what might scare you about your codebase is accidental complexity that’s crept in over time.

Or the fact the whole thing has been vibe-coded with little to no thoughts about software quality.

One area where I see this play out is in how teams think about reuse. There’s a real temptation to build reusable components early. It feels responsible. It feels like good engineering. But I’ve been burned by this enough times to be pretty cautious about it now.

Build the thing first. Use it in one place. Understand it properly. Then think about reuse. Premature abstraction is its own kind of complexity, and unpicking a badly designed abstraction that’s been reused in fifteen places is a painful way to spend a sprint.

The exception I’d make here is around observability and schema design. Getting those right early, and building them consistently, tends to pay for itself pretty quickly.

Making Illegal States Impossible

One of my favourite ways to reduce the surface area for unexpected failures is to make invalid states literally impossible to represent in code.

Take something like an email address. In a lot of systems, an email can be verified or unverified, and that distinction gets tracked with a boolean flag somewhere. Which means somewhere in the codebase, some function that should only operate on verified emails is probably trusting that the caller checked the flag. Or maybe it isn’t. Hard to tell.

A better approach is to make these two things separate types. UnverifiedEmail and VerifiedEmail. The constructor for VerifiedEmail is private, and the only way to get one is through a verification service that you actually trust. Now any function that needs a verified email just takes a VerifiedEmail as a parameter. The compiler enforces the rule. You can’t pass an unverified email in by accident because the types won’t let you.

This is the kind of thing that sounds like a small detail but compounds nicely across a whole codebase. Fewer implicit assumptions. Fewer “I assumed the caller had already checked that” bugs. Fewer surprises at 2am.

These are also the kinds of things that LLM written code will never do without explicit prompting, they just don’t think that way.

You Can’t Predict Everything, So Stop Trying

Coming back to where I started. You can’t predict every failure. You can’t anticipate every question you’ll want to ask about your system when something goes wrong at 2am. And you can’t design away every edge case before it surfaces in production.

There’s no way you could even see all the different edges, let alone track them.

What you can do is build systems that are easier to understand when things go wrong. Invest in real observability so you can ask questions you haven’t thought of yet. Design your application code so that change is cheap and safe. Push complexity down to the infrastructure layer where failure modes are predictable, and keep your application layer as clean and queryable as possible.

The goal isn’t a system that never breaks. It’s a system where, when it does break, you can actually figure out why.

Conquer .NET Lambda Performance with these 3 steps

James Eastham — Tue, 24 Feb 2026 16:01:00 GMT

Every engineer working in serverless has felt the sting of a cold start at least once. There you are, full of anticipation, hitting an API endpoint only to stare at your screen, twiddling your thumbs, wondering if AWS Lambda has gone on holiday or something. When the response finally arrives, it’s often paired with frustration, confusion, and a twinge of embarrassment if there was a team member watching over your shoulder. For us .NET, and Java, developers this problem can be particularly pronounced.

I know of entire organizations who have completely re-written their whole stack because of this challenge.

Cold starts aren’t just a curiosity—they’re an architectural bottleneck, a design constraint, and a UX reality all at once. But like most “impossible” problems in distributed systems, once you peel away the mystery, a set of actionable strategies emerges. Let’s dig into why .NET Lambda cold starts loom so large, and what you can do—today—to whittle those waits away.

Memory Allocation: The Undervalued Lever

There’s a certain badge of honor in minimizing your Lambda function’s memory allocation. It feels frugal, architectural, green. But here’s the kicker: your memory setting isn’t just about RAM. AWS Lambda ties your CPU allocation directly to how much memory you request. Pick the smallest memory footprint, and you’ll get a low powered CPU as well.

That 128 MB Lambda function? It might be cheap, but it’s also starved for compute. That translates, under the covers, to lethargic cold starts—exactly the pain you see when the clock ticks and nothing seems to happen.

Start with a gigabyte. I know, I know, it feels excessive, but you’ve got to think in Lambda’s logic. Jumping from 128 MB to 1 GB can transform a 9.2 second build duration into a 1.2 second response for .NET Lambda. Even at a glance, it’s clear that frugality can cost you more—both in user patience and, depending on your workload, in real money.

Still skeptical or want to optimize further? Try the open-source AWS Lambda Power Tuning tool. Let it hammer your function with various memory allocations. More often than not, you’ll see a sweet spot emerge where performance improves and costs diminish beyond what intuition suggests.

Initialization: The Secret to Fast Successive Invocations

The first invoke pays for all your mistakes. SDK initialization, secrets fetching, client construction—pack those all into your handler and you’ll relive cold start pain every single time. Systems thinking means designing for the lifecycle Lambda gives you: exploitation of “execution environments” that stick around for future requests.

In plain speak: anything that can be set up once and reused across invocations should be. In .NET, that means initializing heavy objects (like AWS SDK clients or configuration data) in the constructor or a startup class, rather than every single time the handler is called. .NET’s Lambda Annotations Framework is brilliant for this pattern. As well as simplifying your handlers code, it also brings in familiar dependency injection principles.

The mechanics are simple. Suppose you’re using the modern Lambda Annotations-based model:

public class MyLambdaFunction(IAmazonDynamoDB ddbClient, HttpClient httpClient, AppConfig config)
{
    [LambdaFunction]
    public async Task<string> Handler(APIGatewayProxyRequest request)
    {
        // Handler code here
    }

}

You’d have a Startup.cs file that looked a little bit something like this:

[LambdaStartup]
public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddSingleton<IAmazonDynamoDB>(new AmazonDynamoDBClient());
        services.AddHttpClient();

        var apiKey = FetchSecretSync();

        services.AddSingleton(new AppConfig()
        {
            ApiKey = apiKey,
            TableName = Environment.GetEnvironmentVariable("TABLE_NAME") ?? "Products"
        });
    }

    private string FetchSecretSync()
    {
        try
        {
            using var secretsClient = new AmazonSecretsManagerClient();
            var secretResponse = secretsClient.GetSecretValueAsync(new GetSecretValueRequest
            {
                SecretId = Environment.GetEnvironmentVariable("SECRET_NAME") ?? "my-api-key"
            }).GetAwaiter().GetResult();

            return secretResponse.SecretString ?? "default-key";
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Could not fetch secret: {ex.Message}. Using default.");
            return "default-key";
        }
    }
}

Objects get initialized once per execution environment, rather than on every invocation. That means after the cold start, followup requests can be lightning fast—single-digit millisecond response times aren’t uncommon.

What’s so powerful here is that Lambda’s initialization phase (the init duration) gets a CPU boost behind the scenes: if your memory setting entitles you to, say, 1.5 vCPUs, Lambda rounds up and gives you 2 during init. That’s a “free” burst you can harness for heavy startup tasks.

The knock on effect, is that you also write more testable handlers as well.

Native AOT: Henry Ford for Your Lambdas

For greenfield APIs and microservices, there’s a not so new anymore tool in the .NET arsenal: Native Ahead-Of-Time (AOT) compilation. .NET typically relies on a JIT (Just-In-Time compiler), converting IL code into machine instructions when the Lambda spins up. Native AOT flips that on its head, compiling down to a native binary ahead of time. The cold start penalty from the JIT almost vanishes.

Caveats loom. Reflection-based features don’t play well with AOT, so you’ll need to retool serialization and similar patterns. For example, JSON serialization/deserialization must use source generators. Here’s a rough sketch of what that looks like in code:

[JsonSerializable(typeof(APIResponse))]
[JsonSerializable(typeof(APIGatewayProxyRequest))]
[JsonSerializable(typeof(APIGatewayProxyResponse))]
public partial class LambdaJsonContext : JsonSerializerContext
{
}

Switching to source-generated serialization means you must explicitly list the types you want handled—no wildcards. But the performance benefits speak for themselves, cold starts dropping to as little as 300 milliseconds, total. For a modern API, that rivals the fastest interpreted runtimes out there—go ahead, compare it to Node.js or Go. Alright alright Rust developers calm down though, you’re still the most performant Lambda runtime.

It All Adds Up: A Systems View

Serverless isn’t “magic.” It’s a set of trade-offs, many of them masked by abstraction. But as you can see, the cold start challenge is best tackled not by a single fix, but by layering solutions: appropriate resource allocation, smart initialization, and native compilation.

It’s systems thinking in action—a holistic approach that sees cold start latency as the emergent property of resource allocation, object lifetimes, and compilation mode.

Want to See This in Action?

Seeing is believing. If you’re hungry for benchmarks, concrete configuration steps, and even more technical deep dives, check out the video version of this post here.

Writing testable, maintable, Lambda Functions in .NET

James Eastham — Wed, 11 Feb 2026 16:02:43 GMT

Writing Testable, Maintainable Lambda Functions in .NET: Beyond the Big Ball of Mud

Moving fast is exhilarating, until it’s 3am and you’re knee-deep in a sea of 700-line Lambda functions, trying to swat a bug that only appears on production. The question that lingers for many .NET engineers venturing into serverless is: does adopting Lambda mean sacrificing testability, maintainability, and all the hard-won architecture patterns from the rest of your career?

If you’ve ever squinted at a monolithic function full of parsing, validation, business rules, database operations, and notification logic—crammed together in a mess that’s impossible to unit test—welcome to the club. But here’s the good news: serverless doesn’t have to mean “abandon all engineering discipline.” In fact, your Lambda functions can be clean, decoupled, and a joy to work with, just like the best of your ASP.NET codebases.

Let’s look at how applying separation of concerns and dependency injection to .NET Lambdas (with a little help from the Lambda Annotations framework) can transform not just your code, but your productivity, test coverage, and on-call sanity.

Surfacing the Real Problem: Untangling the Ball of Mud

I still remember opening what should have been a simple Lambda handler, only to scroll through hundreds of lines of code. Parsing requests. Manually deserializing bodies. Inline validation logic. Database queries and updates interleaved with business rules. “Quick” bug fixes took hours because every change risked unintended side effects, and writing effective unit tests was next to impossible.

Sound familiar? Here’s a snipped-down example of what these functions often look like:

public async Task FunctionHandler(APIGatewayProxyRequest request) {

// Deserialize order
var order = JsonConvert.DeserializeObject<OrderRequest>(request.Body);

// Validate order (customerId, items, email address, etc)
if (string.IsNullOrEmpty(order.CustomerId) || ... )
{
    return BadRequest("Invalid order");
}

// Run business logic
// Query DynamoDB
// More business logic
// Publish notification
// ... hundreds more lines ...

}

Making changes here is like pulling a thread on a wool sweater: you fix one thing, and the whole thing unravels.

A Better Way: Structuring Serverless .NET Like Grown-Up Software

There’s nothing inherent to Lambda or serverless that requires abandoning SOLID principles. Part of the shift is recognizing that your Lambda handler is just an entrypoint—it shouldn’t own all the work.

The solution is to treat your Lambda like you would a controller in ASP.NET: keep it thin and delegate everything else.

Parsing and Routing
The Amazon Lambda Annotations framework brings modern annotation-driven programming to .NET Lambdas. You can decorate your handler so it automatically receives parsed request bodies, dramatically reducing boilerplate.


[LambdaFunction]
[HttpApi(LambdaHttpMethod.Post, "/orders")]
public async Task<IHttpResult> ProcessOrderAsync([FromBody] OrderRequest orderRequest) {
   // We'll fill this in soon...
}

Just like in ASP.NET MVC, you annotate your parameters and let the framework handle parsing.

Validation
Inline validation logic sprinkled throughout the function breeds chaos. Extract it behind an interface:

public interface IOrderValidator {

   ValidationResult Validate(OrderRequest order);

}

This not only cleans up your handler, but lets you swap in different validators or leverage existing libraries like FluentValidation.

Business Logic Isolation
Following the Ports and Adapters (a.k.a. Hexagonal Architecture) pattern, pull all your “core” business logic into well-defined services:

public interface IOrderService {

   Task<OrderResult> ProcessAsync(OrderRequest order);

}

Your Lambda handler simply orchestrates: parse, validate, process, return a response.

public async Task<IHttpResult> ProcessOrderAsync([FromBody] OrderRequest request) {

   var validation = _orderValidator.Validate(request);
   if (!validation.IsValid) return BadRequest(validation.Errors);

   var result = await _orderService.ProcessAsync(request);
   if (!result.Success) return BadRequest(result.FailReason);

   return Created(result.OrderId);

}

Centralized Models & DI
Move your models (requests, responses, validation results) to a shared Models folder, and use a Startup.cs (or modern .NET pattern with Program.cs) to register dependencies with the Lambda dependency injection container.

Now, your codebase mirrors everything that’s good about modern .NET server applications.

Testability: The Payoff

Here’s where the magic happens: once you’ve factored your Lambda in this way, writing tests becomes almost trivial.

Your handler is just a class with dependencies. You can inject fakes or mocks for IOrderValidator and IOrderService and unit test your function logic directly—no need to spin up Docker, mock cloud APIs, or wrangle YAML.

[Fact]
public async Task ReturnsBadRequest_WhenValidationFails() {
    var orderValidator = new Mock<IOrderValidator>();
    orderValidator.Setup(v => v.Validate(It.IsAny<OrderRequest>()))
        .Returns(new ValidationResult { IsValid = false, Errors = { "Invalid!" } });

    var orderService = new Mock<IOrderService>(); var handler = new OrderFunction(orderValidator.Object, orderService.Object);

    var response = await handler.ProcessOrderAsync(new OrderRequest()); Assert.IsType<BadRequestResult>(response);
}

Suddenly, all those patterns you painstakingly learned about isolation and dependency injection aren’t just possible—they’re natural in serverless .NET.

So Why Does This Still Go Wrong?

The temptation is real: with .NET 10’s single-file Lambda features, it’s never been easier to slap a bunch of code into one file and deploy. It feels fast… right until you’re in maintenance purgatory. “Quick and dirty” compounds–the simple demo becomes tomorrow’s production headache.

There is a cognitive pressure in serverless to just ship the function, but production-ready systems thrive on discipline. Clear separation of concerns is just as important in the cloud as anywhere else—arguably more, because you’re often dealing with multiple data sources, event triggers, and team members.

The Principle: Treat Your Lambda Functions Like Any Other Code

When you build serverless systems, resist the urge to compromise on code quality. Use dependency injection, model binding, clear business/service layer boundaries, and write tests that don’t require the cloud to run. Your future self will thank you.

A Lambda handler should feel like a short, readable story: Validate. Process. Return. Everything else belongs somewhere more appropriate.

Don’t settle for spaghetti. Compose your functions with the same care—and you’ll ship faster, debug easier, and sleep better.

Want a deeper dive, or prefer to see it in action? Check out the video walkthrough version of this post on YouTube and see clean Lambda structure come to life!

.NET on AWS Lambda in 20 minutes

James Eastham — Wed, 28 Jan 2026 16:02:43 GMT

Feels Like ASP.NET, Runs on Lambda: Under 10 Minutes to Serverless APIs with .NET Aspire

The line between the familiar and the future in cloud-native .NET

Staring at a greenfield project, most engineers face that twinge of anxiety. You know the feeling: you want to move fast, lean on your experience, but there’s this wall—deploying your .NET API to serverless without losing the developer ergonomics you love from ASP.NET. That’s the heart of the issue. Serverless is supposed to save us operational headaches, but how do you keep your flow when the abstraction starts to diverge from your muscle memory?

Recently, the rise of .NET Aspire and the Lambda Annotations framework has begun to close that psychological (and technical) gap. With the right approach, you can build .NET APIs for AWS Lambda with the confidence and patterns you’d expect from mature ASP.NET—without losing momentum or getting trapped in YAML purgatory. And, if you’re fast, all of this can be live in less than ten minutes.

The Familiarity Principle: Serverless Without Sacrificing Syntax

Software engineers love muscle memory. That’s why so many engineers bounce off early serverless: the friction of learning new SDKs, puzzling over cryptic handler strings, and wrestling with unfamiliar deployment models. What if you didn’t have to?

That’s where Lambda Annotations come in. You set up your project as you would with an ordinary ASP.NET API—no need to break old habits. Here’s a taste:

public class Functions {

    private readonly ICalculatorService _calculatorService;

    public Functions(ICalculatorService calculatorService)
        => _calculatorService = calculatorService;

    [LambdaFunction]
    [HttpApi(HttpMethod = "GET", Path = "/add/{x}/{y}")]
    public int Add(int x, int y) => _calculatorService.Add(x, y);

}

You’re reading it right. Endpoints are just class methods, annotated with [LambdaFunction] and [HttpApi], mapping HTTP paths and verbs just like classic ASP.NET. Path parameters? Handled through names—you don’t fumble with routing tables or hand-rolled API Gateway event parsing.

Want to accept a typed payload? Add a POST endpoint, use a [FromBody] parameter, and let the framework handle parsing:

[LambdaFunction]
[HttpApi(HttpMethod = "POST", Path = "/post")]
public string Post([FromBody] MyData data) => $"You said: {data.Hello}";

The result: refactoring, DI, and endpoint development feel exactly like an old-school ASP.NET project. And all that complexity AWS used to force on you—the ugly handler strings, the cloudformation glue—is boiled down to a thin, readable abstraction.

Configuration Done Right: Dependency Injection and That Startup.CS Nostalgia

For those who’ve grown up with .NET, the Startup.cs file is almost sacred ground. It’s the heart of your app’s configuration, and it keeps your engineering fast and organized. Even as new patterns like minimal APIs have taken hold, there’s value in having a central place for dependency injection, configuration builders, and service registration.

Lambda Annotations respects that. Your Startup.cs lives on:

public class Startup {

public void ConfigureServices(IServiceCollection services)
{
    services.AddSingleton<ICalculatorService, CalculatorService>();

    var config = new ConfigurationBuilder()
        .AddEnvironmentVariables()
        .Build();

    services.AddSingleton<IConfiguration>(config);
}

}

Not quite as magic as DI in classic ASP.NET Core—there’s a bit of hands-on work for configuration—but it’s recognizable. Adding AWS SDKs or custom secrets logic goes here, and IConfiguration flows everywhere it needs to go.

The upshot: moving your codebase from POC to production won’t mean undoing months of “serverless special-case” hacks. Changes stay predictable, understandable, and close to conventional engineering wisdom.

The Reality Under the Hood: Source Generators and Infrastructure Simplified

Here’s the deal nobody talks about with serverless: the glue code is exhausting. Most Lambda guides ask you to hand-craft handler functions, stubbing out JSON parsing of APIGateway Proxy events until your eyes cross. But with .NET 10 and the Lambda Annotations framework, the ceremony is—finally—gone.

What’s actually happening? As you write your nice, human-friendly methods, .NET Source Generators spin out the glue: handler functions, deserializers, and the bits needed to make Lambda “just work.” Even the CloudFormation serverless template is piped out. Curious? Go peek at the generated code or template if you like, but you rarely need to touch them.

Aspire and the Local Feedback Feedback Loop

Local testing is where most serverless enthusiasm dies. Mocking infrastructure, wrangling function URLs, and staring at inscrutable errors is the norm. But this new workflow changes that equation.

.NET Aspire’s Aspire.Hosting.AWS package puts your Lambda in a local development harness. You can spin up everything with:

var builder = DistributedApplication.CreateBuilder(args);

var getRootFunction = builder.AddAWSLambdaFunction<Projects.LambdaAnnotationsDemo>("GetRoot",
    lambdaHandler: "LambdaAnnotationsDemo::LambdaAnnotationsDemo.Functions_GetRoot_Generated::GetRoot");
var getItemsFunction = builder.AddAWSLambdaFunction<Projects.LambdaAnnotationsDemo>("GetItems",
    lambdaHandler: "LambdaAnnotationsDemo::LambdaAnnotationsDemo.Functions_GetItems_Generated::GetItems");
var getItemFunction = builder.AddAWSLambdaFunction<Projects.LambdaAnnotationsDemo>("GetItem",
    lambdaHandler: "LambdaAnnotationsDemo::LambdaAnnotationsDemo.Functions_GetItem_Generated::GetItem");
var createItemFunction = builder.AddAWSLambdaFunction<Projects.LambdaAnnotationsDemo>("CreateItem",
    lambdaHandler: "LambdaAnnotationsDemo::LambdaAnnotationsDemo.Functions_CreateItem_Generated::CreateItem");


builder.AddAWSAPIGatewayEmulator("APIGatewayEmulator", Aspire.Hosting.AWS.Lambda.APIGatewayType.HttpV2)
    .WithReference(getRootFunction, Method.Get, "/")
    .WithReference(getItemsFunction, Method.Get, "/items")
    .WithReference(getItemFunction, Method.Get, "/items/{id}")
    .WithReference(createItemFunction, Method.Post, "/items");

builder.Build().Run();

Fire up Aspire locally, and you’ll get a dashboard showing your Lambda functions and an API Gateway emulator. Hit endpoints; tweak methods; debug as you would a regular web app. Not only does this save hours of frustration, it returns control to the engineer—no more blind deployment-faith.

And when it’s time to ship, toss in a SAM template or the generated CloudFormation and deploy straight to AWS, no translation layer needed.

Evolving with Minimal Friction

Here’s where it all pays off. You need to evolve your API: add a DELETE endpoint, accept new parameters, or branch out to complex workflows. You don’t pay the serverless tax anymore. Just graft on another method, update the annotations, and you’re done.

[LambdaFunction]
[HttpApi(HttpMethod = "DELETE", Path = "/product/{id}")]
public string DeleteProduct(string id) => $"Deleted product {id}";

You’ve maintained a direct line between engineering intent and cloud-native result. Plus, performance is up-to-date with .NET 10—so you’re not losing anything by moving serverless.

Closing the Gap Between API Imagination and Reality

The part that engineers really care about—in my experience, at least—is flow. Not the meditative “in the zone” flow, but the deterministic, reproducible, low-surprise flow from business need to delivered feature. The new .NET serverless story, rooted in familiar ASP.NET paradigms, finally honors that.

Yes, serverless is still evolving, and yes, debugging distributed systems remains a challenge. But today you can build production-grade .NET APIs for Lambda without putting on a different hat, learning a new language, or sacrificing codebase integrity. There’s an elegance to that—a closing of a tradeoff we’ve carried too long.

If you’ve been hesitating to bring your .NET team into serverless, now’s the time: try building that next microservice the Aspire/Lambda way, and notice how little you have to re-train yourself. You might be surprised how much of your classic ASP.NET sensibility is preserved in the cloud.

Want to see this in action—code and all? Check out the video version of this blog post on YouTube.

On AI & Creativity

James Eastham — Fri, 12 Dec 2025 14:00:00 GMT

I’ve always written to help myself think, and I thought it may be interesting to share those thoughts with the world. This post was written in a single stream of consciousness with very little editing.

I love going to watch classical music. Listening to such a large group of people come together playing in such perfect harmony to create music is a sight I think everyone should witness at least once. Last time I attended, I tried to think more deeply about what it was I enjoyed so much. A thought planted whilst my wife and I were chatting about AI.

When I really boil it down, the reason I love classical music is partly because it just sounds great on the ears. But also, and I think a bigger reason, is the amount of work each individual musician has put in to get to where they are on that stage. The inevitable failures, the challenges, the hours and hours of practice. That’s all paid off and there they are on a stage.

If you look further back to the composers who wrote many of the great symphonies. I mean Beethoven was deaf, yet still managed to compose some of the greatest pieces of classical music that exist today.

The humanness adds an extra dimension to almost all things we’d consider creative.

## So how does this relate to AI?

Seemingly, the problems AI companies are trying to target first are the creative ones. Disney selling the rights to their characters to Sora, Spotify generating their own AI music and flooding playlists with it. Nowadays, anyone has art/film/music at their fingertips. Just a prompt away.

And when you think about it, this makes sense. I love this article and how it refers to LLM’s as a bag of words. Without knowing the details of what data AI companies use to train their LLM’s, I would assume a lot of the words in the bag are from films, music, television and books. There is a lot of reference data, which makes generating stuff easy.

When you compare that to science, as the bag of words post does, things are a little different. Most groundbreaking science, by it’s very nature, is stuff that hasn’t been done before. That makes the bag of words pretty shit. There aren’t many words in the bag to describe something that is brand new.

So whilst an AI could generate a bunch of low quality studies, repeating things that have already happened, I’m not convinced in it’s current form it could do anything new and novel. That’s just not how the tools work.

All of this does beg the question though, what exactly is creativity?

What is creativity?

In all of the above; classical music, film, science, where exactly is the creativity? Which specific piece of it is creative?

I was chatting to someone at a conference recently and a family member of their’s was an artist. And one of the things the artist used to do was take paint and throw it in the air above a canvas. Wherever the paint landed was how the piece started. The act of doing that seems pretty creative. But is that too dissimilar for having the seed of an idea and asking an AI to continue from there.

This is where I get conflicted. Very conflicted. And I don’t know, even for myself in my day to day, where the right place to draw the line is. When does creativity, humanness, turn into AI generated slop. To be able to type a prompt into an AI service, you need to have the idea in the first place. Is that not creative? You have the idea, but you don’t have the skill to create the end product.

I feel this conflict deeply when I start to relate these same ideas to software development.

I’ve always been a backend developer, and I’m dreadful at anything frontend or UX related. I know enough to be dangerous, but not enough to create quality front-end experiences.

Now I have a tool at my fingertips that allows me to think of an idea, code the backend and generate a passable frontend layer that is 10x better than anything I could have created myself.

The creativity is in the idea, and it feels great to generate functional applications quickly. But does that diminish the hours and hours and hours of time others have put in to becoming great frontend engineers?

Quick sidebar, the one relatively large frontend codebase I’ve used an AI to generate become an absolute un-maintainable hellscape after a month or two of work and adding new features. Probably due to my lack of skills on how to build quality frontend applications.

Creativity in software development

So what is the creative part of software development? For me, software exists to solve a problem. To make someone’s else life easier, faster, better or more efficient. If you’re building software that isn’t doing that then what is the point in building it.

The creative part, is solving the problem. Is figuring out a new or novel solution to meet your users needs. Whether that’s the specific needs of the user, or the fact you’re user traffic has increased 100x and you need to find a way to scale it.

Most software systems I’ve ever worked on are snowflakes. We talk about microservices, or serverless, or event-driven architecture, but even systems that follow these patterns are unique. Different programming languages, different tools, different stacks, different business conditions, different leadership styles, different team structures. All these things come together to build software.

All of these things, make your system unique. It might be similar to other systems, but without a doubt it’s unique.

And this is where I think things start to get interesting with AI.

Because whilst your system might be unique to mine, there are some similarities. We’ll both have tasks we need to do that are repetitive and tedious.

“Oh crap, I need to add a new property to the class and then make the same change in 10 separate code files”.

“We’ve added this one API route, I need to scaffold 20 others in the same fashion”

“I’ve got an API with 30 routes, now I need to generate an OpenAPI spec to document it”

These repetitive, tedious tasks is where I believe AI coding assistants shine. I can give them clear guardrails, go off and do this things.

But the stuff that makes your software unique, the stuff that solves your users needs in a new and novel way. The creativity that you as a human, with all your experience, your 3am pagers, your big wins and terrible losses. That creativity to solve new and novel problems in new and novel ways.

The bag of words ain’t got a chance.

James

The Simplest Way to Deploy Containerized Apps on Azure in 2025

James Eastham — Fri, 17 Jan 2025 13:30:03 GMT

Picture this: You’ve just built the first version of your containerized backend application. The code is clean, the architecture is solid, and everything runs on your machine. It’s simple, it’s portable and it meets the requirements. And then comes the question… How do you get that application to your users?

If you’re like most developers, you want to focus on building features, not managing/operating/security/battling infrastructure. You’ve heard about Kubernetes, but honestly, you’d rather not deal with its complexity right now. You just want your container to run reliably in the cloud. Sound familiar?

Enter Azure Container Apps - probably my favourite service across all of the cloud providers. Let me show you why it’s become my go-to solution for deploying containerized applications, and how you can get started with it today. Using everyone’s favourite pizza restaurant - PlantBasedPizza.

What Makes Azure Container Apps Special?

At its core, Azure Container Apps is a serverless platform for running containers. While it’s built on top of Kubernetes, you don’t need to know anything about Kubernetes to use it effectively. Think of it as getting all the benefits of a modern container platform without the operational overhead.

Container Apps is incredibly simple. There are only two concepts at the core of all applications:

Container Apps Environment - Think of this as your deployment space. It could be your dev, staging, or production environment.
Container Apps - These are your actual applications, running as containers within the environment.

That’s it. No nodes to manage, no clusters to configure, no complex networking to set up. Just your code, in a container, running in the cloud.

Getting an application running is remarkably straightforward. You only need to specify:

How much CPU and memory your app needs
Whether you want public access (ingress)
Your container image

That’s it. No load balancers to configure, no clusters to manage, no node pools to size. Container Apps handles all of that for you.

And whilst things start simple, you can get more complex if needed. For example, you can define auto-scaling rules. Scale rules can be based on HTTP, TCP, custom data (CPU/memory) and even other Azure services like Azure Service Bus or Azure Event Hubs.

One of my favoutire features is the native integration with Dapr (Distributed Application Runtime). With Dapr enabled, you get:

Service-to-service invocation using just application names
Built-in pub/sub capabilities
State management
Secrets management
And so much more, all without changing your application code

For example, if you have two services – order-service and payment-service – they can communicate simply by using each other’s names. No need to manage URLs, service discovery, or retry logic. Honestly, if you’ve not come across Dapr I’d highly recommend checking it out.

Perhaps one of the most underappreciated aspects is that Container Apps provides a nice off-ramp. Since it’s built on Kubernetes, if you ever outgrow its capabilities, you can:

Migrate to Azure Kubernetes Service (AKS)
Keep your existing container configurations
Maintain similar deployment patterns
Preserve your development workflows

You would still need to actually build the Kubernetes infrastructure, but because under the hood ACA is using open source tooling it’s a relatively safe choice for growing applications – you can start simple and scale up without needing to completely rearchitect your system. And if you do ever need Kubernetes (you probably don’t) you have an off ramp.

Deploying Your First Container App

Let’s get practical. Instead of clicking around in the Azure portal (it’s 2025, after all), we’ll use infrastructure as code. Personally, I’m a big fan of using the same language to define my infrastructure as my application code. Which makes Pulumi an excellent choice for defining and deploying applications to Azure.

Look just how simple it is to define a container app on Azure:

var environment = new ManagedEnvironment("pizza-prod-env", new ManagedEnvironmentArgs
{
    ResourceGroupName = resourceGroup.Name,
    Location = location,
    EnvironmentName = "production",
    SkuName = "Consumption"
});

var app = new ContainerApp("pizza-backend", new ContainerAppArgs
{
    ResourceGroupName = resourceGroup.Name,
    ContainerAppEnvironmentId = environment.Id,
    Configuration = new ConfigurationArgs
    {
        Ingress = new IngressArgs
        {
            External = true,
            TargetPort = 8080, // The port on your local container
            Traffic = new TrafficWeightArgs()
            {
                Weight = 100,
                LatestRevision = true
            }
        }
    },
    Template = new TemplateArgs
    {
        Containers =
        {
            new ContainerArgs
            {
                Name = "pizza-api",
                Image = "your-registry/pizza-api:latest",
                Resources = new ContainerResourcesArgs
                {
                    CPU = 0.5,
                    Memory = "1Gi"
                }
            }
        }
    }
});

This code creates everything you need to run your application. Let’s break down what’s happening:

First, we create a Container Apps Environment - this is where your apps will live
Then, we create the Container App itself, specifying:
- Basic resource requirements (CPU and memory)
- The container image to use
- Whether it should be publicly accessible (external ingress)

Real-World Considerations

Of course, real applications need more than just basic deployment. Azure Container Apps has you covered there too. You can easily:

Scale based on HTTP traffic or custom metrics
Set up authentication and authorization
Use managed identities for secure access to other Azure services
Deploy multiple containers as sidecars (great for logging agents or service meshes)

For example, adding a DataDog monitoring sidecar is as simple as adding another container to your template:

Containers =
{
    // Your main application container
    new ContainerArgs { /* ... */ },
    // DataDog sidecar
    new ContainerArgs
    {
        Name = "datadog-agent",
        Image = "datadog/agent:latest",
        Environment =
        {
            new EnvironmentVarArgs
            {
                Name = "DD_API_KEY",
                SecretRef = "dd-api-key"
            }
        }
    }
}

Getting Started

Ready to try it yourself? Here’s what you need:

A containerized application (if you don’t have one, you can use the example from PlantBasedPizza)
An Azure subscription
Pulumi CLI installed (or if Terraform is your jam, then a Terraform example is available in GitHub as well)

The full sample code, including a complete pizza ordering backend, is available in my GitHub repository.

Why Should You Care?

The cloud shouldn’t be complicated. Azure Container Apps gives you a production-ready platform for running containerized applications without the complexity of managing Kubernetes directly. It’s perfect if you want to focus on building features rather than managing infrastructure.

Best of all, if you do eventually need more control or want to migrate to a full Kubernetes cluster (you still probably shouldn’t), you’re already using compatible technologies.

What’s Next?

Start small. Take a simple containerized application and deploy it using the code samples above. Once you’re comfortable with the basics, you can explore more advanced features like:

Dapr integration for microservices communication
Auto-scaling rules
Virtual network integration

Remember, the goal is to keep things simple while maintaining the flexibility to grow as your needs evolve.

Give it a try, and let me know how it goes! Drop a comment below or reach out on socials if you have any questions.

On Choosing Your Complexity

James Eastham — Mon, 16 Dec 2024 19:47:03 GMT

Software is complex. I don’t think that’s a controversial statement. Building, testing, observing, scaling, failing, evolving. And that’s before you try to understand what it is your users want the system to do.

At times, it can be a bit demoralizing can’t it? Constantly fighting against this ever changing, ever growing challenge of a system that is getting more and more complex. You think back to the days when you built a “hello world” application to learn a new concept and you wonder where all this complexity came from.

Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple. But it’s worth it in the end because once you get there, you can move mountains. - Steve Jobs

Reading Steve Jobs’ biography (which is an excellent read by the way) one of my biggest takeaways is simplicity. One of the reasons Apple became so successful is the relentless focus on simplicity.

Take the well-told story of Job’s second coming at Apple. On his return, the product lines had grown massively. To the point Apple’s own teams couldn’t explain which Mac to pick. Internally, teams thought they were meeting a user needs when actually they were confusing customers with so many choices.

When Job’s came back, he changed the entire product line to just have 4 products. Yep, just 4. 2 desktop computers and 2 portable devices. An obvious example of simplification, but it had the right effect.

When you’re trying to change an entire industry, you are already dealing with lots of complexity. Changing human behaviors, introducing a new way of interacting with technology. This is essential complexity. Introducing 10x more product lines than you actually need, that is a form of accidental complexity.

What does this mean for you dear reader? Many of you will be working on new systems. New systems that will require a change in human behavior. Many of you are likely operating in a highly complex business domain where the problem space itself is highly complex.

Every single software system that exists has a certain level of essential complexity.

Take Plant Based Pizza, the backend for a pizza restaurant. Whilst not the most complex of domains, there is still a decent amount of complexity to think about. Taking orders, managing stock, managing staff, taking payments, refunding payments, preparing orders, organizing delivery drivers and getting the order delivered to the correct address.

And that’s just considering the happy path. What if a payment fails? Or an order is ready but there aren’t any delivery drivers available.

Think about the systems you are working on, they will inevitably have their own essential complexity.

Unfortunately for us, this complexity can’t be removed. Much like Apple trying to change an entire industry, you can’t get around this complexity. Trying to “simplify” this complexity can seem like a challenge.

Accidental complexity though, that’s something you do have some agency over.

Accidental Complexity

Accidental complexity is the complexity that comes from the choices you make. The complexity that you (or your company) has control over.

Every technology choice you make will change the amount of accidental complexity of your system.

When Plant Based Pizza first started, they focused on delivering value quickly. At that time, there was a team of 5 building the backend. They built a monolithic application with a Postgres database to support all the different business use cases.

A single running process and a simple, flexible and well understood database technology == simplicity!

Of course, once they built the application they needed to run it somewhere. One of the team had been reading about Kubernetes recently and got the rest of the team incredibly excited about building a shiny new Kubernetes cluster.

They created a cluster, deployed some virtual machines instances as the underlying compute and used a managed database service for providing the Postgres functionality. Great, we are live.

This isn’t the part of the story where I shit on Kubernetes. It’s merely an example of the complexity you’ve just accidentally introduced into your system. Think about all the things you need to think about running a Kubernetes cluster:

Scaling nodes
Operating nodes
Securing nodes
Dealing with node failures
Upgrading the cluster
Securing the cluster
Dealing with the cluster failing
Scaling your application
Dealing with application failures

That’s before you consider the various 3rd party tools you’ll need to deploy inside the cluster and keep up to date.

Let’s consider an alternative universe where the team made a different decision.

The team still packaged the monolithic backend application as a container image, and still used a Postgres database. They made that decision for simplicity (containers are well understood) and portability (containers can run anywhere, as can Postgres).

Instead of spinning up a Kubernetes cluster though, they used a managed container orchestrator. Something like Amazon ECS (with Fargate), Azure Container Apps or Google Cloud Run. They kept the Postgres database running in a managed cloud database service.

Let’s look at the complexity you’ve just introduced:

Scaling your application
Dealing with application failures

A shorter list, right? Using a managed container orchestrator means you shift a whole bunch of that accidental complexity on to the cloud provider. Operating nodes? Operating clusters? Dealing with node failures? That’s the cloud providers responsibility now.

There’s a trade off here with cost of course, and I talked about that recently in this post on the factors of modern compute].

Continuing this tale, we fast forward 12 months. Plant Based Pizza has grown, and there are now 50 developers across 5 different teams. All teams are still contributing to that same monolithic code base, and are getting in each others way.

This is a form of accidental complexity! The complexity that comes from the decision to grow teams. Complexity you now need to deal with.

Microservices are a reaction to organization challenges. Microservices are a solution to a socio-technical problem. They aren’t a solution to a technology problem.

So the Plant Based Pizza team decides to start to break their application down into microservices. As you might expect, microservices introduce their own complexity. And believe me, microservices introduce a lot of technology complexity.

At this point, there is no good reason to change the complexity of your underlying compute. You are already introducing enough additional complexity by choosing microservices, introducing another layer of infrastructure complexity probably won’t help you too much.

Speeding forward in time again, Plant Based Pizza has now been refactored to microservices. These microservices are communicating with asynchronous events to make them as loosely coupled as possible. And you’re still running the applications (packaged as containers) on serverless compute. You still don’t have a good reason to introduce the complexity of Kubernetes.

What accidental complexity does the system have?

Scaling the applications
Dealing with application failures
Eventual consistency across different microservices
Service to service communication
Observability (it’s hard in asynchronous systems)
Governance and evolving the system safely

Slightly more complexity, but still not crazy amounts right? And all this complexity is something you can have direct control over as a developer.

If you choose to run your application on Kubernetes you still have this complexity to deal with, as well as all the complexities that come from Kubernetes.

Again, this isn’t a post written to bash Kubernetes. It has a place in the world. But you probably don’t need it for your applications.

And if you are building with Kubernetes, you can still apply this lens of accidental complexity. Does every single developer inside your company need to understand Kubernetes? Do they need to understand Helm charts and YAML files. Or can you build your Kubernetes cluster in a way that means developers only care about shipping container images. This is, of course, platform engineering.

When I was at AWS I worked with a company (that I can’t name) doing this. They were building an abstraction layer on top of Kubernetes so that developers could just ship container images and not worry about anything else. To add complexity, this abstraction layer also spanned multiple cloud providers.

Whilst it was an interesting project to work on, it wasn’t for the business. They had put an insane amount of money, developer time and operational responsibility into building a service that wasn’t even nearly as feature complete as Amazon ECS or Azure Container Apps. They even had a side by side comparison chart and the custom built service wasn’t anywhere near.

All of that, to deploy business applications… To put it bluntly what a massive waste of time and effort.

You need Kubernetes if your ability to dynamically scale and manage infrastructure is a core differentiator for your business.

The takeaway from this rambling post is to think about the decisions you make and the complexity that introduces.

Focus on building stateless, portable, containerized applications. Focus on structuring your application code to separate infrastructure concerns from business logic (ports and adapters style). Focus on shipping those applications in an operationally light way as possible.

Reduce your operational/infrastructure complexity by adopting managed platforms and shifting that responsibility onto the cloud provider. Cost is of course a consideration here. The line item cost of running Fargate is more than a comparable EC2 instance. But that doesn’t take into account the human cost required to run EC2.

You can see this line of thinking in action with the fact all the cloud providers now have some form of ‘Kubernetes auto’. Where you can run Kubernetes, and then pay your cloud provider extra to take a chunk of the operational overhead.

If you focus on stateless, portable containers this compute conversation becomes largely irrelevant. Run your application in a place that makes sense right now, and you are ready to deal with future changes.

As a thought exercise, think about the accidental complexity you have in your system today? What complexity are you dealing with that isn’t related to the business problem you are trying to solve. And importantly, what could you do to reduce it?

On Patterns of Modern App Development

James Eastham — Wed, 20 Nov 2024 07:00:00 GMT

We build modern applications on common building blocks. Applications may use one or all of the building blocks. They are:

User-facing applications
Background applications
Storage
Integration
Caches
Observability
Service-to-service communication
Orchestration

This isn’t an exhaustive list. But it gives us a set of building blocks, first principles, that form the basis for thinking about modern applications.

User-facing applications

The first, and probably most obvious, is the web application. I use the term application loosely here. It could be an API, a server-side rendered web app, or a frontend. The critical thing to remember is that someone external consumes these applications. That external person could be another team inside your organisation or someone browsing your site on their mobile.

User-facing applications, regardless of who the user is, have a set of common characteristics. User-facing applications need to be:

fast
resilient
exactly where I (the caller) expect them to be

First, web-facing applications need to be performant. In my experience, users are always unhappy. Things could always be more performant. An application’s performance can directly impact a user’s job performance and satisfaction. If you’ve used Salesforce, you’ll know exactly what I mean.

The second and third options in the list are related. In addition to being fast, if I’m a user calling a web application, I want it to be there and to work. ‘Working’ is a broad term, and what working means will depend on the application, the context, and the use case (a rarely called non-critical service can probably accept some downtime).

But inside whatever the SLA you’ve set, you want the web app you’re calling to work and to exist behind the DNS name you’re calling to access it.

“I’m happy with this slow service that only works some of the time” - No user ever

User-facing applications use synchronous communication. This article would grow exponentially if I discussed the ideas of coupling, so let’s leave it at that.

Background applications

I’m differentiating a background application as an application that doesn’t have another user (remember, the user could be a human or another service) accessing it synchronously. A background application reacts to messages coming from an external location.

Background applications will run, wait for a message to arrive, and then pick it up and process it once a message does arrive. You don’t want your application sitting there idle, consuming resources and costing money when there is no work to do. I’d argue the most important characteristic of a background service is its ability to adjust itself according to throughput—in both directions.

You don’t want to worry about your application if your message volumes increase, and if they decrease, you want your applications to scale back automatically.

Idempotency is also incredibly important. Most messaging services guarantee ‘at least once’ delivery. Some give you the option to switch to guaranteed one-time delivery, but with that, you lose throughput. Honestly, even if you use a messaging system that supports guaranteed one-time delivery, YOU STILL NEED TO THINK ABOUT IDEMPOTENCY.

If you aren’t familiar, idempotency is the idea that the outcome should be the same if the same message/request hits the same service multiple times. If I receive a TakePayment request for order 12345, and then I receive the same request a minute later, you don’t want to take the payment again. Of course, this would also apply to user-facing applications.

As a developer, these are the two components you have direct control over. You can write code that addresses some of these challenges. But the code you write isn’t the only thing that makes up your application.

Storage

When you store data, you want it to be there again later. Whether that’s file storage, blog storage or more typical relational databases, the ability to persist state is an important part of almost every application, particularly if you are building your application layer to be completely stateless (hint, you probably should be).

What that storage looks like would change from system to system, but you’ll likely need to persist things.

Integration

Unless you’re building a monolith, things need to communicate. Even if you are building a monolith, you may still use queues to offload some processing from your main user flow. You can still use asynchronous processing inside a monolithic application.

That said, messaging and integration are more common in a microservices architecture. If you’re a developer working on a single microservice, it’s not your single microservice that makes up ‘the system’. All the individual services work together and form something greater than their parts: the system.

For that to work, your components need to integrate, whether synchronous or asynchronous. Systems need to talk!

Caching

As systems grow bigger and more distributed, data spreads across multiple backend services. Or your users may complain that your app needs to be faster. Caching is one solution to that problem.

Caches aren’t a long-term storage mechanism. Hence, they’re being grouped separately, but they are an excellent lever to pull to increase your app’s performance. A cache could be something more traditional like Redis or an in-memory state you hold inside your application. Think of a cache as an extremely fast temporary store.

Caches bring about their own sets of challenges:

When does something expire in the cache?
How does the cache get updated?
How do you deal with stale data that isn’t current but has yet to expire?
How many cache hits/misses am I getting? Is this helping my system?

There are a bunch of questions you need to answer with caching. And if your cache isn’t set up correctly, it can add latency to your system. It’s not a silver bullet but a core building block of modern distributed apps. Hint: companies like Momento give you fully serverless caching.

Observability

Observability should be number one. If you can’t understand what is happening inside your system, how can you possibly expect to fix a bug or make it better? Observability is the most important part of building modern applications. You need the ability to ask questions about your system to understand individual requests and the context around them. That should be where you start.

Think about it. A product manager comes along and says the system needs to be faster. How do you find the bottleneck? The first-line support team calls you with an odd bug they can’t diagnose. How do you know where to start? You’ve added a cache to your system, and performance has worsened. Why?

Observability helps you answer the questions. A good observability strategy should allow you to play detective and ask questions of your system that you didn’t know you needed to ask.

There’s probably a whole other blog post on this… Stay tuned, or ya’know. You could subscribe…? :)

Orchestration

The final one, which I wasn’t sure about including, is orchestration. You could replace the word orchestration with workflow. Software systems exist to solve problems, most of which are multi-step. If I order a pizza, software in the background is going to:

Check the stock
Take the payment
Send to the kitchen
Wait for the kitchen to prepare the pizza
Find a delivery driver
Deliver the order

Whether you’re building a monolith or distributed event-driven microservices, you’ll need to write some code somewhere in your code base to determine what happens next. After payment is taken, what do I do? If the payment fails, what do I do?

That’s why I’ve included orchestration in this list. Your system is probably orchestrating something, even if you are not explicitly using an orchestrator.

This gives us the building blocks—the overarching ‘patterns’ of modern apps, if you will. Inside each of these components, there is a range of options, services, and decisions. For the vast majority of use cases I’ve encountered in my career, you would benefit from trying to make each component as serverless as possible.

But what does serverless even mean? (If you want to dive deeper into modern compute you might like this article).

There are almost definitely some things I have missed here, and I’d love to hear from you!

Serverless? What does it even mean

Historically, serverless has been defined by a set of core principles. Momento wrote a great article introducing the Litmus Test for Serverless. For a service to be considered serverless it must:

Have nothing to provision or manage
Usage-based pricing with no minimums
Ready with a single API call
No planned downtime
No instances

I like this definition, but I would add one caveat. Viewing serverless as ‘all or nothing’—either serverless or not—can remove some valuable services.

Take a service like Azure Container Apps (ACA). ACA is a container orchestrator that provides an abstraction on top of Kubernetes. You can deploy an application by providing a container image, CPU/memory requirements and scaling behaviour if required. There is almost 0 operational overhead running an application this way.

Looking at the Litmus test, this meets the criteria of 1, 3, 4 and 5. 2 gives us nuance. An application running on ACA won’t automatically scale to zero; you can configure scaling rules, but it doesn’t ‘just happen’. When stopped, they don’t cost you anything. You pay only when your app is running. But your app is running all the time even if no requests are coming in.

This application is still serverless. No, it doesn’t automatically scale to zero. Yes, you would pay for the application running when no requests are coming in. But you can deploy an application with next to 0 operational overhead.

Serverless is a spectrum, not a binary decision. You can be more or less serverless based on the requirements of your application.

In this repository, I demonstrate how to run modern web applications on various cloud providers with little to no operational overhead.

What does it mean for you?

If you’re a developer, at least anything like me, you want to run your application with as little infrastructure worries as possible. “Here’s my application. It needs this CPU and memory, scale it like this, and frankly, I don’t care about anything else.”

If your company has invested time, energy, and money into building a Kubernetes platform, then great. That can feel serverless to you as a developer; leverage it. This isn’t to say Kubernetes isn’t valuable—it very much is. But it’s valuable when your application needs it.

If your ability to dynamically scale and manage infrastructure is a core differentiator for your business and your customers, great. Go for it. Otherwise, you don’t need Kubernetes.

Are you training a machine learning model, doing heavy GPU computation, or ingesting dynamic/large amounts of data? Virtual machines and Kubernetes are probably useful.

The conversation around managed services vs. Kubernetes becomes more interesting when you zoom out and look at the bigger picture. Does your organisation have an excellent reason to invest in building a Kubernetes platform (which is just rebuilding Cloud Run/Fargate/Container Apps)? If you have a good reason to do it (that isn’t CV-driven development), then great.

Otherwise, use a managed serverless, be as serverless as possible, and build your application to keep you portable.

On The Dimensions of Modern Compute

James Eastham — Wed, 13 Nov 2024 11:40:03 GMT

I gave a talk recently at RustLab. The talk focused on building with Rust & Serverless; a couple of slides related to this serverless spectrum triggered some discussion afterwards. This post is me pulling together some of these ideas and trying to make sense of them.

Serverless Is A Spectrum, Not A Binary Decision

The limitations of Markdown don’t allow me to make that point any more strongly. When you choose the compute to run your application, you have a wide range of options, from bare metal instances running in a data centre to serverless functions and many steps in between.

Boxing yourself into ‘serverless only’ or ‘Kubernetes only’ limits your options. Every organisation I’ve worked in has some use cases that better suit a long-running container and some use cases that suit a reactive style of computing.

Of course, it’s possible to run long-running web apps on serverless functions and reactive compute on Kubernetes with services like KEDA. But is that what we want? Aren’t we just forcing square pegs into round holes?

My mental model for compute is thinking about it as a spectrum. Where you choose to sit on that spectrum will depend entirely on what your application needs.

A spectrum is a much more interesting model for thinking about compute choices. Look at your application and its requirements, and then decide what is most important.

Imagine for a moment that you’re building an app with an interface that serves web requests. Those web requests are persisted and sent off for background processing.

Now, imagine that the background process needs to run a complex algorithm that would benefit from having access to a GPU. For that kind of system, you need infrastructure flexibility.

Your application needs a specific piece of infrastructure to function, so you need to take on the additional operational responsibility of running that infrastructure.

The web component, on the other hand, is just a simple web interface that receives requests and passes them on. It’s largely irrelevant how that component runs; you just want it to be reliable and available.

Different components of the same system have different requirements. Let’s make this more tangible for a moment.

That’s the range of compute options you have across the cloud providers. Of course, running your own machine in a data centre also sits off the far left of that spectrum as well.

The web component of your application could sit anywhere. My argument would be to run that as far right as possible.

Your users don’t care about the compute it runs on, your users just want the app to work

The component that requires specific compute, though, clearly needs to be further left. You need control over the infrastructure. How far left, of course, depends. Which brings us onto the second dimension of compute decisions: cost.

This cost slide generated the conversation, and it was actually a last-minute addition to the talk.

The keynote from the previous evening at RustLab was an interesting real-world use case. The speaker briefly mentioned their choice of compute. They were running their Rust application on Kubernetes, and the reasoning was, “Lambda, ECS are more expensive.” I’m paraphrasing there, of course.

My issue with that is that it reduces the cost of a system to the dollar value that AWS invoices you at the end of the month. That’s oversimplifying it.

There’s a reason serverless services have a higher line item cost: You’re paying someone, somewhere, to run your infrastructure. The cloud provider is your platform team.

Running virtual machines and operating Kubernetes clusters have costs beyond the line item cost that a cloud provider charges. As Werner Vogels so perfectly described in the frugal architect cost and sustainability are important considerations for building modern systems. I don’t know Werner personally, but I’m pretty sure he isn’t just referring to the line item cost on your AWS bill.

That isn’t me shitting on Kubernetes; it has a place in the world. But the vast majority of systems don’t need Kubernetes.

Back when I worked at AWS, there was an interesting heuristic that I heard internally. Which was

Kubernetes is useful if your ability to scale infrastructure dynamically is a differentiator for you business

Going back to the app you’re building, the algorithm component has specific infrastructure requirements, and that infrastructure needs to dynamically scale up and down based on load. It feels like a good fit for Kubernetes.

Does that necessarily mean your entire organisation needs to adopt Kubernetes? I’m not so sure. Could the web component run somewhere with lower operational overhead? Something like Lambda? somewhere where you can deploy the app and not worry about it until you need to make a change to it. Definitely.

The counterargument is that once you’ve put time and effort into building a container platform, surely you might as well mandate that all teams use it.

It’s a convincing argument. It’s a good argument. One I’m not sure I have a good argument against.

My only critique of that approach is that it has introduced a single point of failure and a bottleneck into your system.

The single point of failure is the cluster itself. You’re now running multiple (potentially critical systems) on top of a piece of infrastructure that relies on a single team (the platform team) patching and upgrading it every 6 months and in a way that ensures reliability.

That’s entirely possible, of course. But it feels like we are returning to a world where Dev & Ops are split.

One of the core principles of DevOps is “you build it, you run it.” Now, you build it, and someone else (a platform team) runs it. You might be on-call, and you might build observability and alerts. But fundamentally, someone else is running the underlying infrastructure.

This brings me back to the idea of a compute spectrum. In my opinion, a good architect provides guardrails, not mandates.

A guardrail could be your service has to communicate over well-defined APIs.

A guardrail could be that your applications must be packaged as containers and be stateless.

Empower teams to make decisions that suit the specifics of their workloads. Because, at the end of the day, those teams know their workloads the best.

Are you event based or event driven?

James Eastham — Tue, 06 Feb 2024 21:12:03 GMT

You might think the title of this post is purely semantic? Surely if you are using events to trigger functionality, it doesn’t matter so much. James, you’re spending your time writing about something that doesn’t matter.

Until it does.

In his bestselling 2014 book Homosapiens, Yuval Hoah Harari discusses a history of humankind. And one of the key takeaways from that book is the importance of stories. Stories, and our use of words, are what set us apart as a species. Our ability to collaborate at scale, behind a single idea is what makes us different to the animals. Wether that be religion, nationality or the company you work for. These are all human constructs, detailed using language.

Homo sapiens rules the world because it is the only animal that can believe in things that exist purely in its own imagination, such as gods, states, money, and human rights.

Yes, animals work together. But it’s a far more primitive, survivalist kind of collaboration.

So if languages and words are fundamental to human success. And software development is (at least for now, hello GenAI) a human endeavour. The language and words we use in our software is vitally important.

And this is why this distinction is important.

What Is Event Driven Architecture? Event Driven Architecture, in my opinion, is a way of modelling the integration between your systems. And you model this integration using business events. And by business events, I mean things that are meaningful to your business.

Event Driven Architecture allows the language of your business to drive it’s technical implementation - James Eastham

You could look at two businesses in the same domain both building event-driven software. And the set of events in each system could be completely different.

There will be similarities, in function and conceptual ideas, but the language that each of the businesses use will be different. Meaning the events driving the functionality will also be different.

What Is Event Based Software? Triggering functionality via events is not a new thing in software. I started out writing software in Windows Forms. For all you non .NET developers out there, Windows Forms is an old way of building desktop applications for Windows.

Functionality in Windows Forms is driven by events. But these aren’t events that are unique to your business. They are things like ButtonClicked, MouseHovered or WindowResized.

You could look at two systems in the same domain and if they were event based they would have the same set of events. ButtonClicked, MouseHovered so on and so forth.

To bring this into modern, cloud based systems you could look at an S3ObjectCreated in the same way. This is a technical event, that notifies you of a technical thing that has happened. But how many non-technical people in your business would understand what that means?

Who Cares? This might still lead you to ask the question, who cares? If systems are triggering functionality with events, why do the semantics matter.

From a technical perspective, you’d be correct. Both things are immutable facts, notifications of things that have happened somewhere else in the system.

But how many of you have sat in a room with a non-technical person, talking about microservices, REST API’s and SQL databases and had the other persons eyes glaze over? Maybe they mutter under their breaths about ’these nerds talking techie all the time’ and nobody can agree about exactly what the system should do.

Yeah, this is why you should care.

S3ObjectCreated gives you the same problem. This is not the language of your business. InvoiceFileUploaded… now you’re talking.

One of the key takeaways from running an event storming session is to agree on a shared language across your entire business. From technical to non-technical. Don’t waste all that effort by then using different, technical words in your system.

In the same way Domain Driven Design gave us a shared language to talk about our architecture. Event Driven Architecture gives you a shared language to talk about communication. And in my opinion, that’s the real benefit of event driven systems.

The takeaway from this, albeit short rambling stream of consciousness, is to be careful when you talk about being event driven. At it’s core, event driven architecture is about communication. Wether that be technical communication between two services, or the human communication between two people.

Event Driven Architecture allows the language of your business to drive it’s technical implementation - James Eastham

This is the first of a shorter selection of memos I’ll be publishing over the next few weeks. Gathering together my thinking around event driven architecture.

I find writing a key part of developing my own thoughts, so if things get a little messy. I apologies in advance ;)

Hexagonal Architecture with Rust & AWS Lambda

James Eastham — Thu, 07 Dec 2023 21:12:03 GMT

I’ve spent a good portion of 2023 learning the basic principles of Rust. Fascinated by it’s performance, especially within the context of serverless systems. I dove deep, putting my object oriented, 10 years of .NET development to the side and attempted to learn Rust. And it’s been quite the journey.

Early in my career, Domain Driven Design changed everything for me. Up until that point, I was more of a script kiddy than a software development. I look at times in my life as before the big blue book, and after. It sounds hyperbolic, but reading that book changed the trajectory of my career.

Of course, domain driven design is something you apply at the design phase of your system. And once you’ve identified the core domain, you focus all of your efforts into building software in this area. As you move into the development phase, one of the best techniques I’ve found for actually writing code is clean architecture.

Clean architecture, very similar to hexagonal architecture, ports and adapters and onion architecture is a way of structuring your code to separate the business logic from the code that deals with ‘infrastructure’ concerns.

In a more traditional application context, this separates the outer layers of your application like API’s, databases and message buses from your actual business logic. Using AWS Lambda, this would separate the code that handles the Lamdba event from the actual business logic.

I couldn’t quite get rid of all my .NET thinking, so I went ahead trying to bring this kind of programming model to Rust.

In .NET, or other object oriented languages, you have things like encapsulation, inheritance, abstract classes and interfaces. You might define an interface as an IToDoRepository and then implement that interface with a DynamoDbTodoRepository that contains the logic specific to interface with DynamoDB.

Let’s start at the more fundamental level. One of my favourite ideas from clean architecture is hiding certain pieces of functionality inside the domain. Let’s take the example of a ToDo application.

A ToDo is quite a simple thing:

/// Represents the structure of an incomplete ToDo #[non_exhaustive] #[derive(Deserialize, Serialize)] pub struct ToDoItem { pub id: String, pub title: String, pub is_complete: bool, pub completed_on: String, } It has a title and if it has been completed or not. But actually, there might be quite a lot of business logic that goes into the management of a ToDo record. Off the top of my head, things like:

A title cant be changed once a ToDo is completed A ToDo can’t be ‘uncompleted’ The business logic for these 2 things, are related to the ToDo itself and therefore the ToDo should own the logic. There should be no way to update a title, or the completion status without calling a function on the ToDo itself.

#[non_exhaustive] pub enum ToDo { /// Represents an incomplete ToDo item Incomplete(IncompleteToDo), /// Represents a complete ToDo item Complete(CompleteToDo), }

impl ToDo { /// Update the title of the existing ToDo. /// If the ToDo is already completed then the title cannot be updated. /// Returns a new ToDo pub(crate) fn update_title(self, new_title: String) -> Result<ToDo, ValidationError> { let new_title_value = Title::new(new_title);

    if new_title_value.is_err() {
        return Err(new_title_value.err().unwrap());
    }

    let response = match &self {
        ToDo::Incomplete(incomplete) => ToDo::Incomplete(IncompleteToDo {
            to_do_id: incomplete.to_do_id.clone(),
            title: new_title_value.unwrap(),
            owner: OwnerId::new(incomplete.owner.to_string()).unwrap(),
        }),
        ToDo::Complete(complete) => ToDo::Complete(CompleteToDo {
            to_do_id: complete.to_do_id.clone(),
            title: Title::new(complete.title.to_string()).unwrap(),
            owner: OwnerId::new(complete.owner.to_string()).unwrap(),
            completed_on: complete.completed_on,
        }),
    };

    Ok(response)
}

/// Set the ToDo as completed
pub(crate) fn set_completed(self) -> ToDo {
    match &self {
        ToDo::Incomplete(incomplete) => ToDo::Complete(CompleteToDo {
            to_do_id: incomplete.to_do_id.clone(),
            title: Title::new(incomplete.title.to_string()).unwrap(),
            owner: OwnerId::new(incomplete.owner.to_string()).unwrap(),
            completed_on: DateTime::parse_from_rfc3339(&Utc::now().to_rfc3339()).unwrap(),
        }),
        ToDo::Complete(complete) => ToDo::Complete(CompleteToDo {
            to_do_id: complete.to_do_id.clone(),
            title: Title::new(complete.title.to_string()).unwrap(),
            owner: OwnerId::new(complete.owner.to_string()).unwrap(),
            completed_on: complete.completed_on,
        }),
    }
}

} Now, if any part of the application wants to update the title or the status they must use these two methods. The logic is encapsulated, and not spread all over your application.

Enums are one of the features I love most about Rust. You can add implementations to an Enum, and then one of the options of the Enum can have a struct. The implemented methods on the enum can then do different things based on the enum options.

As far as your application is concerned, you’re just calling methods on a ToDo. Inside your domain, the underlying logic differs depending on the type of ToDo you’re using.

Another interesting idea of clean architecture, is separating infrastructure from application code. In .NET, that is typically done using interfaces and the implementation of them interfaces.

Rust doesn’t have that luxury, interfaces aren’t a thing. However, you do have traits. Traits allow you to implement similar functionality to interfaces in Rust.

#[async_trait] pub trait ToDoRepo { async fn list(&self, user_id: &str) -> Result<Vec, RepositoryError>;

async fn create(&self, to_do: &ToDo) -> Result<(), RepositoryError>;

async fn get (&self, user_id: &str, todo_id: &str) -> Result<ToDo, RepositoryError>;

} This is an implementation of a ToDoRepo, for interacting with a persistence layer. This could equally be a MessageBus.

Code in your actual application then only makes method calls on the ToDoRepo trait, notice the use of the dyn keyword. The implementation of a ToDoRepo is passed in at runtime.

pub async fn list_todos(owner: &String, client: &Arc<dyn ToDoRepo + Send + Sync>) -> Result<Vec, ()> { let query_res = client.list(owner).await;

match query_res {
    Ok(todos) => {
        let mut to_do_items: Vec<ToDoItem> = Vec::new();

        for todo in todos {
            to_do_items.push(todo.into_dto());
        }

        Ok(to_do_items)
    }
    Err(_) => Err(()),
}

} At the outermost layer of your application, in this case an Axum web APIm, the implementation of the ToDoRepo is defined and passed into your application state. Notice the use of a DynamoDbToDoRepo, that uses a DynamoDbClient from the newly GA AWS SDK for Rust.

let shared_state = Arc::new(AppState { todo_repo: Arc::new(DynamoDbToDoRepo::new( dynamodb_client.clone(), table_name.clone(), )), });

let app = app(shared_state);

let listener = tokio::net::TcpListener::bind(“127.0.0.1:8080”) .await .unwrap();

axum::serve(listener, app).await.unwrap(); Early days in my exploration of the Rust language, but it’s been an interesting journey trying to marry together the two different programming models I hold in my head. The design principles of .NET, with the syntax and idioms of Rust.

If you’re interested in the code from this article, check out the GitHub repo.

For anyone reading this who is a more experience Rust developer, I’d love to hear if this is the right thing to do or if I’m heading down a long and lonely road.

Until next time, James

Applications, Microservice Mistakes and Getting Things Done

James Eastham — Thu, 05 Oct 2023 21:12:03 GMT

Don’t miss another newsletter by subscribing below :)

Application Architecture as Code One my favourite people in the industry is back again with another featured article. Gregor Hohpe makes an excellent case in this post for how important automation is when you’re building modern systems.

Cloud without automation is just another data center

What I find incredibly interesting about this post, is how Gregor ties the architecture of a system into the actual code you write. I spoke about a similar topic at the recent AWS Community Summit.

With modern. serverless systems, the line between ‘infrastructure’ and ‘application’ code becomes blurry. If you are pulling message from a queue and starting a business workflow, well that’s an application decision.

So how can you define your application architecture using code? Using the power of cloud development kits. These wonderful IaC tools that allow you to define cloud resources using programming languages of your choice.

Now, you have the power to create abstractions that represent your architectural intent. And implement them using real programming languages.

Brilliant! Check out Gregor’s article for a deep dive. One more quote from the article before you do though

Document-oriented automation languages don’t represent data or control flow across application components well.

Enjoy

Read Here

Million Dollar Microservice Mistakes I find this such an interesting view on microservices. From someone who was there in the trenches at the dawn of the buzzword.

Microservices are great in the right hands. But the promise of ‘simplicity’ microservices bring is a misnomer.

It might make your applications simpler, but the infrastructure and integrations to make things work certainly get more complex.

In the land of the monolith, integration is as simple as a method call within the same process.

In the land of microservices, this simple call needs to traverse a network. And networks are hard, just go Google the fallacies of distributed computing.

They create separate “DevOps” squads or teams to manage Kubernetes clusters, centralized CI/CD pipelines, infrastructure, and automation. The outcome? Developers find themselves working in a ready-to-use yet restrictive environment, far from the promised autonomy of the microservices approach.

Any article with a call-out to Conway’s Law is always worth a read.

So before you decide microservices is right for your next project, give this article a read.

Read Here

Getting Shit Done A minimum viable architecture (MVA) is something I am a huge fan of. I’ve spoken with people who have been scarred by enterprise architecture teams, big upfront design and waterfall projects.

Architecture doesn’t have to be that way.

Architecture is an enabler, it allows you think deeply about a problem before you start building. But getting ‘just enough’ architecture is difficult.

This article proposes a set of criteria you can use to define a definition of done (DoD) for your architectures. To ensure you consider the architectural characteristics that are important to you.

So instead of spending weeks and months drawing diagrams and documenting decisions. You can align your architecture to the things that matter most to your business. Once you’ve addressed the things that matter, you have a minimum viable architecture.

Once you have an MvA, you can move on and start building. Safe in the knowledge you’ve thought about all the key components of your system.

The product evolves MVP by MVP. Each MVP should ideally increase customer satisfaction with the product, while each MVA should increase the system’s quality while supporting the MVP delivery

This article also discusses why continuous deployment is vital for success. Honestly, if I was to build a brand new application now. The first thing I would build is my pipeline. Once you can ship changes quickly and safely, well the rest is easy……

Don’t quote me on that.

Read Here

Debugging Cleaning Testing

James Eastham — Wed, 20 Sep 2023 16:00:00 GMT

Debugging

For you architects reading this, you should get involved in debugging and production outages…

I know, I know. Your ivory tower is super comfortable with the reclining chairs and central heating. But the insight you can gain from getting down into the engine room is priceless.

One of the easiest ways to understand the constraints of a system is at the exact point these fringes are being pushed. Production outages and issues tell you that.

In this article, Gregor shows that

A fantastic way in which systems tell us that we are wrong is debugging (including trouble-shooting operational issues). Debugging happens when reality deviates from what you intended it to be, i.e., you were wrong in some way. The task now is to figure out where you were wrong. That means the happiest moment of debugging is when you find out where you were wrong:

Gregor Hohpe, Debugging Architects

Debugging gives you priceless insight. Which you can take back up to ivory tower and use to improve that component of a system, but also the wider system.

One of your jobs as a modern software architect is to think about the wider system, the architecture of it’s entirety and the interplay between the smaller components.

Understanding where one component went wrong gives you a valuable lesson. You can take that lesson to other components to stop them making the same mistake.

Link

Cleaning

Don’t worry, this newsletter hasn’t magically transformed into good housekeeping.

Let’s talk about clean architecture.

I’ve seen various ‘hot takes’ and commentary on social media recently arguing that clean architecture is overkill.

It adds a lot of unnecessary boilerplate and formality etc etc etc…

This post from the Software Alchemy blog gives you an introduction and also shows how clean architecture fits into the wider ideas of domain driven design.

The ‘hot takes’ are interesting though. And I think there is a middle ground to be found here.

One thing I’ve learned over the years is that simple is good. So if you’re building a simple system (I’m looking at all the CRUD API’s out there), DDD and clean architecture are overkill. A simple repository pattern with some API endpoints will work perfectly.

But, and it’s a big but. If you’re working on a core domain that is complex and evolves quickly, putting time into the structure of your code upfront will save you time in the long run.

Initially, it may seem more complex. But as your system develops, the simplicity becomes clear.

As everything in our industry comes back to, it ALWAYS depends. The best heuristic to use? Simple is good, so keep it simple. Adopt the complexity when the business and system requirements need it.

It’s part of the reason I love AWS Lambda, simple.

Link

Testing

Event Driven Architecture (EDA), one of the most common patterns in modern software development.

So common, I’m going to be talking about it at NDC London 2024.

The benefits of adopting EDA are SEEMINGLY infinite. Loose coupling, asynchronous communication, teams operating independently, extensibility built in.

And then you need to test it…

Testing event driven systems is hard. Mostly because of their asynchronous nature.

If you’re testing a synchronous process, you Act, and then once the response comes back you Assert.

With an event driven system, you Act and then have know way of knowing when is the right time to Assert. Of course, you can assert that your original request completes successfully. But how do you ensure that the asynchronous process you expect to run in the future actually happens?

This article from Steve Morland tackles these testing challenges through a deep dive into how his company actually worked through this problem.

It’s AWS specific, but the process used could be applied anywhere.

Link

Learning Software Architecture; Start With Why!

James Eastham — Wed, 13 Sep 2023 14:00:00 GMT

61,698,933!

That’s the number of views (at the time of writing) Simon Sinek’s TED talk on ‘How great leaders inspire action’ has. A fundamental idea from the talk is the idea of a golden circle, and how we default to approaching things in the complete opposite way.

Many of us approach things with an outside in approach. We start with what, take steps to work out how, and rarely worry too much about the why. This is a flawed approach. Starting with why, at least in business, explains your purpose and why you are doing the thing you are doing. People buy your why, not your what. Yet many businesses start with the what, and completely lose the sense of purpose. Your why drives the relationship with your customers.

You might wonder how this short piece of business advice relates to how we teach software. But let’s take this same golden circle, and apply it to how many of us try to learn about software architecture.

People interested in software architecture often ask me which AWS service to learn first. Or which certification to take. Do I learn about serverless or containers? Event driven architecture or microservices? These areas are inevitably ones you will need to learn about at some point, but they are unlikely to be the best the place to start.

You are starting with what! Instead, start with why.

Starting with Why

Build your knowledge from first principles. Technology iterates on itself quickly, and every iteration takes a problem seen previously and makes it better. These iterations stack on top of each other to give us the technology choices we have today.

On-Premise servers -> Elastic Cloud Servers -> Containers -> Container Orchestrators -> Serverless

The most famous saying in software architecture is it depends. Taking a specific technology driven approach, to learning software architecture limits your ‘it depends’ to the specific niche of technology you are learning. If you have a hammer, everything looks like a nail. If you have a Lambda function, everything looks like it’s event driven.

Instead of asking why exactly are we trying to hang this thing from the wall?

Let’s take databases as an example. DynamoDB is a popular, fast and flexible NoSQL database service provided by AWS. It’s actually my second favourite AWS service (just beaten by Step Functions if you’re interested). DynamoDB is largely schemaless. The schema of the items you write is determined at runtime.

What this means is that you can store almost anything in DynamoDB, in any format. It’s an incredibly flexible database that could probably fit almost any application use case.

That flexibility is one of DynamoDB’s biggest strengths, and also it’s greatest downfall if misused. If you spend all of your time learning about the specific API’s of DynamoDB, you understand single-table design, global secondary indexes, sparse indexes, read and write capacity units but never understand the why… Well then there is every possibility you may choose to use DynamoDB in a scenario where it doesn’t actually fit.

Traditional relational databases still have a perfectly valid use case in modern technology. As do graph and time series databases. There is a reason AWS (currently) has 15 different database services. Yet if you are stuck in your little DynamoDB bubble you never even look up to see what else is out there.

I’m not suggesting everyone needs to go deep on every single different type of database, but understanding the history of why we got to where we are is important. Designing Data-Intensive Applications by Martin Klepmann is an absolute goldmine for this content. Even reading the first 3 chapters of what is an admittedly dense technical book, would give you a huge architectural advantage because you understand enough to make an informed decision.

A Generalist Growth Mindset

In the excellent book Range David Epstein discusses how being a generalist is a fantastic skill to have. The cross-pollination of different ideas, from different domains, leads to better decision making and more creative thinking.

Having a general idea of the use cases for the different database options, a set of first principles that guide your decision making, forces you to start from the problem you are solving and work outwards to what database you are going to use. Suddenly, the decision between 16 different AWS database services becomes less challenging. You’re making your decision based on the specific problem domain, applying your general knowledge of all databases to work out to the best technology choice.

Once you’ve gone through the why and reached the what, it’s time to double down on a growth mindset. Professor Carol Dweck proposed the idea of fixed and growth mindsets. A person holding a fixed mindset assumes that their abilities are immutable, whereas a person with a growth mindset acknowledges that they can acquire and develop any new skill with sufficient time and effort.

Fixed = I don’t know Growth = I don’t know, yet

Baseline your thinking as a generalist and start from first principles, before doubling down on your growth mindset to pickup a new technology as and when the time arises.

Starting from why; a skill all architects, aspiring or otherwise, should cultivate at all times.

As always, thanks for reading. James

Thinking Changing Scaling

James Eastham — Wed, 13 Sep 2023 14:00:00 GMT

Thinking

Thinking; something you spend an awful lot of time doing. Whether that be about what you’re working on now, something new you need to build in the future or a particularly gnarly issue, you can’t get your head round.

So surely, having a set of tools you can use to aid your thinking is useful? Right?

One of my favourite frameworks for thinking about problems is using First Principles.

First Principles thinking boils a problem down to its purest form, the smallest potential building blocks. And uses them building blocks to build up a solution.

Imaging you are building something on AWS. For argument’s sake, you’re deploying an application to a virtual machine. You’re trying and trying but your application won’t scale.

You’ve re-written modules, optimised hot paths, you’ve done everything you possibly could.

First principles thinking boils it down to its core. Forget about your application code, the specific language or technology choice. What you need is a software system that scales to meet X demand. That’s the root of the challenge you’re solving.

Now that you know that, you can build back up. “If I was building a system that scales, I will do X”. Now you have a solution to work towards.

This article from the Farnham Street blog is not technology specific. But will walk you through the process of thinking from first principles.

Top tip, it works much better if you do things the old-fashioned way with a pen and notepad.

Link

Changing

Change, the one and only constant in life.

The only thing you can guarantee at any point in the future is that something will have changed.

The business requirements of your system.

The features of a service you’re using.

The structure of your organisation.

Your life situation.

Change is the only constant. So effectively managing change is a pretty fundamental life skill.

Something you will do in your career in software is bringing about change. Sometimes, you’re the person causing the change and requisite stress for the rest of your organisation.

When you find yourself in that situation, how can you do it effectively? Minimising the feelings of upheaval and stress for the people your change is affecting.

In this article, Mary Lynn Manns (author of the book Fearless Change) proposes that you can use emotional connection to help with the change process.

If you connect with people on a deeper level and recognise how they are feeling, the change process will be a lot smoother. I promise 🙂

For some super practical tips, go read the article.

Link

Scaling

You are working within a certain set of scaling constraints. Whether you’re Netflix servicing millions of users, or a smaller organisation just meeting the needs of your first 100 customers.

Banging your head against the proverbial scaling wall is something you’ll inevitably do in your career.

And obviously, you’ll only realise your system can’t scale once everything is on fire.

A lesser talked about scaling issue, though, is one of architecture. How do you scale the practice of architecture across your entire organisation?

At one end of the spectrum, you have the centralised enterprise architecture team. Done badly, this leads to a team of architects sat in their ivory tower dictating decisions on the rest of the organisation.

At the other end, you shift the responsibility to the individual development teams. Let teams architect their own solutions. Which sounds magical, but then what happens when you have an architectural requirement that affects multiple teams or even the entire organisation?

Scaling architecture is difficult.

Thankfully, Andrew Harmel-Law has some suggestions for you. What’s even better, is that Andrew’s proposed process is so simple, I almost fell off my chair in surprise.

Word of warning, this one is a long read. To tempt you in, I’m going to pull a quote from the opening section:

I’ll explain how, within this alternative approach, everyone can do the architecting they need, safely and efficiently, without everything descending into chaos.

Andrew Harmel-Law

Sounds perfect! So grab a cup of your favourite hot or cold beverage and settle in for the read.

Link

Elevators, Modules & Microservices

James Eastham — Sat, 09 Sep 2023 16:00:00 GMT

The Software Architects Elevator

If you’ve been around the software industry for a period of time you’ll inevitably have come across Gregor Hohpe. If not, have you been living under a rock?

Anyway, amongst the many incredible books Gregor has written (including one of my favourite software books of all time Enterprise Integration Patterns), one of the most impactful on my career is the Architects Elevator. The idea of the architects elevator is that to be a successful architect you need to spend time both in the boardroom, and down in the engine room with development teams.

In this talk, Gregor covers the core ideas of the architects elevator and why it is so important.

Link

Modular Monoliths

The monolith gets a bad rap, and can be an incredibly useful tool in your toolbox when build correctly. There, I said it! Please reach out on Twitter if you wish to continue this debate in a healthy, rational manner 😀

One of the ways to ensure your monolithic application is successful is to make it modular. Logical separations in your application aren’t just for microservices you know. The modular monolith is one of my favourite patterns for building net new applications.

There are a number of benefits of doing this, and I won’t cover them all here. Instead, I’ll let the wonderful Milan Jovanovic talk you through. It this article, Milan discusses the communication patterns in a modular monolith covering both synchronous and asynchronous communication.

Link

Some Love For Microservices

To continue with the system design theme, it would be remiss of me to not include some content on the buzzword of the decade… The microservice.

This presentation by Adrian Cockcroft discusses the lessons learned from building microservices at Netflix. One of the most interesting things I took from this talk is that Adrian didn’t just talk about technical. On opening, he discusses people and culture and how important they are in being successful with microservices.

Which is something I’ve seen time and time again in the industry. The technical implementation of a microservices architecture is well documented. But if the culture of your organisation ain’t ready for microservices, then frankly you’re in for a world of pain.

Link

Art, Production and Domain Driven Design

James Eastham — Wed, 06 Sep 2023 16:00:00 GMT

Engineering as Art

Something many people don’t know about me is that I studied art right until I was 18. I was never technically brilliant, but what I had was creating thinking and plenty of ideas.

Sometimes, I struggled to put them into practice. Which is much like my career in software as well.

When you think about software engineering, art is not be the first thing that comes to mind. But actually, this article by David Grizzanti proposes we are artists.

And much like David, I have finished reading Rick Rubin’s book ‘The Creative Act’

And much like David, I was reading it thinking just how many parallels there are between creative work and our work in the software engineering world. Especially as you move to higher level roles.

If you follow some practices that the more typical artist follows, you could be surprised at how that affects your day to day work.

Adopting a beginner’s mind, being open to new ideas, getting outside your comfort zone or trying out new mediums (I’m looking at you Python). These things help cultivate a skill set that will set you apart.

For an incredibly motivating look at our industry, check out the article below.

Link

Building Production Grade Software

Ah, the words ‘production ready’. Something I talk about all. Of. The. Time. It’s great to show a new technology or service using a simple hello world example. But where we fail as an industry is moving beyond that.

Once I have my hello world, how do I apply that to my big enterprise application with 10,000+ users?

CICD, observability, performance tuning, resilience. There are many more things to think about than writing code.

So when I came across this article from Allen Helton on, I was surprised to ask the same question. What actually is ‘production ready’ software?

With an excellent analogy and tangible examples, Allen shows exactly what production grade actually means. As well as pointing out a few key architectural ‘-ilities’. And I agree with lots of his points.

That’s not to say you will never write another POC. I just ask that you keep things like performance, maintainability, operability and resilience in the back of your mind as your building it.

Link

Doman Driven Design in the Real World

And to round out this issue, I wanted to share a post from one of my favourite new writers I’ve discovered in the last few weeks. A mentor recommended hila to me, so I had to dive straight in.

In this article, Hila covers the practical aspects of Domain Driven Design. As a long-time advocate of DDD (the big blue book was the first software book I ever read) it’s great to see it applied in a real world setting.

I LOVE, how the ways of working at Augury are wrapped in ‘when → then’ statements to explain exactly when to use certain concepts.

I also LOVE, how the core ideas of DDD have been tweaked and changed to fit the specific needs of an organisation.

DDD is a fantastic book, and a fantastic framework for thinking about software. But the ideas need to fit YOUR organisation. And these slight changes are a great example of how DDD can work for you.

It also touched on one of my favourite topics, application integration. And patterns you can apply as you build cross-cutting services and shared interfaces you can use across your entire organisation.

It even gives you an example of how to take an existing integration point, and refactor that into a different integration pattern.

Brilliant stuff. Follow Hila once you fit the link below.

Link

The Essential Art of Sustainable Software Architecture

James Eastham — Thu, 27 Apr 2023 00:00:00 GMT

Sustainability consists of fulfilling the needs of current generations without compromising the needs of future generations, while ensuring a balance between economic growth, environmental care and social well-being.

Sustainability; a word we hear an awful lot and one that means different things to different people. What does it mean to be sustainable? The definition above sums it up for me, fulfil the needs of the present without compromising the future.

When we think about sustainability, our minds jump to climate change, making sure we re-use our shopping bags and making sure we recycle our plastic bottles. These are noble causes, as technologists, we have the potential to make a bigger impact.

Software is taking over the world. Every single organisation on the planet is now considering itself a technology company.

This gives us a huge opportunity to have a tremendous impact. Huge portions of the world are using software every single day. Is it not our responsibility to ensure that the software we architect meets the needs of the present, whilst optimising for the future?

What makes up the art of sustainable software architecture, balanced across the environmental, economic and social pillars?

Environmental Software Architecture

The environmental pillar is imperative, and the most obvious when we think about sustainability. To create a more sustainable planet, and aim to mitigate climate change. There is a single word that springs to mind when I think about the environmental impact of our architectures:

Efficiency

The ability to do more with as few resources as possible. The art of having the right resource, in the right place, at just the right time. With some of the recent innovations in software development, efficiency is a practical thing to implement.

Leveraging the cloud for efficiency

A report by S&P Global Market Intelligence discusses the impact of moving workloads to the cloud.

“Simply moving to using green energy is not going to deliver the desired — or the best — carbon reduction. The best place to go from a carbon reduction perspective is the cloud.”

The report estimates that a combination of more up-to-date equipment and a better utilised fleet of hardware amounted to an 85% saving in energy usage. This is before considering the fact that many cloud providers are moving their data centres to use 100% renewable energy.

Architect for Efficiency

Many of you have heard of the cloud, and are likely already running some kind of workload in the cloud. But once in the cloud, how can we architect for efficiency? Thankfully, the architecture patterns that are the most efficient also are the most optimal ways to build modern software.

Serverless-first, event driven architectures are a prevalent architectural pattern when building cloud native systems. Remember, one principle we are looking to optimise our systems is to do more with less. A system that scales to zero when it’s not in use, and reacts to events as they happen… What could’ve more optimised than that?

Let’s unpack the two terms:

Serverless first: A mindset when building software that defaults to leveraging serverless compute options like AWS Lambda. Shifting the operational burden on to the cloud provider. This enables the cloud provider to be efficient with the underlying compute as they can shift workloads around to meet demand.
Event Driven Architecture: A pattern in which business events drive functionality, coupled with an asynchronous first approach. A system which reacts to events as they happen, and defaults to running things asynchronously, provides ample opportunity to optimise for sustainability.

Of course, a serverless event driven architecture may not always be the most optimal or even the most practical. I’m looking at all you GPU compute heavy folks out there. That said, it’s still possible to architect for efficiency.

Defaulting to a single type of compute everywhere in your system is an anti-pattern. Choose the right compute for your use case and optimise accordingly.

If that means running big GPU intensive EC2 instances, then go for it, but consider how you could be more efficient with the compute that you have.

Think asynchronous first, default to using managed services and take intentional step outs away from this pattern where required.

### Code for Efficiency

Code is a fundamental part of any software system. There is so much nuance in an individual business use case that hand written, custom code is inevitable. Many of us (me included) default to familiarity. Debating the merits of a language based on syntax, performance and legibility (I’m looking at you Python).

Something I rarely hear mentioned in programming languages, is the environmental impact. A group of extremely smart people out of various universities in Portugal have ran this analysis for us.

The report linked above analyses 27 well-known software languages and monitors the performance using 10 different programming problems, expressed in each of the languages. An interesting observation from this paper is that:

“Compiled languages tend to be, as expected, the fastest and most energy efficient ones. On average, compiled languages consumed 120J to execute the solutions, while for virtual machine and interpreted languages this value was 576J and 2365J, respectively. This tendency can also be observed for execution time, since compiled languages took 5103ms, virtual machine languages took 20623ms, and interpreted languages took 87614ms (on average).”

I find this even more interesting considering how prevalent interpreted languages like Python and NodeJS are in our modern software world.

Unsurprisingly, the top 5 languages for both energy efficiency and execution time:

C
Rust
C++
Ada
Java

And the bottom 5 for energy:

Perl
Python
Ruby
JRuby
Lua

And execution time:

Lua
Python
Perl
Ruby
Typescript See, my looks at Python weren’t completely off the mark

Quick side note, Rust has also ranked as the most beloved programming language in the Stack Overflow developer survey for 7 years running (as of the 2022 survey).

Now I’m not suggesting you all go away to learn Rust (although it’s a lot of fun) and re-write your entire applications. We will see more on this when we move on to the social pillar of sustainable architecture. But understand the efficiencies of each language is important.

Maybe you don’t re-write your entire application, but you re-write the most commonly used pieces of functionality to use a more efficient language. Taking Rust as a specific example, combined with AWS serverless compute, this is also likely to save you money as well.

When I first looked at the pillars of sustainability, the social one made me stop and think. There is the obvious social impact of software; building software to solve the world’s hardest problems. But can be dive a little deeper into that, what about the day to day social impacts of the people building the software?

I’m sure many of us have worked on applications where the code was an unruly big ball of complicated mud. Equally, I’m sure some of of us have worked inside code bases that are a perfectly modularised thing of beauty.

Not all of us can work on software that is ‘changing the world’, but we can all aim to build software that considers the needs of future generations.

Enjoyable to work on

One of the most important characteristics of social software is a code base that is enjoyable to work on. I want to go to work and have fun solving problems, instead of battling with complexity, un-necessary abstractions and hard to read code.

The boy scout rule is an excellent mantra to use here:

Always leave the code better than you found it.

Simple, easy to remember and forces you to think about continuously improving code. Maybe this is as simple as re-naming a variable to be more expressive. Maybe it’s adding additional services and functions to make the code more intentional.

A rule of thumb I try to follow (and often fail at) is to write code that reads like a story. If you’re using a compiled language, using long easy to read variable names is irrelevant because the compiler will compress them down.

for (var x = 0; x < counter; x++)
{
    var currentIterator = list[x];
}

for (var currentProductIndex = 0; x < maxProducts; currentProductIndex++)
{
    var currentProduct = allProducts[currentProductIndex]
}

I know which I prefer.

Reduce cognitive load

Cognitive load is devastating to a software developer. Our brains can only hold so much information in our head at a time. When working on tasks, it functions more like a stick of RAM than an SSD.

Reducing the cognitive load of a system enables your fellow developers to make more informed decisions about functionality. If I can consume a library with a well defined API taht abstracts away a complex task, that reduces my cognitive load. Giving me more mental cycles to focus on the problem at hand.

Defaulting to a modular design, whether that be a modular monolith or microservices, reduces this cognitive burden. Especially when coupled with domain driven design, defined bounded contexts and organising teams around features and not technical domains.

If I’m a developer working on the product microservice, I can become fully embedded in the world of products within the organisation. I need to get some customer information, well the customer team has a well defined API interface with excellent documentation. They even produced an SDK for communication, excellent! Cognitive load removed.

Fast Feedback

Dopamine; the neurochemical responsible for almost all of our modern addictions. That little burst of pleasure you get when you get a new notification… That’s dopamine in action. It’s not all bad though. We can leverage dopamine in positive ways as well.

Fast feedback loops within the application, typically with a well defined set of test cases, help enable this reinforcement. Imagine working within a code base and continuously running the test cases seeing all of them green icons light up. You can feel that dopamine hit already, can’t you?

These fast feedback loops also help enable mastery of the domain and language. A core component of mastering anything is quickly being able to course correct.

Tests light up green = dopamine hit

Tests go red = an opportunity to increase your mastery

Extensible & Observable

The last topic I wanted to touch on around the social impact is that of extensibility and observability. I want to work within a system in which I can easily extend it to add new functionality, and observe/debug it when things go wrong.

Some of the most frustrating times in my career have been through trying to debug a problem that has little to no logs or tracing.

Building with extensibility and observability in mind will impact the needs of future developers, meeting our commitment to the social pillar.

The extensibility angle ties nicely in with our serverless-first, event driven mindset. An event driven system is extensible. If your system behaviour is driven by a set of business events shared between bounded contexts through some kind of central event stream, hooking into that event stream to add new functionality becomes incredibly easy. Coupling that central event stream with a schema registry and you are in a fantastically extensible place.

The observability angle applies to much more than the social component. Observability will help us environmentally and economically (you can’t measure what you can’t improve), as well as socially (understanding what the heck is going on in a system).

The mindset of the sustainable architect is to be serverless-first, asynchronous and event driven by nature with observability as a fundamental tenant.

The Economic Pillar of Sustainable Architecture

If the social pillar made me pause for thought, this one had me stumped for a little whilse. The definition of the economic pillar is:

This pillar is based on companies’ ability to contribute to economic development and growth.

As architects and engineers, we have little say in our organisations’ wider policies on contributing to economic development and growth. So what does economic development and growth mean within the context of software?

Within this pillar, I see the practical patterns & steps you can take as an architect to contribute to the development and growth of your software system sustainably. There are many ties with the environmental pillar here, with efficiency being a fundamental component. So let’s consider the economics of the different system components.

Compute

We’ve discussed compute already, defaulting to a serverless-first mindset. But there are several additional patterns and practices to help enable sustainable use of compute.

The first is to optimise for stateless compute. Aim for your application layer to hold little to no state. It helps to enable the event driven, reactive compute we are looking for. If your application requires in-memory state, then you restrict the type of compute you can use, as well as how you and your cloud provider can shift that compute around.

Holding an element of data in memory may be useful from a performance perspective, caching for example, but you should not rely on it.

Second, and related to the earlier point, optimise the actual compute itself. This may mean using more optimised chipsets (x86 vs ARM). Or it may mean optimising the type of compute you have. If you’re within the serverless world, this may mean optimising the memory allocation of your function to get the optimal performance vs memory balance. If you’re in the physical server world, this may use the most optimal instance type (compute vs memory vs network optimised) for your specific use case.

There is not a one size fits all in compute, it always depends.

Networking / Data Transfer

The environmental impact of networking and data transfer is difficult to quantify, as there are so many variables. That doesn’t mean we can’t be more economical though.

Ask yourself the question, what is the minimal viable data I need to perform the task I have? The backend for front-end (BFF) pattern is excellent for this, alongside technologies like GraphQL. The aim here - ensure only useful data is transferred.

Compression is another decision point that we will see again when we discuss storage. Many workloads, including almost every single one I have built, default to JSON as the data transfer medium. Understandably so. It’s human readable and relatively succinct. But actually, as we are passing data over the wire, can we not make this even more efficient?

Protobuf and MessagePack are two such technologies specifically targeting efficient binary serialization, whilst still maintaining an element of schema and operability.

Caching and data transfer distance are two concepts that go hand in hand. That is using content delivery networks (CDN) to get the data and compute as close to your users as possible. Caching data at the end, as well as running compute at the edge, reduces the network ‘cost’ of transferring the data.

Caching is also valuable within our backend systems. If you are working within a micro-services architecture, it’s likely you have an element of communication across a network. Consider if you could cache external data within your own service to reduce the network transfer cost and secondarily reducing the compute load on your other systems (a big tick in the social pillar here as well).

Micro-services architectures, coupled with event driven architecture, also bring up an important point of debate around event schema design. There is a probably a separate blog post within this topic, but determining what you include in your event is important. Events that are too sparse result in many calls back to the producer service to retrieve additional data. A ‘fat’ event results in a larger initial cost as it distributes the event to all the subscribers.

That said, if we truly optimising for sustainability, my preference is for a fat event with a combination of JSON & Binary. We compress the main body of the payload into a binary format using a schema-based compression like Protobuf, with additional metadata in readable JSON. That strikes the right balance between a well-known schema, reduced network calls and an optimally sized payload as well as providing the ability for subscribers to filter data.

{
  "metadata": {
    "eventType": "orderCreated"
  },
  "filters": {
    "orderValue": 256.82,
    "geo": "EN",
    "customerAccountId": "123456",
    "deliveryType": "PREMIUM"
  },
  "payload": "<BINARY_DATA_GOES_HERE>"
}

Storage & Analytics

Many of the concepts around compression discussed above apply equally to storage. The most important architectural consideration here is to understand your data access patterns. Regardless of the storage technology you are using, understanding how the data is going to be accessed is vital in understanding how you can optimise.

For example, let’s consider the use of DynamoDB as a primary data store. Used in the most efficient way, access to DynamoDB uses a combination of partition and sort keys. Any other data stored against an item isn’t available to query.

In this scenario, leverage technologies like Protobuf to serialize the main bulk record to binary and store that in a more efficient way. An additional benefit here, you’ll probably reduce your costs as well.

Considering your access patterns will also enable you to optimise the performance of your queries, having the knock on effect of optimising your compute. It’s a wonderful sustainability flywheel we are getting ourselves into, isn’t it?

The other consideration around the data itself is what you store and how long you store it for. Continuously ask yourself the question about the data being stored and if all of it is necessary. And if it is, how long do you need to keep it around for?

And finally, think about your recovery point objective and recovery time objective. Do you really need to be taking full database backups of your product review database every 5 minutes and storing them for up to 3 years? Probably not. Much like the compute option, fit the technology to the use case. There is no such thing as a one size fits all storage solution.

Now some of these ideas become less useful when considering analytics. A nicely compressed piece of binary data is excellent for our storage costs and retrieval efficiency, not so useful for running custom analytics queries. How can we be pro-active about our analytics workloads?

Unlike the compute recommendation of being reactive, only running applications when required, for analytics I would suggest being much more pro-active. Specifically, transform your data into a format that is suited for custom queries. Extract, transform, load (ETL) jobs have been around for a long time. One of my first jobs was developing ETL jobs. Using these pro-active mechanisms for pre-formatting data ensures that when these complex, custom queries come in, they are working in data sorted in the most efficient way.

If you combine this with scheduling your ETL jobs to run during low-carbon intensive periods, you can really push up the sustainability of your analytics workloads.

Charachterize. Observe. Improve

I wanted to close out with a very specific section on architectural thinking. All architectures comprise the architectural ‘-ilities’. These ilities are often at odds with each other and are at the core of the often heard phrase ‘it depends’.

Performance
Scalability
Availability
Reliability
Security
Maintainability
Flexibility
Configurability
Personalizability
Usability
Portability
Conformance to standards
Efficiency
Responsiveness
Interoperability
Upgradability
Auditability
Transactionality
Adminstrability
Sustainability (my addition)

For example, it’s difficult to combine performance and security. Increasing security typically means slower performance. If we consider sustainability to be a non-negotiable, that naturally makes some of the other ‘-ilities’ fall become clear as well. This is very much my opinion, but these are what I would consider the most important ilities for sustainability.

Scalability
Security (it’s job zero, it makes it on to every list)
Flexibility
Usability
Efficiency
Interoperability
Upgradability
Auditability

Once we have defined the charachterisitics we want our system to have, we can use fitness functions to ensure we have continual feedback for conformance to the core tenets of our system.

For example, if scalability and efficiency are our core tenets to enable sustainability, we may write fitness functions to:

Monitor resource usage and how optimally we are consuming the resources we have allocated to a task
Monitor how often we are using compute when there is no work to be done
Ensure no individual piece of compute runs for longer than 100ms, and uses more than 128mb of memory
Monitor scaling behaviour, and how quickly our system adapts to demand

Fitness functions should run both as triggers, as part of CICD, and continually as part of ongoing system monitoring.

For all of that to work, observability is another non-negotiable ‘-ility’ in all of our sustainable architectures. Generating trace and metrics data to understand our systems enables us to be data driven about how we optimise and gives our fitness functions something to work with.

Wrapping Up

If you’ve stuck with it this far, thanks for reading. If I’ve lost you, well, you’ve probably not got this far anyway, so this sentence is becoming irrelevant.

Frankly, this post has been a bit of a brain dump. A coming together of my technical skills and my values to want to inspire change in the world.

I truly believe that as software practitioners, we have immense potential to bring about genuine change in the world.

Let that carry into everything you do.

James

Essentialism in Software

James Eastham — Fri, 31 Mar 2023 06:25:00 GMT

“The Way of the Essentialist involves doing less, but better, so you can make the highest possible contribution. The Way of the Essentialist isn’t about getting more done in less time. It’s not about getting less done. It’s about getting only the right things done.”

Greg McKeown

Essentialism is a book that I frequently revisit, typically at least once or twice per year. The core idea is to focus all of your energy on one thing and do it well, rather than spreading yourself thin across multiple tasks. This approach differs from the norm, where many of us tend to work on multiple things simultaneously, leading to a dissipation of our energy in different directions.

This might be multi-tasking in the well known sense. Focused work, whilst checking email every couple of minutes and being distracted by constant Slack notifications. Or it might be the multi-tasking that sits slightly under the surface. The kind that has us working on multiple different projects, in multiple different disciplines at the same time.

Having interests in multiple different areas is a great strategy, it allows us to cross-polinate ideas and solve problems in innovative ways. This is discussed at length in the excellent book Range by David Epstein. But a clear seperation helps our brains focus.

I’ve been particularly guilty of this myself. In the past, I would sign up for any and everything that even remotely related to my interests. Learning to say no or delegate has been a tough lesson that I’ve had to learn in the last 18 months. While it may work for a little while, and you may achieve some success, it’s not a sustainable strategy. Saying yes to everything and attempting to advance multiple projects simultaneously inevitably results in little progress anywhere.

Have one focus, commit fully, then move on to the next thing. The next idea. The next area of interest.

Essentialism in software

You’re probably wondering how this relates to software. But there are some interesting ideas we can take from essentialism and apply it to the way we think about software.

This isn’t a new idea. In Domain Driven Design, Eric Evans discusses core, generic, and supporting subdomains:

Core: The unique selling point (USP) of your business, the thing that sets you out from your competitors and delights your custoemrs.
Supporting: Areas of the system that are required for your core domain, but aren’t differentiating with the competition and there aren’t any off the shelf solutions available.
Generic: Areas of the system that are completely undifferentiating.

Already, you can start to see how essentialism fits. As much as possible, the focus of a software organisation should be in the core domain. Improvement in this area, will have the most impact on your customers.

Yet, even with this in mind I see many companies getting lost in supporting and generic subdomains forgetting the value they actually want to deliver to customers. They are expending energy in every possible direction, instead of a single minded focus.

Undifferentiated Heavy Lifting

Werner Vogels, CTO of Amazon, is well known for talking about undifferentiated heavy lifting. The cloud enables you to shovel this work on to the cloud provider leaving you to focus on adding value. This is a big step towards essentilism. Gone are the days of racking servers and crossing your fingers that the backup generator kicks in quickly during a power cut.

But even in the modern, ‘cloud native’ world, there still seems to be an obsession with continuing to focus on this lifting. On these supporting and generic subdomains that your customers don’t care about. Yes, your customers care that your application is always accessible. They don’t care if you’re doing that with serverless compute or with a container orchestrator.

Let’s stick with the container orchestrator idea for a moment. I don’t want this next section to feel like I’m shitting on Kubernetes and I fully appreciate there are valid use cases for a container orchestrator. But let’s compare and contrast with something like AWS Lambda, or if containers float your boat then AWS AppRunner.

Let’s consider this for a startup or early stage organisation trying to launch a new idea.

Even when levearging managed services like Amazon Elastic Kubernetes Service (EKS), there is an awful lot of infrastructure management and configuration needed. This is before you even get the first MVP of your product in front of a customer.

Comparing that with a service like AWS Lambda, where I can write a piece of code that focuses only on my business logic, deploy that to the cloud and be confident it will scale when required. No infrastructure management, no setup beforehand. An unparalleled time to value.

Low Code Applications

There is a step even a beyond Lambda and serverless compute. A step that enables applications to be built without a single line of application code. What application code requires the least amount of heavy lifting? Code that is never written.

AWS Step Functions is my favorite AWS service. It offers direct integration with almost any AWS service through its SDK integrations. By leveraging this capability, you can potentially build out large parts of your supporting and generic subdomains without writing a single line of application code. With no need for infrastructure management, configuration, or even updates to libraries and changes to code, your non-core subdomains become about as low lift as they could possibly be.

If you’re interested, I have a repository on GitHub that implements a review translation and sentiment analysis service without a single line of application code. Or if video is your thing, my talk at the AWS Community Summit 2022 where I discuss this repository.

The Essentilist Spectrum

I’m not a purist, and appreciate there are absoloutely scenarios where greater control is required. GPU heavy workloads, developer familiarity, longer running workloads. This isn’t a debate about Kubernetes vs Lambda vs native service integrations. That was meant purely as an example.

However, all of these ideas sit along a spectrum of essentialism. At each stage, you choose to introduce an additional pressure on the energy your organisation has to move forwards. To implement features. To delight your customers.

This spectrum fits nicely with a ‘serverless first’ mindset. My view on serverless first is quite simple. Default to as little undifferentiated heavy lifting as possible and increase as required. I would typically couple this with a view on core vs non-core subdomain.

For example, if I am designing a new feature for a service that exists as a supporting sub domain I would go through:

Step Functions with native service integrations
Lambda
AppRunner
Fargte on ECS
EC2

You can apply similar heuristics to areas outside of compute (DynamoDB vs RDS for example).

I realise many decisions are a lot more nuanced, and jumping straight into the implementation details is not typically where I would begin. But once we are down in the weeds, I find this an excellent way to add business value whilst reducing operational overhead.

Our software is essentialist.

Thanks for reading, James

Thinking in Patterns; The Importance of Language in Software Architecture

James Eastham — Fri, 24 Mar 2023 14:19:00 GMT

Language is the foundation upon which all communication is built. It’s how we share our thoughts, feelings and ideas with one another. And in the world of software, language is just as critical. How we talk about and describe the structures and patterns we use to build software can have a significant impact on the success of our projects.

This becomes even more important when building serverless, event driven applications built up of small, loosely joined pieces of functionality. It’s like playing with LEGO bricks, but instead of creating a colorful spaceship, you’re constructing a well-oiled software machine. And we’ve all been there when we can’t quite understand how two pieces fit together. In this blog post, we’ll explore the importance of language in software architecture and how the words we use can shape the way we think.

Patterns Aren’t a New Thing

Software patterns aren’t a new concept. The famous ‘gang of four’ published Design Patterns back in October 1994. Whilst focused on object-oriented programming languages like C# and Java, it provided terminology to help developers communicate with each other.

If someone mentions the builder pattern to me, I understand what they are trying to achieve.

It also aids the readability of code. If I jump into a new code base and see classes named OrderFactory or ConfigurationBuilder I have a rough idea of what the intended functionality of that object is without needing to work through multiple code files.

Patterns reduce cognitive load, providing helpful guidance to our brains when processing new information

There is a part of our brain (the neocortex for any fellow neuroscience nerds) that recognises patterns. The pattern recognition process involves matching information new information with information already stored in the brain. Using a common language aids that matching process, much like a memory pointer in programming.

Following on from the Gang of Four book, Patterns of Enterprise Application Architecture and Enterprise Integration Patterns were both published in 2012. Both are brilliant books to have as a reference if you don’t have them sat on your bookshelf already.

These books are a reference guide to help us talk about the systems we build. To give us a common language, a set of patterns that describe the systems we are building. Somehow, this got lost along the way.

Lost in a sea of implementation details

When was the last time you saw an architecture diagram or design that mentioned a channel adapter? Or using scatter-gather to meet our business use case. Instead, as architects, we default (and I’m guilty of this as much as anyone) to the implementation details. To the services sat underneath.

I pulled this diagram straight from a repository on my GitHub. For someone unfamiliar with AWS services, how would you understand what is going on here? You could establish there are 5 separate components, each made up of 2 or more individual pieces. Yes, you could deep dive into the code and try to work it out. But this diagram tells me nothing about my architecture, and everything about my implementation details.

Both are important, of course. But we are missing a step. We are jumping straight to the implementation without considering what patterns we want to use.I have a particular liking for Amazon Event Bridge. If I need to communicate between 2 systems asynchronously, that’s my default. This is a bias I’m trying to break. So let’s explore this requirement in a little more detail.

There are 2 systems that need to communicate asynchronously, which might seem like a pretty straightforward ask. Depending on biases you hold from systems you’ve worked on in the past, or what you’re most familiar with, you’ll probably end up with one of:

Amazon EventBridge
Amazon SNS
Amazon SQS
Amazon Kinesis
Kafka
RabbitMQ
Other message broker/streaming/queuing system that allows components to be decoupled

The question then becomes, which one do we pick? Comparing service to service at an implementation detail level will lead to a lot of confusion. It’s the most common question I get asked when talking about event driven architectures. What’s the difference between SNS & EventBridge? It’s confusing, right?

There’s an easier way than getting lost in the features of a specific service, though. That’s through good communication and the use of patterns.

A Better Way

This is where communication is a vital tool in the toolbox of any architect. The answer to the question of ‘I need to communicate between 2 systems asynchronously, which service do I use?’ is, of course, another set of questions:

Do you need to guarantee that only one receiver processes the message?
Do you need to distribute the message to an unknown number of receivers?
Do you want to keep each type of message on it’s own individual channel?

There are a bunch more questions we could ask here. But what began as a simple question that my bias told it could answer with Event Bridge turned out to not quite be that simple. And, in my hypothetical scenario, the business requirement is for strict ordering and a guarantee that only one receiver ever processes the message. My first decision was one that future James would not have thanked me for.

Taking that same list of questions, we can apply a pattern which then leads to the right implementation detail falling out:

Do you need to guarantee that only one receiver processes the message?
- Point to point channel / Amazon SQS
Do you need to distribute the message to an unknown number of receivers?
- Publish Subscribe / Amazon Event Bridge
Do you want to keep each type of message on it’s own individual channel?
- Message Channels / Amazon SNS

Describing our architectures through the lens of patterns & requirements, as opposed to services, simplifies the communication between both existing and new developers. If I join a new team, and I’m told we have a point-to-point integration between these two services that guarantees only one receiver processes the message, I have learned an awful lot about the system function. If I’m shown an architecture diagram that contains a map of service icons, I’m up the creek of implementation details without a paddle.

Knock on Affects

There’s a more subtle benefit to thinking about systems in this way, and that’s one of bias. I might work for weeks on a well-architected system using specific AWS services only to be told that ‘You can’t use Event Bridge because the security team said so’. My architecture diagram falls to pieces. I might push back, or spend hours with the security team discussing what we can do to make Event Bridge available. This might be the right approach, but there’s also a pretty good chance it’s sunk cost bias at play.

If I built my architecture diagram using patterns, instead of an Event Bridge service icon, there would be a ‘publish subscribe’ channel. Event Bridge isn’t usable, ok then let’s use SNS. The CTO mandates we must use non-AWS managed services, ok let’s substitute in Kafka or RabbitMQ.

One might look prettier, but the second much more clearly demonstrates my intentions as an architect.

Think in Patterns, not in services

The risk here, is a re-introduction of the ivory tower architect. An architect who thinks in patterns and abstract concepts but never spends any time building. That is an important point to close this post. Think in patterns, design in patterns, but never lose that builder mindset. The balance is key in the world of software architecture.

Thanks for reading, James

Business Logic First, Implementation Details Second

James Eastham — Sun, 21 Aug 2022 00:00:00 GMT

Start with why?

Have you ever made an implementation decision in an application that turned out to be incorrect? Have you ever got a ‘production ready’ system and realised you have no way to test it? If so, this post is for you.

If I was to ask you what is it that makes your software systems stand out, what would you say? Would it be the fact your entire system runs on AWS Lambda? Or how about that you’ve got a custom built Kubernetes platform that developers can deploy to? NoSQL vs SQL?

Of course, it’s none of these things. What makes your system stand out is its core features. What is the unique selling point that convinces your customers to choose your product over one of its competitors? It’s how quickly you can respond to the changing needs of your customers.

Anything outside of these two points is an implementation detail:

What is the core domain of your system that makes it stand out from the crowd?
How maintainable is it and how quickly can you respond to change?

Making a technology choice because everyone else is doing it is not a convincing reason to do anything. I learned that lesson the hard way. In a previous job, I spent days implementing Kafka to allow 5 loosely coupled services to run independently. It was an interesting technology to work with but, in hindsight, the whole thing would have run better as a monolith. Couple that with the fact I was the only one who understood Kafka, and the maintainability took a nosedive.

In my attempts to reduce the coupling between the parts of my applications, I’d tightly coupled the parts to a more valuable resource. A single developer’s time.

I saw Kafka; I was interested in it, my developer magpie took over and I made an unsuitable implementation decision. I did that because it was interesting. I started from the technology choice, not from the problem space my application was solving.

Serverless or containers? Kubernetes or Amazon ECS? Kafka or Amazon Kinesis?

None of these decisions matter, at least in the short term.

I am not advocating for big upfront design. What I am an advocate of is to start every new software project with your core business logic, forgetting all implementation details. Then let the code drive the design of your system components.

Design with code

What does it mean to do upfront thinking?

Domain Driven Design (DDD) is one of the seminal books on software system design. Written by Eric Evans in August 2003 it should be at the top of the reading list for any architect or engineer.

Amongst the many other concepts in DDD, one of the most useful is the concept of bounded contexts. A bounded context is a logical boundary around a part of your application. Note, this does not need to equal a separate micro-service. It could be a modular monolith.

Each model within a bounded context exists only within that context. Take the simple example of an eCommerce application. The term product has several meanings based on who you speak to. In order processing, we care about the price and the quantity of an order. In inventory management, we care about stock levels.

Whilst these are the same product, conceptually they are different things in our application.

If we were to dive straight in and start building this application, we may end up with a product class like this.

public class Product
{
    public string ItemSKU { get; set; }
    public decimal Price { get; set; }
    public decimal QuantityOrdered { get; set; }

    public decimal QuantityInStock { get; set; }

    public decimal QuantityOnPurchaseOrder { get; set; }

    public void UpdateItemPrice(decimal newPrice)
    {
        //....
    }

    public void AddToPurchaseOrder(decimal quantity)
    {
        //....
    }

    public void Ordered(decimal quantityOnOrder)
    {
        //....
    }
}

Our product class now understands how to manage orders, purchasing and stock. Inevitably, this will lead us down a path of having one big Product class that controls our entire application.

Understanding this up front is a vital part of software maintainability.

We could approach this problem by spending days talking about the system, whiteboarding and drawing up design documents. Whilst this would work, a lot of problems don’t appear until code is written.

There’s a two-step process to get around this, that doesn’t require a lot of designing. A way that gets you writing code quickly whilst not committing down any specific path:

Start with your core business logic, leave all implementation details until the last possible moment
Practice test driven development

Start with your core business logic

Code becomes infinitely more difficult to refactor once you introduce a dependency. Whether that be a database technology, messaging system or a compute provider. At first, only write ‘pure’ business logic code.

Remember, it’s your business logic that is your key differentiator. Getting this right and making it maintainable is going to have a tangible effect on the success of your system.

Going back to our product class for a moment. Writing that code, you may realise that the product class is doing too much. If you had already tied that to a database implementation change becomes difficult. If it is pure business code with any implementations hidden behind well-defined abstractions, change becomes easier.

Seeing this may make you realise that you have a boundary. A product is conceptually different things based on the context it is in. So let’s split that into a separate module.

This strikes the perfect balance. You are getting hands on and writing actual code. No amount of white boarding provides a better way of understanding your problem space. However, it leaves flexibility to change direction.

Test Driven Development

Hand in hand with this approach is test driven development. As you change and develop your core domain, you want to ensure you aren’t breaking any functionality. Practicing test driven development gives you that safety net to change your code whilst ensuring you aren’t breaking any functionality.

Writing code that is testable has the added benefit of producing code that is maintainable.

Let’s imagine you dive straight in and use AWS Lambda because all the cool kids are doing it. We may end up implementing a Lambda function that looks something like this:

public class Function
{
    public async Task<APIGatewayProxyResponse> FunctionHandler(APIGatewayProxyRequest apigProxyEvent)
    {
        if (string.IsNullOrEmpty(apigProxyEvent.Body))
        {
            // Handle HTTP response...
        }

        var command = JsonSerializer.Deserialize<CreateProductCommand>(apigProxyEvent.Body);

        if (string.IsNullOrEmpty(command.ItemSKU) || command.Price <= 0)
        {
            throw new ArgumentException("Invalid command");
        }

        var product = new Product(command.ItemSKU, command.Price);

        using var connection = new SqlConnection("CONNECTION_STRING");

        using var command = new SqlCommand("", connection);

        await connection.OpenAsync();

        await command.ExecuteNonQueryAsync();

				// Additional business logic to update the product quantity in stock...
        // Additional business logic to create an initial purchase order...

        // Handle HTTP response...
    }
}

This code may be functional, but how would you test it? We tied the implementation details up with our business logic, making it almost impossible to test without standing up additional infrastructure.

The second problem with this is that we are now tied to an implementation detail. Here, AWS Lambda. What if we discover during development that the business logic to create a purchase order takes 20 minutes? Given AWS Lambda has a 15 minute timeout, it is unsuitable. But our code is so coupled with the intricacies of Lambda, how would we move away?

If we had approached this differently and started with the business logic, it may have looked like this.

public class CreateProductCommandHandler
{
    private readonly IProductRepository _productRepository;
    private readonly IInventoryService _inventoryService;
    private readonly IPurchasing _purchasingService;

    public CreateProductCommandHandler(IProductRepository productRepository, IInventoryService inventoryService, IPurchasing purchasingService)
    {
        _productRepository = productRepository;
        _inventoryService = inventoryService;
        _purchasingService = purchasingService;
    }

    public async Task<Product> Handler(CreateProductCommand command)
    {
        if (string.IsNullOrEmpty(command.ItemSKU) || command.Price <= 0)
        {
            throw new ArgumentException("Invalid command");
        }

        var product = new Product(command.ItemSKU, command.Price);

        await this._productRepository.Create(product);

        await this._inventoryService.UpdateInventory(product);

        await this._purchasingService.CreatePurchaseOrder(product);

        return product;
    }
}

We now have a core piece of business logic that has no dependencies on anything outside of our code base. This code is easily testable, we can inject mock implementations of our interfaces and use that to understand how different inputs flow through our logic. It also allows us to easily express the intent of our business logic. For anyone reading this who isn’t a .NET developer, I’m sure you can still grasp what the intention of the code.

Test Driven Development won’t solve all our problems. After implementing this, we may still end up committing to AWS Lambda before realising in production that our purchasing service takes 20 minutes. It doesn’t matter though, Lambda is an implementation detail. All our Lambda function is doing is handling the intricacies of Lambda before handing the actual work off to our business logic.

This provides an un-paralleled level of flexibility. Here is my create product process running 3 different hosting options.

Lambda

public class Function
{
    private static CreateProductCommandHandler _handler;

    public Function() : this(null)
    {
    }

    internal Function(CreateProductCommandHandler handler)
    {
        _handler = handler ?? new CreateProductCommandHandler(new SqlProductRepository(), new IInventoryService(), new PurchasingService());
    }

    public async Task<APIGatewayProxyResponse> FunctionHandler(APIGatewayProxyRequest apigProxyEvent)
    {
        if (string.IsNullOrEmpty(apigProxyEvent.Body))
        {
            // Handle HTTP response...
        }

        var command = JsonSerializer.Deserialize<CreateProductCommand>(apigProxyEvent.Body);

        var createdProduct = await this._handker.Handle(command);

        // Handle HTTP response...
    }
}

ASP.NET Web API

[ApiController]
[Route("[controller]")]
public class ProductController : ControllerBase
{
    private readonly CreateProductCommandHandler _handler;

    public ProductController(CreateProductCommandHandler handler)
    {
        _handler = handler;
    }

    [HttpPost(Name = "CreateProduct")]
    public async Task<Product> CreateProduct([FromBody] CreateProductCommand command)
    {
        var createdProduct = await _handler.Handler(command);

        return createdProduct;
    }
}

Console Application

var handler = new CreateProductCommandHandler(new SqlProductRepository(), new IInventoryService(), new PurchasingService());

Console.WriteLine("What is the item SKU?");
var itemSku = Console.ReadLine();

Console.WriteLine("What is the price?");
var price = decimal.Parse(Console.ReadLine());

var command = new CreateProductCommand(itemSku, price);

await handler.Handle(command);

Console.WriteLine("Product created");

You’ll notice the only difference is the specific requirements of the technology. Lambda requires us to parse the HTTP request. ASP.NET takes away a lot of boilerplate and the console app requires us to read data from console inputs.

We can run the same code in 3 different compute options with 0 changes to our business logic. Code that isn’t changed has a much higher chance of stability. Our code is maintainable, flexible and resilient to changes in technology. And one thing that is certain in technology is that things will change.

Writing software in this way makes your core business logic, your unique selling point, more resistant to that change. Who wouldn’t want more of that?

Event Modeling By Example

James Eastham — Wed, 21 Jul 2021 22:12:03 GMT

Good design is a fundamental thing for me when it comes to building applications. Yes, there is a time in a place for a quick functional program that does a simple job ala scheduled batch file. But in almost all cases, committing even a small amount of time to thoughtful design has huge ramifications on the long-term success of a project.

Design for where you want your application to be, not where it is now

An element of this is the principles of Domain-Driven Design (DDD). For those who aren’t aware, DDD is the name of a book written by Eric Evans in 2003 detailing a methodology for designing software. At its core, it maps business domain concepts into software artifacts.

Whilst I can heartily recommend the big blue book, I did find it difficult to take practical steps that I can use when working with clients in the real world. That is where event modeling comes in.

Introduction to event modeling

Event Modeling is a term coined by Adam Dymitruk. It builds upon a lot of the concepts set out in DDD. The main focus is on the events that happen in a business, as all software is at the core is a series of events happening one after another.

If the software focuses on things that the business can understand, the system can be easily understood by anybody looking at it. That includes everyone from C suite execs through the development team and even to customer service representatives with no background in IT.

There are some fantastic resources on Adam’s website https://eventmodeling.org, but what I wanted to-do today was walk through my process for implementing event modeling in a ‘real’ application.

Step by Step

Before I dive into the step-by-step, I want to identify a couple of tools and resources I use for this process.

The first is the article on Adam’s website on ‘What is event modeling’. It adds a lot of context to the steps I’m going to follow in this article. I can recommend going to read that article before continuing with this one.

The second is Miro. Miro is a virtual whiteboard that allows multiple people to collaborate on the same board. Given more and more of the world is moving to a completely remote way of working, it’s an invaluable tool when it is simply impossible to all stand at a physical whiteboard.

Even if you do use a physical whiteboard initially, it’s a great idea to move that to a digital tool for future developments. Online, backed up, and accessible anywhere - what’s not to love.

So now, on to the steps.

Brainstorming

The first step of the process is to brainstorm ALL of the events that affect within the business domain being modeled. It’s important to have stakeholders from all parts of the business in this session as each may have a valuable piece of insight into the workflows.

What you are looking for here is just a brain dump of all the things happening in the business. There is no need to organize or categorize at this stage, just get it all out there. Remember, an event is always in the past tense. SubmitOrder is not an event but OrderSubmitted is.

Timeline

The second step is the first layer of organisation. All of the events should be added to a timeline that shows the journey through the system.

For anyone who has ever ordered food online, this series of events should be familiar.

OrderSubmitted
StockChecked
PaymentProcessed
OrderCooked
OrderDispatched
OrderDelivered

The other thing you’ll notice on this diagram is that I’ve identified a couple of important events. These events are the ones that are fundamental to the business. In this example, taking payments and delivering orders are the two things needed for the restaurant to succeed.

Storyline

The third phase is to begin thinking about user interactions and the different people taking part in the process. Wireframe mockups are a great tool. These mock-ups don’t need to be concrete though they are just to give a rough idea of the interactions with the system.

I find it helpful to split these down into swimlanes for the different people/systems who affect the system. It would also include any external systems (payment processors, CRM systems) that change state, albeit without direct human intervention.

In this example, I’m more concerned about the backend application than the look of the frontend. For that reason, I’ve just included some icons to indicate there will be interactions from either a UI or an external system.

Notice how I’ve included the cog icons for the stock checker and payment processor. It shows that there is a process that happens to cause a change in the system state. The interaction may not be driven by a human, but it is an interaction all the same.

Interactions

Phase four is where things start to get interesting. It is where I begin to link the UI/UX to the events via commands.

Taking the first workflow as a specific example:

A customer interacts with a UX screen that allows them to make an order
The UX sends a SubmitNewOrder command
A OrderSubmitted event is raised

Taking a look at the whiteboard now, there is a clear flow from left to right. This interaction submits this command then this event happens.

It can be beneficial at this point to talk about the command contents. Thinking about what makes up a SubmitStockCheckResults command can help to identify any missing data from earlier in the process.

Informing the users

Phase five is where I start to look at the information users need to make decisions.

I do this using bespoke read models that project the event information into a useful format for users. You can see these below on the green post-it notes.

For the stock checker to reliably check the stock of orders, it needs a view of all the orders that are currently waiting to have their stock checked. For the kitchen to cook the orders, they need to see a list of all the orders that have been paid and are awaiting production.

Again, it’s important to not get bogged down in details here. It doesn’t matter if you’re using a queue, an event stream, or a relational database.

What matters is that the data needs to be exposed in a friendly format that can be used to make informed decisions about the system.

Adam refers to these read models as to-do lists, and I quite like that idea. A to-do list is a concept that almost everybody can understand. If I were to start talking about queues or a messages bus I would likely alienate the non-technical folk.

The system is saying here are some data items that need to have an action performed on them.

I find it’s at this stage that any holes start to become apparent. For example, in the above screenshot, a SubmitPaymentResult command happens results in a PaymentProcessed event. This event has a PaymentSuccess field that looks like it should be conditional in the workflow.

When a PaymentProcessed event happens the order is placed on the OrdersAwaitingCook to-do list. But what if the payment fails?

And this is where the value of event modeling becomes apparent. You have got a room full of people, both technical and not, who can talk the same language and rationalize about the behavior of the system.

Architect: "So, what happens when the payment is processed but it fails for some reason?"

Mark in Order Processing: "Oh, if that happens I would just notify the customer to inform them that their payment has failed."

Architect: "Ok, so how would that interaction happen?"

Barbara in Customer Service: "Well, if the customer was stood in-front of me in the restaurant I could ask them to change their payment method. If they are ordering online, then I need them to know about the failure instantly so they can change their order, or at least know that they aren't going to be getting a pizza delivered."

In a matter of moments, I have learned something about our domain and can amend the flow of events on the fly. In the first iteration of the timeline, I processed payments and then added all orders to the OrdersAwaitingCook to-do list. Now, I have a small branch at this point in the case of an order failing.

Service grouping

Almost there now.

The penultimate stage is to group the events into their service layers. Hopefully by this stage, you should start to see some pretty logical boundaries between the different events in your system.

An important note here, these service layers should be based on the business domain and NOT on a perceived microservices architecture. Remember, you are trying to keep everybody interested and on the same page.

It can be good at this stage to think about the make-up of teams. A team of people, technical and non-technical, can now begin work on an individual service layer. These service layers can then develop independently, but the shared vision is also well understood.

Yep, it sounds a lot like microservices. Resist the temptation, don’t say the word!

Scenarios

The final stage is to build specific scenarios out of the workflows. A scenario is a user story that is built from this set of events, commands, and views.

Now what I have is a list of very specific functional requirements that when pieced together form the basis of the entire system. Each of these individual workflows can be assigned to a specific team and that is the workflow they own.

From here, a more developer-specific specification may be created in the form of a more traditional user story. What is important however, is that this event model is treated as the source of truth that the ENTIRE ORGANISATION can understand.

The development teams may have epics in Jira or cards on a Trello board. It’s largely irrelevant. The importance is the common language the entire system shares and can be used to rationalize different scenarios.

Additional Notes

There are a couple of additional bits of information that I just wanted to tack on at the end to answer some questions I know I had when I first encountered this model.

Implementation is irrelevant

If there is one I would take away from this, it is that the implementation is irrelevant. Whether that’s database providers, to-list implementations, programming languages. Don’t worry about any of it.

The end goal should be for somebody non-technical and unfamiliar with the business to be able to understand what happens within the organisation very quickly.

Imagine starting work at a new company. In your first day, you understand the entire system at a high level and begin to ask questions of both domain experts and developers in a language everybody understands. It wouldn’t matter if you were working in a technical or non-technical role.!

SLA’s can be important

Whilst the implementation is irrelevant it can be useful to note on the diagram any time-sensitive operations.

You can see I’ve marked very explicitly there is a business-specific rule that a stock check must complete in 10 seconds. This is to provide feedback to a user in real-time when they are using the application.

Remember the detail of ‘real-time’ information isn’t important (SignalR, websockets, emails, etc), it is just that there is some form of service level agreement that the system needs to adhere to.

Wireframing

The wireframes on all of my screenshots are icons taken from Miro’s built-in icon sets. My intention with this document is to show the backend architecture of a system using event modeling. I didn’t want to get too bogged down in front-end appearance. At least at this stage.

In a normal working session, these wireframes would be drawn up to look like how you’d actually want the UI to look. If the UI was going to have three inputs, a dropdown, and a button go ahead and create a wireframe in Miro that looks like that.

Wrapping Up

I hope you’ve found this step-by-step useful and that it has given you a good introduction to the concept of event modeling.

I know it’s something I’ll be using in future architecture discussions as another tool to bridge the gap between business domain and technical details.

A huge thanks to Adam Dymitruk for sharing so much information about this process. There is an abundance of YouTube videos out there containing real-world scenarios that Adam works through with businesses.

Resources

Miro Board used in this example

https://eventmodeling.org/posts/what-is-event-modeling/

https://eventmodeling.org/posts/event-modeling-cheatsheet/

Event Modeling with Adam Dymitruk at the DDD Greece Meetup

Serverless Redefined

James Eastham — Sun, 11 Jul 2021 11:43:03 GMT

“Serverless is a way to describe the services, practices, and strategies that enable you to build more agile applications so you can innovate and respond to change faster. With serverless computing, infrastructure management tasks like capacity provisioning and patching are handled by AWS, so you can focus on only writing code that serves your customers.”

Definition of serverless, taken from AWS

For a long time, I had quite a close-minded view of Serverless. I made heavy use of AWS Lambda and DynamoDB and stitched it all together using an API Gateway. Development productivity was through the roof, life was great.

Until it wasn’t.

I was working on a new project which had an extremely variable load (it could go for weeks without being used) but still needed really low latency when it was accessed.

Lambda is great and with provisioned concurrency enabled it was ok from a latency perspective, but the business wanted more. I was out of ideas.

My close-minded view of Serverless kept me banging my head against a wall of cold starts and application tracing to try and eek out as much performance as possible. It was a frustrating time. I knew Lambda was fantastic but couldn’t quite meet the requirements of the customer.my def

Initial simplified architecture

A change in approach

That changed when I listed to an AWS podcast episode, specifically episode #448 talking about AWS Copilot. It struck me for two reasons:

I’ve always been a big fan of ECS, but the cluster configuration, networking and general management was always a blocker. I’ll openly admit networking and server management is a weak part of my skillset, and I’ve never learnt. Deploying from a CLI to an AWS best practice environment removes that blocker completely.
There is more to Serverless than AWS Lambda.

And it’s the second point that really struck me, there is more than one way to deploy an application in a serverless environment. I’m looking at you Fargate, and you App Runner.

Taking a line of text directly from the Copilot documentation - Request-Driven Web Service An AWS App Runner service that auto scales your instances based on incoming traffic and scales down to a baseline instance when there’s no traffic. This option is more cost-effective for HTTP services with sudden bursts in request volumes or low request volumes.

Sudden bursts in request volumes or low request volumes - Doesn’t that just sound like exactly what I am looking for?

So what does that give me? It provides a way for me to easily deploy an application that is ‘always-on’ from an easy-to-use CLI, without the need to worry too much about the underlying infrastructure.

This is when my definition of Serverless changed, and when the opening AWS quote made a lot more sense. Now I’d define Serverless as

“Any managed service that abstracts away operational complexity and allows developers and architects to focus on business value”

From AWS Lambda to Azure Functions to Heroku. All of these services provide a platform that removes a barrier for developers and allows them to just get their code out into the wild.

At a higher scale, there may be an argument for more fine-grained control. But for early startups and small companies, I cannot think of a better and more streamlined way of getting your product out there.

A change in architecture

Whilst there isn’t anything different at first glance, there is one really important distinction.

That distinction is AWS App Runner replacing the Lambda function that is handling client requests.

For people unfamilar with AWS App Runner, it’s a fully managed way of running containers in AWS without the need for managing underlying servers. You simply configure the required amount of CPU and memory and off you go.

The rest of the architecture is identical. Any asynchronous event-driven tasks are still handled by Lambda. This feels like the use case for which Lambda was originally intended. A set of decoupled event handlers that sit doing nothing (and costing little money) until an event happens they are interested in that they react to accordingly.

What this means for the system, is that end users get a low latency API response no matter when they access the app. An App Runner task is ‘always on’ in the same way that running a server would be.

The trade-off here is an increase in cost, in that App Runner is charged per hour. The cost differs slightly when your application is actually processing requests vs sitting and waiting for a request to arrive. For a REST API, that is 24 hours a day 365 days a year. Slightly different than the Lambda model. And what it comes down to is the age-old response of ‘it depends’. Everything depends on the use case.

Lambda is a completely viable way to run an entire application, as long as a higher latency on the initial startup is ok. For high-volume API’s, this is probably not an issue as the functions will always be warm.

However, for my specific use case of a low usage low latency application, the cost trade-off is something we are willing to accept to ensure the end-users have the best possible experience.

The customer always comes first.

For future applications, especially applications built in an event-driven fashion, this is becoming my de-facto architecture. Fargate / AWS App Runner for the client-facing applications, Lambda for the backend services that are less latency driven and more event-driven.

In Practice

Now for a quick example of just how easy it is to get started with AWS Copilot and AppRunner. For that, I’m going to use the order management service in my node-restaurant repository.

Originally, this application was built entirely from lambda functions. The REST API now runs as an express.js application, with any background event handlers running in Lambda. I already had a Dockerfile that I used for some local testing, so I can wire copilot up using that as well. So to get started with copilot, I simply run.


copilot init

The CLI gives a really great step by step walkthrogh, for reference I selected:

Request Driven Web Service
Name the application (order-api) Important note, AppRunner has a max length on the name field which is the application name plus the service name
Select the Dockerfile (helpfully, copilot defaults as it’s the only one I have)
Yes to the test environment

After a minute or two you should start to see the output from the Docker build command and then the beginning of the deployment. A note here, the first time you push out to AppRunner it does take quite a while to provision the infrastructure. Go grab a coffee or something and come back later.


✔ Proposing infrastructure changes for stack node-restaurant-test-order-api
- Creating the infrastructure for stack node-restaurant-test-order-api            [create complete]  [317.2s]
  - An IAM Role for App Runner to use on your behalf to pull your image from ECR  [create complete]  [17.9s]
  - An Addons CloudFormation Stack for your additional AWS resources              [create complete]  [28.7s]
  - An IAM role to control permissions for the containers in your service         [create complete]  [17.9s]
  - An App Runner service to run and manage your containers                       [create complete]  [285.9s]

Once complete, head off to the App Runner UI, and voila the API is ready to serve requests.

To come back around to my earlier point, everything always depends! There are benefits and trade-offs to every service, both AWS Lambda and App Runner are suitable for hosting an application. It just depends on the customer use case.

Front End for the .NET Developer with Blazor Web Assembly

James Eastham — Sun, 23 Feb 2020 17:47:43 GMT

Throughout my entire development career, I’ve always found web to be the best way of deploying an interface to any software I’ve written.

Desktop never really appealed to me, maybe that comes from growing up almost entirely in an age where everything was web-based.

However, coming from a .NET background, there are some downsides to this web-first approach.

ASP Web Forms were ok, and MVC built upon that a little bit. But nothing ever really came close to the magic of the big Javascript frameworks.

That lead me down a rabbit hole I never truly got to grips with, the magical world of Angular and React.

I’ve toyed with both, completed tutorials and written production apps. Still, neither of them really clicked fully (I did massively prefer React, but that’s a personal preference thing).

Typescript improved upon this pain slightly, bringing strong typing and a more C#-esque way of development.

And then, along came Blazor…

Blazor

I’m not going to dive into too much detail around Blazor, there is plenty of other articles around the web regarding that (the official docs are a great place to start).

To quickly summarise though, Blazor is Microsoft’s challenge to the big JS-based front end frameworks. It allows rich and interactive front end apps to be written entirely in C#. Not a line of Javascript insight.

And with that, let’s dive into some code.

Getting Started

To get started with Blazor, I’m going to add a front end to the team-service I’ve been building throughout this series of articles. It’s a learning process for me as well, so stay with me!

Running the below commands is all it takes to get started with a new Blazor app.

dotnet new -i Microsoft.AspNetCore.Blazor.Templates::3.2.0-preview1.20073.1 -o LeagueManager
dotnet new blazorwasm -o LeagueManager

The first install’s the latest Blazor Web Assembly project template (at the time of writing, it’s 3.2.0 Preview 1). The second command creates a new project.

Following the two commands with the below, gets the sample app up and running.

cd .\LeagueManager\
dotnet run

Out of the box, Blazor applications use Bootstrap and are created with some great examples of the key functionality (namely two-way model binding and making external requests).

First things first, let’s add our own color scheme and font. I’ll also remove a lot of the standard Blazor code leaving a much more basic project structure.

I’m also going to apply one of the principles I learned from React, which is to keep stateful components to an absolute minimum and make the majority of the application dumb.

For that, I’m going to add two additional folders on top of the pre-configured ‘Pages’ folder with two new folders:

Components - Dumb, layout components
Services - Stateful components

Initially, I considered disposing of the Pages folder as well. But keeping actual pages (with routes) in one place seemed more logical.

For this initial piece of front end of development, I need to be able to:

View a list of teams
Create a new team
View details about a specific team
Add players to a team

In my head, that gives me two pages.

A list of all teams and the ability to create a new team
A page to view a specific team containing a list of the team’s players and corresponding data entry form for adding new players

Handling State

Before I get into much detail about the page structure, I first wanted to see how HTTP data access works in Blazor.

Turns out, the answer is extremely simple.

The answer is a huge +1 already for using Blazor as a front end framework. The code written to manage to retrieve data from an external HTTP service is almost identical to code that would run in any kind of .NET application.

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using LeagueManager.Models;
using Microsoft.AspNetCore.Components;
using Microsoft.Extensions.Logging;

namespace LeagueManager.Services
{
    public class TeamState
    {
        private string LastSearch = "";

        public IReadOnlyList<Team> TeamSearchResults { get; private set; }
        public bool SearchInProgress { get; private set; }
        public event Action OnChange;

        private readonly HttpClient _httpClient;
        private readonly ILogger<TeamState> _logger;

        public TeamState(
            HttpClient httpClient,
            ILogger<TeamState> logger)
        {
            this._httpClient = httpClient;
            this._logger = logger;
        }

        public async Task DeletePlayer(Player player)
        {
            try
            {
                this._logger.LogInformation($"{player.TeamId} - {player.Name} - {player.Position}");

                var httpContent = new StringContent(JsonSerializer.Serialize(player), Encoding.UTF8, "application/json");

                var postResult = await this._httpClient.DeleteAsync($"http://localhost:8080/team/{player.TeamId}/players");

                this._logger.LogInformation(postResult.StatusCode.ToString());

                if (postResult.IsSuccessStatusCode)
                {
                    NotifyStateChanged();
                }
                else
                {
                    throw new Exception(await postResult.Content.ReadAsStringAsync());
                }
            }
            catch (Exception ex)
            {
                this._logger.LogError(ex.Message);
                this._logger.LogError("Failure adding player");
            }
        }
        public async Task AddPlayer(Player player)
        {
            try
            {
                this._logger.LogInformation($"{player.TeamId} - {player.Name} - {player.Position}");

                var httpContent = new StringContent(JsonSerializer.Serialize(player), Encoding.UTF8, "application/json");

                var postResult = await this._httpClient.PostAsync($"http://localhost:8080/team/{player.TeamId}/players", httpContent);

                this._logger.LogInformation(postResult.StatusCode.ToString());

                if (postResult.IsSuccessStatusCode)
                {
                    NotifyStateChanged();
                }
                else
                {
                    throw new Exception(await postResult.Content.ReadAsStringAsync());
                }
            }
            catch (Exception ex)
            {
                this._logger.LogError(ex.Message);
                this._logger.LogError("Failure adding player");
            }
        }
        public async Task<Team> GetSpecific(string teamId)
        {
            this._logger.LogInformation("Running HTTP search");

            var team = await this._httpClient.GetJsonAsync<TeamSearchResponse>($"http://localhost:8080/team/{teamId}");

            return team.Team;
        }
        public async Task ReRunSearch()
        {
            await this.Search(LastSearch);
        }
        public async Task Search(string searchTerm)
        {
            try
            {
                this.LastSearch = searchTerm;

                this._logger.LogWarning("Running search");
                this.SearchInProgress = true;

                NotifyStateChanged();

                var searchResult = await this._httpClient.GetJsonAsync<TeamSearchResponse>($"http://localhost:8080/team?search={searchTerm}");

                if (string.IsNullOrEmpty(searchResult.Err))
                {
                    this.TeamSearchResults = searchResult.Teams;
                }

                this.SearchInProgress = false;

                NotifyStateChanged();
            }
            catch (Exception ex)
            {
                this._logger.LogError(ex, ex.Message);
                this._logger.LogError(ex, "Failure running search");
            }
        }
        public async Task CreateTeam(Team team)
        {
            try
            {
                this._logger.LogInformation($"{team.Name}");

                var httpContent = new StringContent(JsonSerializer.Serialize(team), Encoding.UTF8, "application/json");

                var postResult = await this._httpClient.PostAsync($"http://localhost:8080/team", httpContent);

                this._logger.LogInformation(postResult.StatusCode.ToString());

                if (postResult.IsSuccessStatusCode)
                {
                    NotifyStateChanged();
                }
                else
                {
                    throw new Exception(await postResult.Content.ReadAsStringAsync());
                }
            }
            catch (Exception ex)
            {
                this._logger.LogError(ex.Message);
                this._logger.LogError("Failure adding team");
            }
        }

        private void NotifyStateChanged() => OnChange?.Invoke();
    }
}

A reasonably straightforward class that uses Dependency Injection to add a HttpClient and an ILogger. From there, it is mostly just a set of different methods handling the various different CRUD operations.

** Note the terrible hardcoded URL, this is a work in progress :) **

There are a couple of front-end/Blazor specific parts that I do just want to point out, however.

public IReadOnlyList<Team> TeamSearchResults { get; private set; }
public bool SearchInProgress { get; private set; }
public event Action OnChange;

These three properties are all relevant to HTTP in Blazor.

The Blazor application runs completely in a web browser. Therefore this TeamState object is specific to the client accessing the site.

Therefore, whenever search runs we can hold a copy of the returned data in a property for easy retrieval if the search needs to happen again.

SearchInProcess holds a value indicating that the search method is running. Imagine a front end in which a loading bar appears during a search, that could be bound to this property.

The OnChange property is probably the most important of the three. It’s linked directly to this line of code here:

private void NotifyStateChanged() => OnChange?.Invoke();

So, the OnChange property can be subscribed to by an external method. NotifyStateChanged then invokes that external method.

Let’s dive into a real example. Here is the code for the Team List page.

@page "/teams"

@using LeagueManager.Services
@using LeagueManager.Components.Teams

@inject TeamState state

<div id="team-search">
    <div id="new-team">
        <CreateTeam OnSave="this.CreateTeam" />
    </div>
    <div id="search-area">
        <Search OnSearch="state.Search" />
    </div>
    <div id="search-results">
        <SearchResults Teams="state.TeamSearchResults" />
    </div>
</div>

@code {

    protected override async Task OnInitializedAsync()
    {
        state.OnChange += StateHasChanged;

        await state.Search(string.Empty);
    }

    public async Task CreateTeam(Team team)
    {
        await this.state.CreateTeam(team);

        await state.ReRunSearch();
    }
}

An instance of the TeamState object is injected into the page, and on PageLoad (OnInitializedAsync is Blazor’s PageLoad event) the state Blazor page StateHasChanged event is subscribed to the OnChange event of the state injected state class.

So, when a search runs within the TeamState object the OnChange event is triggered. In this instance, the StateHasChanged event of the Blazor page will execute. Taking wording straight from the Microsoft docs:

“StateHasChanged notifies the component that its state has changed. When applicable, calling StateHasChanged causes the component to be rerendered.”

Binding any page to the OnChange event of the state will, therefore, cause the page to be re-rendered whenever our state components notify that something has changed. Cool!

Model Binding

Now that we can retrieve data from the server and give that to the front end, it’d be fantastic to actually get it to display on the page.

Displaying data

Sticking with the best front end practices I’ve taken from JS, my Teams.razor page handles all of the state management. It then passes the data down to components to manage the display.

A good example of this is the SearchResults component.

<div class="col">
    <div class="row py-1">
        @if (Teams == null || Teams.Count == 0)
        {
            <p>No results found.</p>
        }
        else
        {
            <table class="table">
                <thead>
                    <tr>
                        <th scope="col">#</th>
                        <th scope="col">Name</th>
                    </tr>
                </thead>
                <tbody>
                    @foreach (Team team in Teams)
                    {
                        <tr @key="team.Id">
                            <th scope="row">
                                <a href="/teams/@team.Id">@team.Id</a>
                            </th>
                            <td>@team.Name</td>
                        </tr>
                    }
                </tbody>
            </table>
        }
    </div>
</div>

@code
{
    [Parameter]
    public IReadOnlyList<Team> Teams { get; set; }
}

The above snippet raises one of the key parts of Blazor functionality. That being, the Parameter attribute.

The parameter attribute indicates that a value can be passed into the component by way of an HTML element.

In this case, there is a parameter that holds a list of Team objects.

This component can certainly be classified as dumb.

It takes a list of objects and either shows ‘no results found’, or creates an HTML table and loops through each Team object to create the table rows.

C#. In the browser. It’s a magical time to be a .NET developer.

Creating a new record

The final piece of this front end puzzle is creating new records.

My absolute favourite feature of both Angular and React is the ability to be able to initialize a new instance of an object and bind that straight to an input in the HTML.

Luckily, Blazor has taken notes from the JS big boys and offers a similar kind of function.

For that, let’s have a look at the CreatePlayer component.

<EditForm Model=@newPlayer>
    <div class="col">
        <div class="input-group mb-3">
            <input type="text" class="form-control" placeholder="Name" aria-label="Search Term" @bind="newPlayer.Name">
        </div>
        <div class="input-group mb-3">
            <InputSelect @bind-Value=newPlayer.Position class="custom-select">
                <option value="GK">Goalkeeper</option>
                <option value="DEF">Defender</option>
                <option value="MID">Midfielder</option>
                <option value="ST">Striker</option>
            </InputSelect>
        </div>
        <div class="row py-1">
            <div class="ml-auto">
                <button @onclick="() => OnSave.InvokeAsync(newPlayer)" type="button" class="btn btn-danger px-5">
                    Add +
                </button>
            </div>
        </div>
    </div>
</EditForm>
@code
{
    [Parameter]
    public EventCallback<Player> OnSave { get; set; }

    [Parameter]
    public string TeamId { get; set; }

    private Player newPlayer = new Player();

    protected override Task OnInitializedAsync()
    {
        newPlayer.TeamId = this.TeamId;
        return base.OnInitializedAsync();
    }
}

When the page is loaded, a new Player object is created. OnInitialization, the TeamId of the newly created player object is assigned from an input value.

A TeamId is a requirement for a player object, so it probably makes sense to implement some kind of null check on the TeamId property just in case. But that’s a job for another day.

From there, an EditForm component is added to the HTML. An EditForm parses out to a standard HTML form. However, it enables model binding between the DOM and the C# object.

Using the @bind element, the HTML becomes reasonably straightforward. An input is bound to the player name property and a select dropdown is bound to the position property.

When the form is submitted, the OnSave event callback is invoked.

What the hell is an EventCallback?

For that to make a little more sense, let’s take a quick look at a component that consumes this CreatePlayer component.

TeamMetadata.razor

@if (TeamData == null)
{
    <p>Loading...</p>
}
else
{
    <h1>@TeamData.Name</h1>
    <h2>Current Players</h2>
    <CreatePlayer OnSave="OnAddPlayer" TeamId="@TeamId" />
    <div>
        <PlayerList Players="@TeamData.Players" OnDelete="OnDeletePlayer" />
    </div>
}

@code {
    [Parameter]
    public Team TeamData { get; set; }

    [Parameter]
    public string TeamId { get; set; }

    [Parameter]
    public EventCallback<Player> OnAddPlayer { get; set; }

    [Parameter]
    public EventCallback<Player> OnDeletePlayer { get; set; }
}

ViewTeam.razor

@page "/teams/{TeamId}"

@using Microsoft.Extensions.Logging
@using LeagueManager.Components.Teams

@inject TeamState state

<TeamMetadata TeamData="@teamData" TeamId="@TeamId" OnAddPlayer="state.AddPlayer" OnDeletePlayer="state.DeletePlayer" />


@code {
    [Parameter]
    public string TeamId { get; set; }

    private Team teamData { get; set; }

    protected override async Task OnInitializedAsync()
    {
        state.OnChange += StateHasChanged;

        teamData = await state.GetSpecific(this.TeamId).ConfigureAwait(false);
    }

    protected override async Task OnAfterRenderAsync(bool firstRender)
    {
        teamData = await state.GetSpecific(this.TeamId).ConfigureAwait(false);
    }
}

The two code snippets above are for the View Team page, and the corresponding TeamMetadata component within it.

Sticking with the practice of having as little components managing state as possible, the ViewTeam page holds all the logic for loading the players and the creation of a new player.

The state.AddPlayer method is passed down into the OnAddPlayer parameter of the TeamMetadata component, and from there into the OnSave parameter of the CreatePlayer component.

So when the save button is clicked in the CreatePlayer component that event bubbles all the way up to state.AddPlayer method. The newly created and data-bound Player object goes along for the ride and is sent off to our backend server for creation.

I will say it again…

C#. In the browser. It’s a magical time to be a .NET developer.

In Summary

Sometimes I can waffle a little bit with these summary sections. This time, not so much.

Blazor is one of the best web technologies I have ever worked with. I’m probably biased, being a .NET developer. But I no longer need to stumble my way through the world of Javascript.

And that my friends is a magical place to be.

Clean Architecture in .NET Core

James Eastham — Wed, 15 Jan 2020 21:13:01 GMT

I recently wrote an article around the importance of automating the coding standards around your codebase.

Whether that be tabs vs spaces, the number of classes per file or the way comments are structured. It makes the legibility of code so much better when there is a standard and common theme.

The standard repo I use can be found here. I’ve now gone a step further with these standards and added my understanding of the Clean Architecture principles.

Clean Architecture

Clean Architecture is a way of designing and building software first proposed by Uncle Bob Martin in his book of the same name.

This image has been repeated many times around the web (all credit to Uncle Bob) but it really clearly defines the way an application should be designed.

Each circle should only know anything about the circles within, no dependencies should extend outwards.

So what does that mean for our software? All the following examples can be found in my GitHub repo

Entities

Entities are the purest representation of a business domain. In my example application for managing loans for a local bank, the entities are as follows.

using System;
using System.Collections.Generic;
using System.Text;
using CleanArchitecture.Entities.Exceptions;

namespace CleanArchitecture.Core.Entities
{
    /// <summary>
    /// Encapsulates all code for managing loans.
    /// </summary>
    public abstract class Loan
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="Loan"/> class.
        /// </summary>
        /// <param name="principal">The principal valie of the loan.</param>
        /// <param name="rate">The annual interest rate percentage.</param>
        /// <param name="period">The term of the loan.</param>
        internal Loan(decimal principal, decimal rate, decimal period)
        {
            this.Principal = principal;
            this.Rate = rate;
            this.Period = period;
            this.Balance = principal;
        }

        /// <summary>
        /// Gets the standard late fee to be charged.
        /// </summary>
        public virtual decimal LateFee
        {
            get
            {
                return 125M;
            }
        }

        /// <summary>
        /// Gets the initial principal of the loan.
        /// </summary>
        public decimal Principal { get; private set; }

        /// <summary>
        /// Gets the interest rate of the loan.
        /// </summary>
        public decimal Rate { get; private set; }

        /// <summary>
        /// Gets the period the loan will be repayed over.
        /// </summary>
        public decimal Period { get; private set; }

        /// <summary>
        /// Gets the current balance of the loan.
        /// </summary>
        public decimal Balance { get; private set; }

        /// <summary>
        /// Make a payment against the loan.
        /// </summary>
        /// <param name="paymentAmount">The value of the payment made.</param>
        public virtual void MakePayment(decimal paymentAmount)
        {
            var newCalculatedBalance = this.Balance - paymentAmount;

            if (newCalculatedBalance < 0)
            {
                throw new LoanOverpaymentException($"A payment of {paymentAmount} would take the current loan balance below 0");
            }

            this.Balance = this.Balance - paymentAmount;
        }

        /// <summary>
        /// Apply the interest for the elapsed period.
        /// </summary>
        /// <returns>The total accrued interest value.</returns>
        public virtual decimal ApplyInterest()
        {
            this.Balance = this.Principal * (1 + (this.Rate / 100));

            return this.Balance - this.Principal;
        }

        /// <summary>
        /// Charge a late payment fee to this loan.
        /// </summary>
        /// <returns>The new balance after the late fee has been added.</returns>
        public virtual decimal ChargeLateFee()
        {
            this.Balance = this.Balance + this.LateFee;

            return this.Balance;
        }
    }
}

A loan is the most fundamental entity that the bank department works with. Within each loan:

A payment can be made off the balance
Interest can be applied
A late fee can be charged

Following the Open/Closed Principle, the class is created as abstract to allow different types of loans to be created. An example can be seen in the concrete BasicLoan class

using System;
using System.Collections.Generic;
using System.Text;

namespace CleanArchitecture.Core.Entities
{
    /// <summary>
    /// A basic loan implementation.
    /// </summary>
    public class BasicLoan : Loan
    {
        internal BasicLoan(decimal principal, decimal rate, decimal term)
            : base(principal, rate, term)
        {
        }
    }
}

A basic loan doesn’t override any of the base Loan functionality but in the future, it may well be extended.

The bank also has Customers. Again these are represented in a way that is as aligned as possible to the actual business use case. Using the same terminology as the business uses for the entities makes communication between developers and domain experts extremely easy.

using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Text;

[assembly: InternalsVisibleTo("CleanArchitecture.UnitTest")]

namespace CleanArchitecture.Core.Entities
{
    /// <summary>
    /// Encapsulates all logic for a customer entity.
    /// </summary>
    public class Customer
    {
        private int _age;

        /// <summary>
        /// Initializes a new instance of the <see cref="Customer"/> class.
        /// </summary>
        /// <param name="name">The new customers name.</param>
        /// <param name="address">The new customers address.</param>
        /// <param name="dateOfBirth">The new customers date of birth.</param>
        /// <param name="nationalInsuranceNumber">The new customers national insurance number.</param>
        internal Customer(string name, string address, DateTime dateOfBirth, string nationalInsuranceNumber)
        {
            this.CustomerId = Guid.NewGuid().ToString();
            this.Name = name;
            this.Address = address;
            this.DateOfBirth = dateOfBirth;
            this.NationalInsuranceNumber = nationalInsuranceNumber;
        }

        private Customer()
        {
        }

        /// <summary>
        /// Gets the internal identifier of this customer.
        /// </summary>
        public string CustomerId { get; private set; }

        /// <summary>
        /// Gets the name of the customer.
        /// </summary>
        public string Name { get; private set; }

        /// <summary>
        /// Gets the address of the customer.
        /// </summary>
        public string Address { get; private set; }

        /// <summary>
        /// Gets the customers date of birth.
        /// </summary>
        public DateTime DateOfBirth { get; private set; }

        /// <summary>
        /// Gets the national insurance number of the customer.
        /// </summary>
        public string NationalInsuranceNumber { get; private set; }

        /// <summary>
        /// Gets the customers credit score.
        /// </summary>
        public decimal CreditScore { get; private set; }

        /// <summary>
        /// Gets the age of the person based on their <see cref="Customer.DateOfBirth"/>.
        /// </summary>
        public int Age
        {
            get
            {
                if (this._age <= 0)
                {
                    this._age = new DateTime(DateTime.Now.Subtract(this.DateOfBirth).Ticks).Year - 1;
                }

                return this._age;
            }
        }

        internal void UpdateCreditScore(decimal newCreditScore)
        {
            if (newCreditScore != this.CreditScore)
            {
                this.CreditScore = newCreditScore;
            }
        }
    }
}

Within the Entities ‘circle’ I also hold the interface that allows interactions with the customer database.

One of the biggest takeaways from Clean Architecture is to keep the business logic and use cases as far away from the details (dB provider, user interactions) as possible.

It’s for this reason, the Entities have a concept of a ICustomers datastore but no actual understanding or care about how the Customers datastore actually works.

// ------------------------------------------------------------
// Copyright (c) James Eastham.
// ------------------------------------------------------------

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;

namespace CleanArchitecture.Core.Entities
{
    /// <summary>
    /// Encapsulates all persistance of customer records.
    /// </summary>
    public interface ICustomerDatabase
    {
        /// <summary>
        /// Store the customer object in the database.
        /// </summary>
        /// <param name="customer">The <see cref="Customer"/> to be stored.</param>
        public void Store(Customer customer);
    }
}

Once we have the Entities mapped out, we can then move on to the next circle outwards.

Use Cases

Use cases hold an actual business use of the application. If the entities hold the core business objects, the use cases hold the logic on how these objects work together.

Uncle Bob puts it quite nicely (paraphrasing here)

“Using a bank as an example. The calculation of the interest on a loan would come under an entity, as that is a rule that is fundamental to the business. It is a critical business rule.

However, a system that determines if a specific customer should be allowed to take out a loan or not is something that would benefit from being automated. That is a use case. A use case is a description of the way an automated system is used, it specifies the inputs to be provided by the user and the outputs returned to the user.”

Keeping that exact sample in mind, let’s add a use case for checking to see if a customer can take out a loan.

First, we will look at the expected inputs and outputs.

From the conversation we had with a domain expert at the bank, we know there are some key decisions that govern if a loan is accepted or not.

The customer must have a valid name, address and National Insurance number
They must be older than 18
They must have a credit score greater than 500

Translating that logic into request/response objects.

// ------------------------------------------------------------
// Copyright (c) James Eastham.
// ------------------------------------------------------------

using System;

namespace CleanArchitecture.Core.Requests
{
    /// <summary>
    /// Gather required data for a new loan.
    /// </summary>
    protected class GatherContactInfoRequest
        : IRequest<GatherContactInfoResponse>
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="GatherContactInfoRequest"/> class.
        /// </summary>
        /// <param name="name">The applicants name.</param>
        /// <param name="address">The applicants address.</param>
        /// <param name="dateOfBirth">The applicants date of birth.</param>
        /// <param name="nationalInsuranceNumber">The applicants NI number.</param>
        public GatherContactInfoRequest(string name, string address, DateTime dateOfBirth, string nationalInsuranceNumber)
        {
            this.Name = name;
            this.Address = address;
            this.DateOfBirth = dateOfBirth;
            this.NationalInsuranceNumber = nationalInsuranceNumber;
        }

        /// <summary>
        /// Gets the name.
        /// </summary>
        public string Name { get; private set; }

        /// <summary>
        /// Gets the address.
        /// </summary>
        public string Address { get; private set; }

        /// <summary>
        /// Gets the date of birth.
        /// </summary>
        public DateTime DateOfBirth { get; private set; }

        /// <summary>
        /// Gets the National Insurance number.
        /// </summary>
        public string NationalInsuranceNumber { get; private set; }
    }
}

// ------------------------------------------------------------
// Copyright (c) James Eastham.
// ------------------------------------------------------------

using System;
using System.Collections.Generic;

namespace CleanArchitecture.Core.Requests
{
    /// <summary>
    /// Response from a successul gather of ContractInfo.
    /// </summary>
    protected class GatherContactInfoResponse
        : BaseResponse
    {
        internal GatherContactInfoResponse(string name, string address, DateTime dateOfBirth, string nationalInsuranceNumber)
            : base()
        {
            this.Name = name;
            this.Address = address;
            this.DateOfBirth = dateOfBirth;
            this.NationalInsuranceNumber = nationalInsuranceNumber;
        }

        /// <summary>
        /// Gets or sets the applicants name.
        /// </summary>
        public string Name { get; set; }

        /// <summary>
        /// Gets or sets the applicants address.
        /// </summary>
        public string Address { get; set; }

        /// <summary>
        /// Gets or sets the applicants date of birth.
        /// </summary>
        public DateTime DateOfBirth { get; set; }

        /// <summary>
        /// Gets or sets the applicants national insurance number.
        /// </summary>
        public string NationalInsuranceNumber { get; set; }

        /// <summary>
        /// Gets or sets the applicants credit score.
        /// </summary>
        public decimal CreditScore { get; set; }

        /// <summary>
        /// Gets a value indicating if the customer has been accepted for a loan.
        /// </summary>
        public bool IsAcceptedForLoan  { get; private set; }
    }
}

We end up with the two above classes. Name, address, date of birth and NI number go in. A credit score and ‘IsAcceptedForLoan’ comes out.

Keeping the request and response in mind, and using the recommened best Clean Architecture practice, we can create a GatherContactInfoInteractor.

// ------------------------------------------------------------
// Copyright (c) James Eastham.
// ------------------------------------------------------------

using System;
using System.Threading.Tasks;
using CleanArchitecture.Core.Entities;
using CleanArchitecture.Core.Services;

namespace CleanArchitecture.Core.Requests
{
    /// <summary>
    /// Handles a <see cref="GatherContactInfoRequest"/>.
    /// </summary>
    public class GatherContactInfoInteractor
        : IRequestHandler<GatherContactInfoRequest, GatherContactInfoResponse>
    {
        private readonly ICreditScoreService _creditScoreService;
        private readonly ICustomerDatabase _customerDatabase;

        /// <summary>
        /// Initializes a new instance of the <see cref="GatherContactInfoInteractor"/> class.
        /// </summary>
        /// <param name="creditScoreService">A <see cref="ICreditScoreService"/>.</param>
        /// <param name="customerDatabase">A <see cref="ICustomerDatabase"/>.</param>
        public GatherContactInfoInteractor(ICreditScoreService creditScoreService, ICustomerDatabase customerDatabase)
        {
            this._creditScoreService = creditScoreService;
            this._customerDatabase = customerDatabase;
        }

        /// <summary>
        /// Handle the given <see cref="GatherContactInfoRequest"/>.
        /// </summary>
        /// <param name="request">A <see cref="GatherContactInfoRequest"/>.</param>
        /// <returns>A <see cref="GatherContactInfoResponse"/>.</returns>
        public GatherContactInfoResponse Handle(GatherContactInfoRequest request)
        {
            if (request is null)
            {
                throw new ArgumentNullException(nameof(request));
            }

            var response = new GatherContactInfoResponse(
                request.Name,
                request.Address,
                request.DateOfBirth,
                request.NationalInsuranceNumber);

            if (string.IsNullOrEmpty(request.Name))
            {
                response.AddError("Name cannot be empty");
            }

            if (string.IsNullOrEmpty(request.Address))
            {
                response.AddError("Address cannot be empty");
            }

            if (string.IsNullOrEmpty(request.NationalInsuranceNumber))
            {
                response.AddError("National Insurance number cannot be empty.");
            }

            var customerRecord = new Customer(request.Name, request.Address, request.DateOfBirth, request.NationalInsuranceNumber);

            if (customerRecord.Age < 18)
            {
                response.AddError("A customer must be over the age of 18");
            }

            if (response.HasError == false)
            {
                response.CreditScore = this._creditScoreService.GetCreditScore(request.NationalInsuranceNumber);

                if (response.CreditScore > 500)
                {
                    this._customerDatabase.Store(customerRecord);
                }
                else
                {
                    response.AddError("Credit score is too low, sorry!");
                }
            }

            return response;
        }
    }
}

I’m not going to talk through the code line by line, but you can see that the business use case is clear and explicit.

A new developer coming into the look at this class can see with reasonably little difficulty what is going on. It’s clear and concise and very close to the actual business domain.

It’s also not a huge stretch to imagine sitting down with a non-programmer and being able to talk through this code file line by line to iron out any issues.

So now we have the core of our business mapped (it’s a very small bank), but how do we actually use the code.

UI

Now we have the business logic worked out, we can start to think about how people at the bank can actually interact with the software.

In this case, the bank are more than happy having a small executable they can run, key in the information manually and wait for a result to come back.

Because all of our business logic is contained in our core library, the console app becomes really simple.

// ------------------------------------------------------------
// Copyright (c) James Eastham.
// ------------------------------------------------------------

using System;
using CleanArchitecture.Core.Requests;

namespace CleanArchitecture.ConsoleApp
{
    /// <summary>
    /// Main application entry point.
    /// </summary>
    public static class Program
    {
        /// <summary>
        /// Main application entry point.
        /// </summary>
        /// <param name="args">Arguments passed into the application at runtime.</param>
        public static void Main(string[] args)
        {
            if (args == null)
            {
                throw new ArgumentException("args cannot be null");
            }

            var getContactInteractor = new GatherContactInfoInteractor(new MockCreditScoreService(), new CustomerDatabaseInMemoryImpl());

            Console.WriteLine("Name?");
            var name = Console.ReadLine();

            Console.WriteLine("Address?");
            var address = Console.ReadLine();

            Console.WriteLine("Date of birth (yyyy-MM-dd)?");
            var dateOfBirth = DateTime.Parse(Console.ReadLine());

            Console.WriteLine("NI Number?");
            var niNumber = Console.ReadLine();

            var getContactResponse = getContactInteractor.Handle(new GatherContactInfoRequest(name, address, dateOfBirth, niNumber));

            if (getContactResponse.HasError)
            {
                foreach (var error in getContactResponse.Errors)
                {
                    Console.WriteLine(error);
                }
            }
            else
            {
                var result = getContactResponse.IsAcceptedForLoan ? "accepted" : "rejected";

                Console.WriteLine($"Credit score is {getContactResponse.CreditScore} so customer has been {result}");
            }

            Console.WriteLine("Press any key to close");
            Console.ReadKey();
        }
    }
}

The UI is only concerned with how the data is presented to the user, it has no concern over how a customer gets validated.

If in 6 months the bank decides they want to allow potential customers to validate themselves through a website instead of manual entry the process is nice and simple.

The EXACT same library can be used, how that interface is served up is completely irrelevant.

Suddenly, this bank can be a lot more agile to changes outside of their control.

If a new technological trend kicks off and banks start identifying potential customers behind their backs (banks haven’t heard of GDPR you see) they can again use the EXACT same library to run the validation.

It’s also at runtime that the implementations of the CreditScoreService and the CustomersDatabase are injected.

Summary

I’ve seen a lot of posts and comments around Clean Architecture stating it can be overkill for a lot of projects.

I do agree with this statement for small utility applications or any ‘one-off’ programs that will be used for a hotfix of any kind. However, I believe any applications that will be used in any kind of production scenario should be built in the right way from the ground up.

Small choices, like decoupling the database from the business logic, make the application so much more resilient to change.

James Eastham | Just a storytelling technologist thinking about systems, software and people.

An Ode To Building What Matters

The Importance Of Arguing Against Yourself!

How To Guarantee You Don't Miss Things When The Pressure Is On?

Becoming Antifragile

What happens to your plans when you are punched in the face?

What Are The Impacts Of Your Decisions? And Then What?

You're A Human, You Can Do It!

What Can Swans Teach You About Making Better Decisions?

What If Today Was The Last Time You Got To Do Your Favourite Thing?

You Must Not Forget About The Entire System

Golf & The Dichotomy of Control

Best Practices Can Be The Worst Decisions

Why I'm Writing More, And Actually Writing

Ergodicity Why Averages Lie and Systems Break

Ergodicity: Why Averages Lie and Systems Break

The migration that shouldn’t have been hard

What ergodicity actually means

The pattern is everywhere

Back to software

The intellectual lineage

One question

Design Thinking and the Question Serverless Should Be Asking

Maybe the Story Is Wrong?

The Container Story is Already Written

Prototyping as a First-Class Citizen

Shifting the Focus Outward

A Few Practical Things I’ve Taken From This

The Internet Gave Us Our Tribe and Took Away Our Neighbours

Social Connection, it’s a good thing right?

The False Promise of Connection

Aim For 1% better

On Flow, Focus & Recovery

The Flow Cycle

Recovery Isn’t Passive

Deep Work Is the Point

The Loop

Why .NET Devs Should Stop Writing Lambda YAML Forever

Infrastructure Shouldn’t Be This Hard

A Better Way: CDK for .NET Developers

Compiling .NET Lambda: No More Zip Files

Permissions: Grant, Don’t Guess

Wiring Up the API

Custom Constructs: Codify Best Practices

Nothing Else to Learn

Practical Takeaway

You're Crazy To Think You Can Predict Your Next Outage!

The Monitoring Trap

Metrics for Systems, Observability for Software

Designing for Change in the First Place

Making Illegal States Impossible

You Can’t Predict Everything, So Stop Trying

Conquer .NET Lambda Performance with these 3 steps

Memory Allocation: The Undervalued Lever

Initialization: The Secret to Fast Successive Invocations

Native AOT: Henry Ford for Your Lambdas

It All Adds Up: A Systems View

Want to See This in Action?

Writing testable, maintable, Lambda Functions in .NET

Writing Testable, Maintainable Lambda Functions in .NET: Beyond the Big Ball of Mud

Surfacing the Real Problem: Untangling the Ball of Mud

A Better Way: Structuring Serverless .NET Like Grown-Up Software

Testability: The Payoff

So Why Does This Still Go Wrong?

The Principle: Treat Your Lambda Functions Like Any Other Code

.NET on AWS Lambda in 20 minutes

Feels Like ASP.NET, Runs on Lambda: Under 10 Minutes to Serverless APIs with .NET Aspire

The line between the familiar and the future in cloud-native .NET

The Familiarity Principle: Serverless Without Sacrificing Syntax

Configuration Done Right: Dependency Injection and That Startup.CS Nostalgia

The Reality Under the Hood: Source Generators and Infrastructure Simplified

Aspire and the Local Feedback Feedback Loop

Evolving with Minimal Friction

Closing the Gap Between API Imagination and Reality

On AI & Creativity

What is creativity?

Creativity in software development

The Simplest Way to Deploy Containerized Apps on Azure in 2025

What Makes Azure Container Apps Special?

Deploying Your First Container App