Evolving with the Changing Requirements: The DDD Dream

00:05 Indu Alagarsamy

What I'm going to talk today, this talk is going to be, is requirements are going to be changing all the time and how can we keep up with those requirement changes and how can we model our systems better to keep up with that change? So I'm going to be making some assumptions in my talk today that all of you are using some form of context or at least striving to get to a bounded context. And I'm going to be writing some code. Hopefully if I had said my prayers to the demo gods, my life code would go well. And, I will be coding in .NET. How many of you are using .NET, C#? Oh, good. Okay. So you'll be able to follow along. And I also assume that you understand messaging or at least understand that in order to do DDD better, you need to have some sort of messaging.

So something about me, yes, I love to rock climb, but besides that, I started getting into venture of an architecture and messaging architecture is about seven years ago. And up until then I used to code everything in WCF. And I thought I was the coolest programmer ever if I could get into the pipeline and look at the inspectors and do some really cool stuff. So that was my thing until, I learned about temporal coupling and then it all went downhill. The temporal coupling sort of like came to me when I was in Starbucks. And when I looked at Starbucks, then how Starbucks really serves their customers, starting from the cashier and how they serve their drinks. You can understand all about messaging and NACE synchrony, and even driven architecture because it reduces all that coupling.

The cashier doesn't wait for the barista to get the drink ready before they come and charge you. So real world operates in a very asynchronous manner. And if we can model our software after the real world where everything is based on events, then that's a good thing. So the whole thing is about change and we want to model our software after the changing requirements. And we're here because we love good design. And the whole purpose of good design is so we can keep up with the business changes. So if the software that we write can't really keep up with the business changes, then we're failing ourselves in our businesses. And the big thing about DDD is it helps us focus more towards the business activities and processes more so than us trying to model the actual data and more like getting away from this entity driven architecture, more towards activities based architecture, where you're focused more on what is bringing the business the money, where are the bottlenecks, where are the trouble points in these business processes and how we can improve them.

So we start to think along those lines, then our software starts to look better already. So let's talk about business processes. If you're wondering what this is, this is a picture of Mycenae, this is Greece and I was there with my kids for vacation. And if you look at this picture, you can see that there is some order or sequence. You can see that some concentrate circles. This is actually a picture of the grave circle in the temple or palace of Agamemnon. So you can see that there is a specific layout and there's a sequence. There's concentrate circles in the architects of those thousands of years ago, they designed it to be this way. And it's like, as soon as you see this picture, everything looks very clear. But then if you try to take it to the business process, it's hard. Where do you start with all these complex rules?

And, one way to look at and model business processes is to use event storming technique. I learned this technique at Metasys workshop, at this DDD conference on DDD and message driven architectures. And so with event storming, you start with the most critical event for that business process. And then you start looking at, what drives that or how you could arrive at that event. And then you start to look backwards. Once you received this event, what are the other possible outcomes and other events that can get triggered as a result of this?

So you start to get a big picture of your business process. And of course, this is something that you should do with the domain expert in the room with you. So you get a better idea and the whole concept here is you're talking about this. So you want to learn about the nuances and ask them, you start the questions with, what is the most struggling point? What is the problem here? Or what are we trying to solve?

So then, you understand the events and you can understand the constraints better. So, as an example of a changing requirement, I want to talk about the American Air Advantage. I am still a proud member of the American Air Advantage, frequent flyer miles program. So the way it works is, before 2017, the way it used to work is, there were three levels of frequent flyer programs like gold, platinum and executive platinum. So all you needed to do was get the 25,000 miles, or like fly how many of our segments? So if you went from San Diego to Denver, 30 times, you're good, you get gold.

Or if you went to London or Europe, a few times, you accumulated those miles you're good. So in this model, if you were to assume that, the loyalty program or the loyalty context had all this data, then it's pretty straightforward as the flight segments, as the customer keeps flying these segments, this context has enough data to compute, Oh, has this customer flown enough segments or enough miles? And you can come to a conclusion and everything is good. And you can publish an event saying like, "Yeah, let's promote this customer to gold." But then in 2017, American Airlines changed their policy. Now it's no longer necessary for you to just fly 25,000 miles, you also need to give them $3,000. And that's how I am no longer a gold customer.

And I flew United all the way to Denver. So there you go. Now, how does this affect our complexity? So the problem now is, that price or the amount that was paid by the customer is not in the loyalty context. It is in a different bounded context. It is in the payments bounded context. So if we are following DDD or a good design principles, we know not to share important data or we don't ask bounded context information like, "Hey, how can I get this?" So you don't do that. So now all of a sudden you're using DDD because you want to do good design practices, but now your data is in three different places. And how do you do this? So this is where messaging comes in and helps DDD. So in order to communicate between different bounded contexts, you use messages, but there are two types of messages. You have commands and you have events.

So to try to understand commands and events, let's take Greece and Spartans, for example. So you have, the Spartans on one hand, and then you had the Persians on one hand. So if you consider them as two different bounded contexts, with completely different rules and stuff, how do they communicate? Well, Persians sent a messenger to Spartans and what did they do? They sent a command. They said surrender. And what happened to the messenger? The messenger got killed, that's right. So you don't send commands between bounded context. What you want to do is, as the Spartans taught us, don't use commands to communicate between bounded context. You want to use events.

So, if you look at commands, the important thing about commands it's, you can use commands within a bounded context because you own the data and you have to understand that commands can fail, so it is also a message. It can fail. So you need to design, what are the compensating actions I need to have if in case I'm sending this command. If this command fails, what do I need to do? So you need to think about that. And on the other hand, events are something, it's like a statement of truth, you only say that when something has already happened. So it's in the past already. So it's like a statement of truth. So how do you go about naming these commands and events?

When you want to model these messages, these are messages. When you want to model these messages, you want to use, of course, a language that is understood by the domain experts and language that implies that context. So, for commands, you use, action verb, like imperative, like what needs to happen? Like the Persians said, "Surrender." So, for events, you want to name them in the past tense, as like flight segment was added or reservation was booked or something in the past tense. You don't need to add tag command at the end or event at the end because you don't need to. Then that's our programmer and us was trying to introduce technology.

Business doesn't need to know whether it's a command or an event. It's a thing. Now you have these three bounded context and a business process for a certain thing to happen, maybe there's a certain sequence of steps. First, a customer needed to have paid this much money. And then they also needed to have flown a certain amount of miles. So it can cross across different bounded context. So this is where in order to model this, a saga pattern can come in handy. There are different, in the internet there's probably a lot of contention around the naming of the pattern. Some purists might argue that it cannot be called a saga, but we will leave that out of this discussion. So basically the saga pattern is where you have this thing and different messages come in and there is state involved because as these different messages come in, the second step of the process needs to know a certain state.

So you have to share the state. And that's where the saga pattern is handy because each message happens like the action of what needs to happen and the message handler it's atomic. It has its own transaction guarantees, but that state can be stored and that state, you can restore or rehydrate that state when the next message comes in to make some decisions on what you need to do. And as a result, you can send more messages out or whatever. So this is in general, like how the saga pattern works. So if I were to show this, an implementation of this pattern. A message comes in and you create the saga instance and the saga instance operates on a state data. So there is some state data involved.

So when the message comes in, you call the right message handler for it. If it's, the customer has paid, then you go and invoke the handler for that message customer has paid. So you dispatch the message invoke the handler. And then if that handler, as part of handling that message needed to send out more messages than you do that. And you persist that information along with the state in some persistent storage. So now let's say, another message comes in that belongs to the same saga instance. So basically like this customer ID one, two, three, like the customer ID has purchased and customer ID that same customer has flown certain miles. So the event that says like, "Hey, I've flown certain amount of miles."

When that message comes in, we need to rehydrate the appropriate saga information back from the database. So, when that message comes in, what we do is we pull based on the ID and the customer ID, the correlation information, we pull the correct saga instance. We rehydrate that along with the state, and then we dispatch the message to the handler. And if at that stage, based on our state data, if we can make that assumption that everything is okay, then we can complete that business process. So how do we actually do this? And this is where I'm going to show you a little demo.

So I coded up this saga last night. Here's the preferred customers gold policy. Basically, as you can see, there are two messages involved. One is segment was flown, which gets triggered whenever a customer has flown a certain segment. And then the other message is that the customer has paid. So the message handlers are very simple. So when this saw the gets, the segment was flown, it invokes this handle method for the segment was flown. So all we're doing here is, we're remembering the state, we're storing the customer ID and we're incrementing the miles. So the state data is actually very simple. It's just a class where we keep whatever information that we want to remember about that process across these messages. So message handlers don't have state by itself. So this is when you want to share the state across multiple message handlers. This is useful.

So whatever information you want to share across these different message handlers, you just put it in your state object. So now, you ask yourself, "Hey, can this customer be promoted to gold?" And if that's the case, we can publish an event saying, "Customer was promoted to gold." And then it's done. And again, keep in mind that, the customer can also be made gold based on the payment. So you need to also check. So if the condition for being promoted to gold is pretty simple. If the miles is greater than 25,000 and the customer has paid X amount of money, then that condition is satisfied. So both these message handlers check for that condition. So if, that goes okay, then the saga is complete. So let me run this.

I hope you can see. So this is the policy code that's going to determine whether or not the customer can be made gold. And then I have this little, X here. That's going to listen to when the customer is made gold, what needs to happen? So what I'm going to do now is, since this is a test application, I'm just going to assume that those messages happen and I'm going to press some buttons. So I'm pressing up. So I'm publishing that segment, the customer has flown a certain of miles. They've flown 10,000 miles. So I have this little tool that lets me see what's the view. Oops, sorry. This is also small. There you go. So if we zoom in, can we, there we go.

So the segment was flown messages, come in and it's updated the state as 10,000 miles. So let me exit this. Now, let me go in and publish a few more. Now the customer is flown at least 30,000 miles. So if we refresh, there is an auto refresh, but it's just not fast enough for me. I sorted it differently. Let me just refresh. Great. So let me just go ahead and just trigger a payment event. Sort by date. Let me sort by date. There we go. Thank you. So now we can see the different messages that participate in this business policy. And as you can see, that the properties are being updated and the miles have been updated and the payment is updated, but still you can't really make them a preferred yet because the customer hasn't paid the entire amount of money. They've just paid 2,500.

So now if we go in refresh one last time, and if I don't click the sort button. You can see that the customer has paid and the amount has been updated to 5,000 now, and that was enough for us to figure out that this condition is satisfied. And we published an event saying, "Customer was promoted to gold." And this other process that was listening to that event has also been notified. So let's say that now American Airlines can say, "Ooh, you have achieved or attained your a gold status." And things like that. So you could add subscribers to do this. Let me stop debugging and switch over.

So this was great, but sometimes we need a model time. Time is a very, very crucial element of business processes. So time can be modeled like using timers, but timers are not durable. If you restart your process, then that customer is never going to be made preferred because you've lost all that information. So how can we model time as a durable thing? We can also use messages to model time. So that's one good way. So let's say the requirement is changed to the point where they're saying, "Yeah, yeah, yeah. You have attained the gold status, but it only takes effect in the next calendar year." So it's great that you've got all this, but your gold status is going to start to kick in like in the calendar year. So now you need to model this sort of time. So let's see how we can do this. And this is where I'm going to do some coding.

Okay. So the first thing we need to do is we need to remember that we need to have this wake up call. So the wake-up call in this case is that a calendar year has occurred or a calendar year has started. So let's just create a class. Can everyone see the code? Okay, great. So let's create a class, this is just to explicitly model this wake-up call. So I'm just going to call this, calendar year has started. So it's just an empty class. The purpose of this is just to model the fact. So the next step is we can't call this business process complete as soon as the customer has attained gold, we have to wait until the calendar year has started. So we need to take out the part where we mark the business process complete. So that's done.

The event there that says customer was promoted to gold is still useful because that's the point in time where the customer attain that status. So that is good to have, and that is still good to publish to the outside world so we can keep that. So now what we want to do is, we want to be able to start this time thing. So what we're going to do is, we're going to request a timeout. And in this timeout, we're going to say... Is it time-span? Nope. I'm just going to use 15 seconds. I don't want to wait for a calendar year. And then I'm going to just pass in the state object calendar year has started. This is an async method, so we want to await this. And because you use async, you got to configure await folds. I wish this was the default. You didn't have to write configure await folds for every time you will wait. Okay, so we've done that.

We got to do the same thing for the other message that comes in. Of course, you can make this code pretty and not repeat yourself and stuff like that. I just am doing this just to keep it simple. So you see that there are two messages, just to make it simple. The saga has this configure, how to find saga. That's just infrastructure magic that in-service bus uses to tie the saga instances together. Sorry, I am using on service bus to implement this saga pattern. And disclaimer, I work for NServiceBus. So you can use Mass Transit if you're in the .NET world and if you're trying to explore other event buses. So, we've requested timeout and now, so wake me up, remind me, but that reminder needs to come back to you. So you need to go in and say, I handle timeouts. And what did we call it? Calendar year has started, and now you need to implement this method. There you go.

This is the method that's going to get invoked when that time has elapsed. So here we can say, "okay, we can publish context." Publish. So there's an event, customer activated to gold, gold status activated. So customer ID equals, our state has the customer ID so we can pass that in. And because this method is async, we need to await this and add the configure await stuff. And because we're awaiting, we need to make this method async and let's just go and write something. I don't know, something. Customers status gold has been activated. How about that? Sofar, so good? Cool. Now let's just run it and see what happens. Let's do this.

Hey, no compilers. That's a good start. Same policy. And then, I will just bring up this subscriber as well. That's listening. Let me get rid of that. So I'll just publish a few messages for the flight and let's see, let me go here. Refresh, note to self, don't click the sort button. So as you can see, the three messages for the segment was a flown has been incorporated. Let's go in and add the new two payments. So this should now have promoted the customer. But now if you see that once we've published that event, customer was promoted to gold. We have also requested this timeout. So here in this case, the timeout is 15 seconds. And so, this calendar year has started. That is just a message that has been dispatched. So that's just to indicate that, "Hey, wake me up after this duration has passed to go and do my operation." And I hope I have talked enough for longer than 15 seconds. So let me refresh. And there you go.

So once the 15 seconds have elapsed, the framework in this case has sent that message, that was told back to you. So this is like your wake up call. And in this case, we now can publish the message single and status activated because it's a new calendar year. So, if we look at the console, now you can see that the other application has also received the event. Gold status is now active. Yeah. How about that? Cool. Thank you. I guess I can leave this. So going back.

As you saw, in whatever, less than 30 minutes, we were able to make some changes and implement a business policy with time calculations and things like that. That wasn't hard. So what is hard about this is the fact that you need to understand the business constraints, you need to ask, what are the rules that is hard to tease out off these domain experts? What is not happening, but is important? For example, let's say, the customer has, in this case me, I got so close. I got 25,000 miles, but I got so close. I paid American Airlines, $2,800, but I was short by $200. So what if the business wants to say, "Hey, we see that you were so close to getting the status. How about you pay us $500 and we will give you this gold status for free."

Well, that's what they do. I got an email from American Airlines saying, "Hey, you can upgrade to gold status and you can just shell out $500." Again, that's a business rule, but you don't tend to see that rule right away because it's not visible or upfront because this is not an event that's happening, but this is an event that you're looking for. So these are the complicated questions that you need to ask your domain experts in trying to tease out these nuances so you can build these into your business policy. So this is the hard part. And also things like, okay, in this context, the goal status was activated. What do we do about people that, let's say, when you moved from the old policy to the new policy, do you just ditch the current people who are already gold, who haven't spent that money? Or do you say, "Hey, let's just send them a notification to let them know that this policy is about to change."

So as a programmer or a designer, we don't have the answers, the business has the answers. So these are some of the things that we need to get out of the domain experts, which is really hard to do. And because we've taught the domain experts our language of how to speak and they try to explain us in user stories or programming language. It makes it even more harder to actually get to the business requirement. And again, if you ask yourself, if you ask your domain expert, what is the troubled point? What are the trouble points that you currently have? Or, if you start looking at the financial aspect of it, of how the business is trying to improve the financial model, how can this business policy affect money?

Because ultimately it's all about the money. Business is trying to make money and they have to change in order to be the coolest business, so they can get more money. And so in order to do that, they have to change. And if they change, then the software that we write has to change. But, if we write software that stays good for three years. And after that, you can't keep up with the change and I'm sure how many of you have at some point thought to yourself, we need to rewrite this all over again? Exactly. So that's not our goal, even though it might be hard in the beginning, we want to make sure we think about these cases. So when that change arrives, we can extend our software. So this is where messaging for me was like an eye-opener because when you start modeling things around events, so business usually says, "When customer became preferred, we want to do something."

Now that is an indication of customer became preferred as an event. So when they say, "When this happens, go do something." Then you can simply just add a subscriber to that event and capture all that logic of what you need to do in that one module. So now, it's easy to change that code. You don't have to touch anything else. Now, this gold policy, you can create one for your platinum, or executive platinum. And let's say they change the platinum policy. You don't have to go and change the gold policy. Everything is separate. So it gives you by modeling your stuff after events and using event-driven architecture along with DDD, it really helps you out. I wanted to give you a link here. This link will take you to Udi's advanced design course, where he talks about long running processes. So there's this access to two videos and you can download them. And I will make my code available on GitHub. It is not there yet because this was written like last night around like 11:30. So I will make it available with sometime today or tomorrow. Thank you very much. Any questions? Yes.

36:47 Speaker 2

So it's interesting that you brought up the idea of a saga pattern. Because we use a similar concept called a process and rather than rely on a timer or a time span to delay certain events from being published, we rely on a timestamp and we rely on another supervisor service that will track for that timestamp to take place. And so, it has its own pros and cons.

Yeah, you're right. The process manager is a very similar pattern, but I think where it differs from the saga, is the saga treats everything as a new instance. So every customer has its own instance. So every customer has this life cycle of things that are being kept track of. And I think the event processor it's just the one thing it looks at collectively everything. So that's the difference. I'm not exactly sure how you would do time in that model.

38:00 Speaker 2

We rely on...

Timestamps?

38:03 Speaker 2

Yeah.

Okay. I'm trying to think. So if in case you need to take actions based on per customer level, then this might make more sense to you. Because the lifespan of this is per customer. So if you have a hundred customers, you have like a hundred instances, one per. So I think that's where it might come handy. Yeah.

38:38 Speaker 2

[inaudible]

Right. Cool. Oh, I'm sorry. One more thing I forgot to mention was, this is a trouble that I had like last night, was, who wants this business process thing that we've written and does the payments context own it? The loyalty own it? Before I called it loyalty, I'd called it like flight segments or something. So I had this struggle of, I have this business policy but it doesn't make sense for flight segment planning thing to own it. And it doesn't make sense for customer payments context to own it. So had this struggle. When we changed that to loyalty, it started to make sense that, yeah, dah, this business processes is talking about keeping track of gold versus platinum. It naturally fits in this context. But sometimes you can have a business process that spans across different contexts.

That could be something like business intelligence where you're making decisions based on a lot of events from a lot of the context. And then, businesses trying to figure out why certain thing has happened or how can I make more money out of this policy? Or what changes do I need to do to this preferred program in order to get more out of this. So they probably did that and that's how they probably added the $3,000 more. So that's one thing I wanted to write.

40:23 Speaker 3

That was my question.

Oh, great. I read your mind. Yes.

40:34 Speaker 4

Since you request timeouts for sagas, when you delay a publisher event. In the event of an outage, how can a saga keep track of how much time has elapsed?

So the timeout is a message. And as part of the saga, it gets persisted. And if for any reason like let's say right when you were trying to save that message into the persistence, it blew up then that particular transaction, it is a transaction, that will roll back and that message that would be in the queue. So, you won't lose the fact that you requested a timeout. And if things fail, like let's say after, where you've persisted the message in the persistence. So now when the process comes back, in this case, NServiceBus infrastructure, the implementation of saga, we'll keep track of that. And we'll scan the persistence and find out what, Oh, these messages have elapsed. We need to send it back to the process.

41:49 Speaker 4

That's good.

Cool. Thanks. Any other questions? Well, thank you very much.

41:57 Speaker 5

Thanks, Indu.

Evolving with the Changing Requirements: The DDD Dream

🔗Transcription