Live Q&A with Udi

00:00 Bob Langley

Thank you for joining us. My name is Bob Langley. Today, I'm joined by Udi Dahan, the author and presenter for ADSD and the person who will be answering your questions today. But just a quick note before we begin. Please use the Q&A feature to ask any questions you may have during today's live webinar, and we'll be sure to address them at the end of the ... Well, we'll address as many of them as possible during the presentation. We'll follow-up offline to answer any questions that we haven't answered during the Q&A. We're also recording this webinar and everyone will receive a link to the recording via email.

00:42 Bob Langley

Okay. Udi, let's take it away. We're going to start with the first question. This question is a question about asynchronous pub/sub as opposed to using synchronous remote calls when you're coupling between services. What is the difference between, what is the advantage of using this over using, say, asynch/await in your controllers to prevent blocking? And should you use HTP calls between services or just use messaging? And if you do use messaging, how do you respond back to the client?

01:28 Udi Dahan

Right. Okay. Thanks Bob. Hi everybody. Great to see so many participants online for this. In the ADSD course I do present the topic of pub/sub as a contrast to the regular synchronous request-response style that most people are familiar with when developing systems. However, the term service that we've discussed later on gets a slightly more refined meaning. A lot of times when people ask the question about whether it's request-response between services or pub/sub between services, part of that means being specific about that definition. And as I understand, we have some more questions that have already been submitted on that point, so we'll get to them a little bit later.

02:23 Udi Dahan

I'd say that the main objective or the main benefit of pub/sub is in representing logical decoupling between things. There are additional technical benefits that I also talk about in the course where we get into things like, again, whether we're holding the thread or not and those types of elements.

02:52 Udi Dahan

One of the topics that gets discussed and it's not a point of confusion or contention is when talking about publish/subscribe versus when we mean data distribution. So for example, if we're doing a kind of SignalR push notification where we're updating clients about something that happened versus those clients that are pulling a given back-end request-response style, essentially it's the same logical relationship between these two pieces. We're just changing the mechanism for the data that is moving between them.

03:37 Udi Dahan

That type of thing I'd say it is better to call that data distribution than to call that pub/sub. So you're talking about logical components that are sitting in different physical places and you're using a mechanism to get data from one place to another. So again, that could be request-response pulling. That could be SignalR notification, data distribution Kafka as another type of infrastructure that supports that.

04:08 Udi Dahan

However, all of those things are physical patterns that deal with where components are on the network, and it's important to distinguish that from the more logically decoupled style of publish/subscribe. So understanding that these are two different types of architecture, data distribution being in the physical architectural world and logically decoupled pub/sub in the logical architectural world. A lot of times people don't realize that they're intermingling those two worlds.

04:47 Udi Dahan

At least for the purpose of this Q&A, what I'm going to be using the term service, I'm going to be referring more to that type of logical entity. And when we're talking about, as we'll get to them later on, components that are talking to each other over a physical network, I'll try to be specific and say, "Okay, that's a component. That is a piece of code that is deployed in a specific place."

05:17 Udi Dahan

So back to the question, the original question now that I've hopefully framed some of that, it's when we have two pieces of code, one that is running on a client that could be a browser or a rich client or a mobile device talking to some other component over HTTP, I wouldn't call either one of those things a service. It's really a question of physical architecture of saying is each one of these client components representing sort of a single user performing a discrete action, in which case a request-response style is perfectly appropriate for that.

06:05 Udi Dahan

Is this a type of case where we want to have sort of a distributive presence? This could be online gaming or chat applications where we want to know are people online, are people offline, are they currently typing, those types of things, where a type of push notification data distribution mechanism would be technically superior because it results in less unnecessary ping pongs and lower latency versus the logical pub/sub world that we'll get to later on. So that would be my summary at least for this point. Over to you Bob.

06:44 Bob Langley

Thanks Udi. The second question is about data. If you have a third party system that is the master source of authority for your data like a CRM system, should services keep a local copy of the data they own and apply updates to and from the CRM? And also, what do you do if you can't update the data? Should you pull and should the service keep a local copy?

07:20 Udi Dahan

Okay. So a bunch of the answer to that question like any consultant will tell you is it depends. There's sort of the tactical, "I'm in this situation. I have limited options. What do I do?" A lot of that is really dependent on the specifics of the situation on the ground.

07:42 Udi Dahan

I will talk now in this answer in broader terms of saying, a given system, whether it's CRM, Salesforce, or otherwise is more of a physical architecture element. It's saying, "There is a given system deployed to a specific set of servers that has a database, and all of that is in a specific place." And now the question is, I need to write code that interacts with that data and/or the functionality in that system. How can I do that?

08:17 Udi Dahan

Now, I'd say the first thing to realize is that a system boundary is not necessarily a separate logical service boundary. So you can think of it as a physical host environment. Salesforce and some other of these types of systems have plug-in capabilities. So you can write code that gets hosted inside that system. And then through that to interact either directly with the data or the functionality that's in there. In terms of the fallacies of distributed computing, that would be preferable rather than doing remote calls. Once we introduce remote calls, that's yet another thing that can go wrong. However, as I mentioned before, the tactical elements of, well, what kind of API, what kind of plug-in abilities does this system afford me, those might be even more limiting than if I were to develop this remotely.

09:18 Udi Dahan

But I'd summarize this by saying don't treat a system boundary like a CRM as a logical service boundary. And I'd say 8 to 9 times out of 10, plugging into one of those systems hosting your code inside it is probably the tactically preferred approach. The alternatives of saying, "Hey, should I pull and create a copy of that data, or should I use some sort of pub/sub mechanism to create a copy of the data?" Creating copies of data is riskier on the balance of it, of things getting out of sync and then users being presented with incorrect information and then all of the negative business ramifications that can come from that.

10:10 Udi Dahan

So I'd say that that's more ... I'm not sure I'd use the term last resort, but it's a I'd want to disqualify a bunch of other options before going and starting to create copies of the raw business data.

10:27 Bob Langley

Thank you Udi. The next question. We have a service like a SMS service for instance and we have several like a capability on the network, like they can send SMS messages and we have several services that need to set to use that capability. How do we use pub/sub to manage that, and how do we make sure that the subscribers, so the results get there, get to only the ones that they're interested in?

11:07 Udi Dahan

Okay. This topic of using the term service loosely is also to some extent present here. It's assuming that an SMS service is a service. But I'd say that it's a piece of code or a collection of components running in one or more hosts that we can interact with. And I know that in the course and in my presentations, I also say, "Hey, look for pub/sub. Try to find that decoupling." However, the challenge is with any good pattern is that people take it and apply it as a best practice and saying, "Pub/sub is awesome, I'm going to use it for everything."

11:58 Udi Dahan

If there is a natural request-response interaction for the behavior that we're talking about. So we have a given piece of code that says, "Under these circumstances, an SMS message should be sent." And before I continue with my workflow, I need to have confirmation that that SMS was actually sent. In a case like that where you have two components, one of them that is able to identify now is the time to send an SMS and a component that actually sends the SMS, it could be that a full-duplex request-response approach is more applicable and that introducing pub/sub it's kind of like scratching your left ear with your right hand, of saying, "Okay, so I'm going to sit, I'm going to publish the send an SMS message requested event and pretend like I don't know who subscribed to that, and then I'm going to subscribe to the SMS sent event, and then I'm going to correlate those things on my end after the fact." You're really doing request-response.

13:25 Udi Dahan

If the nature of the interaction is request-response, model it as request-response. If you're talking about a component like sending an SMS or integrating with some third party that can take some time, that is a good place to introduce messaging so that it is not ... you don't sort of lose an HTTP timeout and then say, "Well, I don't know. Was the SMS sent or not?" So having the reliability of messaging both for the request that I'm sending and for the response that I'm receiving back allows me to simplify how I manage my long running workflow thing, which I understand we have a question later on talking about sagas.

14:12 Udi Dahan

And sagas have a natural, or sorry, sagas in NServiceBus have a natural built-in ability to do a lot of this correlation for you. So I'd say don't just jump to pub/sub immediately if the nature of the interaction is more naturally request-response. And if you find yourself doing a lot of intricate things like saying, "Okay, I'm going to try to do some content-based filtering on the subscriber side based on something," then there's a fair to good chance that you should be doing request-response rather than pub/sub, and that trying to do pub/sub is what is causing or trying to do pub/sub where it's not a fit is what is bringing about that complexity.

15:10 Bob Langley

Thank you Udi. The next question is about incrementing share on our collaborative counters, and the person asks about using an approach where they hand out blocks of resources and allow each person to work on that block as opposed to using it. Is there any other approaches to an atomic increment operation, will this kind of approach work, and is there other approaches other than to use an atomic increment?

15:48 Udi Dahan

Right. I've seen people come up with all sorts of very creative approaches to deal with a case where the load on the system continues to grow and that load is highly localized. So there's some small set of states that lots of transactions are constantly operating on. The example that I give in my course is inventory of a very popular item. So you'll have lots of people that want to buy it at the same time and also back office systems that want to replenish that inventory. So you end up getting a lot of contention on that, and then people try to find ways to kind of redesign or come up with ways to reduce that contention, and one of them is to say, "All right, instead of having everything just operate on one thing, we'll create these separate blocks of things."

16:54 Udi Dahan

I would say that that is a useful and interesting approach but I'm afraid that at the business level more so than at a technical implementation level. One of the examples that I give when I talk about CQRS is for example people buying tickets online for a very popular event. It could be Super Bowl final game or some very popular music concert or something. In that example I talk about creating a business subdivision of saying, "Well, we're going to have our VIP customers," and essentially we subdivided the user population by saying, "Hey, if you want to be a VIP customer, then you need to perform this thing, you need to commit to pre-authorizing a certain amount of money." And then that gives a window of time for those VIP customers. And after that we do group reservations which operate another slightly different way, and then we handle other groups.

17:59 Udi Dahan

So this behavior of taking a big load problem and subdividing it into smaller chunks is a good technique. However, I'd say don't view that as a technical technique. Seek out a business subdivision. That will be your greatest point of leverage.

18:19 Udi Dahan

Now, if there isn't the possibility and you're dealing with truly again sort of the inventory of a single item and it's not subdividable, that's when other persistence technologies like Reac come in. Reac was, I think, the original data store that implemented atomic increments, and they did that for the Facebook Like button because lots and lots of people had started liking things, and doing that in a non-blocking manner was the thing that Reac introduced.

18:53 Udi Dahan

Subsequently, there have been many other persistence technologies that support that and now that's known under the general category of conflict-free replicatable data type, CRDTs. So if you want to check out more about that thing, read up on CRTDTs.

19:13 Bob Langley

Great. Thanks Udi. The next question is about micro services architecture, and that it seems a popular opinion swinging away from it because of its increased complexity and managing the dependencies and relationships between your micro services. And you talk about a layered monolith is another alternative, there's another simpler alternative and service orientation. How do we keep the benefits of a service oriented approach while mitigating the risks of this complexity and dependencies?

19:50 Udi Dahan

Right. So I'd say part of the issue with that's called the popularity around micro services was on the micro side and the misunderstanding of service, specifically the remotely deployable, remotely independently versionable, all of those types of things. And that led us to fallacies of distributed computing territory.

20:21 Udi Dahan

I would say that if you're looking at an architecture where you have a large number of components doing essentially request-response communication over a network, yes, moving back to a monolith, hosting all of those components in the same process would be better. I wouldn't say that either one of those are service-oriented or micro service-oriented. The trick, if you will, is the logical subdivision.

20:53 Udi Dahan

I wouldn't say that service orientation done correctly creates accidental complexity. I'd say almost any architectural approach done incorrectly is most likely going to create a whole bunch of unnecessary complexity. So I'd say the advice for getting the benefits of a service-oriented approach is, number one, don't equate logical decoupling to physical decoupling. Just be aware that these are two different things you can do. You can do an event-driven architectural style between pieces of code and still host those in the same process. That's been a thing ever since I think callbacks were introduced into programming. Yeah, we can create logical decoupling without sticking a network in the middle of it there.

21:50 Udi Dahan

So I'd say a lot of what service orientation is about is the logical decoupling first and foremost. And a lot of what I talk about in the five-day ADSD course is detailed advice on how to go about doing that and example and example after example on what that would look like in practice. I'd say get more people that if you're trying to convince them to watch those videos, and then if needed, talk to one of our folks here at Particular to talk through the details of your situation, and we'll be able to help out that way.

22:31 Bob Langley

Thank you Udi. This question is somewhat related to the last one. So again, service boundaries should avoid ... You say that service boundaries should own their data and they should avoid crossing network boundaries when possible. How does this relate to the layers of your application, the UI to your maybe API gateway and then to your database and the message broker? How does that relate to this advice about service boundaries?

23:01 Udi Dahan

Okay. I'd say a lot of those things, you got your code that's running in the browser on a mobile device, which is then talking through several, I'm going to use the term tiers here specifically, which is crossing network boundaries to then talk to some kind of API gateway, which is then going and talking to some kind of other API code running on a different machine.

23:28 Udi Dahan

All of those things are logically part of the same flow, part of the same service. So maybe here's where I can't avoid it and more say what is a service? A service is the technical authority for a given business capability. So business capability meaning all of the data and all of the rules that form that larger business capability are part of that service. The code on the client that's calling the API gateway that's calling the rest that's calling the database that's ... all of that, that flow request-response along those different tiers, that would be in the side one of logical service.

24:20 Udi Dahan

In that case I'd say don't introduce unnecessary network hubs because of the fallacies of distributed computing. If you can simplify those, they're going to be logically coupled to each other anyway, all right? Because they're dealing with substantially the same business problem, the same data structures, the same flow, the same logic. If things are going to be logically coupled, don't spend huge amounts of effort trying to decouple them. Accept it and bring those things closer together.

24:55 Udi Dahan

When we're talking about different business capabilities where we can introduce a more event-driven or messaging style between them, that after some flow happens in one place it can raise an event and multiple other business capabilities can react to that at some later point in time in a way that doesn't or that can't succeed or fail independently of the first piece, then that's a place to introduce a message broker like RabbitMQ as your service bus, a middleware like Kafka or any other type of pub/sub supporting thing. So that would be where the service boundaries are coming in.

25:45 Udi Dahan

In terms of the data ownership, I want to come back to what I said at the beginning. Again, you might do some data distribution of pushing data sets from your back-end database to your front-end mobile devices as a kind of notification mechanism that's still inside the same logical service. So don't get hung up of saying, "The service owns it, therefore all of its data only sits in a database server." It's more the logical ownership of data. The thing to be cognizant of is if the same logical data definition exists in multiple service boundaries.

26:27 Udi Dahan

One of the things that I'd say is a common mistake but often puts people even on sort of the middle of their SOA journey in a little bit of a bind is that element of status updates. So when a given entity changes its status, a customer becomes preferred, an order is placed, or things like that, an item is no longer for sale. Those business statuses sometimes it's hard and you really need to go deep into a domain to realize am I duplicating this business concept between these two different services and is that an indication that my boundary is wrong, or is this a stable enough business fact that this form of duplicate, this form of logical duplication and coupling to that concept in two different places is unlikely to hurt me?

27:34 Udi Dahan

And it really is on a case-by-case basis to find out which types of statuses are stable enough that you can represent them as events between services versus which kinds of statuses are overly fine-grained and then they should be kept inside the service boundary.

27:56 Udi Dahan

So going back to the example that we mentioned before about the SMS, of saying SMS sent or SMS delivery failed or SMS delivery timeout or all sorts of detailed status changes of a thing, that's probably best kept inside the logical service whereas things that don't change as frequently and that they're about the semantic meaning of that state is quite stable. The business is not going to change what that means often, then that is something that you're likely going to be okay sharing between different services as an event.

28:37 Udi Dahan

So it's kind of volatility of the concept. Not how frequently the event is published, but how frequently is the business going to change what that thing means. All right?

28:52 Bob Langley

Great. Thanks Udi. In your course you said that we do not need entities and a connection between them. Is aggregate the same for you as an entity and how would you tell that if they are the same without using the word entity, since everybody uses the word entity differently?

29:11 Udi Dahan

Right. Terminology is tough. We have a lot of baggage associated to things. Often when people talk about entities, they're talking about the traditional entity relationship diagrams. Back in the '70s was formed by code and friends that kind of served as the idea of the relational database management system. And since then that concept of modeling things in the real world as events. So subsequently object orientation happened after entity relationship diagrams, an idea of objects and entities essentially mapping to each other and representing physical things.

30:00 Udi Dahan

So a customer in the real world would be represented by a customer object which would be a customer entity that has persisted in a database. I'd say that is the historical context of things. And the problem with that is that we tend to get these very large entities or these very large objects that have lots and lots of data fields on them, and then that creates a kind of logical coupling which causes problems for us down the road. So again, whether we're using the term object or entity, I think that that's kind of ... the history and part of the problem around it is when they get big because of lots of stuff that's put on them.

30:45 Udi Dahan

More recently in the Domain-Driven Design book, the term aggregate was used to describe or distinguish a kind of entity, and then there was an aggregate route which was yet again a refinement of this is a special kind of aggregate. The potential problem with the term aggregate is that it implies an aggregation of stuff. Sort of linguistically it's collecting up a bunch of stuff in it. But I think it's better in that it is a different term from entity. So when using it with people that are familiar with entities and objects and say aggregate, it's less likely somebody's going to interpret that as just another entity. However, this DDD thing has been around for long enough that a lot of times people in that community, ultimately when you say aggregate, they're thinking entity, or when you say entity, they're thinking aggregate and the terms are interchangeable. That's part of the risk with these things.

32:02 Udi Dahan

I don't know what would be a better term that is less overloaded and is clearer. I guess, what I'd say is that when looking to define your entity or aggregate or your object boundaries, try to ignore physical representation. So in other words, if a customer is a singular, physical entity in the real world, do not try to create a single software entity representing everything about that. If an inventory item is a single physical entity in the real world, avoid trying to create a single software entity to encapsulate everything about that in the software world.

32:57 Udi Dahan

And a lot of what goes into identifying service boundaries is to say some of that data relates to business capability A, some of that data relates to business capability B, and then essentially by having both of those entities inside their respective services referencing the same identifier. So I can have the accounting customer entity having the same identifier as the sales or marketing customer entity so that I can always connect up the dots in a kind of join, not necessarily a database level join, that I can have eventual consistency there without having data duplication.

33:52 Udi Dahan

So I'd say rule of thumb, again, is don't have one-to-one between physical world to software world. Usually you'll have multiple distinct software entities for that. All of them will have an ID type field on them that will provide you the correlation, and the fields in one of those software entities should be distinct. There should be little to no overlap with the fields defined in another software entity in a different service. Those would be like islands.

34:32 Bob Langley

Thanks Udi. This question almost sounds like planned but we're going to ask it. What are the minimum features you would look for in a message bus and what are some nice to have features and are there any additional criteria you would use to choose a message bus?

34:52 Udi Dahan

I'd say that first of all the question needs to come back to awareness of what you're using it for. As I mentioned at the beginning, this element of data distribution. You're saying, "Look, what I need to do is I need to keep data in sync across a large number of physical places." Someone says, "I'm going to use a message bus for that." I'd say, "Well, what you want is a more data distribution service thing." The problem is the different technologies call themselves so many different things, and say, "Yeah, we do that, and we do that too, and we do all of those things."

35:29 Udi Dahan

So I'd say it's the if you want to do data distribution, if you want low latency data distribution, then be clear which part of your architecture you're wanting to do that for and limit your use of that technology for that purpose. If what you're looking to do is this type of logical pub/sub between distinct business capabilities, then usually latency is not as important because you've defined the service boundaries as a when I publish this event, whatever happens next can happen some longer time later. I'm not really worried about it happening as immediately as possible.

36:17 Udi Dahan

So I'd say low latency is usually not a feature needed from, I'm going to use the term service bus, when talking about these types of business capability type service interactions. Whereas when we're talking about data distribution, low latency is quite important in many of the cases.

36:37 Udi Dahan

I would say that again it's service bus territory because you're talking about highly business capability focused things that represent important business flows. Some amount of transactional integrity is important. You don't want to end up in a situation where an important event gets lost, and then having to figure out what to do about them. And I do think that that tends to be underappreciated. The complexity in making sure that transactionality both of incoming messages, data state changes and outgoing messages all get handled atomically.

37:28 Udi Dahan

So I would say that that's a very important thing that is unfortunately underappreciated or not sufficiently understood broadly in the industry and is usually chalked off as, "No, no, it's eventually consistent, it'll be fine." The distinguishing characteristic of eventually consistent does not mean eventually inconsistent. You need to have sufficient transactional guarantees to be able to end up on the eventually consistent side of things as opposed to the eventually inconsistent side of things. So I'd say that's probably the core thing that when it comes to business capability type services I'd say is non-negotiable.

38:25 Bob Langley

Great. Thanks Udi. Another question is, do sagas always, are they always owned by a single service? And if they aren't always owned by a single service, how do you manage them if they're not owned by a single service?

38:43 Udi Dahan

Right. So I'd say a NServiceBus saga, like any other piece of software, or software classes or a set of data is owned by a single business capability service. The data, the business rules that are encapsulated by that saga belong in one and only one business capability. If you don't think that's the case, something's off there logically in your design and needs to be restructured.

39:26 Bob Langley

Yes, sorry, next question. If you use messaging in a service to store a piece of information about object you subscribe to a change event to keep this data up to date, however, where does the initial state of that information come from? When the service is created new or does it have historical data?

39:45 Udi Dahan

Right. This question comes up and saying I'm creating a new service which is essentially a new business capability that didn't necessarily exist before, and there's a bunch of events that it wants to subscribe to, but there's a bunch of history, the other services have been running for some time. They have a bunch of data. They've published events in the past about significant things that my service would have wanted to have received. How do I go about getting that state into my service?

40:20 Udi Dahan

I'd say you've got two general options. The first and what's called the dirtier one that people, whether people say there should be a more elegant solution for that is ETL. Extract the data from the existing services databases, transform the relevant data to the structure of this new service, and load it up into that services database as a one-time action. The alternative choice and what has made Kafka kind of quite an attractive option, people say, "Well, Kafka holds the entire history of all of the things that have been published." So my service can just go to the beginning of that stream and read everything. Now, that's at least the Siri of it. However, what I've heard repeatedly from people that have run Kafka at scale in production is that invariably we do some kind of compaction on the streams.

41:24 Udi Dahan

Now, when you're talking about crud for lack of a better term of saying entity updated and then entity updated again and entity updated again. So when you're doing that kind of very data-centric event model, then yes, compaction to say I can throw away a whole bunch of all of those update, update, update, update and just deal with the last one is an effective approach. However, when I'm dealing with a business fact type of events that reference a customer entity that say this customer was identified as a fraud, this customer is now a preferred customer, this customer exceeded a certain threshold of whatever, all of those things having the same customer ID, I wouldn't want any of those events discarded due to compaction, because these subsequent events do not contain context that sort of overwrites the previous ones.

42:32 Udi Dahan

And I think that's the mistake that I see when people are looking to Kafka as a solution, is they hear these things from the Kafka folks without realizing the Kafka folks are talking about data streaming territory which is relevant within a single logical service and not between logically distinct business capabilities. And in those places, you don't want the compaction and hopefully you have enough storage in Kafka to retain that history, but most of the time again, the ETL approach is the better one.

43:18 Bob Langley

Great. We're going to switch and try to go to more lightning round questions, so quicker questions, quicker answers. The first one is, is it valid for a service to send commands to other services?

43:32 Udi Dahan

No, it is not. That would violate their business capability responsibilities.

43:42 Bob Langley

Great. Is SOA or Domain-Driven Design always recommended?

43:50 Udi Dahan

I'd say that they are useful analysis and architectural approaches that I would look to see if they benefit the problem at hand. I would say that the most common mistake is that they were not applied early enough by the right people, meaning someone upstream said, "Okay, we're going to build a system to do X for this set of users built by that team over there," and then they give that team a whole bunch of requirements. At that point that team is saying, "Can we do SOA or DDD or is it overkill?" And I'd say it's almost too late. The important SOA and DDD questions were about all of the previous decisions even before that project was kicked off. So I'd say, yes, they're applicable, but they need to be applied much earlier than most organizations realize.

44:55 Bob Langley

Thank you. If something needs to be done in a transaction, does that mean it needs to be part of the same service? Can transactions cross service boundaries?

45:05 Udi Dahan

If it needs to be transactional meaning atomic, isolated, consistent, durable, yes, that means same service. No, you can't have service A and listing service B in its transaction. Here when we're talking about transaction, it's relevant, specifically we're talking about making changes to data. As I mentioned before, there are patterns where we can have logically independent pieces of code meaning from different services taking part in the same host process.

45:44 Udi Dahan

There are special cases like that. We can't get to them in the limits of a lightning round. If you're interested in learning more about those special cases, check out my presentations about micro services and rules engines. So the engine pattern gives more nuance around how services can interact at a higher degree of interaction between them but still it's not enlisting each other in transactions.

46:13 Bob Langley

Thank you. What about entities though? Can entities cross service boundaries?

46:20 Udi Dahan

If by entity we mean a software class that has a bunch of fields on it which is persistent into some database, I'd say similarly that that is data and business rules owned by a single business capability belongs to a single service. Again, if you feel like you need to share it between services, either your service boundaries are on or more likely than not you're thinking of systems rather than services. So think of things that are physically independently deployed to different places for different reasons. So you're thinking of sort of a vertical decomposition rather than a horizontal decomposition.

47:06 Udi Dahan

A given service can have a component in system A and in system B. And another service can have its components also deployed to system A and system B. A lot of times when people ask these questions, the reason they're doing that, they're tying themselves up in knots because of the mistake of a system for a service or vice-versa.

47:28 Bob Langley

When services are communicating, is there a problem if they're doing a lot of communicating in between, if there's a lot of transfer of messages between those services?

47:43 Udi Dahan

I'd say it's the number of logical messages more so than sort of the physical rate, because you could have very high volume type of environment where you're processing millions or billions of things a day and as a result of that services which are publishing loosely coupled events could still be transmitting millions of those things a day. So I wouldn't get hung up on the number of units that are transferring per unit of time. I would be focused more on the number of message types, the number of data structures that are shared between them, the number of use cases that those data structures need to be used to pass things around with.

48:35 Udi Dahan

I'd also be cognizant of the pub/sub masquerading, or sorry, request-response masquerading as pub/sub. That's often a thing that results in a lot more message types. And that's usually an indication of a surface boundary problem or an element of not realizing all of these pieces of code are actually in the same service boundary. So I'd say, yes, there should be a relatively small amount of data structure definitions that are shared and passed between services via messages events, et cetera.

49:18 Bob Langley

Can you maybe explain the difference between a service as you, the term you used at first in SOA versus a micro service, and how does that relate to the autonomous component thing you also talked about in ADSD?

49:34 Udi Dahan

Right. So when we say a service is the business capability, the technical authority for a given business capability, that often results in there being code belonging to that service which gets deployed to a large number of systems. All right?

49:53 Udi Dahan

So I'd say that at some level you could say each one of those components could be a kind of micro service in the sense that it is deployed independently of other components in that same service, and often as a result of that version independently of it. However, a lot of times those components do need to do synchronous blocking HTTP request-response between them.

50:29 Udi Dahan

So I'm not sure I'd go so far to say those things are micro services. I'd say those are components that sometimes remotely call each other, and there's a term in software that was coined in the past called remotely callable components. I'd say that's what those things are.

50:49 Udi Dahan

When those components behave in a specific manner, meaning they don't use synchronous blocking HTTP request-response with each other and they use more of a message-driven event-driven style that is more async in nature and when they end up having some level of ownership of data the way that NServiceBus style sagas do where a saga tends to have its own little set of data that it owns and it communicates asynchronously with things around it, I'd say now you have a component which is much more autonomous in its runtime behavior with regards to the other components that it collaborates with.

51:39 Udi Dahan

So I'd say a saga as an autonomous component inside a given business capability would be the kind of thing that fits the definition of micro services as they're talked about in the industry, but that not necessarily every single thing inside a given larger service qualifies as a micro service/autonomous component.

52:08 Bob Langley

All right. Thank you. What about autonomous components within a logical service boundary? Can they use different database technologies like MongoDB or Cassandra or do they all have to use the same database technology?

52:25 Udi Dahan

I think that if you have an autonomous component that is truly a micro service, meaning it has logical ownership of that subset of data, and this comes back to some of that CQRS data the structure Reac discussion that we mentioned before of saying, given autonomous component micro service that owns its own data that there may be a justification that because of its load patterns and data update patterns and the functionality that it needs to run, that a different persistence technology would be preferable for backing that autonomous component than another one. So it's that element of saying, oh, I've got an inventory management autonomous component thing for my, let's call it, the high-volume items.

53:24 Udi Dahan

I'd say Reac because of its atomic increment decrement functionality is a good choice for supporting that thing, then I might choose that for that autonomous component. Whereas other parts of my system that may be autonomous components, I'd say, "Yeah, I can just use straight Postgres for backing those things or MongoDB if all I need is a fairly simple document structure and a simple transactional model on top of it.

53:54 Udi Dahan

So I'd say yes, absolutely that can happen. And I'd think at scale, so the longer you're doing this and at scale in terms of the amount of functionality and at scale the amount of load that you're dealing with, these types of multiple data persistence technologies become more and more necessary.

54:19 Bob Langley

When you have a composite UI and you have different components in that composite UI that need to coordinate in some way, how do they do that within the UI layer itself? Is there's some sort of mechanism which should be included as part of the composite UI to do that?

54:38 Udi Dahan

Right. So just like how I mentioned before about service bus technology, I said there are certain functionality that you want to have from service bus technology. When doing UI composition, having some kind of UI composition framework that gives you a certain set of capabilities to have logically independent components running together and interacting with each other in a good manner is important. I don't think I can do justice in the context of a lightning round on what those features are, but we will in the notes include a link to that type of UI composition framework and samples or whatever that show how those things do work and how you can compose a single item type page or a list of stuff type of page, a grid type of thing, and how those components would behave with each other.

55:39 Bob Langley

Is communication like email or SMS, those kinds of activities part of the branding service as you talked about in ADSD?

55:50 Udi Dahan

Right. I'd say that the technological part of the actually communicating with an SMS gateway email delivery service, et cetera, I'd describe those as being more IT ops related because there's a very strong strict technocratic component of it. Oftentimes, branding will be collaborating in the formatting or templating of what gets sent. So saying I need to send an SMS message telling this person that their order is shipped, it's likely that branding will have the template to say what do we send for when a person ... when we're notifying them that an order is shipped, or when where we're saying, "Oh, you've canceled your order," and sending you an email confirmation of that, branding would likely have that on the thing. What is those communication templates for those things?

57:00 Udi Dahan

So I'd say, the branding as a service gets composed into the pipeline of the email sending or the SMS sending, but it doesn't own the actual sending, nor does it own the business decision now we should send an SMS. That is usually where you'll have some kind of user preference thing that says how does this person prefer to be communicated to, and part of that is truly IT ops because IT ops holds the information about user IDs and logs and passwords and also the technocratic ability of communication. So those types of preferences of how to communicate them would also be in IT ops at that level.

57:55 Bob Langley

Great. Thank you Udi, and thank you for answering all those questions. Everybody on the call, there was, we got over 90 just during this webinar, plus several questions we weren't able to get to that were pre fielded. So again, as we said in the beginning, we'll be doing our best to answer those questions and send them out as part of the email we respond. Do you have anything else to say Udi before we wrap up?

58:22 Udi Dahan

It's great getting these questions. It's been a while since I've interacted with people in this COVID-19 space. I know this is virtual and not the same thing, but I always love hearing the questions and talking about this stuff. So for those of you that haven't yet followed me on Twitter, that's @udidahan. Feel free to also send me a connection request on LinkedIn. I'm out in the world. I love interacting with all you. So happy to follow up on this in the coming days and weeks going ahead. So thank you.

59:03 Bob Langley

Great. Thank you. Yeah, thank you everybody for joining us. That's the end of the webinar.

59:05 Udi Dahan

Thank you. Bye-bye.

Live Q&A with Udi

🔗Why attend?

🔗Ask Udi your questions about:

🔗Transcription

About Udi Dahan