Brian Vargas and Steve Maine have both weighed in on some of my thoughts on SOA, and their feedback made some very good points.
I think the main thing that their feedback points out is that SOA is about insulating various sections of your application from other portions of the application, through the use of services. This part I understood, the part that was tripping me up was how to implement the actual services and I am not sure the conclusion that I came to in my last post is completely correct. It is best to look at this through an example, in my current application one of the main groups of services will be authentication and user management… here are a couple example services:
AuthenticateUser(username, password, application)
AddUser(name, address, etc)
UpdateUser(userId, name, address, etc)
These services will be used by a number of different applications, and in different parts of those applications. Since I want to be able to revise and modify these services and the various application using them in different development cycles it makes perfect sense to use services. Now, the question is how should I implement these services and build the internals of these services. I do not expect to modify individual aspects of this group of services, so it would make sense to build the internals of these three services using OOP. This would give me the benefits of OOP for the internals of these services, but the benefits of SOA for the interface between these services and the other applications that will be using them. This is the opposite of the conclusion I came to in my last post.
Using an OOP design the internals of my services would end up looking something like this:
AddUser() Web Service would call a Service Interface layer which would:
1) Verify user has permissions to add a user
2) Create an instance of the User object
3) Populate the object with the parameters from the service
4) Call the Save() method on the User object. (Any business rules/validation applied here)
Using an SOA design the internals of my services would end up close to something like this:
AddUser() Web Service calls the Service Interface layer which would act as a Process Service and would:
1) Call an activity service called ValidateUser() which would confirm that the user can perform this action
1.1) The ValidateUser() service would call an Entity Service which would retrieve the data from the database
2) Call an activity service called AddUser() which would:
2.1) Check any business rules using some sort of business rule engine
2.2) Call an Entity Service which would add the user to the database.
If I know that the internals of these services will always be modified as a whole, then is it worth the trouble to use a SOA design for the internals? This is an almost trivial decision, considering the main benefits I am looking for will all be accomplished by implementing these functions as services.. whether those services are built using OOP or the SOA method in my last post… is not really that important to the entire application. I might even try implementing these services using an OOP internal design, and then implemented some of the other services using a SOA internal design… then see how each of them works out.
But I think the main lesson is that the part of SOA that is important is that these services are implemented as services, not how the internals of these services are built.
-James
{ 15 comments }
I think that you’re missing something here. The service should not be implemented as:
AuthenticateUser(username, password, application)
AddUser(name, address, etc)
UpdateUser(userId, name, address, etc)
But rather:
AuthenticateUser(Entities.User u)
AddUser(Entities.User u)
UpdateUser(Entities.User u)
Where Entities.User has the following properties:
int Id { get; set; }
string Name { get; set; }
string Address { get; set; }
etc…
Note that you would implement basic validation checks in the setters – such as checking the string passed in is of the appropriate length.
Now the Entities "layer" is really just a part of the common type system for the application. So, when distributing your services and setting up the calls over web services or message queuing, these entities will be passed as Xml.
This is how you move to the "Share schema, not type" directive, since, all these entities in essence just define a schema. From an implementation perspective, yes, they are a type too, but that’s really just internal to the service.
Maybe I’ll pull all these comments together and make a coherent post on my blog on how to migrate from OO to SO. What do you think ?
I was actually going to suggest that, you have left a ton of good stuff in my blog comments and should really put it together in one big post.
I agree with what you are saying about the entity types, I would definitely use some type of object as the parameter… I just did not want to confuse the example any more.
Thanks for all of your input and advice, I look forward to reading your complete post.
-James
I suggest looking at a post on SOA by Bruce Johnson of Object# Consulting. He gives it a very thorough treatment. You can find it here:
http://objectsharp.com/blogs/bruce/posts/235.aspx
I am quite curious to see where this goes. I am *not* a C expert, though I have hacked around a lot of C code at times so have some familiarity. What appears to be emerging is the system where you pass around transparent data structures to various functions, a la classical procedural programming.
Instead of passing around a struct with the data though, what gets passed around initially are the data structures (and only the primitive ones at that) in the standard library (String, List, etc). In the long run this changes into passing around arbitrary queryable data structures, typically hierarchical — XML. If you can rely on the data structure remaining constant (published schema, get the data structures right and you are mostly there, etc) you can pass this highly portable chunk o stuff (be it a DOM model, or text) around and query into for what you need.
The problem here is that this is no more resistant to incompatible requirements, and late-stage changes than any other technique. As soon as you require data more complex than string tuples you become bound to the structure. To use an earlier example from Udi, late in development you are told customers need multiple addresses. You can hide this in a service, but somewhere, somehow you need to store this, and trasnmit it, and now change every service that needs to know about it.
OO developed partly in response to the inadequacies of the abstract data type, which grew from the inadequacies of passing around tons of primitives (including null terminated string as a primitive).
I don’t think that application design via passing lots of arguments is going to work any better this time around. It works, you can make great applications that way, but it removes an abstraction layer. This is fine if you can handle the esulting complexity, but complexity is the main thing we are trying to do away with.
I will maintain that OO design as we know it now is the strongest way presently known to design large in-process systems until aspects really mature, and even then aspects just serve to reduce coupling, not change basic design techniques.
The push behind web services isn’t RPC (dear glod, I am about to disagree with a guru) as Uncle Bob rails against, but application-as-client in a convenient form. It simply makes more convenient for large systems the old Unix ideal that all input and output should be on stdin/stderr and be line-oriented text.
SOA is about large systems architecture, many full applications working together. It works well at high levels where the publihed services are equivalent to the user interface. It scales down if you design C style apps, but then you get all the classic C problems that Smalltalk, C++, Java, C#, Perl, Ruby, Python, etc have been trying to solve.
The services in SOA are user interfaces, where the user happens to be another application communicating in a (somewhat more) efficient manner. I do not believe the A in SOA is for architecture in the single-application’s-design sense. It is component architecture rebranded without the concept of remote objects.
-Brian
stdout not stderr =)
Brian,
I definitely agree with you that SOA is really about the larger systems, and groups of applications working together. That is why in this article I tried to steer the discussion away from talking about the internals of the service, as that is just not as important as the overall architecture.
-James
I left some comments on the post I referenced above stating how SOA does coexist quite naturally with Fowler’s Patterns of Enterprise Application Architecture.
I shall reiterate.
The set of Service Layer, Remote Facade, and Data Transfer Object patterns are essential components of SOA. The Data Transfer Object pattern is implemented by what I call entities. These entities form the schema in SOA.
Now, as to Brian’s comment, the relationships between entities are not found in the entities themselves. A customer entity, for instance, would not have a collection of order entities as a property. Rather, in order to get the orders for a customer, you would turn to the Association service calling the method GetOrdersForCustomer and pass the customer entity as a parameter.
It is in this manner that entities are all self defined, and the relationships are externalized. Why is this good ? Because the changes in relationships 1-1, 1-many, many-many back and forth are localized, and client code is less coupled to it.
For instance, changing the customer-order relationship from 1-many to many-many requires no changes in client code. The same call as above is used – GetOrdersForCustomer. This creates systems that are more robust to change.
And this, Brian, is how to handle the Address example you threw back at me =) , that is, as long as Address was implemented as an entity in its own right. Had it been implemented as a string property of customer, this would have been more problematic.
Eventually I’ll get around to writing a full fledged post on how to do the whole persistence/relationships thing in SOA.
Udi,
Interesting — though I cannot say I like it, I can see the possible benefit of the externalized relationships, but I don’t think that the way you do it is any better.
If the relationship service always returns a collection, change the object to always have collections as properties so it may have 1, or 100, or none. Either way you have to write code that deals with what the service gives back to you, and that code gets coupled to the number of addresses ( 0-1, 1, 1-N, 0-N). You still have the maintenance headache, but you also now have less expressive code.
Need to think on this some more =)
-Brian
Udi,
Despite the fact that I always seem to argue against you, I thoroughly enjoy it, and cannot (wholly) disgree with (most
of what you have to say.
-Brian
Brian,
You are the yang to my ying, and I thank you for it.
Udi
I don’t have much time to get into this, but what I meant about externalizing relationships is "The Death Of The Foreign Key" – or some other title that will get people up in arms.
Even at the database level where entities are persisted into tables, you won’t find any connection between them. All the relationships between entities are persisted separately from the entities themselves. You wouldn’t have a CustomerID field in the order table, for instance
Imagine, not having to update any CRUD sql because you changed a relationship from 1-many to many-many. Not having to change data access code. This is separation of concerns.
Anyway, all out of time.
Bye
Now *that* will get some people up in arms. The mathematical provability of the relational database is one of its strong points. I look forward to seeing where you go.
-Brian
I can’t say I agree with the direction you are going Udi, but I can’t wait to see what you come up with as well.
-James
This idea of the death of the foreign key is something that I thought up a while ago. Keeping all entities free standing, and managing the relationships between them myself.
I had in mind some sort of configuration file that defined all the relationships between the entities of the system. The business rules engine would use these to check whether an association could be created at runtime. For instance, the Business service call AddOrderToCustomer(o,c) would go check with the BusinessRules service by checking the result of the call CanAssociate(o,c). This call would check if:
1. The relationship between Customer to Order is 1-1 and that each of the entities ( order+id, customer+id ) wasn’t already associated with a different entity. For instance if a Customer with id=1 was already associated with Order id=2, then the call to check if an association between Customer id=1 and Order id=3 can be created should return false.
2. The relationship is 1-many and the given order isn’t already associated to a customer entity.
3. The relationship is many-many and the assocation between the given customer and order doesn’t already exist.
In other words, I working towards creating a generic association service that will work on any and all types of entities, and the associations between them.
The associations are themselves stored in a database, could be the same DB as the system itself. The tables used are as follows:
1. A table for storing binary relationships ( 1-1, 1-many, many-many )
2. A table for storing tertiary relationships ( many-many-many ).
3. I haven’t yet needed quarternary relationships, but you can see where this is going.
The binary relationship table is quite simple:
FirstTableName – string
FirstID – int ( or string for those who like GUIDs )
SecondTableName – as above
SecondID – as above
Tertiary and the rest would be done in the same way just with a Third*.
I’ve tried this method on several development projects with much success. However, these projects have had little changes in the domain model during their lifetimes, so the benefits of the technique were not really felt.
I’m not advocating that this is how you should go and develop a production system, but rather floating this idea that’s been growing on me for some time.
I like the fact that it greatly simplifies the rest of the system and allows changing the "domain model" of a system while it is still running. What I worry about is the effect on performance. Obviously this is slower than optimal, the question being – would it become the system bottleneck ? I don’t know yet. From all the tests I’ve done, its nothing too serious in cases when you don’t have upwards of a million rows of associations. That’s when you have to start doing some serious measurements to see how an optimal, non-generic solution compares.
This isn’t fully baked yet, but I like the way its turning out. I think that just thinking about these things helps me better understand the service-oriented, separation of concerns mindset that I find so necessary when working on large, complex, long projects that have high probabilities for change in major architectural ( that’s OOA ) portions of the system. When working SO, my capability to react to change without impacting my architecture is greatly increased.
Think about it, then tell me about it.
Think about how SQL Server manages foreign keys and tables. Each table is free standing, from SQL Server’s perspective. You can also create assocations between tables – each table being just a row in the systables table. The associations too are just rows in the sysobjects table.
In essence, I’m trying to go meta, and do exactly what SQL Server does, just at a system wide level.
Comments on this entry are closed.