Modeling for Smart Ecosystem Architecture

In a new CBDI Journal report this month I have been looking at the modeling requirements to support our Smart Ecosystem Architecture (SEA) concept.

It is evident there are two major architectural patterns in play – service oriented and event driven. But in addition we need to recognize a new dynamic which we might characterize as “post enterprise” including cloud computing, Web 2.0 and so called smart IT that encourages ecosystem collaborations over internal enterprise processes.

There is a lot of superficial (marketing) noise suggesting a close relationship between Event Driven Architecture (EDA) and SOA, Actually the two domains are in practice islands of automation. SOA is much more widely used, whereas EDA is still largely restricted to narrow focus, point solutions. This is changing, but slowly as there are considerable complexities to deal with in an integrated world. Not least event data availability, architecture contention and difficulties in testing. Of course the solution to broader utilization is integrated modeling approaches.

Similarly cloud, Web 2.0 and smart IT are separate domains and narrowly focused and capturing the requirements for either still tends to exist in two separate domains. However, to recognize the real opportunities in the smart ecosystem requires us to take a more coordinated view of modeling that spans all of these considerations.

CBDI has a vast amount of business modeling guidance and advice; much of it has pioneered thinking around modeling services, events, capabilities, meta data, dynamic business intelligence and ecosystems. See for example the CBDI report series on Business Modeling for SOA that begins specifically with the event-response concept at a high level as part of business modeling , and the report Event Driven Service Architecture that examines the convergence of EDA and SOA.

In this new report Modeling for Smart Ecosystem Architecture I advise on how to integrate these different perspectives across the broader set of architectural views. We also explore the dual requirements to consider both events and services, and also consider some of the meta model impacts.

On that last point it is interesting to note the lack of any 'standard' meta model for EDA, and hence similarly the EDA/SOA relationship. Whilst OMG have been working on a RFP for an ‘Event Model and Profile’ (EMP), this has yet to be issued.
Today, the requirements are quite likely to be captured directly into a format prescribed by proprietary event management products used for the EDA implementation.

We are currently evaluating how we add the necessary concepts to support EDA and SOA convergence in the SAE Meta Model and our associated UML profile, and discussing how we may contribute to emerging standards activity in the same way as we have done with SoaML.

The Shape of Business - Drivers for Smart Ecosystems

A report this week from the Confederation of British Industry (CBI) entitled The Shape of Business – The Next 10 Years provides some useful insight into emerging business drivers that reinforce our concepts of smart ecosystems.

Taking at some of the key headlines in the report, you can see the increasing need for organisations to develop ecosystems, and adding smart behaviour.

Movement to a more collaborative business model. The recession has made businesses much more aware of the complexities and interdependencies in their operations, their financing, supply chains and customers, but they are still not able to fully assess or capture these in their business planning. To gain greater control of these uncertainties, businesses will seek to ‘simplify’ their operations and will enter into more partnerships and joint ventures. In particular, this will be important for businesses moving to a ‘core plus periphery’ model

= a need to develop ecosystems
= the need for a smarter supply chain, hence smart ecosystem. Though supply chain optimization is hardly new, the emphasis on this is clearly going to grow. There are also additional ways of thinking about the supply chain – e.g. as a source of finance, not just ‘goods’. i.e. don’t invest in stock, and certainly don’t lend money to buy stock, but rely more on the support of participants in the supply chain and on its optimisation to lower inventory and provide 'just in time' fulfilment.

Rationalization to the Core. The recession accelerated the need to address inefficiencies and non-core activities across the enterprise. It has also provided the stimulus for companies to re-think themselves and re-evaluate their future.

= a need for greater collaboration across an ecosystem consisting of an internal core and an external periphery (what Moore might term context)

Technology will enable new ways of working. Businesses will increase their use of social networking techniques to solve problems – many more companies will use Facebook, Twitter and other web 2.0 developments

= making sense out of what is going on in complex social networks is going to require a lot of ‘smart’. It isn’t going to save money if it requires an army of people to monitor and participate in social networks

There were also some interesting comments that identify the need for agility.

For example, there is a current trend of “localism” in some organisations or industry sectors, relocating certain supply chain activities back to the UK. What is evident is that the decisions about what should be done locally or globally, or what should be in-house or outsourced, will be fluid. It is not a one-off decision but one of constant re-evaluation in the face of prevailing conditions. Equally, large organisations may find that the decision varies across different product lines, with ‘no one size fits all’ solution. As such, organisations will need to be very agile if they are to quickly capitalise on changes in those conditions.

Or course it is very difficult to predict what will happen across the next 10 years. How many predicted the current situation a decade ago?

And there is of course the need to survive 2012 first!

Time to Eat the Programmer?

One of my colleagues asked the question of whether the work that we do (at Everware-CBDI) could ride the ‘green’ bandwagon. After all he suggested, some of our key services in helping customers to transform their existing systems, and reuse software components and software services via CBD and SOA, might be considered ‘green’.

My first reaction was “that’s a bit of a stretch”. However, some of the news recently that has accompanied the publication of the book “Time to Eat the Dog” by Robert and Brenda Vale highlights just how complex the ‘green’ issue is.

For example, the authors point out that the ecological footprint of a keeping a medium sized dog as a pet is in fact greater than that of driving a large SUV 10,000km a year. This presents a double ecological challenge for my colleague who raised the question as he has both a Labrador and a SUV.

It struck me therefore that if those authors can determine the effect each pet has on the environment, then similarly we ought to be able to determine the ecological impact of each line of code produced by a programmer.

I don’t intend to do so here (determining the equation would itself be a waste of valuable resources – well my time at least). But it does illustrate just how complex the issues of all things ‘green’ truly are.

Most ‘green’ IT efforts today are focused on reducing energy consumption by using more efficient computers. Similarly, many of the ‘green’ messages focused on the population at large are also focused on reducing energy consumption by using more efficient means of transport.

But if as demonstrated by the ecological impact of pets, we need to consider much more complex factors in order to truly establish our impact on the environment, then I guess it is just as valid to ask not just how much electricity does a CPU use, but also what is the ecological impact of programming – or other IT development activities.

Perhaps organizations that are serious about ‘green’ IT ought to better consider the options of software ‘recycling’ before they simply create yet another new system. No longer is it just an economic decision of whether it is cheaper to recycle components of an existing system or build new one, now it is also an ecological decision too.

Yes, it is still a “bit of a stretch”. But nevertheless, not perhaps quite as extraneous as you might have first thought.

(you can read more about “Time to Eat the Dog” via New Scientist, BBC News, Guardian Newspaper)

Using Service Component Architecture (SCA) and SoaML

Service Component Architecture (SCA) brings the formality of SOA contracts and end point separation into the implementation layer. For the many organizations undertaking modernization efforts the standardization of componentization and SOA enablement is an important issue. Whilst its usage may not be widespread, it is likely that growing demand for rationalization of the implementation layer will lead to increased adoption.

In a new report I look at SCA and consider how it relates and maps to SoaML and provide guidance for integration into the broader SAE/SoaML picture.

My conclusion is that in some respects the SCA and SoaML initiatives represent something of a missed opportunity. Though the respective groups will claim they address different requirements or audience, the reality is that it is hard to see why they couldn’t have been based on common concepts where they overlap. One could have extended the other instead of re-inventing. Then the inevitable additional effort and loss of integrity through the mapping and transformations from one to the other discussed in this report could have been avoided. More so given that some organizations participated in both initiatives.

On the positive side, SoaML to SCA transformation tools do exist, and the mapping provided between SCA, SoaML, and CBDI-SAE in the report is at least at a high level relatively straightforward.

SCA is an elegant solution, but somewhat limited by lifecycle scope, and dependence on the SCA runtime, in comparison SoaML. However, it does offer portability across SCA compliant platforms and that is of significant benefit.

SCA does seem to have reached a bit of an impasse. It is there, it works, but it isn’t clear whether adoption is growing, either by end-users or support by more vendors. If you are using or planning on using SCA, please let us know in our CBDI LinkedIn where we have created a discussion topic

An Update to the Example Model based on Version 3 of the CBDI-SAE Meta Model

The October 2009 CBDI Journal contains a follow-up to the previous article presenting the draft version of SAE Meta Model V3. In this new report John Butler provide an update to the UML Profile for the CBDI-SAE Meta Model V3 focusing on the core areas and those that illustrate alignment with SoaML. Given that worked examples are the best way to understand a meta model John has updated the example model based on the fictional company Springfield Parcels, Inc. This should allow readers the opportunity to compare and contrast the version 2 meta model with that of version 3.

SOA - From Vision to Practice

I have just been reading the September 2009 issue of the Microsoft Architecture Journal which is focused on SOA. The foreword starts off by referring back to the report 'Understanding SOA' that David Sprott and I wrote for the very first issue back in 2004. As editor Diego Dagum puts it, "the main difference with the article that we published in the early days is that, this time, thoughts have emerged as a consequence of a practice; in 2004, thoughts had emerged as a consequence of a vision."

The inevitable "SOA is Dead" discussion is raised in the report Model- Driven SOA with"Oslo". But that report and others in the journal really serve as reminders that SOA is in reality still in its infancy, and we are still developing best practice, and in Microsoft's case at least, the modeling tools to support it.

Encapsulating best practice in modeling tools is essential if we are to

• move beyond perceptions by some that "SOA is a technology"
• demonstrate business value by making the SOA investment more relevant and 'visible' to business sponsors

That's a key reason why we ourselves have invested so much in our CBDI-SAE Meta Model for SOA and the associated UML profile that enables users to move from business models to implementation in a consistent and traceable manner.

It would be interesting to see how we might implement that in Microsoft Oslo. Oslo offers the promise of "the model is the code". Not surprisingly, Microsoft do have a slightly myopic view of the world that typically extends only as far as their own technologies. Though it may not be a business modeling language, Service Component Architecture (SCA) that I have been looking at in more detail this month does already provide a model-driven approach to modeling the Service Implementation Architecture, which is a good level of abstraction at which to consider "the model is the code". Unfortunately, SCA isn't implemented on the Microsoft platform. It's a pity that wheel is going to be reinvented I guess.

Nevertheless, it is good to see Microsoft continue to develop the SOA vision and turn it into practice, and in particular to embed that vision deep into their platform and tools, such that SOA becomes just 'business as usual'.

Service Classification

Zapthink recently asked the question "Are Services nouns or verbs", and if they can be designed as either, then which is better?

I don't think it is a case of "which is better". Services that expose a process (verb) address different requirements to those that expose an 'entity' (noun) . One is not better than the other. Instead they complement each other.

In CBDI-SAE we classify services according to the type of capability they provide. By focusing each service on a particular type of capability it enables greater modularity, separation of concerns and so on, enabling them to be more easily shared or composed into new services.

For example core business services (entity) provide operations focused on managing the information about a key business resource - like customers, orders, or products. These are independent of the processes that use that information.

Whereas process services provide operations that enable solution assemblers for example to interact with the different steps in the process. In turn the process services then consume the appropriate core business services.

In this way they can change the process service, add new operations and such, or add new processes, but without impacting the core business services.

If you mix these capabilities together you run the risk for example that the entity information is only provided via a specific process. We see this often and it's just too coarse grained and not a very agile approach. Processes have a different change cycle to entities. Many different processes use the same entities. So keep them apart.

Suggesting, as some are, that there is only one type of service and each provides multiple types of capabilities overloads the service and makes it difficult to use or share in different contexts or solutions..

We go further than just verbs and nouns however, and CBDI-SAE classifies service types into Process, Capabilities, Core Business Services, Utilities, Underlying, and Infrastructure .

Appendix A1 of our CBDI-SAE Meta Model for SOA also provides definitions for each type, and also gives an example policy for dependency rules between service types.

In CBDI-SAE we then detail different techniques that are applicable to identifying and specifying each type of service.

SOA Manifesto – Good, but too late?

This week, a small group of SOA cognoscenti met and published a SOA Manifesto. Thought it is eloquently put, it is difficult to see what has been said in the SOA Manifesto that hasn’t been said before.

The reality is that SOA isn’t new anymore. That some claim it is in fact dead or passé is testimony to no matter how much advocates of SOA protest, we have failed to fully get the message across, at least to the business that is footing the bill. I am not sure how the SOA Manifesto changes that. Those that are already pre-disposed towards SOA will become signatories ( I already have); those who are not will ignore it. Little will have changed.

I am reminded of the current political situation here in the UK, where the current labour government could put anything they wish in their manifesto for the coming general election but they are so unpopular it wouldn’t make much difference. Even if they promised to abolish taxes, they still wouldn’t convert those who have already made up their mind how they are going to vote. (and when did any political party actually deliver on their manifesto?)

The world has moved on. SOA should be considered business as usual. Yes, organizations still need to improve the way they approach SOA . Yes, people still need to adopt and apply the principles and be provided with better guidance. But if you want to get the majority’s attention, then you need a manifesto that excites them about the coming decade, not one that tells them what they should have been doing better in the last.

As I highlighted in my blog on Architecture for the Smarter Planet, SOA is one piece of an architectural puzzle that is going to challenge enterprises and individuals over the next decade. Although I believe the future is intrinsically service-based, I know we are not going to convince organizations to invest by telling them all they need is SOA. Rather, SOA has to be put in context, and positioned as one part of something much bigger.

Something like the Smart Ecosystem seems a much more holistic topic for a manifesto that addresses the needs of the next decade. The challenge for businesses will be how they become successful participants in ecosystems they do not control. SOA plays an important part in that, and can be a key enabler in a genuine business vision.

But just saying that SOA should be “business driven” doesn’t make it so.

Architecture for the Smarter Planet

You are probably aware by now that IBM’s current ‘theme’ is the Smarter Planet.Their basic message is that organizations (both businesses and governments) should take advantage the fact that we live in a world that is,

• Instrumented – 30 billion RFID tags in supply chains
• Interconnected –Trillions of connected devices, and two billion people on the web
• Intelligent – 15 petabytes of new information created everyday

And to do something ‘smart’ with these resources.

This seems in part a reinvention of the ‘Pervasive Computing’ vision. I authored a lengthy report on this some time ago in 2002. I still don’t think we are fully there yet, but you can sense that we are another step closer to it becoming reality, and IBM provide some good real world examples via the link above.

Architecture Scope

Attending the IBM Rational Software Conference recently at which the smarter planet was the theme of the keynote, it prompted me to think about what sort of business and IT architecture was needed.

One thing that is clear is that architects need to think very carefully about scope in this scenario. You cannot take a narrow inward looking project or even enterprise-wide view when the smartness you seek comes from the interconnectivity and behavior of the ecosystem. My colleague David Sprott has coined the term Smart Ecosystem Architecture (SEA) in reference to this.

I believe the key thing here in relation to SEA vs EA (Enterprise Architecture) is for organizations to understand

• what activities are going to be performed by the ecosystem (or the other participants in it), and what activities they need to perform themselves.
• and consequently, what they have to do – e.g. in terms of service provision – to participate in the ecosystem, and hence what capabilities and resources they need to provide their services, and handle the events that require their response

It also seems critical to me to that it is necessary to understand which capabilities are core vs context, but from an ecosystem perspective. (I need to ponder on that further…)

Architecture Patterns

That addresses scope, but what about the architectural patterns that applies?
I am tempted to say it’s all about Service Oriented Architecture (SOA) – in that federated, interconnected ecosystems are inherently service-based. But smart behavior is in a large part about sensing events and responding to them, so we need to add Event Driven Architecture (EDA) into the mix.

Before long I recognized we need several other architectural patterns in support of the Smarter Planet. I started off trying to draw a layered architecture to support of this, but that seemed inappropriate, so for now I will try to sum it up in a simple textual list.
OK, so the architecture for the Smarter Planet involves

• trillions of connected objects and billions of people accessing petabytes of information via millions of solutions, based on an agile,
• Web 2.0 Architecture – enabling billions of people to mash up rapid, user and community driven solutions, in turn assembled from millions of services, and generating trillions of events,
• requiring Event Driven Architecture (EDA) to determine the autonomic response required to sensors and changes in state,
• that are also placed into context by Business Process Architecture (BPA/BPM) and Information Architecture (IA)
• supported by services in a Service Oriented Architecture (SOA) that provides a formal basis for the decoupling of Provider and Consumers resource,
• as well as a Web Oriented, or Resource Oriented Architecture (WOA/ROA) as exchanging information between those trillions of devices in an efficient manner will likely be done in a more lightweight manner than full blown Web Service-based SOA
• with the implementations defined in a Component Based Software Architecture (CBSA) - with the focus on right-grained software enabling federated software delivery, that is running anywhere, anytime on a
• Cloud Based Architecture (CBA) that details the virtualized, federated infrastructure providing scalability, reliability. Which brings us back to SOA, as cloud computing is inherently service-based.

Anything I missed? My point is that architecting for the Smarter Planet is not about choosing one architectural pattern in preference to another – SOA or EDA or WOA or Web 2.0. One of those can't possibly address all the requirements. But rather how to use SOA+EDA+WOA+Web 2.0+xyz together, and understanding how they relate, not compete.

Enterprise Architects will argue these are all just components of EA, and to some extent that is correct. However, the scope now needs to be broadened to SEA.

What we need to avoid is any sense that things just got more complicated thanks to the Smarter Planet. With the application of sound architecture, it ought to get a whole lot simpler.

Some of CBDI Forum's Architecture mashups,

• Extending SOA with Web 2.0 (free access on registration)
• Web 2.0 and Enterprise Architecture (subscription required)
• Event Driven Service Architecture (subscription required)

CBDI-SAE Meta Model for SOA Version 3

The September 2009 CBDI Journal contains a “draft” publication of Version 3 of the SAE Meta Model. It incorporates some changes based on our continued work with customers and standards organizations and provides a mapping to SoaML so that end users can reap the benefits of both the breadth and depth of the SAE Meta Model as well as the capabilities that tool vendors will develop based on SoaML. Everware-CBDI plans to publish mappings to other ontologies shortly and other meta models as they mature.

It’s been some time since the last major revision of the CBD-Service Architecture and Engineering™ (SAE) Meta Model for SOA. Since then Everware-CBDI has published a UML Profile for SAE1 that has received quite broad acceptance and has been downloaded by thousands of members worldwide.

Everware-CBDI is committed to aligning with standards where appropriate and to that end we have contributed to, and worked extensively on SoaML, the service modeling language developed within the Object Management Group (OMG). What we’ve found in the course of our involvement in these efforts is that while there is some convergence in terminology, significant differences remain in the details between the various standards bodies working in this area, ranging from overall purpose of model/ontology of each organization to the exact definitions.

Our intent in the SAE Meta Model is to provide support for these various standards to our membership while not sacrificing the value built into our existing meta model. To that end, we will provide mappings of SAE terminology to these various meta models, profile and ontologies and guidance regarding their use within SAE.

Background

The SOA marketplace continues to mature and at Everware-CBDI we feel a measure of satisfaction in knowing that we’ve had a hand in that evolution through the guidance we’ve been giving over the years. The SAE Meta Model is one of many contributions and was a key input to the Object Management Group’s SoaML specification.

The SAE Meta Model was first released in October of 2006 and updated to Version 2 in 2007 as a result of feedback from our membership and the broader industry. Everware-CBDI created a UML profile based on V2 in 2008 that has since been downloaded thousands of times all over the world. Obviously industry was ready for more mature support for modeling services and the feedback we got was very positive.

In parallel with our efforts the Object Management Group (OMG) issued a Request For Proposal (RFP) for a UML Profile and Meta Model (UPMS) in September of 2006. A submission team formed that included Everware-CBDI, IBM, EDS, HP, Model Driven Solutions, SINTEF and several others. SoaML was finally adopted as a specification in November of 2008 and is currently in the finalization process.

Introduction to V3

Any model remains a work in progress if it is to continue to be relevant over time. The SAE Meta Model is no different. As mentioned above the meta model has been downloaded thousands of times and has provided a practical mechanism for using SAE with existing toolsets. Feedback from that broad membership usage along with our own involvement in standards organizations and client engagements has pointed out some areas that needed improvement and some holes that needed to be filled. Equally, we have continued to refine and extend our SAE guidance. Hence, we saw requirements to address the following:

• Incorrect or cyclical dependencies between meta model packages,
• Only a very loose notion of who or what is providing a service,
• The need for a more consistent concept of Service for both business and software contexts,
• Separation of a concept (or “thing”) from the specification (i.e., the artifact) of the concept,
• The need for refinement of the relationships between Service, Automation Unit, and Deployable Artifact, and
• The need for inclusion of the concept of Internal Architecture.

Version 3 of the SAE Meta Model attempts to address these issues.

We shall shortly be launching a review process and will make an updated specification and UML Profile available to those who would like to participate in the process.
If you are interested in doing so, drop us a note on our CBDI Contact Page

To view the full report

A Unified SOA?

My Colleague David Sprott blogs this week on the Failure of SOA Standards
There is no need for me to repeat his comments.

In his blog, David comments on Navigating the SOA Standards Landscape Around Architecture, a paper recently published by OASIS and The Open Group. It seems to us and many others that this is an attempt on the part of the authors to gloss over the fact they have managed to produce three alternative SOA standards when the world really only needs one. Saying that OASIS SOA RA is ‘similar’ to the Open Group SOA Ontology is a bit like saying Cricket is similar to Baseball. Because they both involve similar concepts of bat and ball presumably this means a cricket team could play a baseball team?

It is only with a more detailed level of mapping of more precise meta models that anything useful can be done in the real world. Even then, the reality is it is difficult to mix two frameworks and meta models.

One of the intriguing things for me is the involvement of common participants in all three SOA standards efforts at OASIS, The Open Group and also at OMG. IBM in particular stands out as a vendor who appears to have put significant effort into them all, and with minimal visible consistency resulting. Why? How does it help IBM customers to provide them with a choice of three alternative SOA standards? Surely that only works to slow adoption of SOA, not to encourage it.

This was one of the major successes of Web Service Protocols – that there is only one standard for each requirement in the protocol ‘stack’. Even though the various protocols are administered by different bodies, they have managed to avoid competing. Well eventually anyway after some initial vendor-based alternatives fell by the wayside. And since then take up has been widespread. Partly this was achieved because vendors such as IBM together with Microsoft participated in only one activity per protocol and not three. Hence others were forced to make decisions to go one way or the other rather than sit on the fence and be endlessly compromised by choice.

And this is what the same groups now need to do with SOA standards. Combine their energies, put aside their differences and produce a set of unified standards – even if different elements within it are administered by different bodies, as with Web Service Protocols, so they can each retain some ownership. We only need one SOA Reference Model, one SOA meta model, one set of principles and so on. But once you have the common foundation, just like the Web services efforts, there can be parallelism. From our work on the CBDI SAE meta model we know that there are multiple views which can be modelled concurrently such as business, specification, implementation, deployment, testing etc.

That said, perhaps more of the competitive approach taken with Web Services would be apt. That is, I would rather see IBM back one SOA Meta Model standard for example, and not three. I am sure the competitive nature of early Web Service protocol standards activity really helped to force the pace – a sort of SOA ‘Space Race’. Now, rather like the pace of the space race today, I get the impression it will now be decades before we land on the SOA moon. Of course, if OASIS, The Open Group and OMG cut the stack up as I suggest and avoid overlap so much the better.

But why is this important? At runtime, clearly the vision of SOA wasn’t going to work without consensus on the interoperability standards. But I believe it would be a mistake to assume a similar consensus isn’t required at earlier stages in the life cycle. Time after time we see that the lack of a common SOA framework is hindering SOA adoption in organizations. Sure different projects can all agree to use Web Services at runtime, but the lack of any common standards before they even get to that point means that much of the SOA vision remains elusive when viewed from an enterprise perspective. It is pretty hard to have full life cycle asset management and governance over SOA deliverables if the concepts change for each phase. In most organizations, users have little choice but to pick a single vendor's proprietary solution to such requirements that complies with no relevant SOA standards.

I am reminded of object orientation pre UML. What is needed now is a Unified SOA.
SoaML, in which we have participated might be one step in that direction, at least at the modelling language level. Even so, TOGAF 9 has its own meta model, but so does Archimate, and there is one implied by the OASIS SOA RA, and so on. Then there are the differences even between The Open Group’s TOGAF 9’s coverage of SOA, and their own SOA Source Book. Do you know for example when you see a concept mentioned in one document it has the same meaning as in another? Often no definition of the concept or reference is even given so it is difficult to be sure. Clearly there is still a long way to go.

And let’s not forget that there is a large government and defence sector who are extending their own frameworks such as FEA, DODAF, and MODAF with yet more unique perspectives on SOA.

It is at least welcome though that OASIS, The Open Group and the OMG have seen the need to cooperate on Navigating the SOA Standards Landscape Around Architecture. Perhaps now they can move forward as I suggest and ensure greater collaboration and consistency between their efforts with a single purpose in mind.

We would be happy to participate in such a unified process. What we can’t afford to do unlike IBM is participate in three (even though they have all invited us to), and more to the point it would be counter productive! In the meantime we will continue to participate in SoaML and refine our own CBDI-SAE Meta model for SOA. To that end we have just published a draft V3 of our model which includes SoaML adoption.

Users shouldn't need to be shown how to navigate their way around multiple SOA standards. They just want to know "are we there yet?"

Microsoft Visio – Suitable for Modeling SOA and EA?

Over the last month, I have been looking at Microsoft Visio as a possible tool for modelling Service Architecture and other SOA deliverables and as a place to capture the Enterprise Architecture. This was prompted in part by a product report I have just completed on Orbus Software’s iServer product, which adds a multi-user repository to Visio.

Given the number of specialist modeling tools there are on the market, it is surprising how many enterprises we encounter that use Microsoft Office as their prime vehicle for capturing various items of metadata and producing the documentation and diagrams required to support their IT architecture and delivery projects. Requirements and specification deliverables are routinely documented using Microsoft Word, while metadata is captured as columns and rows in Microsoft Excel spreadsheets, and diagrams are constructed using Microsoft Visio (or even just Microsoft PowerPoint). However, there are many shortcomings to this approach in an enterprise context, and that is primarily what iServer sets out to address and I have examined in my report.

Until now, I had only been a casual Visio user. For simple diagrams we publish in reports, PowerPoint is sufficient, and also the format we provide to our customers. For more rigorous work – with our clients for example, or to produce our SAE Meta Model, then we use other UML modelling tools.

Once I started investigating though, I was impressed how simple it is to create your own stencils and templates, but moreover how straightforward it is then to connect instances of the shapes to items in a Microsoft SharePoint list. In the example below, I created a simple template for the Service Specification Architecture diagram we use in CBDI-SAE, and then linked it to a SharePoint list containing a catalog of Information System Services (TOGAF) which are classified by Service Architecture Layer.

Hence it would be straightforward to create a Service Catalog in SharePoint, and then visualise the architecture for the services in Visio. My product report looks at how to do this in iServer. Clearly that is a more robust solution than my simple SharePoint example. However, for anyone with SharePoint and Visio already at their disposal, all it takes is some effort on their part.

Hopefully I will find some time to work on a Visio Template and corresponding SharePoint Template for CBDI-SAE and make these available in the same way we have done with our UML Profile.

Let me know if that would be useful.

In the meantime, I am interested to know just how many folk are using Microsoft Office and Visio as their prime tools for capturing EA and SOA deliverables, and what your experiences are.

Underlying Services – Architecture for Existing and Acquired Resources

Most organizations expect to reuse their existing or externally provided systems as the primary resources to support their Service Architecture. Now that most operating environments and packaged applications have been ‘service enabled’, using such resources in the Service Architecture is technically straightforward. However, the native Services provided by these disparate resources will probably have inconsistent specifications, business taxonomy and business information models. Using them directly in the Service Architecture will probably compromise SOA goals. The report we have published considers where the Services provided by existing and acquired resources fit into the layered Service Architecture and how constraints and compromises can be minimized.

Appendix A1 of the CBDI-SAE Meta Model V2.0 identifies nine Service Architecture layers. It is a policy decision for organizations as to which layers they actually want to use. However, there are four basic service layers we would expect to see in all instances of Service Architecture.

• The core functionality is delivered by the Core Business Services. Each Core Business Service is based on a major business type identified in a Business Type Model. Some people may think of these as ‘Entity or Domain Services’.
• The Process Services then orchestrate these Core Business Services to deliver a specific business process.
• In turn, both these service types may use Utility Services that provide common business or technical utility functions, such as Address Formatting or Authentication.
• Finally, the Underlying Service layer classifies those Services which are provided by the existing systems or packaged applications, but which have inconsistent information models that are not based on the Business Type Model.

The Service Specification Architecture then assigns instances of services their respective place in the layered architecture and shows the dependencies between them. The Service Specification Architecture is then used as the basis for the Service Implementation Architecture where the Automation Units that provide the implementation for the Services are identified.

A natural question to ask is, why can’t an existing or external resource just be the Automation Unit or Implementation of the Core Business Service? Why is it delegated to a different layer? After all, as stated most organizations will want to use these resources as providers in their Service Architecture as these will contain the operational data required.
However, there are many challenges to doing this, especially if you want to deliver the agility promises of SOA.

Hence this report looks at those challenges and how they are overcome by using the layered Service Architecture.

The report also identifies policies and techniques for identifying, creating and consuming Underlying Services.

See Underlying Services – Architecture for Existing and Acquired Resources (subscription required)

SOA Exception Management

Our SOA Exception Management Framework provides a structured approach to specifying exception conditions and dealing with them at run-time. In this report we look at some techniques that may be used

The report primarily considers exception management within Service-based solutions. That is, how both Service Providers and Service Consumers should handle exceptions that may occur in the processing of Service requests and responses.

That said, SOA exception management should not be distinct from broader exception management approaches that ought to be in place to deal with solutions that are not Service-based, and elements of the framework presented in this report should be common to both.

However, as we find ourselves routinely saying, SOA brings new challenges. For example:

• The federation of participants implies that the Service Provider and Consumer may be in different organizations.
• Loose coupling implies there is no shared technology or infrastructure. Hence, there may be no common exception handling software, systems management, or error log.
• The Service Provider and Consumer must rely on Service Contracts such as the Service Specification to document exceptions and specify any obligations on them when an exception occurs.

Consequently, the obligation is on both Service Provider and Consumer to independently manage exceptions. It is not the Service Provider’s responsibility to resolve errors in the Service Consumer’s requests. The Service Provider may simply reject invalid requests and do little else. It is up to the Service Consumer to handle such exceptions, log them, notify any interested parties, analyze the cause, and finally take any remedial action if required.

Equally, there is little point in the Service Provider giving detailed feedback on exceptions that represent problems behind the Service ‘façade’ as it is not the Consumer’s responsibility to do anything about it. But of course, the Provider must log and respond to such conditions.

That said, in many cases the Service Provider and Consumer will be in the same organization. As such it may make sense for them to:

• agree on a consistent approach to exception handing.
• standardize exception ‘protocols’. For example, exception codes and messages.
• provide common utilities or infrastructure for exception management.

Even where the participants are not in the same organization, they will still need to agree on the exception handing protocols. Though the detail of this should be part of each Service Specification, ideally there should be some generalized framework in place for exception management so that ‘sets’ of related Service Specifications at least behave in the same way. For example, all Services in a Business Domain. This report therefore sets out to describe such a framework.

Service Implementation Architecture and Automation Unit Specification

The Service Implementation Architecture and Automation Units Specifications provide the transition from the logical Business Service Architecture and Service Specifications to the implementation of Services in software. In our latest report we will look at how Automation Units are identified, modeled and specified, and advise on how they are documented in the Service Implementation Architecture.

This report presents Automation Units as a relatively flexible concept that can be used to represent a variety of different implementation approaches. The degree of documentation may vary by type or by the nature of the provisioner/implementer relationship. We would expect organizations to define policies that more tightly scope the permitted Automation Unit types in their organization and the associated documentation they require. From a process perspective, the Service Implementation Architecture is a deliverable from the Service Oriented Architecture and Design discipline, and the Automation Unit Specification is a deliverable from Service Provisioning. Together with the Service Specification, these will be key inputs to the Service Implementation activity.

See Service Implementation Architecture and Automation Unit Specification

Open Group Release SOA Source Book

We notice that the Open Group have this week released their "SOA Source Book".

The SOA Source Book is available athttp://www.opengroup.org/projects/soa-book/
Ideas in the SOA Sourcebook have been circulating around the SOA world for some time, and there is (not surprisingly) a lot of overlap (with more or less variation) with material the CBDI Forum has published from 2003 onwards. In particular the following have strong affinity to work we published in the past few years.

SOA Source Book	CBDI
A Maturity Model for SOA	SOA Adoption Roadmap Planning Framework
High-Level Perspective of the SOA Reference Architecture	Based upon our early layered architecture published in reports such as Understanding SOA
Service-Oriented Infrastructure	SOI Maturity Model
Introduction to SOA Governance	SOA Governance Framework

For example, compare and contrast the Open Group Maturity Model for SOA:

With that from CBDI Forum. The Streams are almost identical, and the phases whilst named differently have a similar basis.

The Open Group SOA Source Book SOA Reference Architecture. This was itself based upon an earlier IBM model, which was markedly similar to CBDI's.

With the CBDI Model from 2003,

The Open Group SOA Source Book SOI Reference Model


With the CBDI SOI Model from 2006

And finally, the Open Group SOA Source Book SOA Governance Regimes
with the CBDI SOA Governance Framework from 2007

 
Recently, we published a report TOGAF 9 complementing SAE practices and concluded

"TOGAF is a generalized enterprise architecture framework applicable to varying architectural styles. TOGAF 9 provides some useful alignment with SOA architecture development at a fairly high level.

SAE is an SOA specific framework and methodology that covers a much broader life cycle footprint in great detail.

There is no absolute conflict between TOGAF 9 and SAE. There are many opportunities to use SAE assets and work packages to guide and inform detailed SOA activity."

Everware-CBDI Delivers Major Upgrade to SOA eLearning Capability

eLearning portfolio for developing SOA skills across all roles in the organization

• 21 eLearning modules
• Fundamentals and Practitioner levels
• Cost efficient SOA skills development
• For business analysts, architects, designers, developers, testers, governance reviewers and project managers
• Associated certification services
• SCORM compliant modules for hosting in customer Learning Management System
• Available now to CBDI Platinum subscribers in the Knowledgebase

Everware-CBDI announces today that it has delivered a major upgrade to its SOA eLearning portfolio. Twenty-one eLearning modules are now available providing both fundamentals and practitioner level education for a broad range of SOA skills. In addition CBDI certification services are available, which together with the eLearning modules, can address critical skills shortages in SOA efforts.
Though many IT architects are very knowledgeable about SOA there remains considerable confusion and inconsistency of practice and process. SOA skills in many organizations are often limited to the architecture role and focused primarily on technology. As a result SOA projects often spend more time and energy determining how to apply SOA concepts than delivering project results.

The new Everware-CBDI eLearning portfolio is designed to equip a broader set of roles including business analysts, architects, designers, developers, testers, governance reviewers and project managers with consistent, detailed understanding of SOA practice. The e-Learning modules allow practitioners to educate themselves when it is convenient for them in a cost efficient manner.

The CBDI Certification process enables candidates to calibrate and communicate their skill level and allows program and project management to plan more effective and consistent resource utilization. There are two certification levels – Fundamentals and Practitioner. The Fundamentals level can be achieved entirely through eLearning and self study. The Practitioner level can be achieved through various combinations of eLearning, self study, remote and face to face classes.

The eLearning and certification products are delivered either as an integral part of the CBDI Service Architecture & Engineering (SAE) Knowledgebase or as standalone executables that can be hosted by the customer. They can also be provided as SCORM-compliant modules so that they can be used within a customer’s Learning Management System. All CBDI Platinum subscribers and Knowledgebase licensees will have immediate access to all the eLearning modules.

More information:

Everware-CBDI SOA eLearning

CBDI Forum publish Rich Service Specification Template for SOA

The Service Specification is a pivotal SOA deliverable that enables both Service Provider and Consumer to share a common view of a Service’s behavior. However, despite the importance of Service Specification there is no widely adopted standard.

CBDI Forum first developed our Service Specification Template in 2005 and it has since been used on numerous projects by our customers and ourselves.

TOGAF have recently published a Service Specification that, whilst rather more high level, bears strong resemblance to the CBDI template.

Following several requests to make our template more widely available CBDI is making the Service Specification Template freely available to the public under licence.

The template is now available for download


The Service Specification Template is designed to address the following requirements

Contract First Delivery
One of the important principles of SOA is that of contract-first delivery, and this is one of the principles that motivates and drives the need for a Service Specification.
Contract-first means the specification of the service should precede development of its implementation.

Use in Multiple Roles
The Service Consumer will of course want to know exactly what the service does. They will want to know what behaviour it offers, and what they have to do in order to use the Service.
A Service Provisioner whose role it is to locate a suitable Service will need to be able to compare the available Services against a specification to see if it meets requirements. Or if they are commissioning a new Service (or implementation of) they will need to precisely detail the required behavior of that Service.
Hence, the developer in the Service Providing organization who has to build an implementation of the Service also needs to know what requirements need to be meet.
A Service Specification is an implementation-neutral deliverable that contains all the information necessary to meet all these needs.
The Service Specification facilitates exchange of consistent and precise instructions between different participants in the Service supply chain

The Service Lifecycle
The Service Specification is expected to be developed iteratively across the service lifecycle. For example, as a;
Planned Service it may start as a high-level description of requirements
Specified Service it will provide precise detail of the required behaviour
Provisioned Service it may reflect any choices or compromises that have been made in the provisioning process, that diverge from the ideal specification
Deployed Service it will detail the actual endpoints and other aspects required to use the service at runtime

Using the Service Specification Template
CBDI provides templates for various SAE deliverables as Word documents. However, we do not really expect users to then complete and store instances of those documents using Word. Rather, they should treat them as the basis for schemas from which they could create a database to store the data in a more structured way – for example in some form of Service Catalog. Our Word templates might then be better used as a format in which to report from that database, rather than the place in which the data is itself stored.

For example, the Operation Signatures would be held as WSDL documents, and the Service Information Model might be held as a UML Class diagram in a modelling tool.

Further guidance on this is provided in two CBDI Journal Reports
Service Lifecycle Automation
Service Lifecycle Configuration Management

The template itself is only part of the story however. The delivery of a Service Specification is just one step, mid way through the CBDI-SAE SO Process.
Though the Service Specification Template is fully documented, ideally further guidance in its use is required.

CBDI have published Service Specification guidance in the CBDI Journal (gold subscribers), and further process guidance is also provided in the SAE Knowledgebase.
Practical Service Specification and Design Part 3: Specifying Services
Documenting Service Behavior
Process Logic and the Identification of Service Behavior
Produce Service Specification (SAE Process Unit) (platinum suscribers only)

We invite discussion on the Service Specification Template via CBDI LinkedIn