Lawrence Wilkes on SOA, EA, AM and CC

NIST Cloud Computing Reference Architecture

2011-09-16T09:45:00.000+01:00

I see NIST have now published their Cloud Computing Reference Architecture as a recommendation. Little has changed conceptually since earlier drafts, but the document is now more detailed and complete.

In earlier commentary on Cloud Computing Reference Architectures, Models and Frameworks I described how there is little concensus on what form a reference "thing" should take or what its content should be, and that most are a mish mash of concepts that have their own structure, but not a consistent structure that enables them to be easily combined or compared for example. The NIST Cloud Computing Reference Architecture is no different in that respect.

However, it is still a worthy document, and as with earlier NIST output that laid out definitions for core cloud computing concepts such as SaaS/IaaS/PaaS, the reference architecture is bound to be widely referenced as a "standard".

Service Boundaries

2011-07-04T10:36:00.006+01:00

Richard Veryard asks in his blog on Service Boundaries "where did all the boundaries go?"

CBDI Forum was one of the organizations back in those early days that Richard refers to that promoted the concept of service boundaries, but unfortunately not much of our work on that principle is freely available. So, I thought I would rectify that.

Services as Points of Flexibility

The first principle described in our CBDI-SAE SOA elearning material that promotes the concept of boundaries is that services are "Points of Flexibility". That is, services should be designed to provide points of flexibility (which we also call Articulation Points - a term coined by Richard) across functional, organizational or technology boundaries.

The objective is to allow flexibility/choice for the participant on either side of the Service.

So services form an important flexibility point – a place where change can occur on one side of the boundary, with minimum impact to the other.

As the figure above illustrated, services can be placed at the boundaries between:

Organizations – allowing the participants in the service ecosystem to change

This could also be boundaries within an organization, such as business domains or organizational units. Where certain services are placed on the boundaries between them.

Technologies – allowing the implementation technology used by provider or consumer to change
Resources (such as existing applications and other software artifacts) – that allow the implementation of the provider’s or consumer’s application to change.

Automation Unit Boundaries

Another concept in CBDI-SAE that promotes the concept of placing services at boundaries is the Automation Unit (AU). An AU is the collection of software artifacts that provide a service (i.e. its implementation). We use the AU rather than component to avoid the suggestion that an AU must also conform to component-based development (CBD) principles, which it may not do.
The AU concept is a logical grouping of software artifacts. Hence the figure below shows an external view of an AU on the left - that can be derived from looking at its public specification - and an internal view on the right, that is privvy only to the implementor/provider of the AU.

Consequently it can be seen that Orders Service and Customers Service in that example are effectively placed at the boundary of that logical grouping of software artefacts (the AU). Whilst the Sales DataService is not, and is only used inside the boundaries of the AU. The Orders and Customers Services are therefore the "points of flexibility" for the provider and consumer of those services. The provider of the Orders and Customers services would publish those in the Service Catalog for other service consumers to discover and use, whereas the Sales DataService would not be in the catalog (or at least not "public") as this is private to the publisher's implementation.

Whereas the Sales DataService is a point of flexibility for the AU implementor, who could for example migrate the Sales Database to a new database platform without impacting the Orders and Customers Service components, as long as the same Sales DataService is provided.

Other Logical Boundaries

This concept, that there are services placed on the boundary of the AU, as well as services that exist within the AU - could equally apply to logical boundary groupings such as organizational units, business domains, etc, where the service architecture differentiates between the services provisioned for intra-unit consumption and those for inter-unit consumption and hence placed on the boundary.

Over time, services could potentially be re-classified as the boundary lines change.

Further Reading

More on the concept of Automation Units and other key SOA deliverables can be found in

See also Service-Oriented Architecture: Considerations for Agile Systems. An article written by Richard and I for the Microsoft Architect Journal

The Service Oriented Cloud

2011-06-17T20:18:00.003+01:00

Cloud Computing is intrinsically service-based. But this is not just in the highly generalized sense of the term ‘service’, but also in the more specific Service Oriented Architecture (SOA) use of the term, where capabilities are provided via published service interfaces. When Amazon CTO Werner Vogels describes the Cloud as "a collection of services", in AWS terms the capabilities provided are SOA-style software services, complete with published Web Service interfaces.

As well as the PaaS and IaaS capabilities provided by AWS or Microsoft's Azure for example, there are also SaaS capabilities provided by the likes of salesforce.com that can be consumed as software services.

Hence the concept of the Service Oriented Cloud (SOC) illustrated below.


The Service Oriented Cloud

Add in the software services provided by business partners, the government, and others, and it is possible to rapidy assemble agile solutions to support new business requirements from a collection of services from multiple federated providers.

Though many of those business partner and government services might not be deployed using Cloud Computing today, it is important that their providers rapidly consider this option. As more and more businesses become service-based (that is, as a business, not just from an IT, software perspective), then Cloud Computing becomes increasingly relevant to the business, not just IT.

The service-based business needs to be elastic, able to scale capabilities such as manufacturing in line with demand. The Service-based business therefore outsources non-core capabilities to specialist business service providers. It participates in global supply chains leveraging the business services provided by its business partners. It constantly seeks additional or alternative sources of capability to improve quality, reduce cost, or to support new requirements.

To operate in near real-time, and reduce operational expenditure, the service-based business must be highly automated. Hence the IT solutions that enable the service-based business are assembled from the software services offered by multiple providers. In turn, the service-based business must also offer software services to its own partners and customers.

Consequently, the service providers need to ensure that their services are similarly elastic, and hence deployed using Cloud Computing.

Service-Based Thinking

The inherent nature of SOA in cloud architecture means service-based thinking permeates across the architecture views.

Service-based Business. Business products, process and capabilities as a collection of Business Services. Enabling transformation to the virtual business and its processes, assembled from a collection of business services from multiple federated providers.
Service-based Applications. Applications as a collection of Software Services. Where the business is supported by application solutions orchestrating a collection of software services from multiple federated providers.
Service-based Infrastructure. Infrastructure capabilities as a collection of Infrastructure Services. Where applications are deployed by orchestrating a collection of platform and infrastructure services from multiple federated providers.

It might be easy to conclude that SOA was yesterday’s issue. In fact according to some pundits SOA is dead and business transformation, application and IT modernization, and Cloud Computing are more fashionable terms today. But SOA hasn’t gone away! Achieving business improvement, modernized applications and leveraging cloud computing requires a solid foundation based on SOA. Most larger organizations have already adopted SOA to some extent, but few have achieved the critical mass that delivers on the original SOA vision. Moreover, organizations need to understand that all roads still lead to SOA. This is the key architectural style that is intrinsic to all strategic initiatives that will deliver radical improvements in cost and agility.

You can read more on these concepts and the importance of SOA in a research note I have authored.

Read In the Service Oriented Cloud, All Roads Lead to SOA

Cloud Computing Reference Architectures, Models and Frameworks

2011-06-13T13:43:00.002+01:00

There are a plethora of different reference architectures, models and frameworks for Cloud Computing (CC). As well as several vendors such as IBM or CISCO, it seems every standards or industry body has to have their own reference "thing" too. Hence there are architectures from DMTF, CSA, SNIA and the Open Group (which has been submitted by IBM) as well as several seemingly competing federal initiatives. NIST, who have established the de facto definitions of CC and the service and deployment models also have a draft CC Reference Architecture.

So which one should an organization adopt? Of course there’s no straightforward answer to that question and so I have published a research note on Everware-CBDI to provide guidance on how to organize some of the best ideas that are emerging in a practical structure that should stand the test of time.

Reference ‘Things’

A Reference Architecture (RA) “should” provide a blueprint or template architecture that can be reused by others wishing to adopt a similar solution. A Reference Model (RM) should explain the concepts and relationships that underlie the RA. At Everware-CBDI we then use the term Reference Framework (RF) as a container for both. Reference architectures, models and frameworks help to make sense of Cloud Computing.

Unfortunately, such formality is absent from the various reference architectures, models and frameworks that have been published for Cloud Computing; these frequently mix elements of architecture and model, and then apply one of the terms seemingly at random.

In developing the CBDI-Service Architecture and Engineering Reference Framework (SAE) in support of SOA (Service Oriented Architecture) Everware-CBDI separated out various parts as shown in figure 1. We developed a detailed RA for SOA and a RM for SOA, with particular emphasis on a rich and detailed Meta Model for SOA and a Maturity Model for SOA. We also developed a detailed process and task decomposition for SOA activities.

But the RF is easily generalized, as shown in figure 1, where the various elements could be applied to any domain, and explicit references for example to “SOA Meta Model” or “SOA Standards” etc., can be removed.

Figure 1 – Generalized Reference Framework

The benefit of this approach is that elements of the framework can then be mapped to each other in different ways to support alternative perspectives such as different usage or adoption scenarios, or the viewpoint of an individual participant or organization. Whereas most of the Cloud Computing Reference architectures, models and frameworks proposed today apply to a single perspective.

Current Cloud Computing Reference Architecture, Models and Frameworks

As discussed there are many frameworks and models to choose from. It is not our intention to detail and critique them all individually. Credit must go to NIST who have already done much of that in their 2010 Survey of Cloud Architecture Reference Models.

We may classify Cloud reference models as one of two styles, either

Role-Based. Where activities or capabilities are mapped to roles such as cloud provider or consumer. For example,

DMTF Cloud Service Reference Architecture
IBM Cloud Computing Reference Architecture (which has been submitted to the Open Group)
NIST Cloud Computing Reference Architecture

Layer-based. Where activities or capabilities are mapped to layers in an architecture such as application or resource layers or to the service management architecture or security architecture

Cloud Security Alliance Reference Model is one of many layered models showing the cloud ‘stack
CISCO Cloud Reference Architecture Framework is an architecture of architecture, placing Cloud on top of layers of Service, Security and Technology architectures
IEFT Cloud Reference Framework goes into more depth, showing the capabilities for each layer.

Analysis of the these shows that they typically contain,

Roles – that would be better placed in the Organization section of an RF
Activities – which would be part of the Process Model
Layered Architecture – which would be part in the Reference Architecture

Used this way, the generalized RF in figure 1 becomes a useful tool to analyze proposed Cloud Computing Reference architectures, models and frameworks in terms of understanding better what they actually contain, and a basis for development of an enterprise specific framework.

Continued in Everware-CBDI Research Note - Cloud Computing elements placed in generic CCRF, mapping capabilities to roles, process to roles, and recommendations

UML Profiles in Visual Studio 2010

2011-06-09T09:40:00.003+01:00

UML Profiles are supported in Visual Studio 2010 (VS2010). Here I examine how they work and whether our CBDI-SAE UML Profile for SOA (SAE Profile) can be used in VS2010

Creating a UML Profile in VS2010

See MSDN for instructions on how to define a profile to extend UML in VS2010 and then add the profile to a model.

Here’s a simple profile I created in VS2010 covering two stereotypes from the SAE Profile - “Automation Unit” and “AU Dependency”. (an Automation Unit is the collection of software artifacts that provide a Service)

Unlike Sparx Systems’ Enterprise Architect (EA) that we have used to develop the SAE Profile, the Stereotypes don’t appear in the toolbox so they cannot be added directly to the diagram. Instead you add a Class or Dependency to the diagram, and then chose a Stereotype for it. The figure below shows the Stereotypes for Dependency, and the properties and values that can be set.

Use of UML Profile in VS2010

And here is the schema,


<?xml version="1.0" encoding="utf-8"?>
<profile dslVersion="1.0.0.0" 
   name="SAEProfile" displayName="SAE Profile" 
   xmlns="http://schemas.microsoft.com/UML2.1.2/ProfileDefinition">
      <stereotypes>
            <stereotype name="AUDependency" displayName="AU Dependency">
                <metaclasses>
                      <metaclassMoniker name="/SAEProfile/Microsoft.VisualStudio.Uml.Classes.IDependency" />
                </metaclasses>
                <properties>
                      <property name="AUDsource" displayName="AUD source" default="">
                            <propertyType>
                                <enumerationTypeMoniker name="/SAEProfile/AUDsource"/>
                            </propertyType>
                      </property>
                      <property name="AUDtype" displayName="AUD type" default="">
                            <propertyType>
                                <enumerationTypeMoniker name="/SAEProfile/AUDtype"/>
                            </propertyType>
                      </property>
                </properties>
            </stereotype>
            <stereotype name="AutomationUnit" displayName="Automation Unit">
                <metaclasses>
                      <metaclassMoniker name="/SAEProfile/Microsoft.VisualStudio.Uml.Classes.IClass" />
                </metaclasses>
                <properties>
                      <property name="AUhasEmbeddedDataStore" displayName="AUhasEmbeddedDataStore" default="">
                            <propertyType>
                                <externalType name="/SAEProfile/System.Boolean" />
                            </propertyType>
                      </property>
                      <property name="AUisDistributedPart" displayName="AUisDistributedPart" default="">
                            <propertyType>
                                <externalType name="/SAEProfile/System.Boolean" />
                            </propertyType>
                      </property>
                      <property name="AUname" displayName="AUnam" default="">
                            <propertyType>
                                <externalType name="/SAEProfile/System.String" />
                            </propertyType>
                      </property>
                      <property name="AUrequirement" displayName="AUrequirement" default="">
                            <propertyType>
                                <externalType name="/SAEProfile/System.String" />
                            </propertyType>
                      </property>
                      <property name="AUtype" displayName="AUtype" default="">
                            <propertyType>
                                <enumerationTypeMoniker name="/SAEProfile/AUtype" />
                            </propertyType>
                      </property>
                </properties>
            </stereotype>
      </stereotypes>
      <metaclasses>
            <metaclass name="Microsoft.VisualStudio.Uml.Classes.IDependency" />
            <metaclass name="Microsoft.VisualStudio.Uml.Classes.IClass" />
      </metaclasses>
      <propertyTypes>
            <externalType name="System.String" />
            <externalType name="System.Boolean" />
            <enumerationType name="AUtype">
            <enumerationLiterals>
                <enumerationLiteral name="component" displayName="component"/>
                <enumerationLiteral name="quasiComponent" displayName="quasiComponent"/>
                <enumerationLiteral name="wrapper" displayName="wrapper"/>
                <enumerationLiteral name="script" displayName="script"/>
                <enumerationLiteral name="BPEL" displayName="BPEL"/>
            </enumerationLiterals>
            </enumerationType>
            <enumerationType name="AUDsource">
            <enumerationLiterals>
                <enumerationLiteral name="fromServiceSpec" displayName="fromServiceSpec"/>
                <enumerationLiteral name="designChoice" displayName="designChoice"/>
                <enumerationLiteral name="fromAUSpec" displayName="fromAUSpec"/>
            </enumerationLiterals>
            </enumerationType>
            <enumerationType name="AUDtype">
                <enumerationLiterals>
                      <enumerationLiteral name="requiredType" displayName="requiredType"/>
                      <enumerationLiteral name="requiredIntegrity" displayName="requiredIntegrity"/>
                      <enumerationLiteral name="create" displayName="create"/>
                      <enumerationLiteral name="implementationOnly" displayName="implementationOnly"/>
                      <enumerationLiteral name="exclusiveService" displayName="exclusiveService"/>
                      <enumerationLiteral name="exclusiveDeployment" displayName="exclusiveDeployment"/>
                      <enumerationLiteral name="exclusiveInstance" displayName="exclusiveInstance"/>
                      <enumerationLiteral name="nonServiceDependency" displayName="nonServiceDependency"/>
                </enumerationLiterals>
            </enumerationType>
      </propertyTypes>
</profile>

Importing the SAE Profile

In VS2010 you can either create the UML Profile as a Visual Studio Extension as I did above or import an existing UML Profile in an XMI format. As the SAE Profile is available XMI format I tried importing it but unfortunately came up against some limitations.

The first major one is that we use generalization heavily in the SAE Profile. However, this is not supported in VS2010 and consequently generates the following warning,

Message : The following element was not imported: SAE Profile::Automation Unit Specification::Generalization. This element has the following type: Generalization. Line 30, position 4.

As a consequence this means many of the key elements of the SAE Profile are not imported. In the SAE Profile Automation Unit for example is a type of Participant (which is used for mapping to SoaML).

Next it seems that a bug in the import means that enumerations are not imported. Enumerations are used to define the permitted values for properties. We have thousands of enumerations in our profile...

Hopefully Microsoft will improve this in subsequent releases. I am grateful to Steve Cook at Microsoft for helping me to try and import the SAE Profile.

In turn we are also migrating the UML model of the SAE Profile from EA over to No Magic’s Magic Draw. We need to do this in order to get support for XMI 2.2, whereas EA only supports 2.1. XMI 2.2 is required by IBM Rational tools, which has been popular in the past amongst our SAE Profile users, but the requirement for XMI 2.2 means that it cannot currently be imported. We will look to see if Magic Draw can also export a format that imports better to VS2010. With thousands of elements in the SAE Profile, trying to convert it to VS2010’s own proprietary profile format would take a fair bit of effort.

Trialing Microsoft Office 365 Beta

2011-06-07T11:53:00.000+01:00

Currently trialing the Microsoft Office 365 Beta. It seems tailored made for small businesses like us. Like many small businesses, we already rely on several disparate hosted services such as email, a SharePoint for collaboration, Skype, Webex, plus some use of Google Docs, and so on, all of which are supplied by different providers, resulting in incompatibilities, lack of integration, multiple signons, etc.
Bringing together all these capabilities under one roof seems attractive. It will also help to ensure everyone is using Office 2010 (no more need to save as 2003!).
Will update as we use it more.

Everware-CBDI plays key role in developing ACT-IAC white paper on Enterprise Architecture and Cloud Computing

2011-05-27T11:11:00.001+01:00

Under the auspices of ACT-IAC’s Enterprise Architecture SIG, my colleague Dave Mayo, the President of Everware-CBDI, has led a team in the development of a white paper explaining the role of EA in Cloud Computing. The paper explores architectural issues, management issues and tools for decision making regarding cloud deployments. The fundamental finding is that the prerequisite to success with cloud computing is the establishment of a Service Oriented Architecture that identifies the services deployed to the cloud and how they may be accessed.

Click to access the White Paper on the ACT-IAC website. (Word)
or view as HTML on the Semantic Community

Windows Azure - Making Migration to the Cloud Seamless?

2011-05-26T17:58:00.002+01:00

This week I attended a Microsoft UK Tech Days (a sort of local version of TechEd) on building and deploying applications onto the Windows Azure cloud platform. In typical TechEd style, once a few positioning slides had been quickly dispensed with it was down to business with a deep dive into the code and admin tools. Though my day job isn't as a professional developer, as a consultant and architect I like to know enough about how these things really work.

I came away impressed with just how seamless they are making the migration of applications to the cloud.

Fundamentally, what Azure does is provide Windows Server and SQL Server as a "Platform as a Service" (PaaS). Consequently, existing Windows applications, or parts of them, can be redeployed to the cloud with a minimum of effort.

Several deployment 'patterns' were demonstrated through the day that really highlighted the power of Azure. Want the code in the cloud, but keep the data in house? No problem. Want the reverse? No problem either. Want to keep those thick client applications but move the Access database they share to the Cloud? Straightforward too.

What this bought home to me is that the vision that we have promoted for over 15 years now at CBDI Forum that through well designed component and service architectures, then a seamless agile deployment environment should be possible, is now not only finally becoming reality, but is also being provided at low cost, and low investment, rather than via some expensive premium product price tag that you might often expect leading edge technology to come with.

For those building greenfield apps, then this ease with which existing systems can be migrated will clearly not have the same appeal. Google App Engine or Amazon AWS will appear to provide an viable alternative. However, the seamless redeployment of the components of the system that Azure provides as outlined above still applies to new systems as well. The ability for example to bring the database back in-house, or to integrate existing in-house databases with new cloud databases should still be relevant. The ease with which this can be achieved via Windows and Windows Azure was impressive.

The complete feature set of Windows Server and SQL Server may not be fully available in Azure yet, but regardless it still worthy of consideration as is, and Microsoft were able to highlight a number of case studies of customers already taking advantage of Azure.

CBDI-SAE UML Profile for SOA V3 Now Available

2011-05-10T12:22:00.000+01:00

Thanks to the hard work of my colleague John Butler, the latest version of our CBDI-SAE UML profile for SOA is now freely available for download. This impliments V3 of our CBDI-SAE Meta Model for SOA.

This enables the full lifecycle of SOA to be modeled in UML, enabling SOA artifacts to be associated with both business models at the begining of the lifecycle, through to deployment at the end.

John has also authored a useful introduction to using the SAE profile and walks through the process of modeling a service specification architecture. This is also available for download.

Download now at http://everware-cbdi.com/umlprofile-v3

Cloud Computing. Are Utility and Cloud Computing really analogous to traditional utilities?

2011-02-06T14:50:00.011Z

The analogy is often made between utility computing and other ‘traditional’ utilities such as electricity, gas, water, and telephone. The same analogy is now being made with cloud computing. At a high level this makes sense. In the same way there is no need for each individual or company to operate their own power plant, do they really need to operate their own computers? Why invest in the capital equipment to provide a computing capability if it can be piped into the premises and purchased ‘on demand’? The opportunities for economies of scale, the centralization of expertise, increased reliability and scalability all make perfect sense.

However, as Nicholas Carr also recognizes in his book “The Big Switch”, analogies often break down once you move to another level of detail as highlighted in the table below. Here we need to understand the difference between the utility (e.g. electricity) and the application of that utility (e.g. cooking).

Factor	‘Traditional’ Utility	Utility/Cloud Computing	Impact
Co-location of capabilities	Whilst the utility service might be produced ‘off-premise’, its application is very much ‘on-premise’. That is, the electricity might be produced at a power plant, but it powers my oven in my home. Nor do I need visit the nearest water treatment plant to take a bath. (Putting the notions of restaurants and public bath houses to one side...)	Both the utility and its application are ‘off-premise’. That is, not only is the raw utility – compute and storage – located ‘off-premise’, but so are the software applications and data hosted on it.	Disruption in supply means also a disruption of application. Not only is supply of the utility dependent on a 3rd party, so is its application. I can’t swap to an alternative utility source (e.g. batteries), because I have lost the application as well. How this risk compares to having one or both on-premise requires assessment.
Portability	Though utility suppliers might be monopolistic in some countries or states, the application of it (e.g. powering my oven) is pretty much portable between them. Utilities all supply the same raw product. Whilst voltage might vary by continent, electricity is electricity. If I change my electricity supplier at home I do not have to buy a new oven.	Other than in its most basic form – an empty un-purposed server – many utility/cloud capabilities vary from supplier to supplier, even though supporting the same underlying concept. There are few standards in terms of the capability provided, and no standard interfaces.	Lock-in. Moving between suppliers is far from as simple as when changing traditional utilities.
Separation of concerns	There are very good levels of separation between utility supply and its application. If I change my electricity supplier I am not constrained to only cook a certain brand of food as a consequence.	Utility/cloud supplier’s stacks tend to be highly dependent. I may not be easily able to combine a capability in one layer of the stack from one supplier with a capability in a different layer from another supplier.	Lock-in. Buying one capability from the utility/cloud supplier may limit you to buying associated capabilities only from the same supplier. As well as lack of portability, there is lack of interoperability, either at a coarse or fined grained level.

So there is a breakdown of the analogy. Utility implies a fairly generic capability, but Cloud Computing is not always providing this. Applying the term to more specific capabilities might help people understand the business case, but they shouldn’t misunderstand the possible impact of their dependency upon them. However, this does not imply that cloud and utility computing are unable to deliver the promised benefits, providing the risks are assessed correctly.

To understand these risks better, and to help make the assessment, I have provided a framework for understanding the different models of utility and cloud computing and assessing them based on variety of factors.

See my reports Making Sense of Cloud Computing and also Service Portfolio Planning and Architecture for Cloud Services (both free on registration)

Making Sense of Cloud Computing

2011-01-30T09:12:00.001Z

The terms virtualization, utility computing and Cloud computing are often used interchangeably which can be very confusing. A new report I have just authored aims to provide clarification - to identify the similarities and differences in those characteristics, and provide a framework in which organizations can decide which capabilities they require in specific situations – as it is unlikely that one model alone will suit all their requirements.
A visitor from outer space would be forgiven for thinking that virtualization, utility computing and Cloud computing are different capabilities. But most of us understand that many product and service offerings use the terms rather casually and the trend is to assume they are all describing the same thing, where virtualization is synonymous with utility computing which is synonymous with Cloud. Some might say these are just steps in the evolutionary process – where utility has simply evolved into Cloud.
So, are they the same or different? Can the terms be used interchangeably, or are there clear distinctions between them?

The answer is not straightforward. There are clearly some common, overlapping characteristics that allow the terms to be used interchangeably. But at the same time there are other characteristics that enable them to be distinguished from each other.

The report aims to provide clarification. To identify the similarities and differences in those characteristics, and provide a framework in which organizations can decide which capabilities they require in specific situations – as it is unlikely that one model alone will suit all their requirements.

Capabilities

What capabilities are provided by utility and Cloud computing, and where does virtualization fit in? The figure below shows a layered ‘stack’ of IT capabilities and the relationships to Cloud Computing and Utility Computing capabilities.

Utility and Cloud Computing Classification

Utility Computing is the provision of infrastructure capabilities such as the server and operating system, compute capability, and storage on a pay as you go (PAYG) basis. In many cases utility computing utilizes server virtualization.

As the figure illustrates, Cloud Computing equates to Utility Computing in terms of the capabilities provided. As mentioned earlier, some might see this as a trend with Cloud terminology simply replacing utility. But Cloud Computing can provide these capabilities on a finer-grained basis that was typically the case with Utility Computing. Cloud also raises the levels of abstraction, providing a higher level of interaction for the consumer via utilities and a management layer that can make them easier to use than raw infrastructure, and provides complete virtualization of resources, usually with self-service provisioning on a multi-tenancy basis. In Cloud terminology these infrastructure resources are provided as Infrastructure as a Service (IaaS).

Note we are happy to recommend the NIST definitions of Cloud Computing, which we covered in a previous report.

Security capabilities such as authentication and identification, and networking and mediation capabilities such as message delivery, routing and transformation can also be seen as part of Utility Computing and also classified as IaaS. These capabilities can be combined together into a ‘platform’ and classified as Platform as a Service (PaaS). PaaS provides the IaaS capabilities via a higher layer of utilities, and not directly. Hence the PaaS can be said to encapsulate the IaaS capabilities.

Layered on top of the platform are the applications or ‘business’ capabilities. These might be broken down into client, business process and business function capabilities, but more often than not are packaged together as an application. Cloud providers don’t normally distinguish between them, and hence they are typically combined as Software as a Service (SaaS), regardless of the granularity or separation of layers.

The exception to this is Business Process as a Service (BPaaS). This is an emerging layer in which process assembly is offered as a service which allows the consumer to orchestrate services from disparate sources. BPaaS is therefore a specialization of SaaS.

PaaS is also used to characterize a suite (hence ‘platform’) of application functions that are exposed as service interfaces. For example Google Apps or Salesforce.com provide PaaS capabilities in addition to their core SaaS capability, to expose specific interfaces as a ‘platform’ on which consumers can assemble a custom application.

The use of PaaS and SaaS may be abused, and we advise that you interpret the capability offered on the basis of level of access and control - that is SaaS delivers applications with no opportunity to manage or control the underlying infrastructure or application capabilities. In contrast PaaS provides control over the deployed application and possibly the hosting environment.

Their use may also be seen as contextual. That is, as an end user of Google Apps the context would be SaaS, but as a developer using Google Apps as a platform on which to implement a new application, the context would be PaaS.

Use of a higher layer in the stack usually means the lower layers are inherited. Typically there is no separation of supplier down through the layers from the consumer perspective. That is, whilst the SaaS provider may in turn use a different IaaS or PaaS provider, this is transparent to the SaaS consumer. The exception here is BPaaS which we think qualifies it as a specialization precisely because it allows orchestration of disparate capabilities.

The term virtualization spans all of these capability layers. Cloud computing environments virtualize the capability at its highest level of abstraction in order to make the underlying resources location independent, more scalable, resilient, and so forth. In the Cloud context, virtualization is a generic term and it may be expected that server virtualization and server clustering technologies provided by platform vendors will be used to provision individual capabilities and resources supporting the Cloud service.

However, the same virtualization technologies may be employed wholly within an organization’s own private data centre, and so are not necessarily used to support Cloud behaviors (multi-tenancy etc.). Equally in house virtualization which is deployed with multi-tenant elasticity may reasonably be relabeled as a private Cloud.

Enabling virtualization is usually a necessary first step towards utilizing both utility and Cloud computing. If resources cannot be virtualized, how can they be deployed to the Cloud?

Characteristics

Most large enterprises will be commencing use of Cloud, but inevitably they will have a spectrum or continuum of capabilities deployed which span server virtualization, utility and Cloud computing. For most enterprises an early objective of Cloud initiatives will be to rationalize and modernize existing assets. A table I have compiled therefore provides a set of nomenclature which may assist in documenting and classifying existing and planned technology assets.

As well as considering virtualization, utility and Cloud, I have also included the notion of an outsourced data centre as a further option that many organizations will use. How does this compare to Cloud and Utility? I have also distinguished between public and private Clouds. A private Cloud will provide both utility and application capabilities. As the table shows, in some respects it might be hard to distinguish a private Cloud from a traditional data centre, or the outsourced data center if administered by a 3rd party. However, if it exhibits distinguishing Cloud features such as metering usage on a PAYG basis, providing capability on a fine-grained basis through an easier to administer interface, then it can be identified as a different model. You could further distinguish between Private PaaS/IaaS and Private SaaS provisions, but I don’t think that adds much to the model, as the only difference is the capability offered.

But beware provider organizations that may have merely relabeled these deployed assets with a more fashionable term!

Continued in the PDF version of the report (free on registration)

The full report contains the table of characteristics discussed above plus a set of decision criteria and risk assessments that organizations should take into account when considering utility and cloud computing options.

You can also download the complete CBDI Journal which this month contains an associated report looking at Business Driven Cloud Strategy.

ESB - Everyone's Silver Bullet?

2011-01-06T14:44:00.004Z

Given its prominence in any discussions around SOA for several years, you might think that by now everyone has acquired an Enterprise Service Bus (ESB). However, it is clear that there is still ongoing discussion about the need for an ESB, and questions still remain as to what exactly an ESB is. Cloud computing seems to have renewed interest in the topic. In the same way that people asked "do I need an ESB to do SOA?", we now have "do I need an ESB to do Cloud Computing?" (at least amongst those who recognize that Cloud Computing is largely service-based).

I sometimes think a better expansion of the abbreviation might be “Everyone’s Silver Bullet”, such is the perception that all you need to buy is an ESB and all your problems are solved.

I have long promoted the view that an ESB isn't a product as such, but a set of capabilities that you might assemble from various sources - either upgrading existing infrastructure, buying new components, or even building some yourself. Some of the capabilities might be embedded in the operating platform you use, or looking forward, you might consider some cloud-based ESB capabilities too.

This debate promted me to create the following extract from a report I authored some time ago, but which still seems as relevant today as it did then.

The role of SOA Middleware

SOA middleware may seem an oxymoron. SOA is meant to free organizations from the tyranny of tightly coupled implementations, which in my mind includes creating dependences on middleware, not just application and platform-level dependencies.

With support for Web Service protocols embedded in the Application Servers, and WS-STAR providing support for federated, secure, reliable, transactions between endpoints, why should there be a need for additional middleware?

The reality is that existing benefits of a middleware approach still largely applies. SOA middleware can be used to:

Separate concerns – remove middleware capability such as messaging and mediation away from applications. Though a modern platform can also assist in achieving this, such as Microsoft Windows Communications Framework (WCF)
Support Heterogeneity – separate capability away from OS/Platform specific functions
Provide a more agile environment where changes to infrastructure do not impact applications, or vice versa
Form part of a shared enterprise SOA infrastructure, rather than embedded in or specific to each application solution
Manage by policy, and manage centrally. It is easier to deploy and enforce policies through a layer of SOA infrastructure designed for this purpose.
Web Services are not the only protocol. Even when they do become widely used, most organizations will continue to use the existing middleware and other protocols already in use for some time. Hence SOA middleware often provides support for other protocols.

A challenge for many organizations today, is that the capability required for SOA middleware and SOA infrastructure in general is that it is spread, and often duplicated across multiple products and technologies. In addition it is often found in a mixture of point solutions and specialist SOA products plus existing infrastructure that is typically upgraded to support SOA requirements. Whilst there may be some good reasons to consider new infrastructure products to support SOA, there is also a strong reason for many organizations to look at how they upgrade their existing infrastructure to support SOA – namely, that they already have it, and the existing infrastructure must remain in place to support existing requirements.

However, it is not as straightforward as depending on existing infrastructure. Although not an immediate concern for some organizations, SOA will place new demands on infrastructure capability that the existing infrastructure cannot so easily support. Longer term, the SOA infrastructure must itself become Service-based and able to be virtualized in the same way that is required of business capability. Even in the near term, organizations can gain advantage from using a networked approach to some SOA middleware requirements rather than using a hub and spoke approach that frequently exists today. Longer term, the federated SOA will make this a requirement.

The ESB Spectrum

In earlier reports in our CBDI Journal I identified that there was no commonly agreed definition of an ESB, or a set list of functionality one might expect to find. Consequently, so called ESB products offer a spectrum of capability as shown in the table below.

The ESB Spectrum

At one extreme, ESB products may be little more than a broker, providing routing and transformation capabilities. At the other, ESB products can provide most of the SOA Infrastructure required. It is at this end of the spectrum that the most overlap occurs with other infrastructure domains. Ideally, the range of capabilities offered should be modular enabling organizations to assemble their SOA infrastructure from a range of best of breed capabilities. Unfortunately, not all vendors share a similar goal.

A well formed ESB can help organizations by providing the core mechanism to deliver SOA run-time agility. The role of ESB includes;

Providing the mediation layer between service consumers and providers to enable loose coupling
Abstract hard coded transformation and routing of messages away from service consumers and providing resources – making the SOA easier to maintain, manage
Provide a central point of control for mediation and integration policy enforcement
Host Services within the SOA itself
Depending on the ESB implementation

Provide declarative approach to defining mediations
Provide dynamic mediation
Provide policy driven mediation
Provide mediation based on current and emerging open standards such as WS-protocols, WSBPEL, JMS, JBI

Provide on/off ramps for multiple transport and message types
Orchestration of Services

As already mentioned, the capabilities required will be found in a variety of products and technologies. This will include the Operating System or Platform that hosts the services and automation units, or new SOA infrastructure such as the Enterprise Service Bus. Capabilities will also be found in new Web Service Management or SOA Management products, as well as in upgrades to existing Systems Management tools. Orchestration capabilities might also be found in Business Process Automation products. Some capabilities will also be found in upgrades to existing Message Oriented Middleware and Enterprise Application Integration products, some of which have been reborn as Enterprise Service Bus products.

Consequently, the selection and adoption of SOA infrastructure should be carefully managed. You could assemble it yourself, but it may still be more expedient to acquire a product containing the required capability, and hence delegate the ongoing support to the vendor. If cost of aquisition is an issue in these lean times, then there are several excellent open-source ESBs available.

There is a longer version of this available as a PDF for download from our website (free on registration). The complete report is available to our paying subscribers via our CBDI-SAE Knowledgebase for SOA.

SOA in Context

2011-01-05T10:03:00.002Z

SOA does not exist in isolation. Whilst it may be the centre of the universe for SOA aficionados, SOA is just part of a "bigger picture". That isn't to say that SOA can be marginalized. Far from it, as SOA is an important part of, and enabler of that bigger picture as the image below illustrates.

SOA in Context

So, in SOA terms, we can describe the world in terms of the Service Architecture, which requires SOA-specific Service Management, and SOA Governance. That is, managing the services in an SOA-specific context, both across the delivery life cycle and in the operational state. Similarly, defining SOA-specific policies and exerting governance, again across the delivery life cycle and in the operational state.

But SOA is also part of, enables, and supports the IT Architecture and Business Architecture. There are not hard lines where one universe ends and another begins. Rather it is a continuum. For example, IT Architecture should contain elements of, link to, integrate with, (and so on...), both the Business Architecture and the Service Architecture. Often these might be combined into an Enterprise Architecture, but we can still identify within that the the Business-specific, IT-specific and SOA-specific aspects, and separate them if we wish into different views. In those views, we can then add detail that is specific only to that view.

Similarly, SOA Governance is driven by Business and IT requirements, but in turn enables Business and IT governance by ensuring that SOA policies exert governance over Services (again across the delivery life cycle and in the operational state) to deliver both business and IT outcomes, as well as SOA-specific outcomes, as illustrated below.

SOA Governance in Context

Nor should the concentric circles in the diagram be misread as implying 'layers', rather they are just indications of broader context. That is, you don't have to go through IT Architecture to connect the Service Architecture to the Business Architecture. I.e. it should be clear in the Service Architecture which Business concepts (business types, business processes, business capabilities, etc,) the various SOA Services support.

Service Provisioning Contracts - New SOA Training

2010-10-14T15:23:00.001+01:00

SOA is fundamentally a contract-based approach. The concept of ‘loose coupling’ works because the dependency between implementations is replaced by a dependency on contracts. As long as the contract between provider and consumer is met, then the dependency on a specific implementation is removed. However, the devil is always as ever in the detail. What contracts are actually required? How are they documented? How should the level of precision required be conveyed?

To answer these questions, I recently completed work on building, and subsequently delivering, a new SOA training workshop based on our concepts of Service Provisioning Contracts.

This workshop considers the following three prime contracts used in the provisioning of a Service, and the relationships between the,

Service Level Agreement: The quality of service contract between service provider and consumer for the operational delivery of the service at run-time.
Service Specification: The implementation independent behavioral contract between the service provider and consumer, and also the service provider and service implementer, that specifies what a service does but not how it does it.
Automation Unit Specification: The requirements contract between the service provider and service implementer for the Automation Unit that provides the implementation of one or more service specifications.

An understand of the content and the role of each of these contracts can be seen in an earlier CBDI Journal Report - Service Contracts in the Service Oriented Process - which is now freely available on the Everware-CBDI website (on registration).

Since that report was authored I have detailed the Automation Unit Specification, and also developed an Exception Management Framework in later reports and so was able to pull this workshop together.

Also, since publishing our original work on Service Specification (the CBDI-SAE Service Specification Template is also freely available for download), we have also gained a lot of experience from different customer engagements as to the differing needs for Service Specification and the other contracts. Consequently, as well as looking at the details of each of these contract types, and walking through the corresponding CBDI-SAE templates for their documentation, the workshop also considers how the requirement for them or the amount of detail required varies by context and differing project profiles.

This workshop doesn't fully replace our existing Service Specification workshop. That goes into more detail on aspects such as developing the Service Information Model and how to document operation behavior, and also includes individual and group exercises on them.

Those particular modules from the Service Specification workshop might be a good follow on for those specifically tasked with their delivery, whilst not necessary in such detail for the rest of the audience. Similarly, the Service Provisioning Contracts workshop really needs to be proceeded with a good understanding of our Service Architecture approach, in particular the classification and layering of service types as this is referenced in the workshop, and also on understanding the difference between the service specification and service implementation architecture views.

Hence, we often find that the agendas for the workshops we put on for our customers are typically customized to suit their specific needs and prior experience.

More details on our workshops can be found in our Education Catalog

Several CBDI Journal Reports Now Freely Available for Download

2010-09-30T16:25:00.000+01:00

As well as making the CBDI Journal free from September onwards, we have also published several reports from past journals available for free download. Registration is required (but at no costs)

See the CBDI Journal index

Hope you find these useful.

How Long is a Piece of String? The 'Dark Arts' of SOA Project Management

2010-07-08T12:48:00.001+01:00

We are often asked "how long is a piece of string?" Well that's not entirely true, more likely the question would be something more SOA related such as "can you tell us how long it will take to define a service architecture?" However, without some greater precision of what is actually involved it is equally hard to answer either question.

Clearly scope plays a major factor. For example, is it the service architecture for the whole enterprise, a business domain, or just for a single solution? However, careful project management and estimating requires much greater precision.

CBDI-SAE becomes very useful in this respect. It isn't just that it details the necessary process decomposition that provides the work breakdown structure required for project management, but also that it provides via the CBDI-SAE meta model, a clearer definition of concepts involved. Rather than some vague notion of what a service is or what an architecture might contain, there are clear definitions of the objects and their relationships, and how they may be structured into deliverables.
This level of detail provides a much better basis for estimating and project management, as the project can be broken down much more precisely, and at the finer-grained level it becomes much easier to apply some guidelines as to how long an activity might take.
That said, there are still many variables involved, and many false assumptions that can be made.

Project Management and estimating have always been something of a 'dark art'. However, in these days of ever increasing levels of outsourcing and ever so careful cost control, having a sound basis on which to estimate the resources and effort required becomes paramount. So recently, I have begun to document some project management templates to support SOA activities and to provide guidelines for estimating the resources required.

The first set of resources have been published in our CBDI-SAE SOA Knowledgebase and made available to subscribers to assist them with these activities. To support these resources, I have also authored a report that complements them by explaining with variables and assumptions as well as any 'rules of thumb' or past experience that can be applied to estimating.

For example, there is a Microsoft Project File as shown below for the entire Agile Application Modernization Project that contains the process/task decomposition with the predecessors defined, plus also all the inputs and outputs listed as a useful reference for the project manager.

However, with over 300 tasks it might be viewed as rather unwieldy. The intention with these files was foremost to provide the decomposition in a Microsoft Project format as we recognize the usefulness of that and we have had several requests for it.
From these files it is straightforward to extract the processes and tasks required for a more narrowly focused project. The process decomposition in CBDI-SAE lends itself well to that, as many of the process units are already identified as autonomous units of work responsible for some major deliverable.

Whilst these files reflect the full scope of the Application Modernization project, locating and copying the processes and tasks to support other projects, such as the delivery of the Service Portfolio Plan or a Service Specification for example, is easy enough.

I plan on publishing further similar resources for our subscribers over the coming months.

Portfolio Management and SOA

2010-05-28T11:32:00.002+01:00

In the current economic climate, most IT organizations are faced with extremely tight budgets. Yet at the same time, the business is likely to be exploring new opportunities for growth or improvements in efficiency that in turn often increase the demand on IT. Application Modernization for example is not just a simple case of switching old systems off to immediately save money, but more often requires investment in order to improve efficiency of those systems and in the business processes they support in the long term.

You sense that for a long time it has been easier for organizations to take the easy option of giving projects the budget and freedom to do what they want, without regard to the needs of other projects or even the organization as a whole, as long as the project promises success.

However, enterprises of all types, especially the public sector, cannot simply dispense money like this anymore, and now must face up to making difficult decisions to reject project proposals that take such a ‘laissez-faire’ approach, and instead focus on more efficient utilization of the organization’s resources and to “do more with less’ by reducing waste and duplication, whilst increasing sharing and reuse

Consequently, more attention needs to be paid to the portfolio as a whole, rather than the projects in isolation

Service Portfolio Planning

CBDI Forum has long advocated a Portfolio Management approach to SOA, with Service Portfolio Planning (SPP) being a cornerstone of CBDI-SAE. So in a new report, as well as considering the specific case of SPP, I set out to consider other portfolio types and how to extend conventional portfolio management activities.

The following table for example outlines the typical approach to Portfolio Management, together with some of the ways in which this might be enhanced with our CBDI-SAE concepts.

	Typical Approach	CBDI-SAE Enhancements
Portfolio Types	Project Portfolio Application Portfolio	Separation of Asset-based Portfolios Service Portfolio Solution Portfolio (To be) Current System Portfolio (As is)
Decision Making	Financial Funding decisions Costs ROI Risk Strategic Alignment Balanced Scorecard	Architectural integrity Shared capabilities Separation of Concerns Manage cross-portfolio dependency and relationships Asset Agility
Scoping	Project-based Organizational	Asset-based Business Domain

Portfolio Management activity today is primarily centered on projects and applications, or on a program-centric approach consisting of multiple projects and applications

We believe this can be enhanced by additional attention to an asset-based approach, where

there is finer granularity, and separation of different asset portfolios, rather than just the arbitrary application scope
projects are scoped around the delivery of assets, with projects focused on different asset types, separating concerns.
projects delivering shared assets take a business-domain rather than organizational unit perspective
as well as financial decisions, the portfolios are also managed to

optimize the delivery and usage of shared assets.
provide asset agility, by decoupling assets so they are more responsive to change

As a consequence of the finer granularity and separation, the asset portfolios will need to be carefully managed to ensure architecture integrity is maintained.

The following table suggests separating the Application Portfolio into distinct portfolios. This is not to suggest that an Application Portfolio might not today recognize assets such as Services, or classify assets on an As-Is or To-Be basis. For example, the portfolio management may be based on a meta model such as TOGAF. It is more often the case today that they are not managed as distinct portfolios, and hence the investment may be sub-optimal, and typically Application focused where the Application that gets the funding and scopes the project, not the shared capabilities within it.

Portfolio	Content
Service Portfolio	Services provided by the organization External Services consumed by the organization Rich classification of Service Types (as defined by CBDI-SAE)
Solution Portfolio (To-Be)	The solutions provided by the organization to the business users and/or customers The solutions consumed by the organization The To-Be solutions Business Processes encapsulated by the Solution
Current System Portfolio (As-Is)	The As-Is Systems (Applications)
Platform Portfolio	Technology platforms on which Services and Solutions are hosted ‘Business’ platforms offering a coherent set of Services to solutions

There's lots more guidance in the report, covering the Portfolio Management Process, how to balance the top-down concerns of the CIO with the 'bottom up' needs of the individual projects need to support their business sponsors, as well as considering the relationship between Portfolio and Asset Management, but sorry, you will need to use our subscription services to read it...

Service Portfolio Planning and Architecture for Cloud Services

2010-04-29T12:16:00.005+01:00

The concept that Services are provided ‘somewhere in the cloud’ has always been central to our vision of SOA and we often used the cloud metaphor to illustrate this. Some 15 years ago in our early CBD research at Texas Instruments Software we presented the notion of application solutions assembled from a ‘cloud of services’ though this was positioned more as a way to achieve technology independence as the notions of a public infrastructure provided in the cloud were not well developed then.

The arrival of Web Services at the turn of the millennium provided a standardized mechanism by which location independence could be added and at that time we began presenting ideas like this, entitled "Does it matter where Services are Located?" - though in 2001 we will admit the audience was often more than slightly skeptical about the idea!

Does it matter where Services are Located?

Whilst there is now a high level of convergance throughout the industry on classification of Cloud Services as Software as a Service (SaaS) or Infrastructure as a Service (IaaS), as concepts such as Public, Private or Community Clouds, these generalized terms can be a bit misleading and vague when it comes to producing a more exact model of the Service Architecture. What exactly is the type of capability being offered by a Service? Who exactly is playing what role in the Service Supply Chain?

So, in the free report I have published in our CBDI Journal this month, I set out to show how our CBDI-SAE approach can be used and extended to architect for Cloud Services. The current guidance has been extended with new and refined classification systems, diagrams, policy types and techniques designed to promote visibility and good governance over Service Portfolio Planning activities and Cloud Services provisioning. (see also the slideshare following the 'read more...')

Service Porfolio Planning (SPP) is an overarching process encompassing the identification of Services, arranging them and associated artifacts in the Service architecture views, and making planning and policy level decisions about their provision, deployment and usage based on business and IT strategy and requirements.

To some extent, the portfolio planning decisions as to whether to use a Cloud Service or capability should be little different to that for other Services. However, the introduction of Cloud Services will often be a deviation from current standard practices for many organizations, and consequently they may need to apply more rigor to their decision making processes and place greater consideration over factors that today may normally just be accepted as a given, or the de facto state in their organization.
In the report I

provide a set of criteria for making planning decisions on the use of Cloud Services - based on Financial, Agility, Risk, and Capability factors
outline a set of policies for each Service Architecture view that should be set in order to ensure governance over Cloud Service provision and usage.
a mapping of Cloud Service classificiation to the more precise classification of Services used in CBDI-SAE
detail guidance and considerations for modeling Cloud Services in each of the Service Architecture Views (business, specification, implementation, deployment)
show a worked example demonstrating how different types of Cloud Services (including some of the Services from Amazon) are classified and placed in the Service Architectures.

The following SlideShare gives a brief view of some of the Service Architecture views showing the classification of public, private and community usage (see the notes for explanation). The report itself extends to 20 pages and contrains several more diagrams that explain how and why the architectures are arranged as they are.

Service architecture for cloud services

View more presentations from LawrenceWilkes.

Service Portfolio Planning and Architecture for Cloud Services - See CBDI Journal Report (free on registration)

Do's &Don'ts of Legacy Modernization - A conversation with Outsystems

2010-04-20T22:27:00.000+01:00

I recently met with Maysoon Al-Hasso, at Outsystems to talk about how IT departments are addressing their legacy systems.

You can read our conversation in Maysoon's three blog entries

- Q.1 What are the top 3 things you've seen IT do to successfully address the competing demands of new build vs. legacy modernization when faced with limited or shrinking resources?

- Q.2 Is there a role for model-based tools like the Agile Platform in legacy modernization?

- Q.3 What are the top three mistakes people make when addressing their legacy?

The Agile Application Modernization Project

2010-02-06T14:04:00.004Z

This month I have been working with my colleague Denzil Wasson to define an agile project approach to Application Modernization.

In a previous report I introduced the Application Modernization process decomposing it into Disciplines, Process Unit and Tasks. In this report, Denzil and I discuss an agile project structure and organization and provide a detailed breakdown of the Application Modernization process in terms of Project Phases and Work Packages, starting with the Assess and Plan phases.

This approach to application modernization will allow an escalation from a solution specific modernization effort to an enterprise SOA effort over time. It can be viewed as the pragmatic middle ground between a difficult to motivate enterprise level SOA and successive SOA projects that will inevitably lead to service anarchy.

The complete report contains a full process decomposition identifying tasks, inputs and deliverables, plus process diagrams.See CBDI Journal Index (modernization tab) for details

Here's slideshare presentation summarizing the introduction to the report

Agile Application Modernization Project

View more presentations from LawrenceWilkes.

SOA Fundamentals - Now Available as a Book

2010-01-21T14:46:00.004Z

I had requests from some of our certification students as to whether they could get the elearning materials in some other format they could study and use offline. Though elearning aids understanding with animations and voice overs, there is no doubt that having a printed copy to hand also has other uses - such as for reference - as well as convenience and the familiarity of a tried and trusted format.

Consequently, our SOA Fundamentals eLearning materials are now available in printed book form.

SOA Fundamentals is not a pure technology book, but covers a wide range of topics including

SOA Concepts and Principles
Web Service Protocols
SOA Technology Infrastructure
Service Architecture
Service Specification
SOA Project Management
SOA Governance
SOA Adoption
Business Architecture
Business Value from SOA.

SOA Fundamentals is aimed at anyone who needs to understand the basics of SOA. That includes IT architects and developers, IT managers and IT specialists of all kinds, as well as business analysts and other business people working together with IT to deliver a service-based business.

As well as providing a basic understanding of SOA to any reader, this guide particularly supports students who are undertaking CBDI Forum certification in SOA Fundamentals as the certification questions are all based on the content of this guide.

Fortunately, the elearning materials were already well scripted for the voice overs, so it was mainly an editing job to pull it all together and format.

For anyone interested in the process, the PowerPoint notes were sent to Word. Then edited and saved as a PDF. Then I found a very good print-on-demand service in Lulu that enabled us to self-publish the book, but at the same time make it available via Amazon and other book sellers. Lulu is highly recommended.

The book is US Letter sized with hundreds of greyscale illustrations. It typically takes about 3-5 to print plus delivery.

Application Modernization Reports - Freely Available

2010-01-06T18:38:00.007Z

We have recently made two of our recent reports on Application Modernization freely available.

These can be download from the website. See the Application Modernization Tab.

SAE2 - Supporting the Application Modernization Process

2010-01-04T17:00:00.011Z

We start the new decade at CBDI Forum with a major focus on Application Modernization (AM). The need to reuse existing assets in new SOA solutions is not itself new, and we have covered this before in various reports, whilst the CBDI-SAE SOA Process has always included a ‘Legacy to Service Transition Planning’ discipline.

Equally, at Everware-CBDI, we have considerable practical experience in AM via a number of engagements our consultants have worked on, primarily in the CA Gen domain. We also recently announced the SOA4GEN program along with partners Jumar Solutions.

However, we recognize that in many organizations AM has increased in priority, whilst SOA is becoming ‘business as usual’. That doesn’t mean AM is replacing SOA, rather it puts SOA in context. AM is the objective, and SOA is one of the key elements in achieving it. Consequently, we wanted to increase our coverage of AM in our research and guidance, and further explore the relationship between AM and SOA.

One of my focuses in this work has been on the process. To date, our CBDI-SAE SO Process has been focused primarily on disciplines covering the planning and provisioning of the Service Portfolio, and its use in solution delivery. To support AM, we have now begun to document additional disciplines such as Application Modernization Planning and Knowledge Discovery (understand the As-Is system, and extract knowledge of the current assets), and give them equal weight and depth of coverage to these in comparison with the existing SO disciplines and detail how they interact.

An overview of the CBDI-SAE SO Process is shown below. Such is the degree of change that we decided to label this SAE2 as it is effectively a new version of our CBDI-SAE SO Process framework.

(click to expand)

I have created the following SlideShare to provide an overview of the AM process. Subscribers to our CBDI Journal and Knowledgebase can find out more detail of the process units and tasks in the December 2009 Journal.

SAE2 Application Modernization Process

View more presentations from LawrenceWilkes.

There are many approaches to Application Modernization as outlined in the table below, and many factors determining which approach to follow.

Objective	Approach	Description
Replace	Replace (Build)	New build in-house or outsourced
Replace	Replace (Buy)	New COTS
Rationalize		Consolidate and rationalize
Modernize -Component Reengineering	Re-skin	New UI Web 2.0 enablement
	Re-process	New Business Process
	Recode	Reengineer implementation
	Restructure	Componentize implementation
	Re-platform	Migrate to new platform
	Re-host	Migrate to new servers Virtualization/Cloud
Modernize - Service Reengineering	Service Enable	New Service Interface New Data Service Underlying or Exclusive service
Modernize - Service Reengineering	Service Facade	New Core Business Services layer New Process services layer

In many cases, to achieve the desired objective it may require more than one approach. For example, in order to build a new Service Façade it may require that some current assets are also Service Enabled – so they can be consumed as Underlying Services in the Service Architecture of the façade.

Another factor in determining the approach will be the unit of scope of the modernization requirements. With a broad scope, each of the different sub units may require a different approach.
The modernization approach may also be determined by the decision whether the To-Be solution is deemed to be core or context, mission critical or supporting, according to Moore’s quadrant.

Subsequently, the approach determines the type and level of activity required in order to perform the modernization. At the implementation level it may not be possible to discover the knowledge of, or recover the actual artifacts themselves for potential reengineering. For example, with ‘black box’ software where the author/vendor has long moved on, this may force activity to take place at a higher level than was perhaps hoped, and lead to greater amount of forward engineering.

On the other hand, even though knowledge and artifacts can be recovered, decisions to outsource delivery or replace with COTS may mean there is less imperative to do so. Hence it is important that such decisions are made in the planning stage.

The factors determining the approach are summarized below, which also identifies key questions of whether the project possesses the necessary capabilities to perform the activities required, or whether the activities should be outsourced?

Factor	Determinants
Objective	What approaches are necessary for the given objective?
Unit of Scope	What approaches are necessary for the given scope?
Entry/Exit Point	What depth of knowledge can be discovered? What exit point is relevant to the To-Be requirements, and decisions on sourcing the To-Be?
Capabilities	Does the project possess the capabilities required to accomplish the modernization? May determine build or buy, outsourcing.

Modeling for Smart Ecosystem Architecture

2009-11-26T17:46:00.005Z

In a new CBDI Journal report this month I have been looking at the modeling requirements to support our Smart Ecosystem Architecture (SEA) concept.

It is evident there are two major architectural patterns in play – service oriented and event driven. But in addition we need to recognize a new dynamic which we might characterize as “post enterprise” including cloud computing, Web 2.0 and so called smart IT that encourages ecosystem collaborations over internal enterprise processes.

There is a lot of superficial (marketing) noise suggesting a close relationship between Event Driven Architecture (EDA) and SOA, Actually the two domains are in practice islands of automation. SOA is much more widely used, whereas EDA is still largely restricted to narrow focus, point solutions. This is changing, but slowly as there are considerable complexities to deal with in an integrated world. Not least event data availability, architecture contention and difficulties in testing. Of course the solution to broader utilization is integrated modeling approaches.

Similarly cloud, Web 2.0 and smart IT are separate domains and narrowly focused and capturing the requirements for either still tends to exist in two separate domains. However, to recognize the real opportunities in the smart ecosystem requires us to take a more coordinated view of modeling that spans all of these considerations.

CBDI has a vast amount of business modeling guidance and advice; much of it has pioneered thinking around modeling services, events, capabilities, meta data, dynamic business intelligence and ecosystems. See for example the CBDI report series on Business Modeling for SOA that begins specifically with the event-response concept at a high level as part of business modeling , and the report Event Driven Service Architecture that examines the convergence of EDA and SOA.

In this new report Modeling for Smart Ecosystem Architecture I advise on how to integrate these different perspectives across the broader set of architectural views. We also explore the dual requirements to consider both events and services, and also consider some of the meta model impacts.

On that last point it is interesting to note the lack of any 'standard' meta model for EDA, and hence similarly the EDA/SOA relationship. Whilst OMG have been working on a RFP for an ‘Event Model and Profile’ (EMP), this has yet to be issued.
Today, the requirements are quite likely to be captured directly into a format prescribed by proprietary event management products used for the EDA implementation.

We are currently evaluating how we add the necessary concepts to support EDA and SOA convergence in the SAE Meta Model and our associated UML profile, and discussing how we may contribute to emerging standards activity in the same way as we have done with SoaML.

The Shape of Business - Drivers for Smart Ecosystems

2009-11-24T12:43:00.007Z

A report this week from the Confederation of British Industry (CBI) entitled The Shape of Business – The Next 10 Years provides some useful insight into emerging business drivers that reinforce our concepts of smart ecosystems.

Taking at some of the key headlines in the report, you can see the increasing need for organisations to develop ecosystems, and adding smart behaviour.

Movement to a more collaborative business model. The recession has made businesses much more aware of the complexities and interdependencies in their operations, their financing, supply chains and customers, but they are still not able to fully assess or capture these in their business planning. To gain greater control of these uncertainties, businesses will seek to ‘simplify’ their operations and will enter into more partnerships and joint ventures. In particular, this will be important for businesses moving to a ‘core plus periphery’ model

= a need to develop ecosystems
= the need for a smarter supply chain, hence smart ecosystem. Though supply chain optimization is hardly new, the emphasis on this is clearly going to grow. There are also additional ways of thinking about the supply chain – e.g. as a source of finance, not just ‘goods’. i.e. don’t invest in stock, and certainly don’t lend money to buy stock, but rely more on the support of participants in the supply chain and on its optimisation to lower inventory and provide 'just in time' fulfilment.

Rationalization to the Core. The recession accelerated the need to address inefficiencies and non-core activities across the enterprise. It has also provided the stimulus for companies to re-think themselves and re-evaluate their future.

= a need for greater collaboration across an ecosystem consisting of an internal core and an external periphery (what Moore might term context)

Technology will enable new ways of working. Businesses will increase their use of social networking techniques to solve problems – many more companies will use Facebook, Twitter and other web 2.0 developments

= making sense out of what is going on in complex social networks is going to require a lot of ‘smart’. It isn’t going to save money if it requires an army of people to monitor and participate in social networks

There were also some interesting comments that identify the need for agility.

For example, there is a current trend of “localism” in some organisations or industry sectors, relocating certain supply chain activities back to the UK. What is evident is that the decisions about what should be done locally or globally, or what should be in-house or outsourced, will be fluid. It is not a one-off decision but one of constant re-evaluation in the face of prevailing conditions. Equally, large organisations may find that the decision varies across different product lines, with ‘no one size fits all’ solution. As such, organisations will need to be very agile if they are to quickly capitalise on changes in those conditions.

Or course it is very difficult to predict what will happen across the next 10 years. How many predicted the current situation a decade ago?

And there is of course the need to survive 2012 first!

Time to Eat the Programmer?

2009-11-15T15:23:00.006Z

One of my colleagues asked the question of whether the work that we do (at Everware-CBDI) could ride the ‘green’ bandwagon. After all he suggested, some of our key services in helping customers to transform their existing systems, and reuse software components and software services via CBD and SOA, might be considered ‘green’.

My first reaction was “that’s a bit of a stretch”. However, some of the news recently that has accompanied the publication of the book “Time to Eat the Dog” by Robert and Brenda Vale highlights just how complex the ‘green’ issue is.

For example, the authors point out that the ecological footprint of a keeping a medium sized dog as a pet is in fact greater than that of driving a large SUV 10,000km a year. This presents a double ecological challenge for my colleague who raised the question as he has both a Labrador and a SUV.

It struck me therefore that if those authors can determine the effect each pet has on the environment, then similarly we ought to be able to determine the ecological impact of each line of code produced by a programmer.

I don’t intend to do so here (determining the equation would itself be a waste of valuable resources – well my time at least). But it does illustrate just how complex the issues of all things ‘green’ truly are.

Most ‘green’ IT efforts today are focused on reducing energy consumption by using more efficient computers. Similarly, many of the ‘green’ messages focused on the population at large are also focused on reducing energy consumption by using more efficient means of transport.

But if as demonstrated by the ecological impact of pets, we need to consider much more complex factors in order to truly establish our impact on the environment, then I guess it is just as valid to ask not just how much electricity does a CPU use, but also what is the ecological impact of programming – or other IT development activities.

Perhaps organizations that are serious about ‘green’ IT ought to better consider the options of software ‘recycling’ before they simply create yet another new system. No longer is it just an economic decision of whether it is cheaper to recycle components of an existing system or build new one, now it is also an ecological decision too.

Yes, it is still a “bit of a stretch”. But nevertheless, not perhaps quite as extraneous as you might have first thought.

(you can read more about “Time to Eat the Dog” via New Scientist, BBC News, Guardian Newspaper)

Using Service Component Architecture (SCA) and SoaML

2009-11-03T16:01:00.004Z

Service Component Architecture (SCA) brings the formality of SOA contracts and end point separation into the implementation layer. For the many organizations undertaking modernization efforts the standardization of componentization and SOA enablement is an important issue. Whilst its usage may not be widespread, it is likely that growing demand for rationalization of the implementation layer will lead to increased adoption.

In a new report I look at SCA and consider how it relates and maps to SoaML and provide guidance for integration into the broader SAE/SoaML picture.

My conclusion is that in some respects the SCA and SoaML initiatives represent something of a missed opportunity. Though the respective groups will claim they address different requirements or audience, the reality is that it is hard to see why they couldn’t have been based on common concepts where they overlap. One could have extended the other instead of re-inventing. Then the inevitable additional effort and loss of integrity through the mapping and transformations from one to the other discussed in this report could have been avoided. More so given that some organizations participated in both initiatives.

On the positive side, SoaML to SCA transformation tools do exist, and the mapping provided between SCA, SoaML, and CBDI-SAE in the report is at least at a high level relatively straightforward.

SCA is an elegant solution, but somewhat limited by lifecycle scope, and dependence on the SCA runtime, in comparison SoaML. However, it does offer portability across SCA compliant platforms and that is of significant benefit.

SCA does seem to have reached a bit of an impasse. It is there, it works, but it isn’t clear whether adoption is growing, either by end-users or support by more vendors. If you are using or planning on using SCA, please let us know in our CBDI LinkedIn where we have created a discussion topic

An Update to the Example Model based on Version 3 of the CBDI-SAE Meta Model

2009-11-03T15:56:00.002Z

The October 2009 CBDI Journal contains a follow-up to the previous article presenting the draft version of SAE Meta Model V3. In this new report John Butler provide an update to the UML Profile for the CBDI-SAE Meta Model V3 focusing on the core areas and those that illustrate alignment with SoaML. Given that worked examples are the best way to understand a meta model John has updated the example model based on the fictional company Springfield Parcels, Inc. This should allow readers the opportunity to compare and contrast the version 2 meta model with that of version 3.

SOA - From Vision to Practice

2009-11-02T19:37:00.005Z

I have just been reading the September 2009 issue of the Microsoft Architecture Journal which is focused on SOA. The foreword starts off by referring back to the report 'Understanding SOA' that David Sprott and I wrote for the very first issue back in 2004. As editor Diego Dagum puts it, "the main difference with the article that we published in the early days is that, this time, thoughts have emerged as a consequence of a practice; in 2004, thoughts had emerged as a consequence of a vision."

The inevitable "SOA is Dead" discussion is raised in the report Model- Driven SOA with"Oslo". But that report and others in the journal really serve as reminders that SOA is in reality still in its infancy, and we are still developing best practice, and in Microsoft's case at least, the modeling tools to support it.

Encapsulating best practice in modeling tools is essential if we are to

move beyond perceptions by some that "SOA is a technology"
demonstrate business value by making the SOA investment more relevant and 'visible' to business sponsors

That's a key reason why we ourselves have invested so much in our CBDI-SAE Meta Model for SOA and the associated UML profile that enables users to move from business models to implementation in a consistent and traceable manner.

It would be interesting to see how we might implement that in Microsoft Oslo. Oslo offers the promise of "the model is the code". Not surprisingly, Microsoft do have a slightly myopic view of the world that typically extends only as far as their own technologies. Though it may not be a business modeling language, Service Component Architecture (SCA) that I have been looking at in more detail this month does already provide a model-driven approach to modeling the Service Implementation Architecture, which is a good level of abstraction at which to consider "the model is the code". Unfortunately, SCA isn't implemented on the Microsoft platform. It's a pity that wheel is going to be reinvented I guess.

Nevertheless, it is good to see Microsoft continue to develop the SOA vision and turn it into practice, and in particular to embed that vision deep into their platform and tools, such that SOA becomes just 'business as usual'.

Service Classification

2009-10-29T19:34:00.007Z

Zapthink recently asked the question "Are Services nouns or verbs", and if they can be designed as either, then which is better?

I don't think it is a case of "which is better". Services that expose a process (verb) address different requirements to those that expose an 'entity' (noun) . One is not better than the other. Instead they complement each other.

In CBDI-SAE we classify services according to the type of capability they provide. By focusing each service on a particular type of capability it enables greater modularity, separation of concerns and so on, enabling them to be more easily shared or composed into new services.

For example core business services (entity) provide operations focused on managing the information about a key business resource - like customers, orders, or products. These are independent of the processes that use that information.

Whereas process services provide operations that enable solution assemblers for example to interact with the different steps in the process. In turn the process services then consume the appropriate core business services.

In this way they can change the process service, add new operations and such, or add new processes, but without impacting the core business services.

If you mix these capabilities together you run the risk for example that the entity information is only provided via a specific process. We see this often and it's just too coarse grained and not a very agile approach. Processes have a different change cycle to entities. Many different processes use the same entities. So keep them apart.

Suggesting, as some are, that there is only one type of service and each provides multiple types of capabilities overloads the service and makes it difficult to use or share in different contexts or solutions..

We go further than just verbs and nouns however, and CBDI-SAE classifies service types into Process, Capabilities, Core Business Services, Utilities, Underlying, and Infrastructure .

Appendix A1 of our CBDI-SAE Meta Model for SOA also provides definitions for each type, and also gives an example policy for dependency rules between service types.

In CBDI-SAE we then detail different techniques that are applicable to identifying and specifying each type of service.

SOA Manifesto – Good, but too late?

2009-10-28T16:45:00.003Z

This week, a small group of SOA cognoscenti met and published a SOA Manifesto. Thought it is eloquently put, it is difficult to see what has been said in the SOA Manifesto that hasn’t been said before.

The reality is that SOA isn’t new anymore. That some claim it is in fact dead or passé is testimony to no matter how much advocates of SOA protest, we have failed to fully get the message across, at least to the business that is footing the bill. I am not sure how the SOA Manifesto changes that. Those that are already pre-disposed towards SOA will become signatories ( I already have); those who are not will ignore it. Little will have changed.

I am reminded of the current political situation here in the UK, where the current labour government could put anything they wish in their manifesto for the coming general election but they are so unpopular it wouldn’t make much difference. Even if they promised to abolish taxes, they still wouldn’t convert those who have already made up their mind how they are going to vote. (and when did any political party actually deliver on their manifesto?)

The world has moved on. SOA should be considered business as usual. Yes, organizations still need to improve the way they approach SOA . Yes, people still need to adopt and apply the principles and be provided with better guidance. But if you want to get the majority’s attention, then you need a manifesto that excites them about the coming decade, not one that tells them what they should have been doing better in the last.

As I highlighted in my blog on Architecture for the Smarter Planet, SOA is one piece of an architectural puzzle that is going to challenge enterprises and individuals over the next decade. Although I believe the future is intrinsically service-based, I know we are not going to convince organizations to invest by telling them all they need is SOA. Rather, SOA has to be put in context, and positioned as one part of something much bigger.

Something like the Smart Ecosystem seems a much more holistic topic for a manifesto that addresses the needs of the next decade. The challenge for businesses will be how they become successful participants in ecosystems they do not control. SOA plays an important part in that, and can be a key enabler in a genuine business vision.

But just saying that SOA should be “business driven” doesn’t make it so.

Architecture for the Smarter Planet

2009-10-27T16:44:00.018Z

You are probably aware by now that IBM’s current ‘theme’ is the Smarter Planet.Their basic message is that organizations (both businesses and governments) should take advantage the fact that we live in a world that is,

Instrumented – 30 billion RFID tags in supply chains
Interconnected –Trillions of connected devices, and two billion people on the web
Intelligent – 15 petabytes of new information created everyday

And to do something ‘smart’ with these resources.

This seems in part a reinvention of the ‘Pervasive Computing’ vision. I authored a lengthy report on this some time ago in 2002. I still don’t think we are fully there yet, but you can sense that we are another step closer to it becoming reality, and IBM provide some good real world examples via the link above.

Architecture Scope

Attending the IBM Rational Software Conference recently at which the smarter planet was the theme of the keynote, it prompted me to think about what sort of business and IT architecture was needed.

One thing that is clear is that architects need to think very carefully about scope in this scenario. You cannot take a narrow inward looking project or even enterprise-wide view when the smartness you seek comes from the interconnectivity and behavior of the ecosystem. My colleague David Sprott has coined the term Smart Ecosystem Architecture (SEA) in reference to this.

I believe the key thing here in relation to SEA vs EA (Enterprise Architecture) is for organizations to understand

what activities are going to be performed by the ecosystem (or the other participants in it), and what activities they need to perform themselves.
and consequently, what they have to do – e.g. in terms of service provision – to participate in the ecosystem, and hence what capabilities and resources they need to provide their services, and handle the events that require their response

It also seems critical to me to that it is necessary to understand which capabilities are core vs context, but from an ecosystem perspective. (I need to ponder on that further…)

Architecture Patterns

That addresses scope, but what about the architectural patterns that applies?
I am tempted to say it’s all about Service Oriented Architecture (SOA) – in that federated, interconnected ecosystems are inherently service-based. But smart behavior is in a large part about sensing events and responding to them, so we need to add Event Driven Architecture (EDA) into the mix.

Before long I recognized we need several other architectural patterns in support of the Smarter Planet. I started off trying to draw a layered architecture to support of this, but that seemed inappropriate, so for now I will try to sum it up in a simple textual list.
OK, so the architecture for the Smarter Planet involves

trillions of connected objects and billions of people accessing petabytes of information via millions of solutions, based on an agile,
Web 2.0 Architecture – enabling billions of people to mash up rapid, user and community driven solutions, in turn assembled from millions of services, and generating trillions of events,
requiring Event Driven Architecture (EDA) to determine the autonomic response required to sensors and changes in state,
that are also placed into context by Business Process Architecture (BPA/BPM) and Information Architecture (IA)
supported by services in a Service Oriented Architecture (SOA) that provides a formal basis for the decoupling of Provider and Consumers resource,
as well as a Web Oriented, or Resource Oriented Architecture (WOA/ROA) as exchanging information between those trillions of devices in an efficient manner will likely be done in a more lightweight manner than full blown Web Service-based SOA
with the implementations defined in a Component Based Software Architecture (CBSA) - with the focus on right-grained software enabling federated software delivery, that is running anywhere, anytime on a
Cloud Based Architecture (CBA) that details the virtualized, federated infrastructure providing scalability, reliability. Which brings us back to SOA, as cloud computing is inherently service-based.

Anything I missed? My point is that architecting for the Smarter Planet is not about choosing one architectural pattern in preference to another – SOA or EDA or WOA or Web 2.0. One of those can't possibly address all the requirements. But rather how to use SOA+EDA+WOA+Web 2.0+xyz together, and understanding how they relate, not compete.

Enterprise Architects will argue these are all just components of EA, and to some extent that is correct. However, the scope now needs to be broadened to SEA.

What we need to avoid is any sense that things just got more complicated thanks to the Smarter Planet. With the application of sound architecture, it ought to get a whole lot simpler.

Some of CBDI Forum's Architecture mashups,

Extending SOA with Web 2.0 (free access on registration)
Web 2.0 and Enterprise Architecture (subscription required)
Event Driven Service Architecture (subscription required)

CBDI-SAE Meta Model for SOA Version 3

2009-10-02T17:22:00.002+01:00

The September 2009 CBDI Journal contains a “draft” publication of Version 3 of the SAE Meta Model. It incorporates some changes based on our continued work with customers and standards organizations and provides a mapping to SoaML so that end users can reap the benefits of both the breadth and depth of the SAE Meta Model as well as the capabilities that tool vendors will develop based on SoaML. Everware-CBDI plans to publish mappings to other ontologies shortly and other meta models as they mature.

It’s been some time since the last major revision of the CBD-Service Architecture and Engineering™ (SAE) Meta Model for SOA. Since then Everware-CBDI has published a UML Profile for SAE1 that has received quite broad acceptance and has been downloaded by thousands of members worldwide.

Everware-CBDI is committed to aligning with standards where appropriate and to that end we have contributed to, and worked extensively on SoaML, the service modeling language developed within the Object Management Group (OMG). What we’ve found in the course of our involvement in these efforts is that while there is some convergence in terminology, significant differences remain in the details between the various standards bodies working in this area, ranging from overall purpose of model/ontology of each organization to the exact definitions.

Our intent in the SAE Meta Model is to provide support for these various standards to our membership while not sacrificing the value built into our existing meta model. To that end, we will provide mappings of SAE terminology to these various meta models, profile and ontologies and guidance regarding their use within SAE.

Background

The SOA marketplace continues to mature and at Everware-CBDI we feel a measure of satisfaction in knowing that we’ve had a hand in that evolution through the guidance we’ve been giving over the years. The SAE Meta Model is one of many contributions and was a key input to the Object Management Group’s SoaML specification.

The SAE Meta Model was first released in October of 2006 and updated to Version 2 in 2007 as a result of feedback from our membership and the broader industry. Everware-CBDI created a UML profile based on V2 in 2008 that has since been downloaded thousands of times all over the world. Obviously industry was ready for more mature support for modeling services and the feedback we got was very positive.

In parallel with our efforts the Object Management Group (OMG) issued a Request For Proposal (RFP) for a UML Profile and Meta Model (UPMS) in September of 2006. A submission team formed that included Everware-CBDI, IBM, EDS, HP, Model Driven Solutions, SINTEF and several others. SoaML was finally adopted as a specification in November of 2008 and is currently in the finalization process.

Introduction to V3

Any model remains a work in progress if it is to continue to be relevant over time. The SAE Meta Model is no different. As mentioned above the meta model has been downloaded thousands of times and has provided a practical mechanism for using SAE with existing toolsets. Feedback from that broad membership usage along with our own involvement in standards organizations and client engagements has pointed out some areas that needed improvement and some holes that needed to be filled. Equally, we have continued to refine and extend our SAE guidance. Hence, we saw requirements to address the following:

Incorrect or cyclical dependencies between meta model packages,
Only a very loose notion of who or what is providing a service,
The need for a more consistent concept of Service for both business and software contexts,
Separation of a concept (or “thing”) from the specification (i.e., the artifact) of the concept,
The need for refinement of the relationships between Service, Automation Unit, and Deployable Artifact, and
The need for inclusion of the concept of Internal Architecture.

Version 3 of the SAE Meta Model attempts to address these issues.

We shall shortly be launching a review process and will make an updated specification and UML Profile available to those who would like to participate in the process.
If you are interested in doing so, drop us a note on our CBDI Contact Page

To view the full report

A Unified SOA?

2009-10-02T15:48:00.007+01:00

My Colleague David Sprott blogs this week on the Failure of SOA Standards
There is no need for me to repeat his comments.

In his blog, David comments on Navigating the SOA Standards Landscape Around Architecture, a paper recently published by OASIS and The Open Group. It seems to us and many others that this is an attempt on the part of the authors to gloss over the fact they have managed to produce three alternative SOA standards when the world really only needs one. Saying that OASIS SOA RA is ‘similar’ to the Open Group SOA Ontology is a bit like saying Cricket is similar to Baseball. Because they both involve similar concepts of bat and ball presumably this means a cricket team could play a baseball team?

It is only with a more detailed level of mapping of more precise meta models that anything useful can be done in the real world. Even then, the reality is it is difficult to mix two frameworks and meta models.

One of the intriguing things for me is the involvement of common participants in all three SOA standards efforts at OASIS, The Open Group and also at OMG. IBM in particular stands out as a vendor who appears to have put significant effort into them all, and with minimal visible consistency resulting. Why? How does it help IBM customers to provide them with a choice of three alternative SOA standards? Surely that only works to slow adoption of SOA, not to encourage it.

This was one of the major successes of Web Service Protocols – that there is only one standard for each requirement in the protocol ‘stack’. Even though the various protocols are administered by different bodies, they have managed to avoid competing. Well eventually anyway after some initial vendor-based alternatives fell by the wayside. And since then take up has been widespread. Partly this was achieved because vendors such as IBM together with Microsoft participated in only one activity per protocol and not three. Hence others were forced to make decisions to go one way or the other rather than sit on the fence and be endlessly compromised by choice.

And this is what the same groups now need to do with SOA standards. Combine their energies, put aside their differences and produce a set of unified standards – even if different elements within it are administered by different bodies, as with Web Service Protocols, so they can each retain some ownership. We only need one SOA Reference Model, one SOA meta model, one set of principles and so on. But once you have the common foundation, just like the Web services efforts, there can be parallelism. From our work on the CBDI SAE meta model we know that there are multiple views which can be modelled concurrently such as business, specification, implementation, deployment, testing etc.

That said, perhaps more of the competitive approach taken with Web Services would be apt. That is, I would rather see IBM back one SOA Meta Model standard for example, and not three. I am sure the competitive nature of early Web Service protocol standards activity really helped to force the pace – a sort of SOA ‘Space Race’. Now, rather like the pace of the space race today, I get the impression it will now be decades before we land on the SOA moon. Of course, if OASIS, The Open Group and OMG cut the stack up as I suggest and avoid overlap so much the better.

But why is this important? At runtime, clearly the vision of SOA wasn’t going to work without consensus on the interoperability standards. But I believe it would be a mistake to assume a similar consensus isn’t required at earlier stages in the life cycle. Time after time we see that the lack of a common SOA framework is hindering SOA adoption in organizations. Sure different projects can all agree to use Web Services at runtime, but the lack of any common standards before they even get to that point means that much of the SOA vision remains elusive when viewed from an enterprise perspective. It is pretty hard to have full life cycle asset management and governance over SOA deliverables if the concepts change for each phase. In most organizations, users have little choice but to pick a single vendor's proprietary solution to such requirements that complies with no relevant SOA standards.

I am reminded of object orientation pre UML. What is needed now is a Unified SOA.
SoaML, in which we have participated might be one step in that direction, at least at the modelling language level. Even so, TOGAF 9 has its own meta model, but so does Archimate, and there is one implied by the OASIS SOA RA, and so on. Then there are the differences even between The Open Group’s TOGAF 9’s coverage of SOA, and their own SOA Source Book. Do you know for example when you see a concept mentioned in one document it has the same meaning as in another? Often no definition of the concept or reference is even given so it is difficult to be sure. Clearly there is still a long way to go.

And let’s not forget that there is a large government and defence sector who are extending their own frameworks such as FEA, DODAF, and MODAF with yet more unique perspectives on SOA.

It is at least welcome though that OASIS, The Open Group and the OMG have seen the need to cooperate on Navigating the SOA Standards Landscape Around Architecture. Perhaps now they can move forward as I suggest and ensure greater collaboration and consistency between their efforts with a single purpose in mind.

We would be happy to participate in such a unified process. What we can’t afford to do unlike IBM is participate in three (even though they have all invited us to), and more to the point it would be counter productive! In the meantime we will continue to participate in SoaML and refine our own CBDI-SAE Meta model for SOA. To that end we have just published a draft V3 of our model which includes SoaML adoption.

Users shouldn't need to be shown how to navigate their way around multiple SOA standards. They just want to know "are we there yet?"

Microsoft Visio – Suitable for Modeling SOA and EA?

2009-10-02T15:21:00.005+01:00

Over the last month, I have been looking at Microsoft Visio as a possible tool for modelling Service Architecture and other SOA deliverables and as a place to capture the Enterprise Architecture. This was prompted in part by a product report I have just completed on Orbus Software’s iServer product, which adds a multi-user repository to Visio.

Given the number of specialist modeling tools there are on the market, it is surprising how many enterprises we encounter that use Microsoft Office as their prime vehicle for capturing various items of metadata and producing the documentation and diagrams required to support their IT architecture and delivery projects. Requirements and specification deliverables are routinely documented using Microsoft Word, while metadata is captured as columns and rows in Microsoft Excel spreadsheets, and diagrams are constructed using Microsoft Visio (or even just Microsoft PowerPoint). However, there are many shortcomings to this approach in an enterprise context, and that is primarily what iServer sets out to address and I have examined in my report.

Until now, I had only been a casual Visio user. For simple diagrams we publish in reports, PowerPoint is sufficient, and also the format we provide to our customers. For more rigorous work – with our clients for example, or to produce our SAE Meta Model, then we use other UML modelling tools.

Once I started investigating though, I was impressed how simple it is to create your own stencils and templates, but moreover how straightforward it is then to connect instances of the shapes to items in a Microsoft SharePoint list. In the example below, I created a simple template for the Service Specification Architecture diagram we use in CBDI-SAE, and then linked it to a SharePoint list containing a catalog of Information System Services (TOGAF) which are classified by Service Architecture Layer.

Hence it would be straightforward to create a Service Catalog in SharePoint, and then visualise the architecture for the services in Visio. My product report looks at how to do this in iServer. Clearly that is a more robust solution than my simple SharePoint example. However, for anyone with SharePoint and Visio already at their disposal, all it takes is some effort on their part.

Hopefully I will find some time to work on a Visio Template and corresponding SharePoint Template for CBDI-SAE and make these available in the same way we have done with our UML Profile.

Let me know if that would be useful.

In the meantime, I am interested to know just how many folk are using Microsoft Office and Visio as their prime tools for capturing EA and SOA deliverables, and what your experiences are.

Underlying Services – Architecture for Existing and Acquired Resources

2009-09-02T17:29:00.002+01:00

Most organizations expect to reuse their existing or externally provided systems as the primary resources to support their Service Architecture. Now that most operating environments and packaged applications have been ‘service enabled’, using such resources in the Service Architecture is technically straightforward. However, the native Services provided by these disparate resources will probably have inconsistent specifications, business taxonomy and business information models. Using them directly in the Service Architecture will probably compromise SOA goals. The report we have published considers where the Services provided by existing and acquired resources fit into the layered Service Architecture and how constraints and compromises can be minimized.

Appendix A1 of the CBDI-SAE Meta Model V2.0 identifies nine Service Architecture layers. It is a policy decision for organizations as to which layers they actually want to use. However, there are four basic service layers we would expect to see in all instances of Service Architecture.

The core functionality is delivered by the Core Business Services. Each Core Business Service is based on a major business type identified in a Business Type Model. Some people may think of these as ‘Entity or Domain Services’.
The Process Services then orchestrate these Core Business Services to deliver a specific business process.
In turn, both these service types may use Utility Services that provide common business or technical utility functions, such as Address Formatting or Authentication.
Finally, the Underlying Service layer classifies those Services which are provided by the existing systems or packaged applications, but which have inconsistent information models that are not based on the Business Type Model.

The Service Specification Architecture then assigns instances of services their respective place in the layered architecture and shows the dependencies between them. The Service Specification Architecture is then used as the basis for the Service Implementation Architecture where the Automation Units that provide the implementation for the Services are identified.

A natural question to ask is, why can’t an existing or external resource just be the Automation Unit or Implementation of the Core Business Service? Why is it delegated to a different layer? After all, as stated most organizations will want to use these resources as providers in their Service Architecture as these will contain the operational data required.
However, there are many challenges to doing this, especially if you want to deliver the agility promises of SOA.

Hence this report looks at those challenges and how they are overcome by using the layered Service Architecture.

The report also identifies policies and techniques for identifying, creating and consuming Underlying Services.

See Underlying Services – Architecture for Existing and Acquired Resources (subscription required)

SOA Exception Management

2009-06-30T13:30:00.001+01:00

Our SOA Exception Management Framework provides a structured approach to specifying exception conditions and dealing with them at run-time. In this report we look at some techniques that may be used

The report primarily considers exception management within Service-based solutions. That is, how both Service Providers and Service Consumers should handle exceptions that may occur in the processing of Service requests and responses.

That said, SOA exception management should not be distinct from broader exception management approaches that ought to be in place to deal with solutions that are not Service-based, and elements of the framework presented in this report should be common to both.

However, as we find ourselves routinely saying, SOA brings new challenges. For example:

The federation of participants implies that the Service Provider and Consumer may be in different organizations.
Loose coupling implies there is no shared technology or infrastructure. Hence, there may be no common exception handling software, systems management, or error log.
The Service Provider and Consumer must rely on Service Contracts such as the Service Specification to document exceptions and specify any obligations on them when an exception occurs.

Consequently, the obligation is on both Service Provider and Consumer to independently manage exceptions. It is not the Service Provider’s responsibility to resolve errors in the Service Consumer’s requests. The Service Provider may simply reject invalid requests and do little else. It is up to the Service Consumer to handle such exceptions, log them, notify any interested parties, analyze the cause, and finally take any remedial action if required.

Equally, there is little point in the Service Provider giving detailed feedback on exceptions that represent problems behind the Service ‘façade’ as it is not the Consumer’s responsibility to do anything about it. But of course, the Provider must log and respond to such conditions.

That said, in many cases the Service Provider and Consumer will be in the same organization. As such it may make sense for them to:

agree on a consistent approach to exception handing.
standardize exception ‘protocols’. For example, exception codes and messages.
provide common utilities or infrastructure for exception management.

Even where the participants are not in the same organization, they will still need to agree on the exception handing protocols. Though the detail of this should be part of each Service Specification, ideally there should be some generalized framework in place for exception management so that ‘sets’ of related Service Specifications at least behave in the same way. For example, all Services in a Business Domain. This report therefore sets out to describe such a framework.

Service Implementation Architecture and Automation Unit Specification

2009-06-30T13:10:00.001+01:00

The Service Implementation Architecture and Automation Units Specifications provide the transition from the logical Business Service Architecture and Service Specifications to the implementation of Services in software. In our latest report we will look at how Automation Units are identified, modeled and specified, and advise on how they are documented in the Service Implementation Architecture.

This report presents Automation Units as a relatively flexible concept that can be used to represent a variety of different implementation approaches. The degree of documentation may vary by type or by the nature of the provisioner/implementer relationship. We would expect organizations to define policies that more tightly scope the permitted Automation Unit types in their organization and the associated documentation they require. From a process perspective, the Service Implementation Architecture is a deliverable from the Service Oriented Architecture and Design discipline, and the Automation Unit Specification is a deliverable from Service Provisioning. Together with the Service Specification, these will be key inputs to the Service Implementation activity.

See Service Implementation Architecture and Automation Unit Specification

Open Group Release SOA Source Book

2009-04-30T14:50:00.004+01:00

We notice that the Open Group have this week released their "SOA Source Book".

The SOA Source Book is available athttp://www.opengroup.org/projects/soa-book/
Ideas in the SOA Sourcebook have been circulating around the SOA world for some time, and there is (not surprisingly) a lot of overlap (with more or less variation) with material the CBDI Forum has published from 2003 onwards. In particular the following have strong affinity to work we published in the past few years.

SOA Source Book	CBDI
A Maturity Model for SOA	SOA Adoption Roadmap Planning Framework
High-Level Perspective of the SOA Reference Architecture	Based upon our early layered architecture published in reports such as Understanding SOA
Service-Oriented Infrastructure	SOI Maturity Model
Introduction to SOA Governance	SOA Governance Framework

For example, compare and contrast the Open Group Maturity Model for SOA:

With that from CBDI Forum. The Streams are almost identical, and the phases whilst named differently have a similar basis.

The Open Group SOA Source Book SOA Reference Architecture. This was itself based upon an earlier IBM model, which was markedly similar to CBDI's.

With the CBDI Model from 2003,

The Open Group SOA Source Book SOI Reference Model

With the CBDI SOI Model from 2006

And finally, the Open Group SOA Source Book SOA Governance Regimes
with the CBDI SOA Governance Framework from 2007

Recently, we published a report TOGAF 9 complementing SAE practices and concluded

"TOGAF is a generalized enterprise architecture framework applicable to varying architectural styles. TOGAF 9 provides some useful alignment with SOA architecture development at a fairly high level.

SAE is an SOA specific framework and methodology that covers a much broader life cycle footprint in great detail.

There is no absolute conflict between TOGAF 9 and SAE. There are many opportunities to use SAE assets and work packages to guide and inform detailed SOA activity."

Everware-CBDI Delivers Major Upgrade to SOA eLearning Capability

2009-04-20T15:49:00.003+01:00

eLearning portfolio for developing SOA skills across all roles in the organization

21 eLearning modules
Fundamentals and Practitioner levels
Cost efficient SOA skills development
For business analysts, architects, designers, developers, testers, governance reviewers and project managers
Associated certification services
SCORM compliant modules for hosting in customer Learning Management System
Available now to CBDI Platinum subscribers in the Knowledgebase

Everware-CBDI announces today that it has delivered a major upgrade to its SOA eLearning portfolio. Twenty-one eLearning modules are now available providing both fundamentals and practitioner level education for a broad range of SOA skills. In addition CBDI certification services are available, which together with the eLearning modules, can address critical skills shortages in SOA efforts.
Though many IT architects are very knowledgeable about SOA there remains considerable confusion and inconsistency of practice and process. SOA skills in many organizations are often limited to the architecture role and focused primarily on technology. As a result SOA projects often spend more time and energy determining how to apply SOA concepts than delivering project results.

The new Everware-CBDI eLearning portfolio is designed to equip a broader set of roles including business analysts, architects, designers, developers, testers, governance reviewers and project managers with consistent, detailed understanding of SOA practice. The e-Learning modules allow practitioners to educate themselves when it is convenient for them in a cost efficient manner.

The CBDI Certification process enables candidates to calibrate and communicate their skill level and allows program and project management to plan more effective and consistent resource utilization. There are two certification levels – Fundamentals and Practitioner. The Fundamentals level can be achieved entirely through eLearning and self study. The Practitioner level can be achieved through various combinations of eLearning, self study, remote and face to face classes.

The eLearning and certification products are delivered either as an integral part of the CBDI Service Architecture & Engineering (SAE) Knowledgebase or as standalone executables that can be hosted by the customer. They can also be provided as SCORM-compliant modules so that they can be used within a customer’s Learning Management System. All CBDI Platinum subscribers and Knowledgebase licensees will have immediate access to all the eLearning modules.

More information:

Everware-CBDI SOA eLearning

CBDI Forum publish Rich Service Specification Template for SOA

2009-03-24T10:43:00.002Z

The Service Specification is a pivotal SOA deliverable that enables both Service Provider and Consumer to share a common view of a Service’s behavior. However, despite the importance of Service Specification there is no widely adopted standard.

CBDI Forum first developed our Service Specification Template in 2005 and it has since been used on numerous projects by our customers and ourselves.

TOGAF have recently published a Service Specification that, whilst rather more high level, bears strong resemblance to the CBDI template.

Following several requests to make our template more widely available CBDI is making the Service Specification Template freely available to the public under licence.

The template is now available for download

The Service Specification Template is designed to address the following requirements

Contract First Delivery
One of the important principles of SOA is that of contract-first delivery, and this is one of the principles that motivates and drives the need for a Service Specification.
Contract-first means the specification of the service should precede development of its implementation.

Use in Multiple Roles
The Service Consumer will of course want to know exactly what the service does. They will want to know what behaviour it offers, and what they have to do in order to use the Service.
A Service Provisioner whose role it is to locate a suitable Service will need to be able to compare the available Services against a specification to see if it meets requirements. Or if they are commissioning a new Service (or implementation of) they will need to precisely detail the required behavior of that Service.
Hence, the developer in the Service Providing organization who has to build an implementation of the Service also needs to know what requirements need to be meet.
A Service Specification is an implementation-neutral deliverable that contains all the information necessary to meet all these needs.
The Service Specification facilitates exchange of consistent and precise instructions between different participants in the Service supply chain

The Service Lifecycle
The Service Specification is expected to be developed iteratively across the service lifecycle. For example, as a;
Planned Service it may start as a high-level description of requirements
Specified Service it will provide precise detail of the required behaviour
Provisioned Service it may reflect any choices or compromises that have been made in the provisioning process, that diverge from the ideal specification
Deployed Service it will detail the actual endpoints and other aspects required to use the service at runtime

Using the Service Specification Template
CBDI provides templates for various SAE deliverables as Word documents. However, we do not really expect users to then complete and store instances of those documents using Word. Rather, they should treat them as the basis for schemas from which they could create a database to store the data in a more structured way – for example in some form of Service Catalog. Our Word templates might then be better used as a format in which to report from that database, rather than the place in which the data is itself stored.

For example, the Operation Signatures would be held as WSDL documents, and the Service Information Model might be held as a UML Class diagram in a modelling tool.

Further guidance on this is provided in two CBDI Journal Reports
Service Lifecycle Automation
Service Lifecycle Configuration Management

The template itself is only part of the story however. The delivery of a Service Specification is just one step, mid way through the CBDI-SAE SO Process.
Though the Service Specification Template is fully documented, ideally further guidance in its use is required.

CBDI have published Service Specification guidance in the CBDI Journal (gold subscribers), and further process guidance is also provided in the SAE Knowledgebase.
Practical Service Specification and Design Part 3: Specifying Services
Documenting Service Behavior
Process Logic and the Identification of Service Behavior
Produce Service Specification (SAE Process Unit) (platinum suscribers only)

We invite discussion on the Service Specification Template via CBDI LinkedIn

SOA Governance Framework

2008-06-17T11:20:00.000+01:00

At CBDI we have always advocated that governance needs to be rooted in clear policy definition and in fact devoted an article to this. Since that time we have been busy assisting organizations improve SOA governance approaches based on an underlying foundation of clear policy definition. One thing that has emerged vividly from this work is that organizations must move forward at their own pace and in a way that is realistic in terms of their current SOA adoption level.

Moreover the approach taken to SOA governance must be in tune with the overall governance requirements and political climate. We have therefore distilled these experiences into an SOA Governance Framework - embracing policy, process, infrastructure and capability maturity - that can be tailored to each organization's specific needs.

The basics of the CBDI SOA Governance Framework are now publicly available without requiring registration

An overview of the process of creating a SOA Governance Framework is presented in this slide show

| View | Upload your own

SOA Process Reports - now available without registration

2008-06-17T11:14:00.001+01:00

The following SO Process reports are now available on the CBDI Forum website without requiring registration.

A report on the SOA process by Paul Allen (Everware-CBDI) and Paul C. Brown (TIBCO) was published in the CBDI Journal for November 2007.

Architected Solution Delivery: Enhancing the Service Oriented Process (November 2007, available to all )

This is a sequel to an earlier report by Paul Allen from February 2007.

The Service Oriented Process (February 2007, available to all)