The Smile-IT Blog » Blog Archives

Tag Archives: Service Orchestration

Automation-Orchestration: Header Image SmileIT
Read more

Automation and Orchestration – a Conclusion 

This post is part of the "Automation-Orchestration" architecture series. Posts of this series together comprise a whitepaper on Automation and Orchestration for Innovative IT-aaS Architectures.

 

Automation and Orchestration are core capabilities in any IT landscape.

Traditionally, there’d be classical on-premise IT, comprised of multiple enterprise applications, (partly) based on old-style architecture patterns like file exchange, asynchronous time-boxed export/import scenarios and historic file formats.

At the same time, the era of the Cloud hype has come to an end in a way that Cloud is ubiquitous; it is as present as the Internet as such has been for years, and the descendants of Cloud – mobile, social, IoT – are forming the nexus for the new era of Digital Business.

For enterprises, this means an ever-increasing pace of innovation and a constant advance of business models and business processes. As this paper has outlined, automation and orchestration solutions form the core for IT landscapes to efficiently support businesses in their striving for constant innovation.

Let’s once again repeat the key findings of this paper:

  • Traditional “old style” integration capabilities – such as: file transfer, object orientation or audit readiness – remain key criteria even for a cloud-ready automation platform.
  • In an era where cloud has become a commodity, just like the internet as such, service centered IT landscapes demand for a maximum of scalability and adaptability as well as multi-tenancy in order to be able to create a service-oriented ecosystem for the advancement of the businesses using it.
  • Security, maximum availability, and centralized management and control are fundamental necessities for transforming an IT environment into an integrated service center supporting business expansion, transformation, and growth.
  • Service orchestration might be the ultimate goal to achieve for an IT landscape, but system orchestration is a first step towards creating an abstraction layer between basic IT systems and business-oriented IT-services.

Therefore, for IT leaders, choosing the right automation and orchestration solution to support the business efficiently might be the majorly crucial decision to either become a differentiator and true innovation leader or (just) remain the head of a solid – yet: commodity – enterprise IT.

The CIO of the future is a Chief Innovation (rather than “Information”) Officer – and Automation and Orchestration both build the core basis for innovation. What to look at in getting to the right make-or-buy decision was the main requirement for this paper.

 

Published by:
System & Service Orchestration SmileIT
Read more

System versus Service Orchestration 

This post is part of the "Automation-Orchestration" architecture series. Posts of this series together comprise a whitepaper on Automation and Orchestration for Innovative IT-aaS Architectures.

 

One of the most well-known blueprints for service orchestration is the representation as seen from the perspective of service oriented architecture (SOA).

The following figure in principle describes this viewpoint:

Service Orchestration, defined

Service Orchestration, defined

Operational components – such as “commercial off the shelf” (COTS) or custom applications, possibly with a high level of automated functionality (see previous chapters) – are orchestrated to simple application services (service components) which in turn are aggregated to atomic or composite IT services which subsequently support the execution of business processes. The latter are presented to consumers of various kind without disclosing any of the underlying services or applications directly. In well established service orchestration, functionality is often defined top-down by modelling business processes and defining its requirements first and then leveraging or composing necessary services to fulfil the process’ needs.

A different approach is derived from typical definitions in cloud frameworks; the following figure shows this approach:

System Orchestration: Context

System Orchestration: Context

Here, the emphasis lies on the automation layer building the core aggregation. The orchestration layer on top creates system and application services needed by the framework to execute its functional and operational processes.

The latter approach could be seen as a subset of the former, which will become more clear when talking about the essential differences between system and service orchestration.

Differences between system and service orchestration

System Orchestration

  • could in essence be comprised of an advanced automation engine
  • leverages atomic automation blocks
  • eases the task of automating (complex) application and service operation
  • oftenly directly supports OS scripting
  • supports application interfaces (API) through a set of plugins
  • may offer REST-based API in itself for integration and SOA

Service Orchestration

  • uses SOA patterns
  • is mostly message oriented (focuses on the exchange of messages between services)
  • supports message topics and queues
  • leverages a message broker and (enterprise) service bus
  • can leverage and provide API
  • composes low level services to higher level business process oriented services

Vendor examples of the former are vRealize Orchestrator, HP Operations Orchestration, Automic ONE Automation, BMC Atrium, System Center Orchestrator, ServiceNow (unsurprisingly some of these products have an essential say in the field of automation as well).

Service orchestration examples would be vendors or products like TIBCO, MuleSoft, WSO2, Microsoft BizTalk, OpenText Cordys or Oracle Fusion.

System orchestration key features

System orchestrators are mainly demanded to support a huge variety of underlying applications and IT services in a highly flexible and scalable way:

  • OS neutral installation (depending on specific infrastructure operations requirements)
  • Clustering or node setup possible for scalability and availability reasons
  • Ease of use; low entry threshold for orchestration/automation developers
  • Support quality; support ecosystem (community, online support access, etc.)
  • Database dependency to minimum extent; major databases to be supported equally
  • Built-in business continuity support (backup/restore without major effort)
  • Northbound integratability: REST API
  • Southbound integratability and extensibility: either built-in, by leveraging APIs or by means of a plugin ecosystem
  • Plugin SDK for vendor external plugin development support
  • Scripting possible but not necessarily needed
  • Ease of orchestrating vendor-external services (as vendor neutral as possible, depending on landscape to be orchestrated/integrated)
  • Self-orchestration possible
  • Cloud orchestration: seamless integration with major public cloud vendors

Main requirements for a service orchestrator

In contrary to the above, service orchestration solutions would focus mainly on message handling and integration, as its main purpose is to aggregate lower level application services into higher level composite services to support business process execution. Typical demands to such a product would therefore involve:

  • Support of major web service protocol standards (SOAP, REST)
  • Supports “old-style” enterprise integration technologies (RMI, CORBA, (S)FTP, EDI, …) for integration of legacy applications
  • Provides a central service registry
  • Supports resilient message handling (mediation, completion, message persistence, …)
  • Includes flexible and easy to integrate data mapping based on modelling and XSLT
  • Supports message routing and distribution through topics, queues, etc.
  • Integrated API management solution
  • Integrated business process modelling (BPM) solution
  • Integrated business application monitoring (BAM) solution
  • Extensibility through low-threshold commonly accepted software development technologies

As a rule of thumb to delineate the two effectively from each other, one can say that it is – to a certain extent – possible to create service orchestration by means of a system orchestrator but it is (mostly) impossible to do system orchestration with only a service orchestrator at hand.

For this reason, we will continue with a focus on system orchestration as a way to leverage basic IT automation for the benefit of higher level IT services, and will address vanilla architectures for typical system orchestrator deployments.

 

Published by:
The "Babel" of Orchestration SmileIT
Read more

What Is Orchestration? 

This post is part of the "Automation-Orchestration" architecture series. Posts of this series together comprise a whitepaper on Automation and Orchestration for Innovative IT-aaS Architectures.

 

Prior to diving into the architectural patterns for a robust orchestration solution for the enterprise, a few definitions need to be clarified, and as this paper talked a lot about Automation in the first place, we shall begin with outlining precisely the delineation between Automation and Orchestration – to begin with:

  • In any IT architecture framework, the automation layer creates the components necessary to provide atomic entities for service orchestration. Automation uses technologies to control, manage and run atomic tasks within machine instances, operating systems and applications on the one hand and provides automation capabilities for the larger ecosystem in order to automate processes (e.g. onboarding a new employee)
  • Orchestration – in turn – uses processes, workflows and integration to construct the representation of a service from atomic components. A service could e.g. consist of operations within various different systems such as the creation of a machine instance in a virtual infrastructure, the installation of a webservice in another instance and the alteration of a permission matrix in an IAM system. “Orchestration” would provide means to aggregate these atomic actions into a service bundle which can then be provisioned to a customer or user for consumption. The Orchestration layer makes use of single automated service components.

While the above definitions mainly address the system context in IT architectures, it is valid to say that there is another slightly different context – the service context – which demands for another definition of the term “orchestration”:

  • Service orchestration is the coordination and arrangement of multiple services exposed as a single aggregate service. It is used to automate business processes through loose coupling of different services and applications, thereby creating composite services. Service orchestration combines service interactions to create business process models consumable as services.

In order to underpin the danger of confusion when discussing orchestration, here are a few references for orchestration definitions:

  • “Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware and services.” (wikipedia: https://en.wikipedia.org/wiki/Orchestration_(computing) )
  • “Complex Behavior Interaction (Logic/Business Process Level): a complex interaction among different systems. In the context of service-oriented architecture, this is often described as choreography and orchestration” (Carnegie Mellon University Research Showcase 12-2013: “Understanding Patterns for System-of- Systems Integration”, Rick Kazman, Klaus Nielsen, Klaus Schmid)
  • “Service orchestration in an ESB allows service requesters to call service providers without the need to know where the service provider is or even the data scheme required in the service” (InfoTech Research Group Inc. “Select and Implement an ESB Solution”, August 2015)
  • “Orchestration automates simple or complex multi-system tasks on remote servers that are normally done manually” (ServiceNow Product Documentation: http://wiki.servicenow.com/index.php?title=Orchestration#gsc.tab=0 )
  • “The main difference, then, between a workflow “automation” and an “orchestration” is that workflows are processed and completed as processes within a single domain for automation purposes, whereas orchestration includes a workflow and provides a directed action towards larger goals and objectives” (“Cloud Computing: Concepts, Technology & Architecture”, Thomas Erl, Prentice Hall, October 2014)
  • “Orchestration is the automated coordination and management of computer resources and services. Orchestration provides for deployment and execution of interdependent workflows completely on external resources” (ORG: http://cloudpatterns.org/mechanisms/orchestration_engine )

Even though these definitions seemingly leave a lot of room for interpretation of what system and service orchestration really covers, clarity can be gained by looking at a few architectural principles as well as requirements for different orchestration goals, which the last few chapters of this paper will be focusing at.

Published by:
Automic SmileIT
Read more

The “Next Big Thing” series wrap-up: How to rule them all?

What is it that remains for the 8th and last issue of the “Next Big Thing” blog post series: To “rule them all” (all the forces, disruptive challenges and game changing innovations) and keep services connected, operating, integrated, … to deliver value to the business.

A bit ago, I came upon Jonathan Murray’s concept of the Composable Enterprise – a paradigm which essentially preaches fully decoupled infrastructure and application as services for company IT. Whether the Composable Enterprise is an entire new approach or just a pin-pointed translation of what is essential to businesses mastering digital transformation challenges is all the same.

The importance lies with the core concepts of what Jonathan’s paradigm preaches. These are to

  • decouple the infrastructure
  • make data a service
  • decompose applications
  • and automate everything

Decouple the Infrastructure.

Rewind into my own application development and delivery times during the 1990ies and the 00-years: When we were ready to launch a new business application we would – as part of the rollout process – inform IT of resources (servers, databases, connections, interface configurations) needed to run the thing. Today, large IT ecosystems sometimes still function that way, making them a slow and heavy-weight inhibitor of business agility. The change to incorporate here is two-folded: On the one hand infra responsibles must understand that they need to deliver on scale, time, demand, … of their business customers (which includes more uniform, more agile and more flexible – in terms of sourcing – delivery mechanisms). And on the other hand, application architects need to understand that it is not anymore their architecture that defines IT needs but in turn their architecture needs to adapt to and adopt agile IT infrastructure resources from wherever they may be sourced. By following that pattern, CIOs will enable their IT landscapes to leverage not only more cloud-like infrastructure sourcing on-premise (thereby enabling private clouds) but also will they become capable of ubiquitously using ubiquitous resources following hybrid sourcing models.

Make Data a Service.

This isn’t about BigData-like services, really. It might be (in the long run). But this is essentially about where the properties and information of IT – of applications and services – really is located. Rewind again. This time only for like 1 or 2 years. The second last delivery framework, that me and my team of gorgeous cloud aficionados created, was still built around a central source of information – essentially a master data database. This simply was the logical framework architecture approach back then. Even only a few months – when admittedly me and my then team (another awesome one) already knew that information needs to lie within the service – it was still less complex (hence: quicker) to construct our framework around such a central source of (service) wisdom. What the Composable Enterprise, though, rightly preaches is a complete shift of where information resides. Every single service, which offers its capabilities to the IT world around it, needs to provide a well-defined, easy to consume, transparently reachable interface to query and store any information relevant to the consumption of the service. Applications or other services using that service simply engage via that interface – not only to leverage the service’s capabilities but even more to store and retrieve data and information relevant to the service and the interaction with it. And there is no central database. In essence there is no database at all. There is no need for any. When services inherently know what they manage, need and provide, all db-centric architecture for the sole benefit of the db as such becomes void.

Decompose Applications.

The aforementioned leads one way into the decomposition pattern. More important, however, is to spend more thorough thinking about what a single business related activity – a business process – really needs in terms of application support. And in turn, what the applications providing this support to the business precisely need to be capable of. Decomposing Applications means to identify useful service entities which follow the above patterns, offer certain functionality in an atom kind-of way via well-defined interfaces (APIs) to the outside world and thereby create an application landscape which delivers on scale, time, demand, … just by being composed through service orchestration in the right – the needed – way. This is the end of huge monolithic ERP systems, which claim to offer all that a business needs (you just needed to customize them rightly). This is the commencing of light-weight services which rapidly adopt to changing underlying infrastructures and can be consumed not only for the benefit of the business owning them but – through orchestration –form whole new business process support systems for cross-company integration along new digitalized business models.

Automate Everything.

So, eventually we’ve ended at the heart of how to breath life into an IT which supports businesses in their digital transformation challenge.

Let me talk you into one final example emphasizing the importance of facing all these disruptive challenges openly: An Austrian bank of high reputation (and respectful success in the market) gave a talk at the Pioneers about how they discovered that they are actually not a good bank anymore, how they discovered that – in some years’ time – they’d not be able to live up to the market challenges and customers’ demands anymore. What they discovered was simply, that within some years they would lose customers just because of their inability to offer a user experience integrated with the mobile and social demands of today’s generations. What they did in turn was to found a development hub within their IT unit, solely focussing on creating a new app-based ecosystem around their offerings in order to deliver an innovative, modern, digital experience to their bank account holders.

Some time prior to the Pioneers, I had received a text that “my” bank (yes, I am one of their customers) now offers a currency exchange app through which I can simply order the amount of currency needed and would receive a confirmation once it’s ready to be handed to me in the nearest branch office. And some days past the Pioneers I received an eMail that a new “virtual bank servant” would be ready as an app in the net to serve all my account-related needs. Needless to say that a few moments later I was in and that the experience was just perfect even though they follow an “early validation” policy with their new developments, accepting possible errors and flaws for the benefit of reduced time to market and more accurate customer feedback.

Now, for a moment imagine just a few of the important patterns behind this approach:

  • System maintenance and keeping-the-lights-on IT management
  • Flexible scaling of infrastructures
  • Core banking applications and services delivering the relevant information to the customer facing apps
  • App deployment on a regular – maybe a daily – basis
  • Integration of third-party service information
  • Data and information collection and aggregation for the benefit of enhanced customer behaviour insight
  • Provision of information to social platforms (to influence customer decisions)
  • Monitoring and dashboards (customer-facing as well as internally to business and IT leaders)
  • Risk mitigation
  • … (I could probably go on for hours)

All of the above capabilities can – and shall – be automated to a certain, a great extent. And this is precisely what the “automate everything” pattern is about.

Conclusion

There is a huge business shift going on. Software, back in the 80ies and 90ies was a driver for growth, had its downturn in and post the .com age and now enters an era of being ubiquitously demanded.

Through the innovative possibilities by combining existing mobile, social and data technologies, through the merge of physical and digital worlds and through the tremendously rapid invention of new thing-based daily-life support, businesses of all kind will face the need for software – even if they had not felt that need so far.

The Composable Enterprise – or whatever one wants to call a paradigm of loosely coupled services being orchestrated through well-defined transparently consumable interfaces – is a way for businesses to accommodate this challenge more rapidly. Automating daily routine – like e.g. the aforementioned tasks – will be key to enterprises which want to stay on the edge of innovation within these fast changing times.

Most importantly, though, is to stay focussed within the blurring worlds of things, humans and businesses. To keep the focus on innovation not for the benefit of innovation as such but for the benefit of growing the business behind.

Innovation Architects will be the business angels of tomorrow – navigating their stakeholders through an ongoing revolution and supporting or driving the right decisions for implementing and orchestrating services in a business-focussed way.

 

{the feature image of this last “The Next Big Thing” series post shows a design by New Jersey and New York-based architects and designers Patricia Sabater, Christopher Booth and Aditya Chauan: The Sky Cloud Skyscraper – found on evolo.us/architecture}

Published by:
Automic SmileIT
Read more

(It’s) The End of the Cloud (as we know it)

“It’s the end of the world as we know it”, they say, “and I feel fine”. I do indeed. Things change. And change is always moving us and all around us. And in the best case it’s moving us forward.

So, let me start with a very moving (and maybe touching) statement:

 

“The Cloud has ended!
You can now stop talking about it.”

After about 5 years (if you think of the Middle-of-Europe geography – maybe some more in the US), the Cloud has finally come to its end. That’s great! Because eventually we can luckily move on to the really important things in our day-2-day IT life!

So – why is that? Why do I have the strong believe, that this hype is now finally over?

Some 5 years ago I participated in a panel discussion. And the main area it circulated on was whether Cloud can ever be secure or not – multi tenant or not – reliable or not – compliant or not. The panel discussion as such went quite well and eventually attendees of some real industry giants, like e.g. Siemens, for the first time felt assured that this was

 

The next Big Thing to Come …

And it was! Indeed! It remained to be perceived as the next big thing for the next – well – let’s say: 2 to 3 years.

That occasion back then was only one example – however – of how conversations went going forward. People lost themselves in arguments of security, compliance, controlability and segregatability of a cloudy, foggy and fuzzy monster which none could really grasp for a long time. Discussing these was just more easy than to understand the big advantages to come …

Still, it remained not only a hype but took its turn into our everyday life! Even more than that – Cloud Computing became the basis for what we today call the third IT revolution. And many talks today aim at spanning everybody’s perception regarding this revolution – one within our everyday lifes with IT being planted into any kind of small or big device – one within our industry lifes with production processes being shifted by 90° to create a complete new way of delivering applications – and eventually also a revolution for the big players as many of them might lose traction when remaining one foot on the platform, the other on the train for too long …

Soon after that panel discussion back in 2009, me and my then team entered the Windows Azure “Technology Adoption Program”. We created a first Cloud-based software distribution platform; you could compare it to those Systems Management architectures which have been very common for long in enterprise IT. By using Cloud patterns, we were able to address a broader reach and distribute more scalable and more flexible. Azure – back then – had its really major leaks and intensively improved down the path – together with fellow teams like ours who recommended changes and enhancements to the platform without end. Amazon – by that time – had already its place in the market. But what were they doing? Nothing more and nothing less than providing servers – more or less. Infrastructure-as-a-Service, it was called (question is: would you claim this to be disruptive today?).

Everybody talked about

 

The Big 3

Amazon, Microsoft and of course — ? — Google. Question: What’s one common thing of all 3 of them??

Exactly: All are US-based, spreading their capabilities over the world pretty quickly – backed by major investments – but still US-based. And every enterprise also back then knew: This was not going to work in compliancy to their respective country’s law or their anti trust regulatory.

So what remained the major discussion over years? Still? Security. Compliance. Controlability. Segregatability – still the same. Resentiments were omnipresent and the Big 3 kept having a hard time generating adoption of their brilliant new technologies (whereas admittedly Microsoft had the hardest time, because they leaked a $-rollin ecosystem such as brilliant search or brilliant book-sells).

Still, that Cloud-hype was not killable. Despite all concerns of private and public endavours, of small businesses and large enterprises, Cloud Computing remained and even strengthened its position as the basis for more to come: Technology ecosystems that evolved solely and purely because cloud took its turn into our everyday life and everyday enterprise IT.

And how couldn’t it?

How many of us are using eMail, some kind of file sharing facility, a social network of any kind? Anyone not using any remotely hosted collaboration product, authoring tools, design helpers – like the Adobe family – etc. etc. …

Who’s been using that within one’s work environment daily? For how long? 2 years, 3 years, …

We have all long ago started to enhance our day-2-day work experience by adding usefull little helpers into our IT-wise behaviour – sometimes without even noticing, into how far away premises we’re providing our data to. Haven’t we?

And many times we’ve all done that long before our enterprise IT provider offered us the same convenience, the same workforce enhancements from within our company’s premises. Either privately or even on our company PCs. Enterprise IT departments have been facing the worst time of their existence with sleepless nights for CIOs asking themselves the very same question ever and ever again: How the hell can I stop that Cloud thingy to happen to my enetrprise IT when it is so unsecure and uncompliant and at the same time, all my CxO’s are using it on their iPads? How can I adopt it without losing control completely.

Well – Ladies and Gentlemen – let me assure you: The nightmare has an end as it’s

 

The End of the Cloud as we know it!

(BTW: Ever had a look to a recent Gartner hypecycle and its positioning of “Cloud Computing”: They kept having it on the declinign edge for the last 2 years or so …) So I think we can rest assured: Our struggle is over.

But why? Why can a hype that big, that it managed to become the basis for all new and disruptive power in IT – social, mobile apps, data management and analytics – finally end?

Because it finally became utterly boring! Because it simply isn’t Cloud anymore, we’re talking about! Stop talking Cloud. Stop talking SaaS. It’s not retaining the importance it used to have. It has ceased to exist as a self-contained technology arguing its relevance by itself. It has ceased to exist on its own.

Hence – and here comes another good news – we’re now finally able to adopt this great new technology precisely for what we really neeed it: To accelerate and grow the businesses, which we – the IT guys amongst us – are bound to support. To integrate it into the devices we’re creating and stay as connected as needed. Or to accelerate our delivery into continuity. Or to simply consume the Services, we need to consume. Right now. Right as much and rapid as we need’em. And ultimately: To bridge the great IT systems we’ve all been managing for decades with the elasticity and scalability and accuracy of what was formerly called Cloud.

Let me talk you through a few examples:

  • Salesforce started in 2001. Claiming the decline of Software. They were facing the same concerns, the same weak adoption (at least in the beginning), the same hurdles for their entirely subscription based model. Today, Salesforce offers an ecosystem of products – even a platform to create your own: force.com – which are virtually friction-freely integrateable with your on-prem IT; mostly by just a few configurational mouseclicks. And a huge many enterprises trust Salesforce just enough to provide them one of their most critical business assets: their customer and opportunity data.
  • Or take user management: Many of today’s enterprises still struggle with providing their businesses a seamless login and authorization experience. At the same time, many roll out an identity provider which bridges on-prem IT systems with SaaS-provided systems and thereby offer a totally new single-sign-on experience not only within their on-prem IT. It’s a way of shifting – or even vanishing – the borders.
  • Or think of the most obvious of all examples: Infrastructure and Application Provisioning. Who has not yet introduced Virtualization in its IT? Every larger enterprise has, and the smaller ones leverage it from some out-of-prem providers. Finally, the technology that was formerly called “Cloud” has evolved into a level of maturity that it lives up to the promises of some earlier days: There’s means for you to control and manage your on-premise IT infrastructure seamlessly in a joint way with some Infrastructure provided – well – somewhere (I’m not stressing the word again).

However,

 

One Piece is Missing

One extensively important technology item that has the ability and purpose of glueing it all together: Automation. Or to be precise: Automation and Orchestration.

We are entering the era of Orchestration. We’re leaving the self-fullfilling technology-focussed island of pure “Cloud” behind us and can finally commence to create the really interesting stuff: By bridging our solidly designed and implemented IT systems and architectures with some even more solid, secure and reliable, scalable, elastic architectures, we are accelerating the business value of technology. We start to shift our thinking towards “Orchestration” and “Service”:

So, trust me: It’s the End of the Cloud as we know it! And we indeed can feel entirely fine about it. Security concerns have been addressed all the past years. Compliancy has been factored into vendor’s platforms – as has multi tenancy. And we are able to control where our data resides. Hence, the change can finally move on. We can stop talking “Cloud” as it became so boring as a standalone technology.

 

The Hype is Over

We’re entering the era of Orchestration – of Service Orchestration and entirely Service-based delivery. And Automation is the glue to make it happen. Automation is the glue between those valuable, well-defined, secure IT architectures and whole-new ecosystems of platforms – for the benefit of Service Orchestration.

So: Let us stop the “Cloud” talk. And start to

Automate to Orchestrate

 

 

Published by:
%d bloggers like this: