The Smile-IT Blog » Blog Archives

Tag Archives: process management

System versus Service Orchestration

This post is part of the "Automation-Orchestration" architecture series. Posts of this series together comprise a whitepaper on Automation and Orchestration for Innovative IT-aaS Architectures.

 

One of the most well-known blueprints for service orchestration is the representation as seen from the perspective of service oriented architecture (SOA).

The following figure in principle describes this viewpoint:

Service Orchestration, defined

Service Orchestration, defined

Operational components – such as “commercial off the shelf” (COTS) or custom applications, possibly with a high level of automated functionality (see previous chapters) – are orchestrated to simple application services (service components) which in turn are aggregated to atomic or composite IT services which subsequently support the execution of business processes. The latter are presented to consumers of various kind without disclosing any of the underlying services or applications directly. In well established service orchestration, functionality is often defined top-down by modelling business processes and defining its requirements first and then leveraging or composing necessary services to fulfil the process’ needs.

A different approach is derived from typical definitions in cloud frameworks; the following figure shows this approach:

System Orchestration: Context

System Orchestration: Context

Here, the emphasis lies on the automation layer building the core aggregation. The orchestration layer on top creates system and application services needed by the framework to execute its functional and operational processes.

The latter approach could be seen as a subset of the former, which will become more clear when talking about the essential differences between system and service orchestration.

Differences between system and service orchestration

System Orchestration

  • could in essence be comprised of an advanced automation engine
  • leverages atomic automation blocks
  • eases the task of automating (complex) application and service operation
  • oftenly directly supports OS scripting
  • supports application interfaces (API) through a set of plugins
  • may offer REST-based API in itself for integration and SOA

Service Orchestration

  • uses SOA patterns
  • is mostly message oriented (focuses on the exchange of messages between services)
  • supports message topics and queues
  • leverages a message broker and (enterprise) service bus
  • can leverage and provide API
  • composes low level services to higher level business process oriented services

Vendor examples of the former are vRealize Orchestrator, HP Operations Orchestration, Automic ONE Automation, BMC Atrium, System Center Orchestrator, ServiceNow (unsurprisingly some of these products have an essential say in the field of automation as well).

Service orchestration examples would be vendors or products like TIBCO, MuleSoft, WSO2, Microsoft BizTalk, OpenText Cordys or Oracle Fusion.

System orchestration key features

System orchestrators are mainly demanded to support a huge variety of underlying applications and IT services in a highly flexible and scalable way:

  • OS neutral installation (depending on specific infrastructure operations requirements)
  • Clustering or node setup possible for scalability and availability reasons
  • Ease of use; low entry threshold for orchestration/automation developers
  • Support quality; support ecosystem (community, online support access, etc.)
  • Database dependency to minimum extent; major databases to be supported equally
  • Built-in business continuity support (backup/restore without major effort)
  • Northbound integratability: REST API
  • Southbound integratability and extensibility: either built-in, by leveraging APIs or by means of a plugin ecosystem
  • Plugin SDK for vendor external plugin development support
  • Scripting possible but not necessarily needed
  • Ease of orchestrating vendor-external services (as vendor neutral as possible, depending on landscape to be orchestrated/integrated)
  • Self-orchestration possible
  • Cloud orchestration: seamless integration with major public cloud vendors

Main requirements for a service orchestrator

In contrary to the above, service orchestration solutions would focus mainly on message handling and integration, as its main purpose is to aggregate lower level application services into higher level composite services to support business process execution. Typical demands to such a product would therefore involve:

  • Support of major web service protocol standards (SOAP, REST)
  • Supports “old-style” enterprise integration technologies (RMI, CORBA, (S)FTP, EDI, …) for integration of legacy applications
  • Provides a central service registry
  • Supports resilient message handling (mediation, completion, message persistence, …)
  • Includes flexible and easy to integrate data mapping based on modelling and XSLT
  • Supports message routing and distribution through topics, queues, etc.
  • Integrated API management solution
  • Integrated business process modelling (BPM) solution
  • Integrated business application monitoring (BAM) solution
  • Extensibility through low-threshold commonly accepted software development technologies

As a rule of thumb to delineate the two effectively from each other, one can say that it is – to a certain extent – possible to create service orchestration by means of a system orchestrator but it is (mostly) impossible to do system orchestration with only a service orchestrator at hand.

For this reason, we will continue with a focus on system orchestration as a way to leverage basic IT automation for the benefit of higher level IT services, and will address vanilla architectures for typical system orchestrator deployments.

 

Published by:

Mind: DevOps isn’t a role model!

This blog post (“How ‘DevOps’ is Killing the Developer”) airwaves a bit at the moment. It reached me right at some really cool, breakthrough conversations on how DevOps will lead change of culture and role perception … and it fully truly nailed the opposite of those highly positive talks. To say it in the words of one of the commenters: “I couldn’t disagree more!” I even would go as far as to consider it dangerous!

Why?

Because the post reflects a totally wrong perception of DevOps! The article claims that DevOps would transform the role and responsibility of a particular person – a developer in this case. I would be surprised if literature really postulates this – the change of a role. DevOps is the transformation of HOW things are done, not WHO does it. Firstly, you have to lay the basis for a DevOps company transformation. Do developers change their expertise by that? No. Do OPS guys do? Neither. BUT: They do get closer together, get better understanding for each others challenges.

Secondly: The post misses another highly important – maybe the most important – investment along with DevOps introduction: Automation! Along with the cultural change, you’ll have to invest in automation of processes for artefacts which would formerly have taken you days and weeks to create/setup/deploy/run.

90°

So – let’s be clear here: DevOps isn’t the change of a role! DevOps is a 90° turn of a modus operandi. The whole movement derives from manufacturing where the importance lies in getting rid of any blocker in a production pipeline. Neither would a continous production change the role of the screwmaster (to name just anything) nor would DevOps change that of a QA expert or buildmaster … or – well: developer (as exemplarly taken here)!

The article is dangerous in another aspect: It claims developers to concentrate on development and nothing else. It is – but – another important aspect of DevOps as a cultural tranformation: To bring understanding for everybody else’s responsibility in the process to everybody. And thereby encourage Automation even more to take its place in it all. This importance is totally missed out in the post!

Bottom line

Let’s be crystal clear on a few things with DevOps:

  • It’s a cultural and organizational change; not a role and responsibility change for single individuals
  • It is a 90° turn of a modus operandi. It turns vertical silos of responsibility and action into horizontal pipelines/chains of continous work-flow
  • It’s a way to create role and responsibility awareness throughout the whole chain of collaborating individuals
  • And it surfaces the need of Automation to support cultural and process transformation, stability, security, repeatability, speed, continuity, …

There’s – however – a really positive DevOps-supporting aspect in that post: It does indeed drive discussion into a good direction … just browse through the comments there … 😉

 

( This post was also published in the official Automic company blog: http://blog.automic.com/devops-not-a-role-model )

 

Published by:
SmileIT

Change “the Change”?

I’ve made up a little story.

The story is fictious. Characters in the story are fictious. Any resemblance to any existing persons are purely in the reader’s mind. Any resemblance with your company’s processes are mere coincidence.

Change?

Is it the process or is it communication?

Characters:

  • George: a very helpful infrastructure project manager located somewhere in Europe
  • Francis: an application manager located somewhere else in Europe
  • Olaf: an even more helpful datacenter operator geographically close to George
  • Dakshi: a very talented cloud operations engineer in/from India
  • Hans: a knowledgable senior cloud technician, European
  • Bob: the senior architect for infrastructure engineering (US)
  • Eveyln: the infrastructure engineering project manager (US, close to Bob)
  • cloud-team.operations@company.com: a support mailbox

On Feb, 4th Francis writes:

Hi Hans, Dakshi,

I currently have an issue you’ll probably able to solve easily.

Our company has signed a deal with a European government’s ministry to host an instance of our social enterprise application. For obvious reasons, that instance needs to be hosted in the customer’s country (not in our cloud)

As this project has a very tight schedule, and to be sure we can deliver in time, I need a dump of a few virtual machines from our existing V2 implementation so we can import them on our local VMWare infrastructure.

Machines needed are:

  • webserver01
  • dbserver01
  • accesssrv01
  • worker01
  • viewer01

I’m not sure what would be the best and fastest way to transfer them. A network transfer may be too long so maybe we can use a hard drive sent to us by UPS as we did for earlier transfers across the ocean.

I hope we can do this very quickly as the schedule is very tight, and I’m a bit late due to the time spent trying to fix the current V1 issues.

Best regards,

Francis


Let us briefly elaborate on the geographical circumstances of this request:

  • The datacenter running the V1 and V2 versions of the application under question is located in Europe
  • The customer is located in a different European country from the datacenter
  • The DC operations team is located in India

Dakshi is a very talented and quick operations guy, who picks up the request just 5 minutes later and responds willingly to export the requested VMs but being in need of a contact geographically close to the cloud DC to x-plug (and deliver) a USB harddisk . For this he redirects to Hans. Hans is known to know virtually everything which in certain organizations sometime means that Hans would be the one also doing everything – not so here: Hans redirects to George. George is a project manager located close to and responsible for activities in the DC in question, hence supposed to be best choice to coordinate the HDD x-plug and delivery process.

* FULLSTOP *


On Feb, 13th Francis writes:

Hi George,

I’m Francis, in charge of our social enterprise application. We’ve been working with our cloud team on the company’s “DriveSocial” project (V1 and V2). I’m writing to you on behalf of Hans. Our company has signed a deal with a local government’s ministry to host an instance of our social enterprise application. For obvious reasons, that instance needs to be hosted in the customer’s country.

As this project has a very tight schedule, and to be sure we can deliver in time, I need a dump of a few virtual machines from our application’s V2 implementation so we can import them on our local VMWare infrastructure.

Machines are:

  • webserver01
  • dbserver01
  • accesssrv01
  • worker01
  • viewer01

Can you please have them dumped and sent on a hard drive to:

DC1GER – Markus Verdinger, Sackstrasse 240, 99999 Praiss, Germany

The latest snapshot of the virtual machines, even though from last week would be perfect.

Thank you very much in advance,

Francis


A few minutes later the same day, Hans (in a supportive manner) shares a diagram of the V2 implementation with everybody; included is a detailed directive how to discover the right VMs within the “complicated jungle” of virtual organizations and virtual appliances [2(!) ORGs; 5(!) vApps].

George, the project manager and always eager to exactly specify the right activities, now kicks off a conversation about which server to plug the harddisk into – which ultimately involves Bob, the infrastructure architect.

On Feb, 13th Bob writes clarifyingly:

George,

They are looking for physical jump server in the EU1DC rack 0815.  On the rack elevation diagram, you are looking for CWRWTs001 in RU99.

Looks like these UCSs have the old US names on them.  We need to review the names for the UCSs in Rack 0815 and make the corrections.

@Evelyn, please set up meeting to talk about and correct this issue.  We should double check EU2DC too.

Thanks. Bob


Evelyn confirms instantly. Case closed for today.

  • Total #mail: 11
  • Total #mail today: 8

The next day passes by with Georges proactive attempts to get the right HDD into the right server. He’s supported by Olaf who gives regular live reports from the actual situation in the DC (i.e. precisely explaning which HDD is plugged into which server and asking for admittance to x-plug).

On Feb, 15th Geroge writes:

Hans,

Can you let us know who will be baring the costs for sending this HDD over to Praiss, Germany?

cheers G


Hans in all honesty responds not to know this and directs back to Francis “coram publico” assuming, that the cost question will for sure be no issue here as it all is about a very important customer and a very urgent request.

For the first time in our story things become “a bit complicated” here, as George has to ask admittance to book costs for x-plugging a HDD, putting it into an envelope and filing it with UPS. The representative of the respective delivery organization for this customer kicks in and asks whether sales has a budget (and PO) for this effort. George suggests to use a cloud development PO for the sake of simplicity, Hans again suggests – for the sake of even more simplicity – to invoice the department responsible for the social enterprise application directly (as this will be the benefitting party in this whole story anyway).

On Feb 15th, late in the evening (after having patiently watched the emails on the matter so far), Francis writes:

Jesus Christ…

Please invoice our department

In any case this is still our comapny’s money – so what!

Best regards

Francis

… and George kicks off the task of x-plugging and exporting with Dakshi by — — — asking Hans to open a Change Ticket in the Change Management system for this activity!

… which now – for the first time – leaves Hans standing in complete awe and totally leaking to understand his involveness in the case (especially as Hans is a future oriented agile minded technician who disbelieves in the flexibility of traditional change processes based on ITIL; ITIL was great some 10 years ago – Hans believes, that this is the cloud era which asks for more rapid process definitions and especially executions – but that’s a different story …)


This is the last we hear from our story’s heros before the weekend begins …

  • Time elapsed: 11 days
  • Total #mail: 34
  • Total #mail today: 17

On the following Monday, Hans and George spend some time (writtenly and verbously) to clarify how to rightly kick off such an activity and George (who asked for a change ticket just a few days ago) suggests, that the right way would be to engage with the cloud operations team in India directly. … Wait. … Directly with India. … I need to scroll up to the beginning here … Wasn’t that what Francis … did …

On Feb, 18th late in the evening, George writes:

Hi Dakshi,

could you please give me a feedback of the status regarding the transfer of the desired files?

Thanks and Best Regards

George

Dakshi now asks his colleague Abhu to act on the request (giving information on which VMs needed), Abhu asks Prahti to start the export. And Prahti re-queries the right ORG and vApps. Wait. … I need to scroll up again … Didn’t Hans … provide this very same information … directly to … Dakshi; well – it’s only copy-paste anyway and Hans is known to have an everlasting rapidly-searchable email-archive. Info delivered. Now to Prahti.

It’s Monday, Feb 18th.

  • Time elapsed: 14 days
  • Total #mail: 41
  • Total #mail today: 7

* FULLSTOP *


On Feb, 25th, Francis writes:

Hi Hans,

Sorry to disturb you but do you know if the virtual machines hard drive has been sent to our DC? I just had a call with them and apparently they had nothing delivered yet.

Best regards,

Francis

Hans in all honesty responds not to have any new status and Francis redirects his question to George. George in turn asks Dakshi. Dakshi confirms to have started the export but reports, that he had issues with some of the VMs (their export causing high load on the servers, which is why he did (does) not want to continue in order not to disturb productive environments). This leads Francis to ask the blunt question why it wouldn’t be possible to just use latest backups of the very same VMs.

cloud-team.operations@company.com thereafter confirms that backups can be restored to a seperate location and the export can then be started. Wait. … ? … Let’s contemplate briefly on why a restore … … …

On Feb, 26th, Prahti writes:

Hi Francis,

We have exported following machines to the attached hard drive:

  • webserver01
  • dbserver01
  • accesssrv01
  • worker01
  • viewer01
  • loadbalancer01

Prahti


  • Time elapsed: 22 days
  • Total #mail: 49
  • Total #mail today: 2

* FULLSTOP *


On Mar, 15th, Francis writes:

Hello Geroge and Hans,

I just had our DC people on the phone, and they’re still waiting for the VMs. It’s been one month now. Do you have a status on this please?

Best regards,

Francis

George, some hours later, replies:

Hi Hans, Hi Francis,

sorry for the delay, but there was no change ticket in place.

But in this case the colleagues will do an unconventional approach. However, the arrival of the disk at the agreed shipping address is expected during next week only.

Best Regards

Geroge

* FULLSTOP *

On Mar, 27th, Francis writes:

Hi George,

I’ve been contacted by the DC, and they’re still waiting for your shipment to be delivered. Do you have any update on this please?

Best regards,


After this we lose track and the emails trickle away.

  • Time elapsed: 51 days
  • Total #mail: 53
  • Total #mail today: 1

* FULLSTOP *



On June, 6th, Francis writes:

Good afternoon,

Our DC has mounted the machines and it appears you did not provided the right machines. Actually, you sent us virtual machines that belongs to another client and that are not even running WindowsServer.

Beside the fact that it is unacceptable to get the data from another client, the delay introduced by this makes our company at risk with that very important client (government ministry for employment).

For the record, virtual machines needed to be cloned, copied onto a USB drive and sent to our DC are:

  • webserver01
  • dbserver01
  • accesssrv01
  • worker01
  • viewer01
  • loadbalancer01

The client’s DC address is: DC1GER – Markus Verdinger, Sackstrasse 240, 99999 Praiss, Germany

Please, fix ASAP.

Regards,

Rewind to start …


* FULLSTOP *

Questions:

  • Where is the leak of communication?
  • Where is the process leaking clarity?
  • What could have been done by whom to improve the result of this operation/request?
  • Who should consider their job attitude?
  • How would ITIL support such a case?
  • Would this be possible to happen in your company? Why not?
Published by:
%d bloggers like this: