Dont let the Enterprise Service Bus lead to Context Bleeding

In this article I would like to discuss bad usage of integration patterns and SOA tools, and why I often favor the Anti-Corruption Layer pattern of Domain Driven Design. I observe that many SOA projects does not end up with something more service oriented, but actually an event bigger ball of mud. Suddenly many more systems must be live, projects slow down and things get more complicated. The ESB becomes One Ring to Rule them All, which is not a good thing.
Why is this?


Good Service Orientation should bring clear separation of concern, easier maintenance, less code, independent deploy and release cycles, more frequent releases, easier sourcing, and a higher degree of flexibility (among others). The idea of a bus (ESB) is good enough, but it does not relieve you from the real challenge; complexity and functional dependency. Where you previously had FTP files separating silos, they now must both be up and running. When things break you have 100.000 broken messages on the bus to clean up. These messages are a long way from home; they break out of context. Probably they are better understood within their domain.

The challenge is to find a design and migration strategy with lower maintenance cost in the long run. You should make things simpler and testable, by using DDD on your system portfolio.

The intention with these integration patterns are good; the Aggregate and Canonical patterns promise encapsulation but often end up with handling complexity outside of its context. That leads to a tough maintenance situation.


Scenario

The initial stage where silos send and depend on information directly from each other:

Silos supporting and depending on each other

An ESB tries to make things easier
, but the dependency is still there

Secondly the ESB comes to the "rescue". We just put a product inbetween and pretend that we now have loose coupling. We may get technical looser coupling and reuse of services/formats, but the functional dependency is still there. Most probably this is not a situation with less maintenance, you have just introduced more architecture. In most integration scenarios people with deep knowledge of their silo talk to each other, directly. The canonical format supported by an integration team is just a man in the middle, they will strive to understand the complexity behind services and messages. Then your total maintenance organization does not scale, the functional throughput of projects slow down, because the integration team need to know "everything".

Black Octopus tentacles

There is also another problem with this approach. Most tools (and actually their prescribed usage as taught in class) let the ESB product make adapters that go into the silo. Many silos have boundaries, like CICS, but others offer database connections so that adapters for the ESB actually glue into the implementation (of the silo). Now we are getting into en even more serious maintenance hell. Each silo has a maintenance cycle and organization supporting the complex systems it is. By not involving this organization and not letting this organization support the services they actually offer to the environment, you will have trouble. The organization must know what they silo is being used for. How else are they going to support SLA, or make sure that only consistent data leave the silo? This is illustrated as a black Octopus with tentacles into the systems having different parts of the organization tied up even closer.

Context Bleeding

This very fast leads to context bleeding. Every EBS vendor has tools and repositories for maintaining the Canonical format. The problem with this is that it is maintained outside its domain, and the organization supporting it. Now many Entities and Aggregates are outside their Bounded Context... Or even worse they are replicated outside, endorsed in a more generic representation, where the integration team has put an extra layer of complexity on top of it. This generic representation also hides the ubiquitous language, making communication between organizations even harder. And just to add to this; how testable is it? This is your perfect "big ball of mud". You do not want to handle complexity outside its domain.
It is a much better situation for those with the deep knowhow of the silo to construct and support the services and canonical messages they offer. The integration team should mostly be concerned with structure and not content.

The same can be said about business processes orchestrated outside of their domain; it may get into a "make minor" approach that does not enhance ease of maintenance. Too often there is high coupling between process state and domain state. (see Enterprise Wide Business Process Handling)

Better approach

Dark green ACL maintained by each silo

So I think a better approach is to use Domain Driven Design and the Anti Corruption Layer. This pattern better describe the ownership and purpose of integration with another domain, while keeping clear boundaries. Maintenance and release cycle is now aligned with the silos maintenance cycles. I also think there is a better chance for higher level services where systems cooperate. This leads to simpler integration scenarios, illustrated by a slimmer ESB.

This is not complete without emphasizing the importance of functional decomposition between the silos, so that they have a clearer objective in the system portfolio. But this takes time, and often you need an in-between solution. ESB-tools are nice for such ad-hoc, but don't let it be your new legacy. Strive for granular business level services, so that you limit the "chatting" between systems and make usage more understandable (but this standardization is more a business challenge, than an IT challenge). Too many ESB's end up like CRUD repositories; illustrating only an open bleeding wound of the silo.

The objective is: Low Coupling - High Cohesion. Software design big or small - the same rules apply.

Dont let the Enterprise Service Bus lead to Context Bleeding by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Tax Norways Proof of Concept

Why a Proof of Concept?

Continual Aggregate Hub and Processing
In our Target Architecture there is one central part (we will test out concepts from Continual Aggregate Hub, "Restaurant" and Aggregate Storage) and it has to do with tax and fee calculation. It does collection, assessment and calculation in one scalable architecture, and this has to be flexible as to collecting new types of information and putting new calculations on them. We believe it is unlikely that we find some COTS in this area, as this data and its rules are highly domestic and decided by our fellow politicians.

Our priorities:

1. Maintenance - Modular, clear and clean functional code
2. Testable - Full test coverage
3. Speed - Liner scalable. Cost of HW determine speed, not development time.

Yes we can!
We have seen that there are so many properties in the large scale financial architectures (as the ones John Davies has talked about in several talks over the years) that are similar to ours, that we have to find out if our domain and challenges can be solved with this.

This is really the core of the PoC; we have to play, test and learn what this architecture is like for our domain and how it affects our ability to change. We need to show this to our stakeholders; business, architects, operations, programmers, designer etc. We must have this experience to leverage risk in planning, understand the cost, to be able to communicate between these groups, and to be able to describe in more detail.

In practice we do a full volume test for 2009 and 2010, but with limited amounts of basic data and business rules, and not fully end-to-end. We will tackle the core challenges, with a small rule set, and also have a "version 1" blue-print for new business initiatives we know will be coming. We target 2 years in memory, and assessment and tax calculation at 5000 tps. We target HW costs at 10% of todays level.
It is a playground for new technology. We are testing the domain solved in this type of architecture, and it is not a test of a product. Any acquisition of product will be done later.

Just do it
Late spring this year we opened a bid for participating in this Proof of Concept, where we presented our thoughts about the target architecture (We have gotten a lot of good feedback on this :-) ). EDB/Ergo, Bekk won the bid, and they teamed up with Incept5.
We are starting with Gemfire as the processing architecture, and as much plain vanilla pojo as we can. We will be working on this through January 2012. This will rock!

2012.01.22: The results are here, and will be presented at Software 2012.



Tax Norways Proof of Concept by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Enterprise wide business process handling

Introduction
(October 2014. This article is updated with the Case Container)
Understanding the requirements, functional and non-functional, is vital when the architecture is defined. The theme in this article is about enterprise wide business process handling. The challenge is to make different types of systems participate in the business work flow and understand what happened for later insight. BPM, Business Process Management, is a huge field with many tools, notation forms, patterns etc. (See process handling in Continual Aggregate Hub and cooperating federated systems). (A system in this text is the same as a Module, although it may reflect that our current systems are not good Modules. We want our current systems to cooperate as well as our new Modules.)
We have chosen to think Domain Driven Design and to elaborate on these main issues:

• how to automate the business process between cooperating Modules
• how to ensure that some manual (human) task is done
• how to achieve high degree of flexibility in changing business processes
• how to know the current state of a Party
• how to know what has happened to a Party
• how to measure efficiency in the business activities over time 
• how to make very different types of systems cooperate

 And some non-functional requirements:

• how to handle the operational aspects over time (batch vs events)
• how to best support that a manual task is now automated
• how to make it testable
• how to have it scalable
• how to change and add Services and Modules without downtime
• how to change process without downtime
• how to re-run a process, if somethings have failed for some time
• how to handle orchestration (defined process) and choreography (event drive) side by side

Observations of SOA tool state and of integration patterns
I will not discuss this too much, but just summarize some observations (I'll get back to how in some later blog):

• In-flight processes is a pain
• Long lasting processes is a pain (both: software needs update now and then)
• Often not loose coupling between process and the state of some domain entity. A lot of processes occur and function within the domain ("make-minor" approach by some process-system is not good)
• Processes are hard to test
• Scaling and transaction-handling gets complex
• Tools have too much lock-in
• The promise of BPEL visual modeling to communicate with business fails
• The canonical integration pattern often leads to context bleeding and tight coupling
• The aggregate integration pattern often is a sign of complex integration, that probably should be addressed with a system by itself
• Business process state is hidden, and history of events is lost or is drowned in technical details
• Message brokers are great to move messages, but bad in history, and not a good tool in operational flexibility
• Too many parties are involved so that maintenance gets slow

Our logical 5 level design
Main goal is to have full control of all events that flow between cooperating Modules, but not achieve an uncontrollable event driven system. (An event-driven system may just as well be diagnoses as "attention-deficit/hyperactivity disorder (ADHD)" system.)

• Enterprise Process Log
  This is purely a log component, with services for registering business and answering queries about them. It has no knowledge of the business process, but of course has some defined keys and types that defines a valid business event (a business activity leads to a business event). It is the Modules that emit domain events to the log component, and the domain defines what the events are. It is a place to store the business level CQRS type of events (or Soft State if you like from BASE), the more detailed events are kept the Modules. Any type of system can emit events either live or in large batches. The implementation effort to send events is little, and the events may be informational and not necessarily used in an automated process. This log will give insight into what happened with a Party. The lifetime principle is taken care of, as this log must be very long lasting. So we want it as simple as possible. (see The BAM Challenge)
• Global Task-List
  This is a simple stand alone task-list whose sole responsibility is to assign tasks to groups of Case Handlers (CH) that do manual tasks in the enterprise wide domain. The task-list has no knowledge of the overall process. The tasks the task-list receive contain enough information to explain what the task is, and to redirect the CH to the right GUI for doing his task. The tasks are defined and maintained in the Module that has a need for manual work, but dispatched to this common task list. These tasks are tasks that are well defined in a work-flow. When a task is done, or no longer relevant (the Modules that owns the task decide), then the task is removed from the task list.
• Process Flow Automation
  When we first have the events in the event log, automating the next step is quite easy and can be done in a lightweight manner. A simple Message Driven Bean may forward the message in a event driven manner via some JMS-queue, or a large legacy system may query for a file once a week because that is the best operational situation for it (operational flexibility is also discussed in CDH). Also events may be logged that only come in operational use a year later, making maintenance flexible, and history robust.
• Case Container
  This is discussed in this article about a generic Case Container representing all Cases handled in our domain. Its purpose to to contain the Case with all its metadata, the process state and references to all incoming and outgoing xml-documents.
• Case Handling System (super domain)
  This is usually called Case Handling and consists of a case which is outside the existing Modules or systems, and that need the completion of many formal sub-processes, but at startup of the process it is not possible to foresee how the case will be solved. This is typically where know business process ends, and a more ad-hoc process is applicable. Also this system support the collection of different types of information relevant for such case-to-case systems. This information may very well be external information collected in a manual matter.
  (by 2014 we have not gone any further into this part of the overall design. It seems that the CAH and the Case Container is sufficient)

Above is the logical design, and this is what we think we need. You might say it follows a hub-spoke design, where the Modules are the spokes and these 4 elements comprise the hub. These are all 4 discrete components that interact in a services oriented manner, with each other and with other Modules or systems. The main idea is that this will enhance maintenance and reduce the need for customizing COTS.

Ill 1. Basic flow and service orientation

Illustration 1. Just to show a fire-and-forget situation (green) where the Tax Calculation and Collection are interested in House events. Tax Calculation wants them live, and the processing forwards the event, while Collection every 2 months, via file, and then issues service requests (yellow) to Party to get details. The EPL event holds a reference to the Case Container. The notified Module then uses the Case Container to open all relevant information for this Case and the process state the Case is in.

Ill 2. Application layer interacts with EWPH

Illustration 2. Show a Module of DDD where  the application layer interacts with the enterprise wide process handling. The green line show how the Case Container is shipped to the Module where its reference are opened by the Application layer and sent to the Domain layer for handling. The Case container is really the container that it is in the logistics world; bringing a complete set of good from one place to another.


We are implementing this with REST and XML, where feeds play an important part in transporting events and data. URI represent addresses for Modules, and are linked to specific Case Container Types.

We do not mandate the usage of the Task-list. If there is some need for task-list internal to a Module that is more efficient for the users that handles tasks solely in a Module, it is OK when it gives a better maintenance in the long run.
Also we do not mandate what technology that does different work in the Process flow automation, it may be Message Driven Beans forwarding to some queue, Mule, Camel, BPEL for some orchestrated process, or simple Query to file.

Design and overview
Of course you still have to have a good design and understanding of the business processes. And it must be there to communicate between business and IT (BPMN is great for this). But as in other areas of systems development: Design is to communicate between people, and implementations is to communicate to machines. Therefore a combined design-implementation (eg. BPEL) will have a hard time achieving both.
The business process is not fragmented, but I argue that the implementation of the business process is best handled in the above-mentioned manner; The process will sometimes occur within a Module (system), and sometimes in-between.



Enterprise wide business process handling by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

The BAM challenge

One interesting issue arose when we looked into the requirements for Business Activity Monitoring (BAM). The main task to answer is: "How is the organization performing?", "Are we more effective than last year?".  Key Performance Indicators (KPI) is a well established jargon, but how to achieve them?
A KPI can only be answered over time, and for large governmental organizations, that's a long time. Just reorganizing may take a year, and making the new organization perform better takes longer than that.

To measure you have to have some measuring points, and they are to be compared over time. It is not about a business performance measuring tool by itself, it is about what business activities where performed, and how much effort some business entity put into one such activity.


There are three challenges in this:

• First a reorganizing usually affects how account dimensions in the economical systems are organized, that makes the accounts discontinuous and disturbs the measurement of the effort put in. There must be defined some long lasting Business Effort measured in cost (Business Cost, BC). When the account dimensions are defined, they must also map to this BC, so that we keep continuity.
  There are challenges to this; "How much does an IT system cost?", "What is the price of this automated task". (Many organizations have a hard time identifying this).
• Secondly the Business Activity (BA) itself over time is performed in different IT systems or done manually. From experience with performance tuning, I believe it is obvious that the IT systems must support a long lasting BA that survives various implementations of that activity. That is what the Enterprise Process Log (see Enterprise Wide Process Handling) is all about. This is where we collect and keep the BA's.
• The BC and BA must have a comparable period of time.

The BAM-tool is not the solution by itself, and we don't want too much tied up on some SOA implementation. Also BAM-tools are often concerned with what happens on the ESB, there are so many other places that may emit BA's and BC's. The solution must be simple and must last long.

KPI1=BC1/BA1 for a period
KPI2=BC2/BA2 for a period

In our domain a BA would be "Number of tax statements processed" or "Complaints handled" during some time period. The BC would be "Cost of people and systems for assessing tax statements", and "Cost of complaint department", during the same time period.

We will collect BA's from the Enterprise Process Log, and use our data warehouse for the compilation with the BC's and the analysis. The analytics might be in Excel, although we may buy something for this analysis and reporting.
It is the long lasting measuring points and a standardized period of time that is the real value.

The BAM challenge by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Skatteetatens Arkitekturprinsipper

Hensikt
Dette dokumentet beskriver obligatoriske IT-arkitekturprinsipper (som en del av SKEs overordnede virksomhetsarkitektur).

Regler for anvendelse av arkitekturprinsippene:

All utvikling, videreutvikling og anskaffelse av IT produkter og systemer skal følge disse prinsippene.

Arkitektur- og designavgjørelser samt valg av løsningsalternativer skal lenkes opp mot prinsippene.

Unntak og avvik må begrunnes og godkjennes iht. den definerte virksomhetsarkitekturprosessen.

Bakgrunn
Arkitekturprinsippene er utarbeidet høsten 2009 som en del av virksomhets­arkitektur­prosjektet. Prinsippene er utledet fra SKE’s IT-strategi og prinsipper gitt av Difi (”Overordnede IKT-arkitekturprinsipper for offentlig sektor” versjon 2.0 utgitt av Direktoratet for forvaltning og IKT 8. oktober 2009).

Innhold
De første prinsippene er brukerorienterte og beskriver hvordan funksjonalitet skal tilbys brukere (innbygger, skattyter, partnere (offentlige etater og 3. part) samt interne medarbeidere) og går på:

• Tilgjengelighet
• Brukervennlighet
• Integritet
• Samhandling

De resterende prinsippene er mer teknisk motivert og beskriver hvordan IT løsninger skal utformes og velges

• Endringskapasitet
• Gjenbruk
• Livsløp

Tilgjengelighet
Beskrivelse	Funksjonalitet skal være lett tilgjengelig for brukerne der og når de trenger det.
Motivasjon og mål	I dagens elektroniske samfunn har eksterne brukere en forventning om å ha tilgang til offentlige tjenester og informasjon uavhengig av tid og sted. Målet med dette prinsippet er både å gjøre det lettere å handle riktig, samt at man gjennom stor grad av selv­betjening sparer etaten for arbeid og ressurser.
Underprinsipper, fokus og avgrensning	Hovedfokus for dette prinsippet et at tjenester for innbyggere, skattytere og partnere som hovedregel skal være tilgjengelig: 24/7, men merk at denne forventningen gjelder muligheten til å kunne utføre sin del av arbeidsprosesser med det offentlige (sende inn skjema, svare på forespørsel, spørre om informasjon etc.), og at det ikke medfører krav til at forespørsler om [saks-]behandling i etatene skal bearbeides og svares utenom normal arbeidstid. Over de kanaler som brukeren forventer/ønsker (multikanal - dette gjelder for tiden spesielt internett og papir samt noe begrenset SMS). Underprinsipper: Det er viktig å være tydelig på hvilke tjenester som skal være tilgjengelig, over hvilken kanal, og til hvilken tid. (F.eks. "send inn skjema" er tilgjengelig hele døgnet, mens "ring meg tilbake" er innenfor 8-16) Det er viktig å være tydelig på hvilke QoS "Quality of Service" som brukeren kan forvente. (F.eks. det innsendte skjerma har avregning i midten av Juni dette år). Avgrensning: Tjenester for interne medarbeidere har ikke like strenge krav til tilgjengelighet. Her er hovedregelen at tjenester skal være tilgjengelig på dagtid og på arbeids­plassen til medarbeideren.
Konsekvenser
Referanse og relasjon til andre prinsipper	Dette prinsippet tilsvarer Difis prinsipp ”Tilgjengelighet”.
Anbefalinger

Brukervennlighet
Beskrivelse	Elektroniske brukertjenester skal være tilpasset brukeren og dennes bruksmønster, legge til rette for effektiv bruk, samt sikre god kvalitet på oppgaven som utføres.
Motivasjon og mål	Målet med dette prinsippet er å gjøre det enklere for brukeren å handle riktig. For interne brukere er dette viktig for å oppnå høyere effektivitet og motivasjon. Dette innebærer at det skal være lett å finne korrekt og relevant informasjon, og å orientere seg i hva brukeren kan gjøre enkelt å oppfylle sine plikter, å gjøre brukeren i stand til å orientere seg i hva som forventes av han lett å unngå unødig rapportering, dvs. bruk av forhåndutfylling og unngå dobbeltrapportering
Underprinsipper, fokus og avgrensning	Brukervennlige tjenester og deres arbeidsflater skal følge disse underprinsippene: universell utforming av arbeidsflater enkel og oversiktig tilgang på nødvendig og nyttig informasjon arbeidsflater eller skjermbilder skal for brukeren inngå i en "dialog", selv om skjermbildene hver for seg tilbys fra forskjellige tjenester i forskjellige systemer (enhetlig teknisk plattform for GUI) enhetlig teknisk plattform for GUI er også viktig for å tilby brukeren enklest mulig håndtering av sikkerhet. Avgrensing: For interne brukere er det viktigst å fokusere på at arbeidsprosesser skal kunne gjennomføres enkelt og "smidig", f.eks. uten at brukeren skal måtte starte og logge seg inn på mange forskjellige systemer og arbeide i mange forskjellige typer skjermbilder innenfor en og samme aktivitet.
Konsekvenser
Referanse og relasjon til andre prinsipper	Dette prinsippet tilsvarer Difis prinsipp ”Tilgjengelighet”.
Anbefalinger

Integritet
Beskrivelse	Funksjonalitet skal tilbys i henhold til lover og regler, sikkert og konfidensielt, og med en høy kvalitet slik at den sikrer integritet for både Skatteetaten, partnere og skattytere/innbyggere.
Motivasjon og mål	Skatteetatens systemer må sikre at etaten gir et godt og rettferdig tilbud til innbyggerne. Skatteetaten skal behandle informasjon konfidensielt og ha sikker identifisering av innbyggere slik at Skatteetaten har troverdighet. Skatteetaten skal framstå som en seriøs aktør som er bevisst sitt samfunnsmessige ansvar.
Underprinsipper, fokus og avgrensning	Hovedfokus: Prinsippet innebærer at sikkerhet og funksjonalitet iht. lover og regler har høy prioritet. I tillegg skal det være enkelt å få et fullstendig bilde av en ekstern brukers / partners saker for å gi en best mulig service.  SKE bør gjennom Grønn IT vise bevissthet ovenfor miljøansvaret. Dette stiller krav til oppfyllelse av følgende underprinsipper: sikker identifisering i forbindelse med bruk av løsningen sikker identifisering i forbindelse med beregning / behandling i systemet (oppgaver / beregning av skatt etc.) konfidensialitet av informasjon skal garanteres tilgangskontroll på både dataelement og dataverdier sporing av endring og oppslag på data (audit / historikk). sporing av systemmessige beslutninger (transparens) sterkt fokus på kvalitet (korrekthet) og etikk. Korrekthet sikres best ved å forstå og å kunne etterprøve utfallsrommet til en beregningsmodul
Konsekvenser
Referanse og relasjon til andre prinsipper	Dette prinsippet tilsvarer Difis prinsipper ”Sikkerhet” og ”Åpenhet”.
Anbefalinger

Samhandling
Beskrivelse	Funksjonalitet skal tilbys på en slik måte at det er enkelt å samhandle innen og med etaten.
Motivasjon og mål	Det stilles krav om økt standardisering, samordning, fellesløsninger og automatisering på tvers av etater og sektorer. Det må være enkelt å utveksle data mellom løsninger (både internt i SKE og eksternt med partnere) for å kunne sette de sammen i en større helhet. Funksjonalitet (tjenester) må sees på som en del av tversgående virksomhetsprosesser. Arbeidsprosesser må samordnes for å sikre riktig, enhetlig og oversiktlig behandling.
Underprinsipper, fokus og avgrensning	Prinsippet medfører høye krav til interoperabilitet, som kan beskrives i følgende underprinsipper ift.: Teknisk interoperabilitet: fleksibel, enkel og gjenbrukbar integrasjon mellom løsninger at systemer kan samhandle via felles tekniske standarder felles logisk syn på dataene mellom løsninger og mellom forskjellige tjenester (spesielt i en og samme arbeidsprosess) at tjenester skal tilbys i en forståelig kontekst, dvs. som del av en overordnet arbeidsprosess (hvor dette er naturlig) forståelig sammenheng mellom informasjon, andre tjenester, hvilken kontekst eller prosess den er en del av, brukers muligheter og plikter samt etatens oppgaver. Semantisk interoperabilitet: at systemer forstår betydningen av en tjeneste eller data. Organisatorisk interoperabilitet: at systemer deltar i arbeidsprosesser i en konsistent kontekst. Det kan også innebære organisatoriske endringer for å gi effekt av samhandling.
Konsekvenser
Referanse og relasjon til andre prinsipper	Dette prinsippet tilsvarer Difis prinsipper ”Interoperabilitet” og ”Tjenesteorientering”.
Anbefalinger

Endringskapasitet
Beskrivelse	Alle løsningers funksjonalitet skal være fleksible nok til å kunne følge forventede endringer i samfunnet og i etatens tjenester innen rimelig tid og prisramme.
Motivasjon og mål	Målet med dette prinsippet er å oppnå en høyere fleksibilitet i funksjonaliteten til virksomhetens IT systemer. Samfunn og teknologi er i stadig utvikling og forandring. Det er ikke realistisk at IT-løsninger skal kunne forbli statiske over lang tid, og samtidig kunne oppfylle sine mål på en hensiktmessig måte. Målet for fremtidsrettede løsninger er dermed ikke hovedsaklig å kunne vare "evig", men heller å kunne endres enkelt for raskt å kunne følge utviklingen.
Underprinsipper, fokus og avgrensning	Stor endringskapasitet er spesielt viktig innen rekkefølge og utforming/gjennomføring av forretningsprosessene. rask gjennomføring av regel- og lovendringer. innføre og ta i bruk nye data (oppgaver) innføre og ta i bruk nye IT tjenester. Merk at endringskapasitet ikke bare har med sammensetning av tjenester å gjøre, men også i selve konstruksjonen av tjenester og støttesystemer for disse, samt datamodeller. Avgrensning: Dette prinsippet gjelder de funksjonelle kravene, de ikke-funksjonelle kravene håndteres under Livsløp-prinsippet. Det er ikke viktig å kunne raskt innføre nye forretningsprosesser eller helt nye typer tjenesteområder, da man kan gå ut i fra at Skatteetatens virksomhetsdomene ikke kommer til å endre seg vesentlig i overskuelig framtid. Underprinsipper: Det finnes endringskapasitet på mikro- og makronivå; innenfor en tjeneste eller i ett system, og i forretningsprosessene som bruker tjenester. På mikronivå bør hvert system må foreta en bevisstgjøring rundt hva som må være fleksibelt (unødig fleksibilitet er dyrt). Tilrettelegg for metoder og teknikker som reduserer risiko ved kodeendringer. Tilrettelegg for metoder og teknikker som effektivt bekrefter vellykkede endringer. Skill utvalgskriterier (hvem), forretningslogikk (hva) og lagringslogikk (hvor) slik at det er tydeligere hvor endringer skal implementeres Vær klar over at alle beskrivelser av ett system (kode, spesifikasjon el.) som brukes i en automatisk sammenheng, like mye er del av helheten og kan være kilde til feil. Tydelige ansvarsområder / tjenester gjør at endringer isoleres til delsystemer (og ikke litt her og der) Viktig å raskt kunne prøvekjøre endringer slik at man ser effekten på de dataene som tjenesten omhandler (testing / simulering).
Konsekvenser
Referanse og relasjon til andre prinsipper	Dette prinsippet tilsvarer Difis prinsipper ”Fleksibilitet” og ”Tjenesteorientering”.
Anbefalinger

Gjenbruk
Beskrivelse	Gjenbruk skal prioriteres, både ved å velge allerede gjenbrukbar funksjonalitet ovenfor ny utvikling/anskaffelse og ved å investere i gjenbrukbarhet ved utvikling/anskaffelse av ny funksjonalitet.
Motivasjon og mål	Målet med dette prinsippet er å oppnå høyere kostnadseffektivitet på lang sikt gjennom redusert kompleksitet og mindre redundans som igjen reduserer utviklings-, drift- og forvaltningskost, og muliggjør raskere utvikling. I en typisk prosjektstyrt IT virksomhet som hos SKE vil prosjekter bli målt opp mot leveranser innen tid og budsjett. Motivasjonen for å investere tid og ressurser i løsninger som er gjenbrukbare for andre er dermed ikke høy nok til å sikre gjenbruk uten at dette settes opp som et eget mål.
Underprinsipper, fokus og avgrensning	Prinsippet innebærer følgende underprinsipper: Hvert enkelt prosjekt og videreutviklingsaktivitet skal, i samarbeid med etatens arkitekturmiljø, lage IT-komponenter som andre (både egen etat og andre) kan benytte senere (”bygg eller begrunn”). Hvert enkelt prosjekt og videreutviklingsaktivitet skal, i samarbeid med etatens arkitekturmiljø, gjenbruke IT-komponenter som allerede finnes (internt i etaten eller hos andre offentlige etater) i stedet for å lage/anskaffe nye (”gjenbruk eller begrunn”). Kapsling av systemspesifikk logikk skal gjøres så nært kilden som mulig.  Masterdata, dvs. etatens viktigste kjernedata, skal så langt som mulig og hensiktsmessig hentes direkte fra og oppdateres i en opprinnelig kilde som kontrolleres av en eier. Informasjonsmodellen for slike data skal være felles for virksomheten. Gjenbruk skal også gjelde skjermbilder som skal kunne delta i en arbeidsflyt. Brukeren trenger ikke å vite hvilket system som tilbyr ett skjermbilde. Gjenbrukbare tjenester utvikles etter konkrete behov i de systemer som naturlig vil tilby tjenesten. Det er viktig å vurdere de ikke-funksjonelle kravene siden forskjellig bruk av en tjeneste kan kreve forskjellig implementasjon.
Konsekvenser
Referanse og relasjon til andre prinsipper	Dette prinsippet tilsvarer Difis prinsipper ”Fleksibilitet” og ”Tjenesteorientering”.
Anbefalinger

Livsløp
Beskrivelse	Ved utvikling og anskaffelser skal gevinst- og kostnadsbildet ta høyde for helheten i SKEs systemportefølje, hele livsløpsbildet for løsningen, samt de ikke-funksjonelle kravene.
Motivasjon og mål	Målet med dette prinsippet er å få god effekt av IT investeringer over lengre tid. Ofte er tidshorisonten det tas høyde for, for kort. SKEs kjernesystemer har en levetid på flere tiår og valg for disse må dermed ha høyere fokus på fremtidsrettede løsninger. Målet er å kunne frigjøre størsteparten av ressursene og tiden til å gjøre nye ting, og ikke måtte gjøre ting på nytt (fordi løsninger og teknologier har blitt for gamle).
Underprinsipper, fokus og avgrensning	Prinsippet innebærer følgende underprinsipper: Det er meget viktig å satse på åpne og anerkjente standarder, pga. flere tilbydere (unngå avhengighet til leverandør, god tilgang på kompetanse), stadig videreutvikling samt bredt tilbud for overgangsløsninger. Det er spesielt viktig å lagre og håndtere data på formater som er framtidsrettede, da det alltid vil være lettere å oppdatere funksjonalitet enn format på historiske data. Åpen kildekode er også en faktor som bør vurderes. Da man ofte har et aktivt "community" som sørger for videreutvikling samt at man har mulighet til å selv utføre endringer. Alle løsninger skal så langt som mulig og hensiktmessig understøtte: - god og enkel driftbarhet - god og enkel vedlikeholdbarhet Ved valg av løsninger skal alltid: - ferdigløsninger vurderes før utvikling/skreddersøm - outsourcing vurderes ovenfor intern håndtering - monopol og sterk leverandørbinding unngås - moderne og fremtidsrettet, men velprøvd teknologi prioriteres Det er viktig å forstå de ikke-funksjonelle kravene, slike som antall samtidige brukene, datavolum, transaksjonshyppighet, samtidighet (concurrency), sikkerhet, krav til oppetid, krav til recovery / backup, internasjonalisering etc. I dag og i morgen. Disse er viktige for å oppnå en så enkel arkitektur som mulig og man skal ha ett bevisst forhold til hvordan dette utvikler seg over tid.
Konsekvenser
Referanse og relasjon til andre prinsipper	Dette prinsippet tilsvarer Difis prinsipper ”Åpenhet” og ”Skalerbarhet”.
Anbefalinger

Lifetime principle. Continuous migration

We have a Lifetime principle defined for our Enterprise Architecture.

So what does this principle (and requirement) actually state?

It states that we should choose components and software systems that has the best cost/benefit over its lifetime. That is nice. What we all need. Problem solved, right?
The main problem is that we do not know the answer of either of them; we do not know the lifetime of the technology we choose, and we do not know the lifetime of the business we are supporting. We probably know more about our business overall; tax-systems are here to stay, and certain elements are fixed (someone must pay their tax). But we do not know the details on how our processes will be in 20 years, neither on the details on how to calculate and what type of tax we will be imposing (CO2 footprint tax?).

We must make sure that the important stuff for us - requirements (see: Requirements), data, code base, and business state (as discussed in other blogs) - is handled and stored in a way that makes them versatile. Requirements by themselves are interesting, because the systems in which we store requirements actually must last longer that the systems they define. Just think about it; what system would you trust your requirements? (Systems Architect, Troux, Mega, Qualiware) Which will be there in 20 years?

When we have defined our target architecture it must have certain capabilities to support our domain; but it also must comply to the lifetime principle. Distinct domains define systems that cooperate (they could be called mega bounded contexts, just to extend the Bounded Context of DDD. In our case they would be Party, Tax calculation and Money, which in turn have smaller Bounded Contexts), it is the most important feature; then these can be exchanged as time flies by. Well this is not Lego, we should add "with as little effort as possible". This also helps in isolating custom functionality from COTS. In both cases we find Domain Driven Design to be a great EA approach.

The overall architecture must adapt to continuous change in technology and business. Surely it is a hard nut to crack, but it still is the main goal for the target architecture. It must support continuous change, that is the normal case, we as an organization must support and endorse change.

With the lifetime principle posed on our systems, the overall effect is that:
The architecture must support a condition of continuous change.

Lifetime principle. Continuous migration by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Steering projects; Where is the improved in "new and improved"?

Enterprise architecture is all about improving, continuously. Our target architecture is our aim, and every step towards it may alter the aim. The new aim is a better aim. That is what it is all about: Improving as we walk to a better architecture. The steps in this walk are our projects. We need to identify, define, implement and steer our projects. We totally depend that a project understand is part of the whole. Every task in the project will go in greater detail. These details will bring us a better understanding; either some part of the goal of the project is a bad path, or it is a good one. Maybe the traversal of the projects tasks really find a golden egg, or a pit of snakes.

Projects live a life of their own. The team is dedicated to their goal, and they will continue to walk within the mandate and quite a bit of self-focus. Very uncommon, or never, has a project said to the steering committee "our task is unnecessary, please stop this project". It is not that the project not is trying to reach its goal, or being loyal, the team is honest in doing what they believe is correct. The purpose of the steering committee is, well, to steer. The steering committee in this context is any entity who is responsible for steering the project. I remember a project manager once said that informing the steering committee was like growing mushrooms "keep them in the dark and feed them horse shit". And maybe this illustrates the challenge; the projects must not obey the project-mandate or architectural blueprints alone.

In this context the mandate must be extended with the target architecture and a description of what the project must participate with. This includes components which are there already - they may be extended - , and new components that later projects will benefit from. The later is a tougher one. It is then vital that projects understand the purpose (why and for what) of this component in the target architecture, and have a steady set of principles. The message must be repeated and repeated, the enterprise architects must be absolutely sure that the architects in the project understand why. Remember that experience the project gain, may eventually alter the target architecture, there must be close cooperation between architects at many different levels.

So this brings me back to the headline: "where is the improved in "new and improved"? When we staff projects, most often we choose people who know the functional domain and someone who know the new technology. And what do you get? To often the existing domain in a new technology. Does this bring us closer to the the target architecture? Can the steering committee see what is coming? Too often they haven´t.


In addition I think the overall process must be agile so that the project mandate and budgeting may be adjusted along the way. Also we have the good opportunity of having project demos as the sprints go by. This makes it easier for the steering committee and the enterprise architects. I think Gartner has coined this overall process EAD "Enterprise-class Agile Development". Using project as changers on the path to an improved enterprise architecture.

All you get is new, but not improved, if you don't make sure that the project does understand the purpose of the target architecture.

Steering projects; Where is the improved in "new and improved"? by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.