The Case Container

This article fills in the last piece, and completes previous articles (linked in this text below) of how we structure Case Handling. It is the design of "Process state and handling" of Continual Aggregate Hub article of 2010.

At the core
A central part to any typical Enterprise Application is the Case or Dossier, and the process handling this. The information going in to a Case, the business logic applied to it, and the subsequent business decision(s). It all has to be filed with accuracy. Case handling get complex because information changes over time, business decisions are made, and the business logic and the information going into it are also complex. Just look at financial institutions and insurance systems, as well as government systems. These have a load of legislation and business rules - that change over time – and every business decision must comply to the rules and information that was valid at that point in time. Otherwise that decision does not have integrity.
Legislation states that filing in such systems must be applied after some sort of standard; in Norway it is called “NOARK”, in the UK “RMS” and in the US “NARA” (and related).

Immutable xml-documents
In our systems design we already have stated that business decisions are immutable, and are stored as XML documents with a set of metadata. The design of the CAH has proven to have excellent qualities with regards to integrity, ease of maintenance and ability to scale. This component is now in production (see BIG DATA - Definite content).

Searching for the answer
We used 3 years looking for this design. And we started out in the BPM (Business Process Management) area. Now that we have the design, we realize that BPM is the wrong end of this challenge. There are a lot of BPM products out there, but they miss out on one main point. It is not the processing framework that is the main challenge, but the Case content (information and rules) and the state of the Case itself. That is why most of these effort are hard to maintain, and often could be solved in a much simpler way. A key point for us is that any Case may be reopened for the next 14 years. We do not want to depend on any specific BPM product, as it is a vendor lock in (enterprise wide process handling).

The missing piece
Let me introduce the CaseContainer, a generic container representing any Case in our domain. It is also an immutable xml-document, and acts as a “binder” between the Case Type and its metadata, the information going into the case, the business logic applied to it, the process state the Case is in, the resulting business decision, and as access control.

It is basically divided into two parts: 1) a set of metadata describing the Case, and complies the NOARK-5 standard. And 2) a set of references to xml-documents it consists of. The document references are divided into two groups: documents going into the case and documents representing the resulting business decision. The metadata makes the CaseContainer be part of a hierarchy of cases, is described by the Case Type, and it describes the case state; both general (eg. Open, Pending, Closed, Reopened) and specific to the case type (eg. The states of this specific case handling process). The Document Archive would be the CAH with its typed documents, or a Archive of binary documents such as PDF (for manual read by a case handler). It is all implemented with REST and XML notation. You can’t find anything more open and robust.


The Case Container described (main attributes, there are more):

Attribute	Description
UUID	Universal Unique ID for this document
PartyId	The Id of the party this case concerns
Timestamp	nil
Case number	The number of the case. Most often this has some formatting case year, sequence number and version (eg. 2014/45235/4)
Case title	The title of the Case (eg. "Tax case 2014")
Status code	Generic case state that complies to your local archiving standard (new, open, pending, closed, rejected...)
Folder	This is also called “File”. It is structured as a flat “namespace” notation to contain “Fonds”, “Series”, “Class”, and “File”. And any hierarchy between them (eg: no.nta.folder.tax.2014) The Case Type may be the whole or a subset of the "Folder".
State	It is structured as a flat “namespace” notation to contain any state in any process. (eg. no.nta.process.tax.2014.complaint)
Case responsible	The case handler or organization responsible for the decision. (eg. "no.nta.authority.username")
Date of invocation	The date for opening this Case version
Date of decision	The date for closing this Case version
Incoming documents	A list of references to all documents going into this case. We have a unique reference to every version of every xml-document. Usually this is called “Registry entry type” or “Document object reference”.
Business decision documents	Same as above, but here is the list of the documents produced in handling this case. Often these are outgoing documents and can contain an Invoice or a written statement.
Audit	A list of all changes made to this case since version 1.

[*) NTA is Norwegian Tax Authority]

Processing the Case
The processing framework handling this is also simple. A Case Type has a set of documents that the Case can consist of. When a new document comes into the CAH is it pre-defined what type of Case should be opened. The Case Type and its State is linked to a processing URI that handle it (see Enterprise Wide process handling for details in how we cope with event driven processing). In that way we connect the valid business rules to this Case Type. Connected to the event is a reference to the Case Container, and its is the Case Container that is transported to its Module for handling. The Application Layer (defined in DDD) then starts processing on it. Any open Case will either be handled automatically or show up in the Task List for manual handling. It will be handled manually if the business logic states so (by some validation), or if some of the documents going into the case is not strongly typed, and must be read by a human.
Processing may also be restarted just by creating a new version of the Case, for example when your code has been fixed and you need to ad-hoc re-run the business logic. This will result in a new version of the Case Container, and a new version of the business decision document. Handling Cases where there is new information, but no new decision, is archived as a new version of the Case Container, referencing the new incoming information, but maintains the last business decision. (see the Restaurant and the role of the "waiter", he is the one composing Case Container when events happen, at a timed event or at ad-hoc need for running a process)

Access control
Security and access handling has a better chance with this design. The user is linked to a set of Case Types and hereby valid Document Types. Now the XACML access control needs only check if the user has access to the Case Type or the Document Types. The metadata containing Case Types and Documents Types is now a PIP (Policy Information Point) for the XACML's PDP (Policy Decision Point). The access control has a high level of granularity and the makes it possible to understand and configure. An otherwise tough challenge in a service oriented environment.

Separating concerns
The CaseContainer is a design that separates the business information and logic, from the decisions made over time. In most systems these are intermingled. Now they form two separate implementations, making the Case Handling System much easier to implement, understand and maintain. The Case Container runs in the Application layer, while the Business logic works on the information in the Domain layer (as defined in Domain Driven Design)

I hope this helps out in your quest. Separating concerns is really what you need in a complex domain.

The Case Container by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Cloud prototyping and Kolb's Learning Cycle

There is a lot of attention on Cloud platforms (IaaS, PaaS and SaaS), but not so many present how they actually will move to it. This blog is about prototyping core business to such a platform (eg. Amazon Web Services' Beanstalk). It’s also about utilizing Klob’s Learning Cycle in organizational development. An even harder problem, until you realize that your organization has to experience a common vision.

I have earlier presented results from our Proof of Concept (Tax Norways PoC Results). That prototype was about designing software for the "cloud" (In-memory, Big Data); to make it easier to maintain, cost less and scale linearly. (I had a talk at QCon London 2013 about this)
This time its about how we can simplify our business and still make sure we comply to legislation. Its about using Domain Driven Design to establish the ubiquitous language and aggregates. Its about engaging business and stakeholders.

Innovation emerges from the collaboration between disciplines.
This is what Enterprise Architecture is all about: Improving business though IT.

What business?

The need to clean up a very complex set of forms representing mandatory reporting from the population to the Tax Administration. Simplifying for small businesses is our focus, by addressing 13 forms (the total is 50 forms, but larger businesses and other tax domains is not addressed now). Main strategy for this simplification is to collect timely facts through a responsive and “smart” wizard made available for the public. There was already a business project working on this involving 20-30 people. They had a concept ready, but was unsure how to proceed. And I had blueprints of our future systems. 

Is the Public Sector able to innovate?
Yes We Can!

Why Prototype?

Kolb's learning cycle

Lets experience something together! Ever noticed how people understand the same text or the same specification, in different ways? That is one of the major challenges in Computer Science, expectations and requirements most often are not aligned. We used Kobl's to achieve a collective understanding of our vision in this area, combining the individual skills of our organization. With this support we managed to combine deep knowledge to create something new and highly useful.

We prototyped to make this cycle come alive. Making very different disciplines; people from business (lawyers, economists and accountants), user interaction (UI designers), IT, Planning and Sponsors come together over a running application. We have run demos every 14. days (Active experimentation and Concrete experience) and discussed calculations, concepts and terms (as will be part of the new and improved ubiquitous Business Language). After the demo the participants used their Reflective observation and Abstract understanding to write new requirements to the Backlog. This is a “silver bullet” for a team fighting a complex domain. No Excel sheet, presentation or report would have made this possible.

In technical terms its more than just a prototype. It is a full blown information model, implementation, and a full fledged stack (although without database). And we know it will perform with massive scale. We have now defined the Java classes and the xml for the document store (Continual Aggregate Store).

But for business its a prototype; as we have not covered all areas or all detail, but shown how the majority of complex issues must be tackled. The prototype will be used as a study within new areas. Also remember that for business, IT is just one piece. There is also timing, financing, customer scoping, migration, integration, processes and organisational issues.

The beautiful prototype

The domain has 13 forms with some 1500+ fields. The solution now consists of 6-7 aggregates and

Navigation at left, income facts at right

some 600+ fields altogether, but any user will only fill in a subset of these. We have utilised the latest in responsive design, to give as much as possible on a web platform. Everything is saved as you work and "backend" business logic is run for calculations and validations. Logic is not duplicated, the user is working towards the same backend as our own case handlers. (There are no Form anymore, as the user is working synchronously with the backend. Old legacy systems need an asynchronous front end, and get a costly duplication of the business logic.)

Previously there was a “back-and-forth” work process through 11 hard to comprehend steps, now the structure is sequential through 5 simple steps.
Previously the user had to do a lot of manual calculations and book-keeping, now the prototype collect facts and calculate for them.

Expert in your hands

Previously the user had to know what forms to use, now the prototype guides the user though it.

Making the impossible possible
At the end there is adjustment GUI of sliders that solves what was previously hard to comprehend. Only experts with many years experience could do this. A goal for us is that normal people can report their own tax-form in an optimal way. We have done user testing and got very positive feedback. It is much easier for a user to report on real world things (assets, income, costs, dept), and let the application do the calculations.
Now that we see the result, it looks so simple, sublime and beautiful.

The key artefact is the new set of fully unit testable java classes (information, logic and aggregates) of the core domain. Ready to be deployed on some PaaS. This core is now much simpler, simplifying all aspects of systems development and integration. This is how we increase the ability to change and decrease maintenance cost.

The platform

We are using Amazon Web Services (AWS) and have deployed the prototype in Apache and PHP Beanstalk containers. This prototype continues where the last prototype stopped, and porting that code has proven to be simple. Also we are using plain Java and HazelCast on the backend. The backend contains all business logic and information. There is very little duplication of code, and the backend is used all the time as the user works his way through the wizard.
The front end has HTML5/CSS3, Javascript, JSON and REST.
AWS has been really simple (time saving) and cheap. The bill is about $100 for the test environments after 5 months! :) Also we have proven (again) that if you do your design right, you can quite easily move to another "cloud" platform (see deploy looking good).
Testing is now a dream compared to before, mainly because the aggregates are great units to test, and because business provide calculations as spreadsheets, which then sub-sequently are used as tests.
(We are deploying our private cloud, and the production-stack will be different)  

The experience
We teamed up in late March 2013 with a team of 5 from Inmeta.no. They where experienced in Java, Infinispan, GUI design, and web-development. They had no experience from Tax Administration or Accounting. The business had a concept and a plan; by starting with simple cases and adding complexity demo-by-demo. And I had a rough design for the aggregates replacing the forms. Late August we finished. At that time we had covered a lot more than we anticipated and also worked out new concepts (which had been impossible to foresee without the prototype). 

• The theory and practices of Kolb's Learning Cycle are helpful in Computer Science. 
• Prototyping is a silver bullet in many aspects
• Use the prototype also on all other parts of you organization
• Our modernised business can run with many 10s of thousands of concurrent users
• EA-perspective and Organizational Development: Business is engaged, drive change and stakeholders are behind the modernisation
• Our business processes can be run in new ways, eg. being much more efficient or providing transparency for the public
• A prototype will result in a mutual understanding of information model and business logic
• Do not implement paper forms into xml schemas, but re-structure in aggregates 
• Your legacy systems should be moved to a “cloud” platform by using Domain Driven Design, Java, and a systems design approach that I talked about at QCon London 2013
• If you understand HTLM5/CSS3, Javascript, JSON and REST, it is not that important what framework to use on the client side
• Java can be really verbose, you dont need a rule-engine
• Aggregate design (of Domain Driven Design) really rock
• PaaS really saves time and cost

This show the innovation-power of small multi-discipline teams, with the right competence and ambition.

Cloud prototyping by Tormod Varhaugvik is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.