Leave all that can be spared behind. We travel light. Let’s hunt some orc. ― Aragorn, The Lord of the Rings
David’s success depended on surprise and accuracy. He knew he could not defeat Goliath on the giant’s terms, which is why he rejected Saul’s armor and with it the conventions of this form of combat. ― Lawrence Freedman, Strategy: A History
How an illuminated customer, carefully planned design decisions and a well chosen bunch of tools turned one of Nubisware’s most challenging commitments into a success story.
Late 2017 we’ve started to work at a very ambitious project for a large logistics company which had to face a complex refactoring of one of its core business applications.
The application had been built with an approach that at the time was considered traditional. A huge, web-based monolith rooted on top of a relational database.
Only very few technicians knowing what was going on under the hood and what were the design choices that led to particular solutions.
Up to when the supplier discontinued support, the application performed well enough. With the exception of a particular functional segment which had been causing headaches for years.
Our customer’s IT responsible was a very clever guy with a huge domain specific knowledge and a solid background in computer science. He was definitely tired of all the complaints raised by colleagues and customers so he asked the company’s CEO to invest in a refactoring of the indicted functionality. And that’s where Nubisware comes into play.
It must be said that, the functionality we were asked to refactor is a complex part of an already complex domain like logistics. The border between a rule and an exception is rather blurred. Thus, the work required a very careful analysis and planning which involved several meetings and a cyclic reworking of the project’s documentation.
Besides the inherent difficulty, the application also suffered from erroneous design choices that caused a general inflexibility making it very hard to correct weaknesses and limitations.
Relational persistence and the object-oriented data model proved to be major logical bottlenecks. We decided to go with a more data driven approach and helped the customer understand why a document oriented representation of the domain rules was better suited in this case.
From an architectural viewpoint the application was bulky, slow and to some extent unmanageable. We decided to rewrite the code as a set of cooperating microservices which are mostly stateless and thus perfectly suited to exploit inherent parallelism available on the customer’s very good infrastructure. Moreover, it enabled for a progressive migration towards the final solution involving several small and risk-limited integration steps.
Every major functionality (the algorithmic core, the data transformation services, the document repository, the graphical user interface) was split apart, wrapped with a REST API, rethought, rewritten, intensively tested and then put into staging. This was so surprisingly smooth that, to my feelings, there has not even been an official staging-to-production phase. At some point, we just have been told that users were using the new functionality with great satisfaction.
The serialization model for the documents was chosen to be XML because its formal solidity and the availability of validation and modeling tools granted the robustness that was needed for the backbone of the project. JSON is in use but limited to the interaction between the User Agent, the GUI and some support services providing integration to existing databases. The use cases were very heterogeneous but we wanted to keep the syntax (i. e. XSD) simple and manually manageable. Thus we decided to go for a very simple yet flexible XML structure with the possibility to enrich the semantics of elements through RDFa.
In order to manage the complexity and the continuously evolving specifications we decided to model all the application semantics (rules, constraints, relations, mappings, …) with an OWL Ontology designed with Protegé. Keeping most of the domain specific logic stored in an external, lightweight yet structured model, allowed for late refactorings and enhancements of the application logic and proved to be also a great support for the implementation of the User Interface.
All the code was written in XQuery 3.1 modules and BaseX has been used as the runtime for the scripts and the REST APIs. The seven services were packaged into WAR archives (each less than 20MBs in size) and deployed inside Tomcat (7, 8, and 9).
In conclusion, the services have been running, nearly without intervention, for about 9 months by now. The customer is very satisfied about the huge improvements in flexibility and performance and, besides all the other metrics that we’ve taken (footprint, memory usage, throughput, …), this is the only actual one that counts at the end!