Despite the global acceleration in digital transformations across organizations in various sectors, many banks and bigger financial institutions are still using monolithic legacy applications designed to perform multiple functionalities. Mature, stable, and reliable, these applications often provide several feeds to downstream systems, and as such may be prime sources of regulatory reporting, thus making them very critical for the banks.

Problems may arise when these applications are acquired by third parties which are no longer providing services to banks, and/or when applications have hidden complexities in terms of process flow and calculations. Yet, Trading desks and regulators expect exactly the same calculations from any new system intended to replace legacy applications. Often, these older applications behave like the center of a spider web, feeding all other systems, thus requiring all other dependencies to be removed before switching the application. Decommissioning legacy applications is normally a multi-year project which involves moving all other dependencies to the newer platform/system. In this case, the old application remains live for many years, if not decades, during the decommissioning phase.

These systems, running in batches (batch-mode), are often set up to perform a particular functionality specific to a financial instrument or payoff type. Over time, the overall load on batches increases drastically, along with its memory consumption. 

Because these memory-constrained environments are driven by mostly legacy 32-bit applications, with a system limitation to use no more than four gigabytes (GB) of memory, they often struggle to cope with instrument loads, leading to system crumbling and crashes. Additionally, these applications often depend upon third-party libraries that may also suffer from technical challenges, so building a 64-bit version may not be a feasible solution. The problem is further complicated if these applications are using a lot of memory on loading and processing instruments that are not needed functionally, potentially building up unnecessary memory. One possible strategy to mitigate such a scenario would be to reduce the overall memory footprint of the application, thus stabilizing it and making it sustainable for the lifespan of the application.

Profile building reduces memory load

In this case, to cut down on the memory drain, one solution would be to access the application profile of instrument types being loaded, categorize them as per payoff, and build out an adapter that can exclude or eliminate unnecessary products from being processed.

Reviewing the instrument type being loaded is a far more critical step, as it indicates the effectiveness of the process, with a bird’s eye view of the number and type of financial instruments being processed by various batches. Another important benefit is that this review gives an indication of the possible improvement of batch time. For instance, if the overall count is high for undesired instruments, then eliminating them will save significant processing time.

The categorization or classification process should be able to label each instrument as per batch process requirements, and should pave the way to attach or de-attach them according to need. Defining clear criteria for categorization will help build this system effectively. Once defined, a process can then be built very quickly using these criteria to label instruments. Qualified labeled instruments are then fed into the existing system for processing.

A simple adapter can use these labels to control injection of the instrument into legacy applications. These adapters can be written at the database level, or within legacy code, depending upon convenience. The ideal implementation is to make label inclusion and exclusion as the configuration base for quick writing turnaround.

Depending upon application legacy functionality and the instrument type being processed, the benefits of optimization can be tremendous. For example, a Swaps financial instrument—which can hold a few payoffs—is generally quite heavy from a memory perspective, so excluding them reduces the application’s overall memory consumption.

Applications which consume too much memory often lead to a 60% to 80% failure rate at startup, and are often manually intensive, as the only immediate solution is to keep restarting the application and hoping that the operating system (OS) will somehow manage other critical apps running on the machine to free up space for the target application. There is generally no immediate patch available, and the solution for refactoring or memory management within an application fix can be costly, as it progresses through the typical cycle of development, Quality Assurance testing, and release—which may also be subject to errors.  Conversely, the proposed optimization utilizing profile building has a smaller release cycle with little or no change in legacy applications, making this an attractive solution.

Additionally, because redundant instruments are being excluded from processing, applications instrument coverage, functionality, and accuracy are not hampered.

The table below highlights the benefits of optimization, observed after excluding instruments/payoffs that account for 30% of the overall population.

Industry benefits

For memory-constrained environments, optimizing processes to reduce the memory load has a related benefit in reducing the operational overhead usually required to maintain the legacy applications. Financial institutions have operational units to report for any system breakdowns, and often legacy systems and/or applications are prime candidates for downtime.  The proposed optimization improves the stability of the systems and ensures lower operational costs for institutions.  Furthermore, by reducing the overall load on systems, optimization can save hundreds of thousands to millions of dollars by eliminating the need to build new systems, when existing legacy systems are plagued by persistent instability.

To enhance the optimization, institutions can couple the profile building solution with a monitoring system that can continuously measure the load on applications, and provide valuable insights into necessary actions that will expand their life, to make them more manageable, durable, and cost-effective  in the long run.

An optimal alternative to replacing legacy applications

In the current regulatory environment, because many of these legacy applications are producing time-sensitive regulatory reports, their stability is imperative. The optimization solution described here aims to keep the legacy system healthy and functioning to meet institutional needs, while providing a tool that enables tech teams to quickly resolve systemic memory issues.   

By controlling which product types feed into an application, this optimization keeps troublesome ones out—helping systems stay stable and online. In the financial industry, where organizations handle multiple and often fast-changing products, that kind of control can be a game-changer when building new applications.

Process optimization using profile building provides an alternate scalable, cost-effective, functional solution to manage high-volume, resource-intensive, win 32-bit legacy processes without impacting the technical aspect of an application. Besides benefits including reductions in memory and manual labor, and quick turnaround, this process also can drive further optimization—and if coupled with a monitoring framework, can transform a vulnerable application into a stable, crash free, and resilient application that is an acceptable standard comparable to any new applications.

Author: Pramod Kumar Singh is a software developer, business analyst, and financial consultant for a leading global financial institution. Based in Princeton, New Jersey, he designs and implements innovative technical solutions to create process improvement across various functions, from onboarding new products to the trading desk and regulatory compliance. He is additionally responsible for managing and prioritizing complex projects and overseeing technology and cross-functional teams. Pramod earned a bachelor’s degree in Technology from the Indian Institute of Technology (BHU) Varanasi, and over nearly two decades has pursued advanced technical expertise in multiple tools and systems including machine learning, data collection, and data modeling. As a Certified Financial Risk Manager (FRM) also certified in Quantitative Finance (CQF), Pramod has developed significant experience working across multiple business lines in the investment banking domain.