Few years back I was super excited after increasing the performance of a system by scaling it. However, there was no real performance gain but we increased the ability of the system to do more. So by scaling an application we increase the throughput while keeping the performance figure constant. The effect of scaling an application can confuse someone into believing that the performance of the system has improved; mainly when she/he measure performance using metrics like transactions per second i.e. when we are measuring amount of things being processed in a given time. Just to reiterate, with scaling we are able to do more transactions in a given time but the time taken to complete one transaction remains same as before. With scaling we also gain on reliability and availability.
On the other hand, we increase performance of a system by decreasing the time taken to complete a transaction. This has a dual affect, by decreasing transaction time we increase performance and also increase the number of transaction it execute in a given time. However it is not easy to increase performance and at times scaling up is a quicker and easier solution.
To better understand scaling and performance increase, consider a system which contain 1 pipe with a radius of 2 centimeter. Water flows through this pipe at constant speed of Y meter per second. This pipe is used to fill a tank of size X and it takes 15 hrs to fill it. Now if our requirement is to fill the tank faster, we have three options.
The easiest option is to buy a better pipe; let’s assume it has less friction than before. With new low friction pipe, the water starts flowing at a greater speed. May be the old pipe was leaking from time to time and required frequent patching, a new pipe also ensures less disruption in flow. That’s an additional motivation for this approach. What we did here is vertical scaling. These types of scaling can be costly from hardware perspective but, can give you a quick win. If you are already using the best pipe you may not even find a better one.
The next option would be to add another pipe or multiple other pipes of any radius and make the water flow through them and let them help us fill the tank. This will speed up the process of filling the tank without improving the systems performance. However, there is a gain in overall performance, now the tank is getting filled faster. Here we did not increase the performance of individual pipes. Also to scale our output in this way, we really don’t need same size pipes; we can increase output even with multiple pipes with smaller radius. What we did here is horizontal scaling.
Lastly, a little harder option: find a way to increase the speed at which the water is flowing through the existing pipe. This too will make the tank fill faster than our initial system. It is not easy to do this, the work is more challenging than the other two methods stated above. This is where we really do performance improvement.
In the software world there are many examples of scaling and performance improvement. When an application upgrades its regular Oracle database to Oracle Exadata database it is vertical scaling. When we migrate an application from an older physical server to newer and powerful physical server, its vertical scaling. But if every few years your application faces performance problems which in the past was solved by moving to newer, bigger and faster machines then it’s time to rethink this strategy. Even though Moore’s Law states that we can expect the speed and capability of our computers to increase every couple of years and we will pay less for them, it is becoming difficult and costly to buy a bigger and faster server.
This is why the world is turning towards horizontal scaling. Spread your computing needs over multiple machines. As we have seen in our pipe example, we don’t need faster or even similar machines to run our application as long as we have many small commodity machines. This is where micro services or micro service based architecture and cloud computing are playing a pivotal role.
Here is an example where many of us have used horizontal scaling: Say you want to convert a non-partitioned table with 60 million rows into a table with 30 partitions. It may take a long time if you simply copy all rows from your original table to a partitioned table, drop the old table and rename the new one. Instead we scale the process by creating 30 different copy script each working on a separate partition and run them in parallel. This way, the copy step will be many times faster.
Lastly, increase the performance of your system. The team working on performance optimization should have a deep understanding of the application. They must have a way to measure performance of each component and visualize the measure in descending order. The team will also need a performance test environment where they have repeatable result. Next they identify bottle-necks, prioritizing the optimizations that will give maximum benefit and optimize them, measure its impact using performance test environment. Most steps of this process is hard and intellectually challenging. However a project solely focusing on performance optimization should be undertaken at least once in 5 years. It will help in keeping the application in shape.
Many years ago while working with IBM WebSphere Transformation Extender (WTX), I learnt that if some data in the input is not needed, don’t even map or parse them before sending them to the next processing step. I guess this is true in every context. Recently, in a completely different and unrelated project we found a huge block of data getting passed to a module and the module was not at all using that data. Just by stopping this block of data getting passed to this module, we got a considerable boost in performance. You will be amazed to see how much time applications spend in preparing data which are eventually not used.
Even though I mentioned that performance optimization is the hardest. It’s not completely true. While scaling an application horizontally I found it equally challenging. It is considerably easy if your system is processing stateless transaction. However, there can be transactional dependencies, caching and obviously monitoring challenge as you start scaling horizontally.
So what should we do to our applications? If possible, it is best to do all in cycles. Every 5 years try to do a performance optimization projects where you reduce all fats that gathered over the years. Think if it will be possible to scale your application horizontally. If it is possible to scale horizontally, start working on it. Most likely no one will talk about vertical scaling instead there will be push to move the application in cloud. If you are on a dedicated physical server, see if you can fit your application into multiple Virtual Machines (VMs).
Performance optimization projects can bring down your applications CPU and memory usage. This will eventually help you, fit your application or part of it into VM/s. As you start scaling an application horizontally, your centralized database is going to become the biggest bottleneck. Scaling a database is one of the hardest part. Space-based Architecture shows how this can be done, once you unlock this for your application your application will join the league of infinite scaling.