Monday, September 3, 2018

From Hardware to Serveless

By: Alon Cohen, EVP/CTO, Sep 3, 2018

Amazon AWS, Google Cloud, and others are now chasing each other to see who will win the Serverless revolution. Amazon calls it Lambda Functions, Google named theirs Google Cloud Functions, and Microsoft calls it simply Functions. The generic name is Serverless Architecture.

The idea is simple to understand: instead of buying a physical server and placing it in the data center, or buying a virtual server instance, configuring the server, installing the needed software and hoping that it will hold the expected load, a developer can now split the traditional monolithic code to a defined set of business logic tasks that can be invoked by a URL, and invoke those tasks as many times as the application requires without thinking about scale, load balancing, networking aspects and more.

It sounds simple, and it is, once you get the hang of the Dashboard provided by Google or Amazon to manage, monitor and define the proper components you need in your solution, and the component access rights. When the configuration is done, all you need to think about is how to break the monolithic process you had in the past to a nimble set of small tasks.

In fact, there are few programming paradigms that you need to leave behind in order to take full advantage of this new architecture.

In the past developers used to build large data objects to hide information from other tasks and yet provide unified access to the object state and properties. This approach in most cases dictates a large monolithic application with nested data structures that are hard to maintain, requires long release cycles and long regression testing. Every change you make affects large parts of the program and requires you to go through rigorous version management and re-tests.

To mitigate some of the above issues, people built a service-oriented architecture where different layers abstract different functions and allow the developers to make changes in one layer without affecting other layers. However this approach, as clean as it might be, still keeps the dependency of each layer on the other layers and so a change in one layer requires a unit test but also yet again requires the QA to go through a full regression testing of the whole system prior to every release. All of that translates to long release cycles.

Serverless for the open-minded opened up new opportunities. You still want to make sure you write a service once, but if you work correctly you can remove the global dependency of the code on a version of any specific function. In other words, in the monolithic architecture when you change a function, that change also affects every part of the code that calls that function, object or service. That dependency is the key problem. So how can we change that?

Instead of creating abstraction layers or services that depend on other services, you create a shared library of global functions. And here’s the key: when you pack a Serverless Task you include the latest version of the shared library code, test that small Task, and deploy. None of the other serverless Tasks that are running and using that Shared Code are affected. The other Tasks are using the version of the function code they were tested with. In fact, they do not even have to be taken down as you update other Tasks. Most likely, if you built it correctly, a Task will be completely independent of any other Task in the system.

If you adhere to that flat architecture, you will now enjoy rapid bug fixing. You will have a very small blast area, meaning that a bug in a newly updated Task affects only the availability of that Tasks and may not disrupt any other Tasks in the system. You also do not need to maintain a Master Branch for the whole app, but rather manage each Task version independently. Since the program is split into small Tasks, there is very little need in merging code managed by a few people. Most Tasks are written by a single person. And, by maintaining a good set of Code Style conventions in the organization all developers are now fungible, meaning any developer can understand and fix other developers’ code.

How much does it cost?

At we used to handle call events in our Core API in a traditional data center. We needed about four instances and two load balancers between the API service layers, to support a barrage of call state events coming from our telephony servers, millions per day. Once we moved the event handling to our serverless environment, where you pay on a per Task invocation, we reduce the cost by a factor of 10.

Amazon AWS also offers a serverless database called Aurora, which is still in Beta. I cannot wait to see how they price that in comparison to other database options. In the instance-based architecture, one will pay about $70 per month for the smallest database instance before data has even started collecting.

There are more interesting aspects that contribute to system stability in case of a crash. If a Task crashes it affects only that Task, the next request will launch a new instance of that Task and will run the Task again.

Is it Secure?

Security aspects are handled by creating a Virtual Private Cloud isolated from the outside. By using an API Gateway module, you allow public access to specific functions. The API gateway lets you define your own authentication and authorization mechanism.

Additional Gains

Instead of Cron jobs, you can now tell a function to start at given intervals. If you need to run a large report that normally takes a long time to generate, you can create a recursive structure where you split the report to hundreds of small segments each one spawning the same Task all running in parallel. As the Tasks finish they aggregate the report segments back to one large report. All that happens in seconds without the need to consider scalability aspects. An overnight type report is now being produced in seconds.

Once a function finishes it lingers a bit, for free, to allow for faster start time and if not invoked for a while they just die and release AWS resources.

Reduce OPS overhead.

Source: is still in the midst of that transformation, however, we have already harvested the benefits in the form of happy customers sending our employees huge edible arrangements of fruits and thank-you letters of appreciation for the quick turnaround off almost any request they come up with.