This is the second article in this series – If you haven’t read the first, you are going to want to go back and start here.
In our first article, we got into some of the operational issues SaaS companies face as they try to reach a positive, profitable cash flow. I asserted that with standard technical architectures, SaaS companies will have a serious problem trying to scale and operate profitably.
In this segment, we’re going to lay the groundwork for a solution. The term groundwork is carefully chosen, because to understand the solution, you will have to rethink everything you know about software development for the Internet and SaaS applications in particular. If you are a software developer then, the first question you should ask is, “Why should I care? We’re using proven architectures and building reliable, industry-standard applications. Why are business operations a concern for developers?”
The answer is simple.
The solution is in the hands of the application architect and developers.
For practical purposes, it is an impossible task for a business to specify the operational scaling characteristics of a SaaS application for a cloud implementation. The business can provide strong insight, if the development team can ask the right questions, especially in the areas of client usage, market characteristics, and business automation aspects including pricing, billing, client implementation, etc. But the most important thing developers can do is to specify a flexible architecture that can leverage aggregation in the right places and puts direct costs in the right pockets. If that is done correctly from the beginning – operational costs will be scalable and manageable. If it is not, application redevelopment is inevitable – assuming the business survives to a point where it can afford to redevelop.
But before we get there, what is wrong with traditional architecture and implementation?
In the simplest sense – a traditional architecture like we laid out in the first article can only scale in two areas. The application itself can be replicated and managed behind a load balancer to its optimum performance. The database can be scaled by using zones and database replication techniques. If the implementation is on traditional managed servers, deploying new servers has a cost in both the time and staff required. Decommissioning servers to capture lower costs for relatively short periods of lower user base activity (hours or a few days) isn’t cost effective in traditional managed hosting. It costs more to redeploy the servers than the offset is worth.
The application tier can and will aggregate usage, but precisely because of that, a point will come where performance begins to suffer. This leads to a performance cycle for scaling applications. Performance will be very good right after scaling the infrastructure one step. As we wait for the right time and cost/benefit point to make the next step, performance begins to suffer again. The size of the swings is a function of staff availability and cash to make the investment. Staff costs will always trump hardware costs, at least until scale reaches a point where implementation automation can begin to make a dent in costs. If we’re cash poor and are in a SaaS business, we’re caught in a difficult situation. Poor performance increases churn. Increased churn lowers income. Increasing scale costs money and requires skilled staff.
We have to scale, but we’re on a treadmill going nowhere.
In the early stages of growth, one-off server implementation can be managed. Later, clustering and server replication will begin to make more sense. But the application is still – one monolithic application. If the biggest load in the early part of the daily work cycle is reporting as an example, all other components of the application will both feel the impact of that usage and need to be scaled to have decent performance because they are sitting within the boundaries of the same server box. This accentuates the impact of the build-to-peak strategy. The peak usage of the busiest application component becomes the scale driver for the entire application. If that component happens to be extremely busy during some periods (such as paycheck compute at the end of pay period for a payroll app), the entire app will have to be scaled to meet that requirement – even though the payroll administration component (for instance) rarely has that kind of usage.
There is also another aspect of the monolithic application that we usually don’t consider. Even when it is sitting idle, an application designed for a monolithic implementation has higher memory and overhead requirements just because of its total code footprint. It takes up more resources because it is a big application. We could break it up into smaller components on multiple servers but in a managed hosting implementation that has one big limitation – the cost of operation for multiple servers. The staff and implementation time for each server in the application system multiplies quickly. Managing the network and server availability becomes very complex. It is literally unmanageable without some level of automation. Thus, until the advent of virtualization, there was just no reasonable alternative. As an industry, we locked ourselves into a monolithic box paradigm because of the standard hosting pattern and associated costs.
Ok – Let’s virtualize traditional architecture…
Virtualization does make server implementation easier and faster. But if we are locked into a monolithic application, how much can we actually save? We don’t have a lot of leeway. We can only scale the application and the database. The reason we’re doing that is to keep performance where it has to be to provide good service. And remember – in a monolithic application the most used component is going to drive scaling of the whole application. We going to have to use large instances so we’re going to have more implementation lag and we’re going to have to lead and follow usage more than we would like.
Again – a SaaS business needs to as scalable as possible and drive the cost of operations down as the client base increases to be successful and worth the investment.
In this scenario, we’re going to quickly reach a point where our potential business scalability is marginal at best. We just don’t have enough levers to pull. Scaling a monolithic architecture will reach a point where the cost per client is flat – it isn’t scaling anymore. We can raise the number of servers we’re using but when the number of clients we can handle per server in the application and database becomes regular we’re stuck. Plus, as we pointed out in our first article – larger virtual machines can actually cost more than hosted machines. Depending on the use pattern and scaling plan, the cost/benefit ratio may not work out and even if it does, the benefit is short-lived. You’re still locked into a couple of narrow dimensions for scaling. Traditional architecture plopped on top of a cloud hosting system doesn’t really serve the SaaS vendor as well as it should.
Ok. So what can we do differently?
We need to break out of the box. Maybe we could change things if we used elastic scaling but how? How can we maximize its impact on operational costs?
Let’s take a minute and explain our terminology. In a pure sense, scalability means we can scale up to capture growth at a reasonable cost because we’re not changing the basic system configuration. The traditional architecture we discussed in the first article is scalable. But it doesn’t scale down elegantly and it the number of dimensions it can scale in are quite limited. Elastic scalability means we can scale in two directions – up and down. When you put that capability into all the aspects of cloud hosting you start to have some really interesting opportunities. But you can only leverage them to a limited extent with traditional application architectures because it was designed to avoid using a lot of machines to do different jobs as much as possible. It wasn’t a practical direction to go on standard, managed hosting.
To understand what we could do differently, let’s put more flesh on the bones of our original scenario. Let’s assume we’re going to develop a retail management application to be sold as a service – SaaS. What is the basic functional outline of this application?
- Client market is individual stores of small to moderate size, in chains or not.
- Stock replenishment is weekly and may be adjusted by store personnel
- Inventory is taken twice yearly
- History is necessary for seasonal inventory management
- Stores have standard item management tools (item scanning at registers, for inventory and receiving)
- We think the market will take a price of $100/store/month.
We now have a basic functional breakdown and usage pattern. This is critical in breaking down an application to be operationally efficient. The assumed price is good to know to model our scaling pattern on – but it isn’t critical to know. We can drive backwards into cost into price, but that would assume we know very little about our target market. That is a risky proposition at best, so we will assume we know our market for this application. But what do we know?
- The number of items are limited by the size of the store, but we need to track items – so we’re going to need a database
- We have scheduled usages – on hand adjustment from the registers, order development, inventory receiving, and inventory
- Store personnel have to manage inputs at a high level at least
- There is a workflow because store personnel have review and adjust stock replenishment after weekly item sales adjust stock on hand and again, after inventory is verified.
- The weekly busy periods are going to be when data comes in from the registers and from store warehouse receiving for stock replenishment. That data will need to be applied to the items in the client database to update stock on hand and orders.
- Inventories will be updated twice a year which will set base stock on hand for the next period.
- The target inventory will change seasonally in all or most items. The change will be made by department or store managers.
What can we tell about our back-of-the-envelope scenario from the first article?
The servers will be busy managing inventory on Sunday night through the middle of the day on Monday as item updates come in from the registers for the week and reports are prepared and pulled by users. There will be moderate user input Monday afternoon until Tuesday morning as individual items in the order list are adjusted. Wednesday, input will come in from store receiving to update stock on hand. By Friday morning, all the stock will be on the shelves for the busy sales period. Not all stores will adhere to that schedule, but most will.
From Wednesday afternoon through Sunday night, the level of activity on the servers is going to be very low. At nights, except for Sunday, there will be very little or no activity. A few reports will be pulled. A few stocking levels will be adjusted. A few stores will have a different shipping and receiving schedule. When the servers are busy, they will be very busy. When they are slow, they will be very slow. They will need the IT person standing by on busy days to take care of any issues, but on slow days and at nights nothing more than an emergency will need response.
So – if you were going to put this application on Microsoft Azure or Amazon Web Services (as only two of the many elastic hosting options that are available) – how would you do it? What could you do differently than just scale a application server and a database server? Think about that and we’ll go into one simple solution in the next installment.
That’s it now – go on – think about it.
Recent Comments