The Cloud, SaaS and the Total Cost of Operations (TCOp) – Part 3

This is the third article in this series. If you are just joining us and haven’t read the previous segments you can go to Part 1 or Part 2 and catch up with us there.

At this point, we have identified a problem with developing an operationally efficient SaaS application on traditional architecture.

• We can scale a SaaS application on managed infrastructure, but it is costly in terms of time and resources.
• We can use virtualization, to ease implementation and allow a degree of elasticity, but we’re hampered by the monolithic nature of the architecture. It requires us to scale the entire application or database to address even very localized component loading or high transaction loads in just one portion of the database. It also results in higher total overhead just because of the size of the application itself.

So, we know the best solution is to break the application down into separate parts and allow those parts to scale as they need to without loading other components. What do we need to do to make our solution as operationally efficient as possible?

Let’s make a few assumptions:

• We will use a cloud hosting provider that gives us access to a range of virtualized services through APIs and web services so we can automate scaling and decommission of services. The API will allow services to be monitored for performance and usage parameters; which will give us a way to manage the application scale programmatically.
• Our hosting provider provides utility-based pricing so we can see benefits from scaling up and down over limited periods.
• Load balancing, fail-over redundancy, and inter-application communication are all transparent.
• We will leverage virtual machine templates so implementation of services will be standard and transparent.
• We will be the account holder with the hosting provider, aggregating costs and billing our subscription and/or utility usage to our clients. This gives the client one bill, keeps us in charge of how we manage operations and accrues the benefits to us as our business scales successfully.

To be successful, there are some key points we must incorporate into our design:

• Application scale management for all components needs to be based in a dedicated instance with an administrative interface that will allow us to adjust performance and scaling rules over time. This is critical. The extra small instance we are using for a component might be just fine at the beginning, but we might find out over time we would be better off with a slightly different size. We don’t want to stop the application or change program code to make that happen.
• Scale management must include monitoring and scaling all the services in our design. It should not be based in just one aspect of instance performance (server memory for instance) that is expected to stand for all dimensions of usage.
• The design will allow for monitoring and metering throughout. Ideally, this should be based in the scale management component so it can aggregate usage patterns across the entire application cloud. We must be able to tell how usage is impacting our operations and use it as part of product management.
• The design needs to allow usage aggregation in ways that allow us to leverage use patterns and functionality.
• Where it is possible to identify direct client costs, our architecture should break out those services and use metering, monitoring and limits to keep them within the expected range for the service package. If a client is buying xGB of storage as part of their subscription we should be able to use metadata to determine average usage, low and peak usage, data transfer and transactions over a period.
• Automation for pricing, billing, settlement, client implementation, user access and role management, support and product management will all be part of the application or integrated from outside services. Operational costs include staff requirements so we want to do everything we can to lower the need for direct staff involvement in ordinary service functions.

For the purpose of this exercise, we’ll use Microsoft Windows Azure as our reference. Amazon AWS and similar services would also work, but differently, and it is easier to understand if we don’t to try to generalize our example too much.

With that in mind – what might we propose for the application we outlined in Part 2 of this series?

You will notice we’re calling this an application cloud. That term does not refer to a so-called private cloud. It is a grouping which allows the virtual servers and services to communicate transparently without having to set up a special network. Using a private cloud configuration is possible, but it raises the cost because we are no longer leveraging the commodity effect of the public cloud.

The component breakdown in our example is quite simple. There are a lot of other ways we could break the application up, but we’ll keep it simple enough that we can see the principal function and usage alignment.

• Web/Application Server – Available full time, scaling on small instance on performance – Provides user access and all direct user functionality. Also receives and stores data uploaded from client sites. Aggregated scaling across all clients and users.
• Scale Manager – Available full time, scaling on extra small instance in zones – Manages component scaling based on performance and usage characteristics. An admin interface allows access to reports, to change management rules and parameters. Aggregated scaling.
• Compute (headless) – Available on demand, scaling with small instance on jobs in queue and performance – Grabs compute jobs from dedicated queue, processes them and stores them as required. Handles many job types so it can be part of aggregated scaling.
• Reporting (headless) – Available on demand, scaling extra small instance on queue and performance – Grabs reporting jobs from dedicated queue, processes them and stores them as required. Handles multiple report job types and stores jobs directly in client simple record storage as well as making them available for users.
• Application Database – Available full time, a federated database for application data on small web instance – scales through federation in various dimensions.
• Client Database – Part of core database federation, but dedicated to each client – scales by moving history to simple record storage after each inventory cycle.
• Queues – dedicated queues for jobs performed by the Compute and Reporting components.

Six components with additional services? Yes, but because we used minimal footprints for each of the servers the minimum monthly operational cost is quite small.

You will notice that the base component set does not include the client database. That is because in this case, we can allocate the cost of each federated client database for inventory directly to the client. We also know that queues, access and storage will have a cost for each client. On Windows Azure, the database will cost $9.99/month and we will allocate $5/month of the client subscription to storage and communication overhead.

The direct cost allocation idea may seem strange at first. But, because we know that each client will require a small database for their inventory – we have leveraged cloud hosting to isolate the transaction load and storage required for each client from the application database. This raises both scalability and the total availability of the application at the level of the client inventory database which would otherwise be a weak spot. We have also limited the dedicated database size by using the inventory cycle to remove data from previous periods and store it in simple table format to reduce total costs. Federation keeps this scheme simple from an application point of view.

So how does this scale in comparison to typical architecture?

You will remember in our scenario, the client register system will send item adjustment directly to the application once a week. Assuming most of this happens on Sunday night, the web/application server will store the data and load the compute queue for the compute server. The compute server will run inventory adjustment jobs, store the results in each client’s inventory database and add a report job to the reporting server queue. The report server will grab the job, build the report and store it for store personnel to access Monday morning.

This activity will scale the application server, compute server and report server but they will each scale separately based on demand. So if the load on the report component is less over all, it will scale less. This kind of lumpy activity will aggregate because these components are shared across all client instances but separately. On Monday, when store personnel are pulling up reports for review, the application component will be busy, but compute and reporting will only have minor usage. Tuesday, users will adjust inventory as needed and send stock orders. On Wednesday, another batch of uploads will come through as store receiving checks in the ordered items. Compute will scale to adjust inventory again and the entire application cloud will scale down on minimal usage from Thursday through Sunday night.

The result is a usage pattern that requires high scaling less than a third of the week. The rest of the time the work is minimal. If some stores have a different use pattern, the system will scale to what it needs to handle those situations.

What is the cost of this type of architecture?

The monthly cost is quite a bit lower overall going from a low of $194 to a high of $1742. It scales directly in response to use patterns from the begining and the total client load based on the performance rules managed by the server manager. This means IT staff is not managing servers directly. They are pulling reports and adjusting performance rules in the scale manager as needed to adjust performance and scaling.  Positive cash flow is being achieved sooner which means more cash on hand over a given period.

The cost for the client database, storage and data transfer overhead is significant but because it is broken out, we can subtract it from the operational costs we are able to manage and can aggregate. Our cost per client is scaling well over the growth cycle.

There is another hidden advantage here.

The application cloud can scale over a very large range without redevelopment. If we want to enhance a component, we don’t have to rewrite the entire application. Each component is independent and can be handled separately. If we decide we have a need for more reports, we could add code to the reporting server and change the instance size without risking the entire application. In the front-end application server, we would only need to add the additional report types for users to select.

The number of servers that each component can scale to is uncapped to keep performance even from the user perspective. We can control the scaling of a component like reporting or compute by changing the parameters for queue performance at different times if we build that capability into our scale manager. So, we might allow more bursty scaling when we have a shorter window of time to complete all jobs. Schemes like this have negligible impact on a small number of servers – but may have very large impact at large client loads. This is one example of why you need reporting and performance administration on the scale manager from the beginning.

This is of course, just an example. But you can start to see, there are a lot of possibilities if we can develop to leverage elastic hosting to achieve operational efficiency. A lot more than we could ever achieve on a traditional architecture.

I hope you’ve enjoyed this series and it will spark some new ideas.

Here are a few resources you might want to review:

Cloudonomics.com – Joe Wienman’s excellent and detailed discussion on how cloud computing creates value. If you read nothing else, set aside some time to digest this nearly book length group of articles. You should also use his ComplexModels.com and see how scaling works in a cloud environment. Remember though, the benefits are strong if a large pool of servers are involved in an enterprise situation or if an application cloud approach is used in a SaaS implementation. They don’t apply as strongly for one application in a traditional architecture.

Cloud Computing = Elasticity * Availability – A great discussion of the value of cloud computing when you leverage it in a way that promotes operational efficiency in applications from a leader in the field of cloud database sharding – Herve Roggero.

Cloud Computing Economics: 40-80% Savings in the Cloud? – An article with links to several other resources from Ray Depena in the Cloud Computing Journal.

SQL Azure Federations: Building Scalable, Elastic and Multi-tenant Database Solutions – From Microsoft TechNet.

Follow Me on Twitter