2008:SE401:Hypothesized Solution

The goal of the project is to autonomously scale a web service to provide a consistent quality of service(QOS) at a minimal cost. Using Amazon Web Services, it is fairly easy to start additional physical machines, which can hold a replica of the service and can be used to help balance the load upon that service(It may be that bottlenecks such as databases affect the ease of which services can be replicated and synchronized). As the utilisation of these services decreases, the number of physically active machines can be decreased allowing us to reduce the running cost.

We propose to create an application in Java (due to its OS generic nature, native network capabilities and experience with the language and tools held by the developers) to control the replication of a web service. This application will monitor the quality of the service being provided and start additional machines with a replica of the service when the QOS starts to deteriorate. As the load on these services starts to decrease, while maintaining our standard of QOS, our application will also scale down the quantity of physical machines, thus reducing the cost of providing the service (this is charged by Amazon Web Services at an hourly rate for each physically active machine.)

It is assumed that in some cases the load on a service will be predictable, an example of this may be a web service advertising the rate of inflation; it is likely that there would be a lot of interest on the day the reserve bank is to announce the newly calculated rate. In such a case our system could preemptively scale up the number of available machines before the service becomes exhausted by the scale of requests.

To determine the quantity of service, we will likely have to use a combination of CPU usage, latency of messages (in RTT). Message queue length? Ram/Total User Footprint usage.

We want to decentralise our system, so our system itself is not a point of failure. We propose to replicate our application on every machine in the cluster providing the service. One instance will act as a master (assuming the services are well load balanced) and based upon its own quality of service, scales the service. Using a heart beat mechanism to keep track of each instance, if the master becomes incapacitated, then a new master will be elected and will take over the scaling of the service.

To test our system we will use JMeter.

Our system should ...

Problems;

• Delay in server start up times, introducing windows of possibly poor service; this could be controlled by increasing the sensitivity of the system so it starts services sooner, however this increases cost.
• Overriding predictions about the use for a system are hard to determine. Both overestimates and underestimates could be costly (cost of running servers and losing customers respectively).
• Election system, still have no idea how it will work algorithmically.

Report Structure

• Introduction
  • Motivation
  • Requirements of the system
  • Existing solutions
• Main
  • Load measuring techniques
  • Load balancing
  • Testing
  • Possible approaches
  • Look at a similar web system and why ours will be better
  • Our approach
  • Planned structure
  • plug in modules for settings
• Conclusions
• Acknowledgements
• References
• Appendices
  • Gantt Chart