Prior to working with SKIPJAQ, Baking IT, in common with the vast majority of businesses which transact online, had not attempted to optimise its application stack - i.e. the layers of software running ‘underneath’ its application. In Baking IT’s case this meant that it was using default set-ups for its operating system, web server and runtime.
Tuning the settings in these layers of code is always likely to transform an application’s performance because these layers include the vast majority of the code actually responsible for the application’s overall performance. SKIPJAQ tunes these settings with advanced machine learning techniques; it just isn’t possible to tune them successfully by hand.
When measuring performance, it is crucial to look at the whole distribution of latencies (‘latency’ refers to the delay before an application responds to a user’s request) for all of the requests made by all of an application’s users.
Within this whole distribution SKIPJAQ always takes particular interest in the ‘tail latencies’ - the worst or slowest group of response times experienced by our customers’ customers. We are especially interested in the 95th percentile latency, which is a particularly useful measurement for describing the worst user experiences delivered by an application. An improvement to this measurement is a sure sign that an application’s overall performance has been improved.
How do you work out what the 95th percentile latency is? Well, if you had 100 response times (typically rendered in seconds and milliseconds), you would rank them from fastest to slowest, and the 95th ranked time would be your 95th percentile latency.
Before bringing in SKIPJAQ, Baking IT’s 95th percentile latency was more than 45 seconds; a significant number of requests were even slower - at over 50 seconds. Response times of this length have the potential to seriously hurt a business, as users click, swipe, or tab away frustrated - possibly never to return.
Overall, the performance of Baking IT’s application, pre-optimisation, was extremely varied, with users experiencing a wide range of response times. Baking IT had much to gain from reducing latency (i.e. running faster) and from offering its users a more consistent experience.
Baking IT linked its AWS account with SKIPJAQ’s, created an AMI (a snapshot of its application stack), and then SKIPJAQ’s machine learning engine began to search for the optimal configuration for settings in the OS, web server, and runtime layers of its stack.
SKIPJAQ’s machine learning engine used both the Latin Hypercube method and our proprietary Bayesian optimisation technique (which enables Bayesian optimisation to work out optimal parameters for 50 or 60 settings rather than just 4 or 5) to suggest possible configurations for Baking IT’s application stack. These experimental configurations were launched and tested in SKIPJAQ’s AWS account meaning that the cost of the optimisation process was not charged to Baking IT (an option we offer to all of our customers with a compatible stack).
In a matter of hours, SKIPJAQ’s engine had discovered an incredible performance-enhancing configuration for Baking IT to ‘put into production’. To put that in context, the sun would run out of fuel before a human being, working on the problem manually, would have achieved the same result.
When Baking IT applied SKIPJAQ’s recommended configuration to its production stack, the results were immediate and striking. The 95th percentile latency was slashed by over 27 seconds - a crucial improvement of 59%. In addition, average throughput increased by 20%. In other words - Baking IT’s system could handle more users simultaneously and give those users significantly faster and better experiences.
By making use of SKIPJAQ’s advanced machine learning capabilities, Baking IT was able to optimise its application and deliver an experience that continues to enable its customers to run their businesses in an effective and hassle-free manner, and spend more time doing what they do best - baking!
We’re a rapidly growing business, but we have a small team and most of our time is taken up with adding the new features our bakers need to make their lives that little bit easier. Of course, our customers are also just like everyone else in 2017 - they expect lightning quick website and application performance.
What SKIPJAQ offered sounded almost too good to be true - a huge improvement in performance in exchange for barely any work from us - but they really delivered. After applying the configuration SKIPJAQ discovered to our stack we discovered very quickly that what they say is one hundred percent true: application performance is business performance.