​SaaS Labs, a leading provider of innovative software solutions, faced challenges in scaling their API calls efficiently while keeping costs under control. Seeking to improve their operational efficiency, they partnered with Simplismart. This collaboration resulted in a remarkable increase in throughput by 4x and a reduction in costs by 6x, with average latency now 3x lesser than their previous setup.

The Story of SaaS Labs

SaaS Labs specializes in creating robust, scalable software solutions for businesses worldwide. With a diverse portfolio of clients, they provide services ranging from API integration to data processing, focusing on delivering high-performance and cost-effective solutions. Recently, they hit internal milestones by expanding their customer base significantly, which led to an increased demand for their services.

The Challenge Faced by SaaS Labs

As SaaS Labs grew, so did the demand for their API services. Initially, they used a mix of internal deployments for OpenAI Whisper and Deepgram for their API needs. However, as their load profiles increased, they found the solutions increasingly cost-prohibitive and unable to handle the required throughput efficiently. SaaS Labs needed a solution that could not only manage higher loads but also significantly reduce operational costs.

By examining the load profile distribution, it is evident that speech-to-text APIs need to run for calls having lengths between 5 and 15 minutes. This accounts for 58.35% of the total usage, with smaller percentages distributed across other time ranges. The peak load data shows significant activity every day at Hour 15 with more than 474 API calls and a rate of 0.13 requests per second (RPS), followed by lower but still notable peaks at Hour 14 and so on. With a projected need to reach 1 RPS within the next 4-5 months, SaaS Labs faces a clear challenge in requiring a more scalable and cost-effective solution to handle the anticipated increase in demand while reducing operational costs significantly.

Why SaaS Labs Chose Simplismart

After evaluating multiple alternatives, SaaS Labs chose Simplismart due to its cost-saving benefits and higher throughput capabilities. Simplismart's proven track record of handling large volumes of API calls efficiently and economically made it the ideal partner. Additionally, the flexibility and support offered by Simplismart's team were key factors in their decision-making process.

How Simplismart Responded

Upon partnering with SaaS Labs, Simplismart first conducted a thorough analysis of the existing API usage and load profiles. Simplismart's engineers then optimized the infrastructure to handle the projected increase in load efficiently. The initial phase focused on improving throughput and reducing latency, ensuring a seamless transition without disrupting SaaS Labs' services.

Baseline Metrics before Simplismart:

  • Throughput: 0.004401216163325131 RPS
  • Latency Percentiles:
    • p25: 109.45800447463989 ms
    • p50: 175.56439554691315 ms
    • p90: 417.10812923908236 ms
    • p95: 418.9832783818245 ms
    • p99: 420.48339769601824 ms

Metrics with Simplismart:

  • Throughput: 0.017397074077760376 RPS
  • Latency Percentiles:
    • p25: 29.930317163467407 ms
    • p50: 60.817490577697754 ms
    • p90: 121.80891602039337 ms
    • p95: 122.28273643255234 ms
    • p99: 150.3053272604943 ms

The Results

The collaboration with Simplismart resulted in a remarkable transformation for SaaS Labs. The throughput increased by 4x, enabling SaaS Labs to handle more API calls efficiently. Furthermore, the cost of operations was reduced by 6x, significantly lowering the financial burden. The speed of API calls became 3x faster, greatly enhancing the overall user experience.

Are you looking to reduce ASR costs?