In WebRTC PaaS, Should Vendors Count Registered Developers or Active Apps?

How should we measure the size and popularity of a WebRTC PaaS?

During his keynote at Signal, Jeff Lawson, CEO of Twilio shared some numbers with the audience. The idea? Stress the point that Twilio is the dominant player in cloud communication API with no close competitor in sight. No one can take that away from Twilio.

I’d like to look at some of the numbers presented, as I hear them and read about them related to other WebRTC PaaS and communication API platforms. The numbers sometimes say very little and can be quite misleading.

So let’s take a view of “Twilio by the numbers” and see what they mean. Please note that I am picking Twilio not because they are doing anything wrong – all vendors act the same – I am picking them because they were the last ones announcing any numbers AND they are the biggest player of them all.

700,000 Twilio Developers

For any communication API platform, 700K developers is the largest. I’d be happy with 10% of that number as my customers.

Platforms tout hundreds of thousands of developers, but this begs a question: How many of them are active?

Matt Pope published a post last week about app stats that can be applicable here. In it he wrote:

An app company that considers downloads as users is the equivalent of an e-commerce company that considers one-time site visitors (who never sign up or purchase) as customers. It doesn’t make any sense.

Downloads are comparable to Developers in our case. The number is meaningless, besides offering some sense of magnitude. The reality is, that a small portion of that number will be active developers. In the same way that VoIP OTT services migrated to MAU (Monthly Active Users) from registered users, so does the Cloud Communication API industry need to shift towards engagement based metrics.

Here are a few suggestions:

• How many of these registered developers logged in to the backend in the past month?
• How many active developer keys have been used to make an API call in the past month? How many did that in production versus sandbox?

1 Billion Voice Minutes

That’s the number of minutes that went through Twilio’s Client (WebRTC voice minutes). A VERY impressive number.

For a platform though, it isn’t enough. Here are a few questions about these minutes:

• How many of them were WebRTC to WebRTC?
• How many WebRTC to PSTN? How about PSTN to WebRTC?
• In how many sessions?
• What’s the average session length?
• Overtime, does the session lengths increase or decrease?
• Is the session length similar across customers using the platform?

Why is this important?

• The metrics that WebRTC PaaS vendors publish today give a hint of the magnitude and the size of their service. That said, it is a far from accurate number:
  • They accumulate history, which may or may not be relevant
  • They lack information about engagement and activity, which is key
• It is time we as an industry shift towards engagement based metrics instead of the measuring the naive values we do today
  • This may hurt some of the players in the market who try to show off big numbers
  • It will help customers make better selection decisions for their platform of choice
• Twilio, as the behemoth in this market, should take the lead
  • They should be the ones changing the tune to a new set of metrics
  • Their numbers are high, so my guess that the new set of engagement metrics will be just as impressive
  • I am sure the industry will follow
• As a potential customer shopping for a platform, don’t be fooled by metrics. Check their meanings and how they skew what it is you think they may mean