Everyone knows that a bug in any software product costs. The question is how much?

There is no easy way to quantify this, perhaps a macro method would be to find the yearly cost associated with QA organization and the cost of time spent by support and developer organizations.

But what about the question of when a bug is found and fixed and the opportunity cost of the time that could have been spent on new and better features leading to new revenue? What about the time and energy spent by countless people in the organization who have to spend time looking at the bug because it became an escalation and they have to “know” about it? Or perhaps more importantly what about the time spent by the customer and the resulting loss of face of the vendor?

As a developer is writing code he/she is not only writing the code for the desired behavior but is also (inadvertently) writing bugs. Most of the issues are (or should be) sorted out while the developer writes the unit tests alongside (or even before) the code. Finding and fixing a bug at the development/Unit Testing/IntegrationTesting stage is fairly easy and is cheapest. One developer or a small team is involved and this time is accounted for within the development budget. Let us call it stage-1

However some bugs remain unknown till a person or a team dedicated to finding bugs (aka QA department) hunt for them. Even though the boundary of these two organizations is blurring, QA will always be pillar in the house of software development. They perform various kinds of tests using various tools and methodologies. Finding of a bug by QA department involves some more people; there are bug reports, bug triages, some exchanges between developers and QA engineers (sometimes acrimonious), re-assessment by developer, simulation, fixing, bug verification and reports generated by QA managers. Going by the increased number of eye-balls, additional processes, intra-departmental interaction, communication gaps, not to mention some heated debates, a bug cost increases here. Even if you just account for an additional QA head involved it is at least 2x of stage-1 in this stage-2.

However, some bugs still sneak past this pillar, this bulwark of quality, and get shipped.

Now comes in new set of people or roles in this stage-3; like sales engineers, support engineers, fields engineers etc. A bug found during interop testing in a field lab or customer lab involves even more people. This means more people have to spend time to report, to understand and communicate on this bug. QA team is called in to simulate this in their lab; developers are summoned to fix it. Overall cost I think at least 1.5x of stage-2

The software then gets deployed in the customer location, running in the real world. Of course there will be bugs, that is the nature of human creation called software. Unlike a faulty bridge or building, software bugs are not apparent to the naked eye (though code reviews help), so after all these stages a new scenario is exposed in field and thus stage-4 bugs are reported. The cost here goes very high. First of all there is a larger set of people involved, a number of people from the customer’s organization, then on the vendor’s side besides the usual suspects seen in the above stages, there almost always is an expensive senior management resource in the escalation matrix. Then there are communication gaps, limited log files, difficulty in simulating the fault in the lab. Usually the development/support teams move on to next releases and find it hard and time consuming to simulate the problem. So the cost in this stage-4 is at least 2x of stage-3.

These stages also have a temporal scale, usually stage1-3 are not that far apart in time but stage-4 is an open ended (from right) scale. The later into the deployment a bug is found the costlier the whole affair is. Support engineers have to excavate an old branch, then find and fix the bug and merge it ahead in later branches. While all this happens a number of people either wait anxiously biting their nails occasionally writing mails or get involved in brainstorming for the fix. It almost qualifies for a new stage: stage-5 with 2x the cost of stage-4.

So if you pay your average developer $Y then you may end up paying $12Y for a stage-5 bug. That is 12 times more that what you would have paid if it were found by the developer. This may look inflated and too simple a calculation but in practice if you do an analysis of a bug at these different stages you will tend to agree with the order of magnitude. And in all of this I am not including the opportunity cost of lost time that could have been spent on R&D and the loss of face in front of a customer which may potentially affect future revenue.

A good testing and simulation tool reduces this cost and improves your overall bottom line. At every stage you need a good functional testing / simulation tool. When developers write their code they can do a better job with unit testing if they have a tool which helps them write regression test suites quickly and efficiently. Most of the time the reason why developers cut corners while unit testing is because they are up against tough deadlines and unit testing almost stands like an avoidable step.

But what if they have a tool like SIPr which (at least for SIP development) makes writing functional unit tests fun. So much fun that developers begin to enjoy writing tests and may even overdo them (j/k). Tools like SIPr make it very easy to create elaborate QA test suites because converting their test plans into test code becomes very simple. In fact SIPr test cases contain all the documentation and so there is “no need” for a separate test plan a simple script extracts all the documentation and create a nice report. The test suites can be run automatically or at a touch of a button.

For later stages also such tools help different set of people simulate various entities involved to isolate the problem. So as in SIPr you can simulate Proxy, B2BUA, UAC, UAS, Registrar, IMS entities, Redirect server etc with ease. What if a network element is misbehaving? No problem with SIPr you can simulate the problem in your labs in minutes, as you can generate any negative scenario without any limitation.

Even the end customer can benefit by signing off on an acceptance test plan, written on SIPr which can be put together quickly and thereby reduce later problems in field.

Even when problems do surface through these various safety nets created at various levels in the form of regression test suites, unit test suites they will be caught faster and solved faster.

If you have a stage-5 bug now, take a moment to try SIPr, simulate the bug with SIPr and see how it helps you to isolate the problem. Then after you finally fix the problem write a regression test case with SIPr. Then look at the regression test and ask yourself if you would have written this test before if you had test tool like SIPr if your answer is ‘yes’ then you would understand the value proposition instantly and our efforts would be rewarded. Even if the answer is ‘no’ then look at the time saved in just solving this problem. Finally we are sure you will enjoy using SIPr so in your moment of joy of fixing a stage-5 bug the SIPr experience would definitely be a cherry on the cake.

SIP is similar to HTTP in structure and a lot of protocol concepts are borrowed from HTTP (and SMTP) work.  A SIP request has a URI followed by a number of SIP headers and then an optional content. This is similar to a HTTP request in structure. However, the goals of SIP are quite different from that of HTTP, while HTTP is basically a protocol for content access and management; SIP is a protocol for two or multiparty session establishment where various type of media can be exchanged. In simple terms SIP is a protocol used to establish IP calls between two end points for voice and other rich multimedia exchange, SIP can also be used for other purposes like exchange of call related information, subscription and notification of state between two user agents, content and capability negotiation etc. In essence SIP is peer to peer while HTTP is essentially client server.
The quest for new and powerful applications have brought the telecom world closer to the web world and has led to opening up of new interfaces to these applications.  For now let me talk about converged applications.

Converged applications

A converged application is one that speaks several protocols and enables a powerful application traversing these disparate protocol interfaces. As an example consider a click to call application, here the application receives an HTTP request and sends out a SIP request (or two SIP request in 3rd party call control situation), when the call connects it may send the result back to HTTP browser in response or even send periodic updates using Ajax. Or as another example on getting a SIP request the application issues a Diameter Sh get request to access some data on the HSS in IMS use case and then on receipt of the data completes the call accordingly. The core idea is that the application itself is capable of managing these different protocol interactions. A good example would be a JSR 289 capable SIP Servlet application that can receive both HTTP and SIP requests in the context of same application session. The application session maintains state on a protocol session basis and allows application to use the information across.
Typically these converged applications are useful within the provider network to talk to various network elements that speak these different protocols. When it comes to exposing a telecom application to outside applications some more safeguards are required and typically would be accomplished in a different framework. However, converged application and converged container would play a key role in these situations as well. This leads us to our next topic mashups.

Mashups

Mashups have been around for a while and are basically enabling exciting new applications on the web. The Web2.0 is finally delivering the promise of webservices where one application behind the scenes aggregates information and data from various other webservices and overlays the data with interesting intersection and association to present a new facet to the user. If you search for the location of your favorite departmental store on their website and see the stores on a Google map then you are seeing an example of a simple mashup where location information is overlaid on Google maps.
The essential constituent of a Web2.0 mashup is a webservices (most likely REST style) API that allows the client (in the above example your departmental store website) to access content (Google map in this example).  The common trend of these APIs is simplicity and ease of use and most of these APIs have chosen REST based interfaces. So it is a simple HTTP request with URL defining the target resource and request method defining the action.
I previously mentioned that mashups typically involve something more than just converged applications and that is because of the security considerations.
If you are exposing your telecom application to be used in a mashup then you will expose it as a webservices  interface (most like RESTful) where the request first comes to an authentication proxy that manages these webservices interactions and after authenticating the user, applies any kind of policy to the request, only then the request may be forwarded to your application. The webservices request may not be sent over as is but may be converted into a simpler HTTP request or even something else like a message posted to a queue towards your application. The way in which these requests are handled internally by the application is opaque to the external client who only uses the webservices API to create a mashup.

What can SIPr do?

Well SIPr as you know is a converged SIP testing framework and it enables you to on one hand manage SIP and HTTP interaction from the same session, on the other hand it is also capable of generating webservices requests. This makes SIPr an ideal choice for testing complex applications that involve mashups.
So if I were to test or simulate an application that involves a webservices exposure layer to a telecom application – say the much used click to call then -

1. I would use SIPr test script to create a couple of SIP UAs that register with the provider registrar
2.  From the same script I would generate a HTTP based REST request using SIPr HTTP APIs for Click to call
3.  I would then receive SIP INVITEs to both my UAs which I will then accept completing 200/ACK SIP flow and thus verify the completion of the call.
4. I could then write another test that checks out other APIs like call forwarding, do not disturb etc
5. I could also access a database from the same script and validate the call flow based on some database interactions.

Bottom-line is that with SIPr it is extremely easy to write converged test cases and exercise mashups with simple scripting and write them fast.

SIPr (Sipper) is now officially launched! I just came back from Scottsdale AZ where we announced SIPr at Broadsoft Connections show. Overall it was a great show with a very focused group around applications. So for one we did not have to explain the role of SIP application server, different type of applications, application components, interfaces etc. to people visiting us, it was a refreshing change from other conferences. People had either deployed telecom applications or were going to do that in short term. We just focused on how SIPr can accelerate their application deployment by cutting down on overall development, testing, interop and actual deployment.

The takeaway from the overall conference was some future looking initiatives in the world of Web 2.0 by Broadsoft, called Xtend program. Basically enabling communication applications to participate in Web 2.0 environment. They have started a developer program around it allowing developers to use and try out the XSI APIs that expose BW platform to Web 2.0.  This could not have been a better time and better place for SIPr because we are industry’s first and only converged application test framework built ground up with application’s need in mind. While SIPr can generate “any” SIP call flow with ease (even involving the guts of SIP stack), we can also create converged SIP test cases from the same test script form where you can run and manage both SIP and HTTP/XML call flow.

So you can easily generate a call flow where you may use an XSI interface to send a REST request to Broadworks and if that results in SIP INVITE, get that in the same test script and validate the end to end call flow. That in my mind is extremely powerful and would facilitate rapid development of next generation applications.