We are pleased to annouuce the general availability of SIPr 2.0 a major release in the direction of IMS and other important features as below.

The downloads are available from Rubyforge Sipper project website for 2.0.
The downlaods are in the form of platform specific gems for RH Linux, Windows and Solaris.

This is so because there is a native library that is not bundled as a native extension to leverage
multi-core, multi-cpu boxes that are so prevalent in telecom labs.

If your platform is not listed above then you can check out the source from SVN and build SipperMedia binary under /media/ and copy the binary to /bin. Then build the gem using “gem build sipper.gemspec” or rake task

Installation
============

The installation is just a gem installation. “gem install SipperXXXX.gem” where the gem file is your platform gem file.
If you are using an older version of SIPr you can safely uninstall older version using “gem uninstall Sipper” and also remove any Sipper installation from windows using \programFiles\Sipper\uninstall.exe
Please refer to the latest Sipper book from http://sipper.agnity.com/http://sipper.agnity.com/documents.php

Changes since last release
================================

1. Feature to pause and resume the load on Linux is added. CTRL^z is the key used to pause and resume the load.[June 23 2009]

2. The default log directory for non project executables like ssmoke etc has been shifted to a temporary file

3. If Session is attached to a transport, IP and Port (created using the right Session type and IP and Port provided e.g create_udp_session(ip, port)) then it (the transport/ip/port tuple) is used for all purposes and never changed unless user changes IP and Port from the controller. If the session is detached (e.g create_session() or create_udp_session()) then the destination is ascertained by normal SIP rules taking into account SIP URI, pre existing routes, Route headers etc and is also changed/modified as a result of SIP signaling like remote target update during the call establishment or subsequent to call establishment. [June 12 2009]

4. Changes done to simplify SIPr installation. Now SIPr is just one gem install. [June 9 2009] Note the details below.

5. <a>For new projects after the gem install it works out of the box <b>.For old projects (backwards compatibility), just copy the new Rakefile in the project directory root and everything should work fine. (To get the new Rakefile, just create a blank new project and copy the Rakefile).

6. For running from the source directly. (Developers only), assuming no Sipper gem installed on that machine, just Rubyforge SVN checked out and also no sipper in windows ProgramFiles.  <a>.You shall need the SIPPER_HOME env var and point to the root of your SVN check out (one dir above “sipper” or “sipper_test”)   <b>.Also need to copy the commands in your ruby/bin or /usr/local/bin for sgen, srun, sproj and ssmoke. Take the example for sgen as reference “ruby $SIPPER_HOME/bin/sgen “$@” <c>.Both old and new projects should work without any change with source installation.

7. Fix for –n 1 in bulk. In new scheme if rate was 1, with RunLoad option, controller was not running. [May 24 2009]

8. Modified SDP class. Added the ability to add media attributes other than “a”. So far using add_media_attribute_at() function always added a=xxx parameter. Now for all media attribute functions add/remove/get media_attribute(s)_at functions we have an optional third argument that takes the type of attribute, one of :m, :i, :c, :b, :k, :a. The default is :a. So in order to add a “b” attribute on the first media line (index 0) you would do - sdp.add_media_attribute_at(0, “AS:12600″, :b) [May 24 2009]

9. Bug fix for #22249. Recording name now rolls over while bulk testing [May 24 2009]

10. Fix for bug #23943. Added the bin dir in the gemspec. Also included /usr/local/bin in the search path for media (for backwards compatibility). [May 11 2009]

11. If protocol compliance is set to lax then early media can even be established with non reliable provisional response. Of course only answer will work here, offer cannot be made with non-reliable provisional response anyway. [May 11 2009]

12. emit_console is also made a controller directive now. So you can set “emit_console true” in the controller and ALL messages will be printed. This is useful for UAS case when setting on session could be done only when first message was received and so it was not printed. [May 11 2009]

13. Simplified load testing without any modification to the test or controller. In order to run the test in bulk mode, all you need is a new configuration option :RunLoad set to true, or alternatively have a –l (minus lowercase L) flag passed on to srun. You can optionally pass –b <number of calls to run>or –n <call rate in cps> option to srun in addition to –l as before to override the number of calls or call rate, otherwise the defaults are picked from configuration from :NumCalls and :CallRate [April 13 2009]

14. Fixed WaitForSignaling effect as the test was not exiting. [Mar 22 2009]

15. SIPr media API rehash to have simplified APIs for all media access. Also included the past work for voice activity detection. [Mar 17 2009]

16. Added ability to exit from srun (from within the controller code) using SIP::Locator[:Sipper].sipper_exit will also need to require ‘util/locator’ in controller. This is in addition to -rf <n> or run for option from srun commandline. [Mar 17 2009]

17. Added the ability to print messages as the test progresss. Using session.emit_console = true for a particular session. [Mar 17 2009]
18. Added the ability to alter the SDP body (session-lines, owner-version, media-lines and media-attributes)after creating the offer or answer.[Mar 17 2009]

19. Feature #24328. The ACK to non failure response is now delivered to controller, such that it can clean up and perhaps more importantly finish the test in some cases. [Mar 15 2009]

20. Fix for bug #24327, not sending cnonce when no qop in challenge, also the ability to not send qop in the challenge. [Mar 12 2009]

21. Fix for bug #24439, ProxyController::create_proxy_request doesn’t handle Record-Route correctly. [Mar 12 2009]

22. Network Asserted Identity implemented as per RFC 3325. [Jan 05 2009]

23. Support for integrated webserver provided. Now the same controller can also act like Webserver and handle HTTP and SIP requests. The association of SIP and HTTP Sessions is automatic and seamless. A pattern similar to SIP handling has been used to have same look and feel in controllers. [Dec 24 2008]

24. Digest Server authentication added in SIPr. Earlier we just had Digest client support. Now users can also challenge a request from a UAS. Two levels of API added one purely directive based and other gives complete control over challenge creation and response matching. [Dec 21 2008]

25. PIDF generator implemented as per RFC 3863 and example test case as per RFC 3856 added.
pidfdata = XmlDoc::PidfTuple.new(”open”, “sip:nasir@sipper.com”, “0.2″, “Welcome to SIPr World !!”)
r.content = session.create_pidf_doc(”sip:joe@example.com”,pidfdata, “Registration done !!”).to_s
[Dec 19 2008]

26. SIP Registration Event - API to generate reginfo doc and example test case as per RFC 3680.
r.content= session.create_reginfo_doc(’sip:joe@example.com’,0).to_s
[Dec 18 2008]

27. Private header extensions implemented as per RFC 3455 [Dec 11 2008]

28. Privacy mechanism implemented as per RFC 3323, 3325 & 4244. One can apply the complete privacy service by calling a simple API called “apply_privacy()” in SIPr b2bua_controller.
request = apply_privacy(request)
[Dec 11 2008]

29. Sipper Load balancer - a SIP UDP load balancer to scale up the Sipper load, implemented as external process [Dec 10 2008]

30. Included support for TCP transport according to RFC 3261 [Dec 03 2008]

31. Improved interface for verify_call_flow(), the direction is now optional [Dec 03 2008]

[released as patch - 1.1.3.2 Sipper-1.1.3.2.gem Nov 11]

32. Handling of new headers for PUBLISH method and example test case as per RFC-3903. [Nov 13 2008]

33. Fix to speed up the recvfrom_nonblock read from UDP socket on Windows platform. [Nov 11 2008]

34. Service-Route Header implemented as Address Type header. Example test case as per RFC-3608 added. [Nov 11 2008]

35. Helper methods for Replaces header creation and finding session based on Replaces Header value. Rejection of improper Request with replaces header as per RFC-3891 in strict mode. To create a replaces header for a session and find the session from the replaces header.
request.replaces = session1.create_replaces_header
session1 = session2.find_session_from_replaces
[Nov 10 2008]

36. Session recorder file name fix to read the files in order of creation in platform independent way.[Nov 9 2008]

37. Support for PATH Header (RFC-3327) [Nov 9 2008]

38. Support for Media keepalive to keep NAT binding open for media channel when SDP negotiated to Inactive or RecvOnly state. [Nov 7 2008]

39. NAT discovery and traversal code based on config and/or controller directive or session config, helper method for REGISTER [Nov 7 2008]

40. Enhanced HTTP API to support - Basic authentication, PUT request, addition of custom header, arbitrary body [Nov 3 2008]

[released as patch - 1.1.3.1 Sipper-1.1.3.gem Oct 21]

41. Fix to add an empty header (without value) to a SIP message [Oct 21 2008]

Ruby 1.9 is officially released.  http://www.ruby-lang.org/en/news/2009/01/30/ruby-1-9-1-released/    This is a major language upgrade releases and first stable one in 1.9 series, perhaps most important feature is the native thread support and jit like compiler for Ruby. This should lead to perceptibly higher performance particularly for multi-threaded system like SIPr. This has come at a time that coincides well with our release plan as well.  Even though we had planned to release SIPr 2.0 on RUby 1.8.7 originally but now we are looking to releases one verision of SIPr 2.0 on 1.9.1 as well, at least for Linux as this is where we are focusing on for our porting.

The bad news is that the port is not seamless, there are several fundamental language changes that we have to take care of in our code to remove the warnings etc. but hopefully we will have a succesful port in time for 2.0 release towards the end of Feb.

Now that our feature development work is almost over we are focusing completely on performance improvements. Several changes haev already been done that have enhanced the performance in plain call processing and others are underway.

We are marching towards a major SIPr release towards the end of Feb 09. In this release a number of new and important features are being added. In fact in the edge SIPr (as available from subversion) a number of features are already there - like TCP support, Webserver support, Digest Server, SIPr load balancer and a number of RFCs implemented for IMS support. In order to see the features as they are added please refer to the online release notes at http://sipper-wiki.agnity.com/pmwiki.php?n=SipperWiki.ReleaseNotes 

Everyone knows that a bug in any software product costs. The question is how much?

There is no easy way to quantify this, perhaps a macro method would be to find the yearly cost associated with QA organization and the cost of time spent by support and developer organizations.

But what about the question of when a bug is found and fixed and the opportunity cost of the time that could have been spent on new and better features leading to new revenue? What about the time and energy spent by countless people in the organization who have to spend time looking at the bug because it became an escalation and they have to “know” about it? Or perhaps more importantly what about the time spent by the customer and the resulting loss of face of the vendor?

As a developer is writing code he/she is not only writing the code for the desired behavior but is also (inadvertently) writing bugs. Most of the issues are (or should be) sorted out while the developer writes the unit tests alongside (or even before) the code. Finding and fixing a bug at the development/Unit Testing/IntegrationTesting stage is fairly easy and is cheapest. One developer or a small team is involved and this time is accounted for within the development budget. Let us call it stage-1

However some bugs remain unknown till a person or a team dedicated to finding bugs (aka QA department) hunt for them. Even though the boundary of these two organizations is blurring, QA will always be pillar in the house of software development. They perform various kinds of tests using various tools and methodologies. Finding of a bug by QA department involves some more people; there are bug reports, bug triages, some exchanges between developers and QA engineers (sometimes acrimonious), re-assessment by developer, simulation, fixing, bug verification and reports generated by QA managers. Going by the increased number of eye-balls, additional processes, intra-departmental interaction, communication gaps, not to mention some heated debates, a bug cost increases here. Even if you just account for an additional QA head involved it is at least 2x of stage-1 in this stage-2.

However, some bugs still sneak past this pillar, this bulwark of quality, and get shipped.

Now comes in new set of people or roles in this stage-3; like sales engineers, support engineers, fields engineers etc. A bug found during interop testing in a field lab or customer lab involves even more people. This means more people have to spend time to report, to understand and communicate on this bug. QA team is called in to simulate this in their lab; developers are summoned to fix it. Overall cost I think at least 1.5x of stage-2

The software then gets deployed in the customer location, running in the real world. Of course there will be bugs, that is the nature of human creation called software. Unlike a faulty bridge or building, software bugs are not apparent to the naked eye (though code reviews help), so after all these stages a new scenario is exposed in field and thus stage-4 bugs are reported. The cost here goes very high. First of all there is a larger set of people involved, a number of people from the customer’s organization, then on the vendor’s side besides the usual suspects seen in the above stages, there almost always is an expensive senior management resource in the escalation matrix. Then there are communication gaps, limited log files, difficulty in simulating the fault in the lab. Usually the development/support teams move on to next releases and find it hard and time consuming to simulate the problem. So the cost in this stage-4 is at least 2x of stage-3.

These stages also have a temporal scale, usually stage1-3 are not that far apart in time but stage-4 is an open ended (from right) scale. The later into the deployment a bug is found the costlier the whole affair is. Support engineers have to excavate an old branch, then find and fix the bug and merge it ahead in later branches. While all this happens a number of people either wait anxiously biting their nails occasionally writing mails or get involved in brainstorming for the fix. It almost qualifies for a new stage: stage-5 with 2x the cost of stage-4.

So if you pay your average developer $Y then you may end up paying $12Y for a stage-5 bug. That is 12 times more that what you would have paid if it were found by the developer. This may look inflated and too simple a calculation but in practice if you do an analysis of a bug at these different stages you will tend to agree with the order of magnitude. And in all of this I am not including the opportunity cost of lost time that could have been spent on R&D and the loss of face in front of a customer which may potentially affect future revenue.

A good testing and simulation tool reduces this cost and improves your overall bottom line. At every stage you need a good functional testing / simulation tool. When developers write their code they can do a better job with unit testing if they have a tool which helps them write regression test suites quickly and efficiently. Most of the time the reason why developers cut corners while unit testing is because they are up against tough deadlines and unit testing almost stands like an avoidable step.

But what if they have a tool like SIPr which (at least for SIP development) makes writing functional unit tests fun. So much fun that developers begin to enjoy writing tests and may even overdo them (j/k). Tools like SIPr make it very easy to create elaborate QA test suites because converting their test plans into test code becomes very simple. In fact SIPr test cases contain all the documentation and so there is “no need” for a separate test plan a simple script extracts all the documentation and create a nice report. The test suites can be run automatically or at a touch of a button.

For later stages also such tools help different set of people simulate various entities involved to isolate the problem. So as in SIPr you can simulate Proxy, B2BUA, UAC, UAS, Registrar, IMS entities, Redirect server etc with ease. What if a network element is misbehaving? No problem with SIPr you can simulate the problem in your labs in minutes, as you can generate any negative scenario without any limitation.

Even the end customer can benefit by signing off on an acceptance test plan, written on SIPr which can be put together quickly and thereby reduce later problems in field.

Even when problems do surface through these various safety nets created at various levels in the form of regression test suites, unit test suites they will be caught faster and solved faster.

If you have a stage-5 bug now, take a moment to try SIPr, simulate the bug with SIPr and see how it helps you to isolate the problem. Then after you finally fix the problem write a regression test case with SIPr. Then look at the regression test and ask yourself if you would have written this test before if you had test tool like SIPr if your answer is ‘yes’ then you would understand the value proposition instantly and our efforts would be rewarded. Even if the answer is ‘no’ then look at the time saved in just solving this problem. Finally we are sure you will enjoy using SIPr so in your moment of joy of fixing a stage-5 bug the SIPr experience would definitely be a cherry on the cake.

SIP is similar to HTTP in structure and a lot of protocol concepts are borrowed from HTTP (and SMTP) work.  A SIP request has a URI followed by a number of SIP headers and then an optional content. This is similar to a HTTP request in structure. However, the goals of SIP are quite different from that of HTTP, while HTTP is basically a protocol for content access and management; SIP is a protocol for two or multiparty session establishment where various type of media can be exchanged. In simple terms SIP is a protocol used to establish IP calls between two end points for voice and other rich multimedia exchange, SIP can also be used for other purposes like exchange of call related information, subscription and notification of state between two user agents, content and capability negotiation etc. In essence SIP is peer to peer while HTTP is essentially client server.
The quest for new and powerful applications have brought the telecom world closer to the web world and has led to opening up of new interfaces to these applications.  For now let me talk about converged applications.

Converged applications

A converged application is one that speaks several protocols and enables a powerful application traversing these disparate protocol interfaces. As an example consider a click to call application, here the application receives an HTTP request and sends out a SIP request (or two SIP request in 3rd party call control situation), when the call connects it may send the result back to HTTP browser in response or even send periodic updates using Ajax. Or as another example on getting a SIP request the application issues a Diameter Sh get request to access some data on the HSS in IMS use case and then on receipt of the data completes the call accordingly. The core idea is that the application itself is capable of managing these different protocol interactions. A good example would be a JSR 289 capable SIP Servlet application that can receive both HTTP and SIP requests in the context of same application session. The application session maintains state on a protocol session basis and allows application to use the information across.
Typically these converged applications are useful within the provider network to talk to various network elements that speak these different protocols. When it comes to exposing a telecom application to outside applications some more safeguards are required and typically would be accomplished in a different framework. However, converged application and converged container would play a key role in these situations as well. This leads us to our next topic mashups.

Mashups

Mashups have been around for a while and are basically enabling exciting new applications on the web. The Web2.0 is finally delivering the promise of webservices where one application behind the scenes aggregates information and data from various other webservices and overlays the data with interesting intersection and association to present a new facet to the user. If you search for the location of your favorite departmental store on their website and see the stores on a Google map then you are seeing an example of a simple mashup where location information is overlaid on Google maps.
The essential constituent of a Web2.0 mashup is a webservices (most likely REST style) API that allows the client (in the above example your departmental store website) to access content (Google map in this example).  The common trend of these APIs is simplicity and ease of use and most of these APIs have chosen REST based interfaces. So it is a simple HTTP request with URL defining the target resource and request method defining the action.
I previously mentioned that mashups typically involve something more than just converged applications and that is because of the security considerations.
If you are exposing your telecom application to be used in a mashup then you will expose it as a webservices  interface (most like RESTful) where the request first comes to an authentication proxy that manages these webservices interactions and after authenticating the user, applies any kind of policy to the request, only then the request may be forwarded to your application. The webservices request may not be sent over as is but may be converted into a simpler HTTP request or even something else like a message posted to a queue towards your application. The way in which these requests are handled internally by the application is opaque to the external client who only uses the webservices API to create a mashup.

What can SIPr do?

Well SIPr as you know is a converged SIP testing framework and it enables you to on one hand manage SIP and HTTP interaction from the same session, on the other hand it is also capable of generating webservices requests. This makes SIPr an ideal choice for testing complex applications that involve mashups.
So if I were to test or simulate an application that involves a webservices exposure layer to a telecom application – say the much used click to call then -

1. I would use SIPr test script to create a couple of SIP UAs that register with the provider registrar
2.  From the same script I would generate a HTTP based REST request using SIPr HTTP APIs for Click to call
3.  I would then receive SIP INVITEs to both my UAs which I will then accept completing 200/ACK SIP flow and thus verify the completion of the call.
4. I could then write another test that checks out other APIs like call forwarding, do not disturb etc
5. I could also access a database from the same script and validate the call flow based on some database interactions.

Bottom-line is that with SIPr it is extremely easy to write converged test cases and exercise mashups with simple scripting and write them fast.

SIPr (Sipper) is now officially launched! I just came back from Scottsdale AZ where we announced SIPr at Broadsoft Connections show. Overall it was a great show with a very focused group around applications. So for one we did not have to explain the role of SIP application server, different type of applications, application components, interfaces etc. to people visiting us, it was a refreshing change from other conferences. People had either deployed telecom applications or were going to do that in short term. We just focused on how SIPr can accelerate their application deployment by cutting down on overall development, testing, interop and actual deployment.

The takeaway from the overall conference was some future looking initiatives in the world of Web 2.0 by Broadsoft, called Xtend program. Basically enabling communication applications to participate in Web 2.0 environment. They have started a developer program around it allowing developers to use and try out the XSI APIs that expose BW platform to Web 2.0.  This could not have been a better time and better place for SIPr because we are industry’s first and only converged application test framework built ground up with application’s need in mind. While SIPr can generate “any” SIP call flow with ease (even involving the guts of SIP stack), we can also create converged SIP test cases from the same test script form where you can run and manage both SIP and HTTP/XML call flow.

So you can easily generate a call flow where you may use an XSI interface to send a REST request to Broadworks and if that results in SIP INVITE, get that in the same test script and validate the end to end call flow. That in my mind is extremely powerful and would facilitate rapid development of next generation applications.

Recently we ported Sipper to Solaris, the port went smoothly. The only native libraries in Sipper are for media handling (Sipper RTP and RTCP implementation). Sipper is now available on Windows, Linux and Solaris. We have only tested on Fedora / RedHat on Linux but I don’t see a reason why there would be any problem on other Linuces, given that Solaris port was straighforward compilation.

One of the often asked question in interviews (for positions requiring SIP knowledge) is – “What is the difference between a Proxy and B2BUA”

Of course there are several fundamental differences between the two SIP entities, notwithstanding that B2BUA ‘behavior’ is not defined by SIP standard. Both however are used on the Server side either as routing entities or for creating applications.

Another interesting question I have seen is “If or How can we find out by just looking at the SIP traces, if the request was sent by a Proxy or a B2BUA”.

Some people say that Max-Forwards would be 69 or less in case of a proxy while 70 in case of B2BUA, while true in most cases this cannot be the rule because the value 70 is just a recommendation.

Another answer I have heard is that only Proxied requests should have a Record-Route header, this may be true for record routing proxies but not so for non record routing variety.

The (almost) right answer is that in case of a Proxy you will have at least 2 Via headers while in case of B2BUA there should be only 1 Via header, that coming from the UAC part of B2BUA. I said ‘almost’ because nobody can stop a B2BUA from adding another Via header in the outgoing request (though most do not).

Sipper had a B2BUA support for a long time which is simple and intuitive, you can write a simple B2BUA with very few lines of script [http://pastie.org/254732], recently while testing for remote target update tests we added support for a rudimentary Proxy in Sipper. We based the implementation off the B2BUA base class having two Sessions in contrast to a single SipSession in case of Proxy in SIP Servlets. Having 2 Sessions not only enabled us to re-use a lot of behavior but also retained the flexibility that the developers on Sipper deserve. Of course the Proxy is rudimentary and is there for simple test scenarios not involving forking etc. but it works.

Oh and if you want to add another Via in your B2BUA or remove your own Via from Proxy then you can easily do that in Sipper before sending the request…