The Cloud Bit …

The challenge for many enterprises has been that website performance and load testing is prohibitively expensive, both in terms of kit and automated test tool licenses.  But can you afford not to do it?

With the prominence of event driven marketing causing large load spikes, as well as the web delivering increasingly rich media such as high definition video, testing those non functional requirements is … well, challenging.  There are some well publicized stories of failure in addition to some jaw dropping statistics out there.  Witness AT&T’s catastrophic iPhone 4 launch crash in 2010 caused by a (predictable I think!) surge of customers rushing to get their hands on Apple’s latest offering.  Or consider that NetFlix (US VOD provider) is now responsible for 30% of ALL peak time North American internet traffic.   Let’s test, right?

But how can enterprises simulate that kind of load without bankrupting their company?

Cloud based testing is increasingly being seen as the answer.  Specialist companies, such as SOASTA, can provide infrastructure on demand that enterprises could never afford to build themselves, let alone maintain.  That infrastructure is highly scalable – simulating thousands to the hundreds of thousands of users, even capable of emulating  real world web conditions from different geographies.  SOASTA do it by providing global infrastructure that is highly utilised, supported by automated deployment and their own specialist analytics tools.

My thinking: There are affordable and proven “on demand” performance and load testing solutions out there today which can be provisioned in minutes and save you from the pain and expense of building an environment yourself – assuming you could even get close to affording a representative environment.  Oh, and it could save your business from embarrassing failure too!

The Crowd Bit …

More and more companies are looking at crowd sourced testing solutions. This can be particularly suitable for consumer facing solutions, such as web based and mobile where diversity of users and platforms is prevalent. uTest, a specialist crowd sourcing test company, presented and shared with us some impressive case studies including Google and here in the UK, the Met Office.

How does it work? Typically, a company will prepare a brief (what the application does, what the user profile looks like – browsers, mobiles, language, location) and uTest will source a crowd to test the solution. Rather than specify detailed, individual tests, crowd testing works by:

• identifying and sourcing a suitable set of test professionals which match both the skills, experience, technology and demographics the assignment requires
• throwing a large number of professional testers at the problem from around the globe over a much shorter period of time.
• Allowing testers to decide their own tests within the parameters of the test assignment, using their own experience, knowledge and professional skills.

uTest say that the results are:

• quicker (fewer tests to specify and quick to setup a test cycle);
• improved bug finding (much larger and more representative group of professional testers, who mirror prospective users);
• cheaper – even with larger numbers, the shorter time and global sourcing of resources make this a very cost effective alternative than a large in-house test team

My thinking: As an option, this seems attractive especially if you’re in the usual IT position of having your test timeline “squeezed” by expanding development timelines or suffering with an under-manned test department. Be aware though this is no “throw over the wall” solution and demands the right level of specification and management from your in-house QA team.

Hope this has been useful. Please feel free to comment or contact me if you have any further thoughts or questions. Lastly, a thank you to Stewart Noakes and the guys at TCL for organising another great event.

Simon Hayes

Useful links:

1. TCL is a specialist consultancy in software testing. As a company, their core purpose is to develop and deliver world class solutions in Software Testing that are innovative, structured and professional.
2. SOASTA Tom Lounibos is the CEO of SOASTA and spoke at the TCL event. Check out his blog.
3. uTest Roy Solomon is a co-founder of uTest and spoke at the TCL event.
4. Watch video from the event here

http://twitter.com/mosaicisland

http://www.linkedin.com/groups?mostPopular=&gid=1580797

At Mosaic Island not only have we seen this dilemma in many organisations but we have been successful in constructing the right technical and business strategy for our clients and, perhaps more importantly, have executed that strategy.

Do you recognise this problem set within your organisation?   Why not contact us to learn how we might be able to help – we might just be able to make your legacy systems a thing of the past.

Check out a sample of the action in the gallery below.

We’re delighted with it as we feel it far better reflects our capabilities and style.  A
big thank you to our business partner ‘Common Agency’ who worked with us to put it
together.  Please take time to have a look around and don’t forget to re-visit every
now and then to check up on what we’re doing.