[Update 1-14-07: You’ll see a couple notes appended to the review in regards to new information obtained directly from iTKO. With this new information, I hope to re-review WS-Testing in another 4-6 months and report back.]
In short, iTKO touts LISA WS-TESTING as a comprehensive way to test web services without having to write a single line of code. So how does it work? Good question. I couldn’t find any “real-world” reviews and my guess is that you may be having the same problem. Therefore, I decided to post my findings to the web.
Before I got my hands on WS-Testing, I was tinkering with Ruby to test web services — making slow, but sure, progress. During that time, I managed to get LISA WS-Testing because iTKO was giving away completely free and fully functional licenses to the first 50,000 users to download and register the product. Coincidentally, my boss and I were interested in this product prior to the free release — so we were quite happy.
The Good:
• As they claim, the product is “codeless”.
• Videos are available on their website to help get you started. You’ll need them. There’s also an installation/getting started guide.
• It takes a while, but once you learn how to use it, you can generate tests from scratch relatively quickly for new web services. Then again, once you learn to code a scripting language you can recycle a lot of that too, so… this may not be as powerful as it seems at first. Plus, when you’re done, you’ve picked up a skill — learning to some extent, a programming or scripting language. Saying that you’ve learned WS-Testing doesn’t make you look as intelligent and the skill isn’t as transferable to something else.
The Bad:
• It takes a while to get started. Hopefully there’s someone you work with, just as confused, that you can bounce ideas off of because the UI is confusing.
• To compound the problem, there’s no manual, which would be immensely helpful. When you open the product, there’s a link to a manual that’s suppose to be on your hard drive, but alas, it does not exist. Anywhere. This was the first bug I found in the product, by the way, but they’ve since fixed it (by deleting the link).
• WS-Testing isn’t the end-all iTKO makes it out to be. I believe I found a very significant limitation of the product. That is, the ability to dynamically update a complex data type, such as a structure or an array, from one method and feed it to another. Then again, I could be wrong, but it certainly wouldn’t be from lack of time or effort. I posted my findings on their web forum well over 2 weeks ago, and have yet to get a response — and the forum is heavily moderated. So for the time being, I’m assuming I’m right.
[Update/correction 1-14-07: iTKO graciously illustrated that this *is* possible using LISA WS-Testing.]
• I’m not sure how “fully functional” this free version is. I ran into some options that were not available with my version/release of the product.

• Frequent updates — and decent sized, at about 104MB each.
• The only place to get help for this product is from iTKO — I couldn’t find another forum or resource anywhere else on the web. There’s just not a lot of people using this product yet to help each other.
And The Ugly:
So, aside from no manual and a clunky UI, I have a few other beefs:
• First and foremost, WS-Testing is a little buggy. Not super bad, but enough to matter. For example, once I got a firm handle on the app, I would occasionally run into a roadblock, knowing that something *should* be working and wasn’t. If I started a new project from scratch, or just blew away the sections that were not working and recreated them, they worked fine. But not before thinking that I was doing something wrong, wasting way too much time troubleshooting the app, and wanting to pull my hair out. Talk about frustrating. This happened three or four times.
• Second, not only does the product have some significant issues, but so does the website. I’m guessing I found some places on their website that were not yet tested. I couldn’t bring up certain articles due to http 500 errors, I tried to use their “forgot my password” feature for the forums only to get more http 500 errors, and another time I tried to contact them using a form on their website. I included a screenshot of that so you can see the results — plainly, there is no “state” field. Has anybody tested this? It doesn’t appear so.

Of course, no one is perfect, but this is from a company that claims to be an authority on quality and testing. And web-based stuff is pretty much up their alley, right? I mean, these guys look really cool, but… I don’t know… what’s going on here? It seemed contradictory that a company preaching quality in theory was greatly lacking it in practice. Is it reasonable to expect companies like this be held to a higher standard? Were they even meeting the standard??
• Finally, the biggest hurdle to using this product is getting help — at least with the “free” version. The only online “community” which offers the faintest hope is iTKO’s own web forum. You’ll notice that hardly anyone posts there. I found the forum frustrating becuase it took, on average, a day just for my post to appear on the board. If I remember right, only one of my posts got a response and it was from an iTKO employee, but only after I had already solved the problem myself. Imagine if an actual peer/user on the web wanted to respond — it would probably be another 24 hours before that got posted.
So essentially, if you run into trouble and are “stuck”, your best hope of getting help is probably about 48 hours away. That’s assuming someone responds. One time it took OVER 72 hours just for one of my posts to appear in the forum. 72 hours! I mean, come on!! And “no” that did not include a weekend or any holidays.
Ultimately, the forum added no value for me. If anything it took value away because it was a waste of time.
[Update 1-14-07: Good news — iTKO assured me that the “time delay” issues surrounding the forums have been remedied.]
Final Words:
The ultimate goal in giving away LISA WS-Testing is to increase revenue for iTKO — and it’s an admirable move. But I feel I need to see some improvements to be comfortable with *my company* spending money on this product and a support plan.
In conclusion, I’m disappointed. More accurately, I’m a little pissed — I don’t like flushing time down the toilet and at this point it’s hard to say if WS-Testing is a complete loss for me. I might revisit this application in the future (and report back), especially if I’m wrong about not being able to dynamically update complex data types. It will definitely be worth reconsideration when iTKO comes out with a more stable release, a web board that doesn’t take 72 hours to post to (or 24 hours for that matter), and some documentation to help with the UI. In the meantime, it’s back to learning Ruby.