Practical Software Testing Resources? 42
rhartness asks: "I've been a software engineer by profession for a few years and a programming enthusiast for much longer. As my experience has increased, so has the size of the projects that I have had to work on. My software testing method involves trying to do everything I can think of that the end user might try to do. Hopefully, this will break the application if there is a bug within my code. The current project that I am working on involves numerous tiers within a smart client environment. Trial and error testing is no longer sufficient — there is simply too much that could happen. Searching the Internet for software testing resources provides an abundant amount of information but it's often quite philosophical and verbose. What are some practical resources that Slashdot readers use for testing your software projects?"
release it! (Score:4, Funny)
TDD, JUnit, FIT (Score:2)
--Robert
First, understand testing (Score:5, Informative)
Re: (Score:1)
Yes. (Score:2)
automatic unit testing (Score:5, Insightful)
You still need to do additional testing, but this will catch the underlying errors that can cause flakiness -- and worse, bad workaround -- at the higher levels in your code.
Automate (Score:3, Insightful)
Re: (Score:2)
Automate, automate, automate. (Score:5, Informative)
Step 2: automate. This isn't optional if you're planning to maintain a project beyond the 1.0 release. For specifics, it depends on your project. Is it mostly a Java app (http://www.junit.org/)? Is a lot of the logic in the database (http://www.dbunit.org/)? Is it a web app, regardless of language (http://webtest.canoo.com, http://wtr.rubyforge.org/ [rubyforge.org])?
Step 3: run your automated tests and laugh at how much easier it makes your life than manual testing.
Re: (Score:1)
Step 2: automate. follow the above advice if you want to create our own harness. But for an easier way check out the commercial tools (trying to be non-blatant, i am a vendor, check my profile)
Step 3: run your automated tests and laugh at how much easier it makes your life than manual testing.
An update to the previous poster.
Re: (Score:2)
Commercial tools are definitely worth evaluating, yes, but as a programmer, I have yet to find one that actually makes my life easier (let alone be worth the usually exorbitant prices). They're great for places like banks, which might have a large budget and relatively low technical expertise, but (generalizing here) they don't sound like what the poster is looking for.
Re: (Score:1)
It is up to the individual development group, but getting the fox out of the henhouse is as important as developing a test plan for accurate results. Programmers are taught to think in a certain way, that is not a bad thing. I can always tell when a programmer developed the test plan, or the subsequent tests. This is not to say that it is all bad. Programmers make great unit testers. But when it comes to wholistic testing, and what your users are going to subject your application to, a pro
Re: (Score:1, Informative)
Step 1.5: show the test plan to someone else who's reasonably familiar with the project being tested. Let them ask questions about the test plan. Add any new test cases they think of (or that their questions cause you to think of) that you f
Hire a tester. (Score:3, Insightful)
While full-time testers are often made fun of, ("Those who can't code, test."), or it is considered a secondary role that developers can perform on their own, there is no replacement for an experienced tester.
Testing is an important part of the development process. If you have an expensive and complex application, do yourself a favor, and hire a test team. Don't try to do it yourself, unless you have no choice. You may do an OK job of it, but it sounds like you don't have the experience to do it properly. This isn't a critique of your skills, it is a fact that you have a different skillset. You don't expect an embedded programmer to develop web apps or vice-versa. Could they do it? Sure. But it will be a long and difficult road as they make the mistakes an experienced person already known how to avoid.
If you have decided that comprehensive testing is important, (and some people decide it isn't), then do the right thing and get professional (test) help.
Re: (Score:1)
I've run across a few excellent QA people. They're extremely valuable if you can find them.
a few I've tried (Score:2)
Of these, I actually like SimpleTest the best so far, but I don't have more than a passing familiarity with any of them.
In terms of code quality, I think I've got more mileage out of goodish coding practices
Another favorite book (Score:1)
TDD and Continuous Integration, also hire testers! (Score:2)
well as Continuous Integration as discribed here: http://www.martinfowler.com/articles/continuousInt egration.html [martinfowler.com].
The next thing you need to do is hire some testers. TDD and CI can help make fantastic improvements in the quality of your code, but you will never replace the time and effort of another human working to break your code.
TDD and Continuous Integration, also hire testers! (Score:1)
Production (Score:2)
Short of 'production', Mercury makes some excellent tools, but they're more for automated regression or load than for full coverage.
Re:Production -still need a methodology (Score:1)
Also, many unit and functional tests don't need a full blown production environment either -at least not until integration testing...
If the production environment consists of $1 million worth of HW and dedicated outbound feeds to banks and
Re: (Score:2)
My take (Score:2, Interesting)
solution. You described the nature of your projects to a degree, but
not the nature of your employment.
Do you work for a company or are you a contractor? Is your
responsiblity just for your code or the application as a whole (which
is another way of asking how big the team is in most cases)? Are the
users of the software someone close to you, you can ask them "does this
do what you want" or is this going to go on the shelf in a store and
you have no idea wh
Re: (Score:1)
Re: (Score:2, Insightful)
Willy-nilly changes are a huge risk to your success. Your management needs to focus on a goal, and let you reach it. It's like running twenty-five miles of a marathon and then having them move the finish line one you: Pretty soon, you drop dead from exhaustion. You can help them focus by making a list of all the stuff that they don't keep
Re: (Score:3, Insightful)
I also agree with jt2190 about the significance of requirement changes. This is absolutely the area where you stand to make the most gains. If your boss does not have professional software devel
A change and a read (Score:1)
But the biggest reason I see for developers not being so good at testing, is that they are of the mind set that they try to show how the SUT does work. Not really in the least bit helpful, that is what your unit tests should have been doing. No, the mind set is 'In what ways does this fail?'. All software fails, you just have to find out how. That is probably one of the reasons you think so much of the literature is of a philosoph
Re: (Score:1)
I have no idea what "SUT" stands for, but testing basic functionality is also important at higher levels than units. Integration testing seems to be the common phrase, though Wikipedia also mentions systems testing (not that I can discern a difference).
write a test plan, and/or hire a pro (Score:3, Insightful)
I'm going to second the notion I've seen in other posts that if your project really is big, there should be a programmer or analyst dedicated to testing it. You can hire someone who is less of a programmer and more of a tester if you want, but you'll get the best coverage if you can find someone with real programming chops who is able to design and implement tests. If you're doing your project right, that QA team will be every bit as involved in every portion of your project as you are.
Without knowing more about your particular project, it's very difficult to suggest appropriate tools. Different types of systems require different approaches to testing.
Verification? (Score:3, Insightful)
Re: (Score:1, Insightful)
Re: (Score:2, Insightful)
In the real world very little software is formally verified. It's very expensive and time-consuming, and the vast majority of programmers don't know how to do it. Most projects just cannot afford it.
You're making too big a deal out of his "software engineer" label. There's no such thing as a software engineer, in
Coverage tool (white box testing) (Score:2)
Ahem... try this (Score:3, Informative)
Just google +"software testing" you will find tons of links.
you might try these guys: http://www.softwareqatest.com/ [softwareqatest.com]
They have some nice links as well.
Or would you rather start with a newsgroup? Go to google news groups and seach on software testing again. Lots of knowledgable and helpful folks hang out on those boards.
BTW, a colorful phrase for "trying to test everything a user might do" is "soap opera testing". Google it, the story behind the name is sort of fun.
Practical resource (Score:1)
The effort to implement something will approximately double, but you'll have a continuously regression tested system/project.
...also... (Score:1)