3,017 reputation
1629
bio website linkedin.com/in/ethelevans
location Seattle, WA
age 31
visits member for 3 years, 7 months
seen Nov 12 at 0:40

SDET with 5 years of experience in software testing & BS in Computer Science. I'm also not very good at filling out "About Me" sections. I'm not as much of an expert as I'd like to be, so am trying to soak up knowledge as fast as I can while balancing work and "real life".


May
11
answered How do you provide proper tests when there is little documentation
May
11
comment What is meant by “Automated tests don't find new bugs”?
But . . . I usually didn't test the code before creating them. It's pretty rare that the code was finished before I made my tests. I thought that parallel development of tests and code was standard?
May
11
awarded  Scholar
May
11
accepted What is meant by “Automated tests don't find new bugs”?
May
11
comment Testing interactions with external systems
I wanted to add . . . a well-designed fixture can be used for both mock and E2E tests; just make the API service configurable, so you can point it at either your mock service or the real one. This also allows you to test for possible changes to Twitter's API, e.g., what if that method that currently never returns null now does someday? You can't test that with a real API call! @glowcoder, sorry for referencing you in the last comment, I misread your comment :)
May
11
comment Testing interactions with external systems
@Traroth, @glocoder, there is definitely a difference in philosophy between mock testing and standard testing. This article touches on it: stephenwalther.com/blog/archive/2008/06/12/… I like to mix the philosophies: I verify that the API is called correctly in the mocked tests (the majority of my tests), which generate fast tests that can often be run on build and are more useful. I verify that the API does what I expect in E2E tests, which are fewer in number and slower - but I'm covering less surface area, because I covered the rest in my mocked tests.
May
11
comment What is meant by “Automated tests don't find new bugs”?
yeah, I'm picking that up. I'm finding it frustrating, because automated testing can and should be thought of as being - at least potentially - so much more. Anyways, I think I've found a new pet peeve :) Time to go laugh at myself!
May
11
answered How do I convince management that we need a formal QA department?
May
11
asked Is a distinction between “testing” and “checking” useful?
May
11
comment What is meant by “Automated tests don't find new bugs”?
The statement, "If you run a test with the same input, it should have the same output" breaks down for other types of testing besides fuzzing - stability testing often is "run until it breaks", testing features with inherent randomness can't have this feature (e.g., I'm testing a random sampling feature right now), etc. Someone distinguished between "testing" and "checking" once, but I can't find the articles - but I thought that was a useful distinction.
May
11
awarded  Commentator
May
11
comment What is meant by “Automated tests don't find new bugs”?
Thanks, Bruce, I thought that was the distinction you were aiming at. I appreciate your patience in clarifying these terms for me. I think the concepts got conflated because testers so naturally added in semi-random data with their random testing (since it's a fairly small set of changes to the fuzzing tool to have it run semi-random data as well).
May
11
comment What kind of testing should a developer do before passing a build to the QA team?
The one change I would make is to make step 4 be step 1. Do the reviews on the plans first. This allows the developer to build in testability features and helps the tester avoid writing useless tests or missing key areas for testing. A review after coding might also be good, but I think a pre-review is much more important. @glowcoder, with the modification I mentioned, we do this on our (small, Agile) team, and I saw this at Microsoft as well (large, branch-and-merge). Both hire management with experience as senior developers, which seems key.
May
10
answered Testing interactions with external systems
May
10
comment What is meant by “Automated tests don't find new bugs”?
Thanks, I wasn't getting the idea that the bug 'isn't a bug' if it's fixed so quickly that it doesn't really spend any time in the code. Do you have more information / good links to read on 'test data generators'? I'm trying to figure out if the people I've worked with have used the term 'fuzzing' loosely when they were doing what you are talking about at least some of the time, or if there is a greater distinction that I'm just not familiar with. When I've seen people 'fuzz' in the past, they did have x% of the tests be category A, y% category B, etc., where A & B are defined by a regex.
May
10
answered Are your UI automated tests dependent or ordered?
May
10
awarded  Student
May
10
asked What is meant by “Automated tests don't find new bugs”?
May
10
answered Has automated UI regression testing helped you in finding critical defects in system?
May
10
comment How can I report coverage on an n-dimensional test matrix?
I second Tangurena's comment. Drilldown reports are very handy for displaying this kind of information.