4

If I understand correctly, the only thing that makes this scenario a challenge is the use of the "live feed". Thus, stub out the live feed. Replace it with your own "non-live feed" and you end up with predictable states where your expected calculation results are truly predictable. Your "non-live feed" can be generated by hand, or if you are automating ...


3

I think you have several directions you can potentially take here, based on the information you've given, but all of them require a degree of modification to standard unit testing structures. Some unit test frameworks allow you to identify a data source and will run a given test once per record. If these read each record one at a time, this will work for ...


3

ui element tagging tends to be brittle. The tool you end up using will find elements in different ways. So you'll likely change up based on the tool you use. Hopefully the tool you pick makes it easy to change 'selectors' (find element logic). testers tend not to have some of the technical skills or experience that developers do. Look into using Telerik ...


2

The issues in the question sound pretty typical of Selenium users and development/QA teams in general. ui element tagging tends to be brittle. Automated testing make the most sense when the interface stays the same but the implementation changes. If a page tends to change a lot, it may make more sense to test it by hand, or to test it with a ...


2

using your favorite script language would probably be the best solution, Peter suggested Perl and I'll add Python to the list, both have excellent modules to parse and analyze CSVs and XMLs and a lot of capabilities to help you in your tasks.


2

In my experience, something like a csv compare can be done programmatically without too much of a speed hit - so long as there aren't many errors. The way I'd handle it is to have your baseline CSV files being the data you expect each table to contain, one for each table (excluding any timestamps or other time-sensitive data - you'd need to check that ...


2

The Big List of Naughty Strings is another good one - it's not just diacritics, but a whole lot of different unicode strings and a bunch of other strings that are known to cause problems in different applications. (I am not affiliated - I just use it a lot)


2

A list of data for testing diacritics can be found at http://www.madore.org/~david/misc/unitest/#accents


2

I would add to an answer from Joe that you may randomize data that you send to an app from the live feed stub. With this technique you may be able to find bugs that would not be found if you only used the same data all over again.


1

which I think might make it a lot harder to automate tests like this. You'd effectively have to test that your underlying data is correct, and for games of this scale testing all of the data at a glance seems wildly implausible due to how much it really is and how volatile this data can be. But so is the code base- it's huge and contains a lot of logic. To ...


1

This will depend on a number of factors, including: time/budget, data quality policy of your organization, data quality of the source dataset, complexity of each layer, etc. A general guideline is, as with any testing, to ask yourself "what could go wrong here?" at each step. For example, you could do data integrity at 1 and 2. But if you had a low budget ...


1

In this case, there are two pieces. First, there should be documentation somewhere on what the data in the state abbreviation database should look like. Worst case, it's in the code. That's what you write the tests against. Second, the data you use should be a copy of the production data. You shouldn't be maintaining your own test databases that could ...


1

Another tool that is very popular is RedGate, I have used it in the past with good results: http://www.red-gate.com/products/


1

Talend Open Studio for Data Quality is a great tool for data comparison, data validation and lot of other data testing features. You can upload data from different type of sources into it and perform the checks. But note that it is comparatively complex tool to begin with, so learning curve could be an important factor while choosing it.


1

An answer to your second bullet point, from one project I've experienced recently: Allow the QA department to draft developers as extra testers for UI tests. Allow the developers to spend some time on automating UI tests. Suddenly a lot of boring, repetitive tests got automated. Make the testers/test analysts available to help with unit tests, especially ...


1

I would hope since this is a financial application there is some sort of auditing you could utilize. If not I would ask to see if it could be injected into the app if you can't control the feed in some manner. You should be able to determine what the selected or triggered price was. You can use the total transaction value against the quantity to see if ...


1

Instead of testing with csv / xml level. You can load the data into a database and run queries and obtain accurate results Create a Results DB with Run_Id (ex-ResultMMDDYYY_1) Load the source and destination data into the DB Run Queries for each source / destination tables and create a results table Fetch the data from results table and publish results You ...


1

Maybe you could modify your schema (many times, you may want to automate this), then generate bad data from the bad schema, and validate the bad data against the good schema to see if the data's bad. Let me know what you did for a solution. Thanks! Another thing to do is to generate a "bad" schema from a metaschema, and then generate XML from the ...


1

dzieciou is right, generating usefully broken XML cases you need to separate feeding in plain oldbroken XML and XML which breaks your schema (the application.) The latter is probably the most useful, because it is hardest to guard against, and a test program that generates valid xml using all the known elements and attributes your application recognizes by ...


1

Try using SchemaCrawler. SchemaCrawler is a free and open-source command-line tool that produces text (text, JSON, CSV, HTML) output that is designed to be diff-ed, using standard diff tools. It can compare both structure and data. Sualeh Fatehi, SchemaCrawler


1

I found an open source command line tool that can compare databases, csv, xml etc. http://www.diffkit.org/


1

I would guess that the app uses log files ( if not ask for them) or records the exchange rate somewhere for auditing


1

The ISO 9126 has some kind of templates for its own implementation: metrics that can be used for each quality characteristics, guidelines for the measurement process and usage of the standard. You will need a full version of the standard to get it, because these are rarely described in the comments on the standard. So buy it, or grab a copy in your company ...


1

Purely in relation to ISO/IEC/IEEE 29119 (The New International Software Testing Standard): There's mixed opinion on ISO/IEC/IEEE 29119, but it does provide a versatile framework that can help/suggest structure to a QA/test team. Standards aren't everything - efficient and effective testing should be your first priority. If ISO-29119 is of interest, there'...


1

Based on the information on wikipedia and searching online (there seem to be a lot of diagrams) it seems like the standard tries to set up a model using five (to 8) characteristics that are sub-divided into sub-characteristics. If I were you I'd mindmap the characteristics and sub-characteristics (although you could also just list them in a document) and ...


Only top voted, non community-wiki answers of a minimum length are eligible