Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

While automating multi locale application I had to stop using -

assertTrue(selenium.isTextPresent("success text message"));

where success text message would change for each locale.

to -

assertTrue(selenium.isElementPresent("constant_id"))

where id would be constant for each locale. Though tests works now, I would be losing on checking the text it self for different locales. What approach do you suggest here?

share|improve this question

4 Answers 4

up vote 6 down vote accepted

I have done this one of four ways and I can share my experience and opinions with each of these ways.

  1. Check for non-text to verify messages. Often with error messages or other important informational messages, they are coupled with an error (or message) code that is available as an attribute on the html tag. If they are not, you can ask your developer to include them so that you know every time message ID 1234 shows up that the text is the appropriate text for the situation and the locale. This is fairly inexpensive to implement and gives you a decent probability of success.

  2. Use the resx bundled in the server code to get the locale appropriate text for your automation. This requires finding and setting up a tool or code that will parse the resx file for you. In addition, you will need to pull down a current version of the resx along with your automation each time you run against a new build. This will give you the most complete coverage and ensure you are always checking for the text that the developer has assigned for the locale. The downside is that this doesn't actually test anything that matters. You still have no idea if the text is appropriate for the context and the likelyhood that the text you're looking for is going to be different than what you get from the resx is almost 0 since you are testing the web site with the same data that the web site uses.

  3. Basically the approach that Siva describes in his answer. The problem with this approach is the maintenance cost. Every time a string changes you have to go modify your XML with strings for each of the locales that you are running in.

  4. Verify text in English version, but skip it for other locales. This actually is the one that I would suggest, possibly in addition to 1. Most likely you will run your automation against the English web site once every build. This way you can ensure that the text appears correctly and that it is correct for the context. Testing this in every other locale isn't really going to buy you anything since you don't understand the text anyways. You can still do stuff like make sure that the div (or whatever tag that holds the text) exists and the style is correct (not hidden, etc) as it would be if it were displaying the text. One additional check you can do is ensure that on non English runs the English text does NOT appear. Often if the text has not been localized, it will still appear as the English text (since most localization engines fall back to that).

    In summary, testing each locale exhaustively is not going to buy you much and is likely a waste of resources. You should focus on testing the functionality of the site and hopefully your localization team is doing their job well and testing their own strings in context.
    Another note: if you are relying on text to identify elements, that is a bug and you should ask your developers to instead provide ID's, or find some other way of identifying the element - perhaps using relationships with other elements that do have ID's or other unique identifiers that don't change in different languages.
    One additional note: You can and should still test languages that have different character sets and/or are bi-directional either manually or automated. I honestly prefer to do these tests manually since often the errors you will see have to do with the layout of the page and/or correct rendering of the character sets (if the encoding is incorrect) which are difficult to test in an automated fashion.

share|improve this answer
    
Thanks for detailed answer. I would most definitely take option 4 especially non english locale not having english text, if I had to check for presence of text. –  Tarun Oct 14 '11 at 3:37

This is a common issue that comes up in automated test design.

Since textual strings in forms are likely to change, and if the purpose is functional validation of 'text' for whatever langauge being present then I would recommend using the control ID as you have done.

Although it is possible (and Siva explains one possible way), in my experience validating the content of a string in automation doesn't seem like a worthy pursuit IMHO and is a recipe for constant maintenance and numerous false positives.

If testing the content of a string is important, then you can always call an API to detect the current OS UI locale/language in your test (rather than passing an arg as a param), and then using either fuzzy match, or taking a sample of the string a comparing the char code points against the range code point values for chars for that given locale/language.

share|improve this answer
    
I am going to stick with ids for now. Product owner has agreed to it as well. Thanks for your response. –  Tarun Oct 16 '11 at 5:01
    
Yep, Testing the API would be good enough to validate locale specific content is returned. I agree to this approach (80% Test cases be checked at API Level). Sample testing can be done for basic cases to required check content displayed for locale. –  Siva Oct 17 '11 at 5:44

I acknowledge that it may be appropriate to use automation for verifying multi-language user interfaces. My company's user interface is English-only. I use automation for verifying parts of the user interface, but I verify messages manually. I believe it is important to view each message (and other static text) not just in isolation but in the context of the rest of the user interface, and this is best done by an actual human.

share|improve this answer
    
looks like it becomes one more case of question - "can every thing be automated" :) –  Tarun Oct 16 '11 at 5:00

Yes, We do verify. From automation suites I have worked on, I have observed it.

Ids would be same in each code, Based on the locale we can pass the expected text.

We can Have a function GetExpectedText which accepts Id and locale. It would return the text applicable / to be checked for that locale.

You can have Custom verify Method

VerifyText(GetActualTextBased by ID, GetExpectedText("Id","Locale"))

share|improve this answer
    
what would be cost to maintain such a system? How has you experiences been? –  Tarun Oct 13 '11 at 9:53
    
The logic is - these text content would be changed time to time for a locale / all locales. We read it from XML based on locale, id from an XML file. This method would help you run your test across locales. This would help you to scale your automation across locales. If you pass locale value as parameter to invoke tests. This should be used for picking right strings to be checked for that locale. I have seen it successfully used last couple of years in my previously organization. –  Siva Oct 13 '11 at 10:27
    
So do you maintain those XMl files? During you experience have you found localization defect using test automation this way? I am just eager to know on efforts it needs to localize automation tests and if it is worth it. On another forum I was given couple of suggestions but they sounded daunting to me and decided to stick to text object identification than text it self, you could find it here - sqaforums.com/… –  Tarun Oct 13 '11 at 11:07
    
Yes. There were defects. Example1 - In a particular locale text was changed before feature launch, Example2 - Localization strings were missed in certain locales only English was displayed. –  Siva Oct 13 '11 at 13:43
    
thanks for sharing your reviews –  Tarun Oct 14 '11 at 3:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.