How to approach towards the powered product

1. First your Org. need to propose the Study phase.Customer should mutually agree on this with your org. for the schedule and the billing the resources during this activity.

2. Need to analyze the existing application from the feature, technology ,architecture , usability and maintenance point of view. Document the existing as well as new requirements which are not addressed by the current application. Document all the issues categorizing them as major and minor.

3. Check that what is the needful support and mechanism presently available to address the existing issues and new requirements.

4. Propose the reverse engineering phase to support the existing application. Propose documentation project for existing application from maintenance point of view.

5. Check if your org. has executed any such projects in the past history. This is to understand the the issues in prior implementation and risk associated. You should able to project the domain expertise needful for this assignment by studying the past project history.

6. Check customer has any budget allocation for new software development and any plans to upgrade / change the existing application.

With all this analysis , you will be in the position to propose new project to the customer. You need to present the case studies ,domain expertise , the cost effectiveness for the solution that your org. is suggesting over the existing products in the market and timeline to the solution. Iterative discussions with your internal teams and the customer will help you to go ahead.

What is the advantage of running the virtual user as thread?

VuGen provides the facility to use multithreading. This enables more virtual user to be run per generator. If the Vuser is run as a process, the same driver program is loaded into memory for each Vuser, thus taking up a large amount of memory. This limits the number of virtual user that can be run on a single generator. If the Vuser is run as a thread, only one instance of the driver program is loaded into memory for the given number of virtual user (say 100). Each thread shares the memory of the parent driver program, thus enabling more virtual user to be run per generator

Regarding testing techniques in the field of space research

mmmmmmmmmm, really a different !!!!!!!!!! see we can test operations on different temprature conditions,and we can test it in minus gravitational force. We can test it further by giving different virus(!!) to it. Virus means infected programs that may cause severe damage,atleast there will be such chance in some % atleast.We can do adhoc testing 2 test operation so that we get to know,how they work in chaotic condiotions.By giving stress we can get to know the exactly that what system do exactly??

DbFit 0.92: Better error reporting and easier regression tests

A new release of DbFit is available for download from SourceForge (http://sourceforge. net/project/ showfiles. php?group_ id=191053& package_id= 224326&release_ id=556930).

DbFit is a free extension library for FIT/FitNesse testing framework. It allows developers to manipulate database objects in a relational style, making it much easier to manage and test database objects than with xUnit-style tests. DbFit supports .NET and Java integration tests, and Oracle, SQL Server and MySQL database engines.

Release 0.92 (2007-11-26) brings:

- Better error handling, DbFit now prints out missing column/parameter names and warns if table/procedure could not be found new Inspect fixture to help with regression tests: DatabaseTest has three methods to automate it: InspectQuery, InspectProcedure and InspectTable. These methods will quickly print out entire query results, structure of a procedure or a table/view in a form that can be easily converted to a regression test, or used as a template for
ExecuteProcedure, Insert or Update tables. support for Oracle procedure and package synonyms Type details are read from SqlServer in a more efficient way,
supporting objects with dots in names and fixing few edge cases where
duplicate records were retrieved with 0.91 null keyword handled correctly in Java

For more information on DbFit, see http://gojko. net/fitnesse/ dbfit

QA Time Estimation

There are a great many people who will tell you there is a "standard" ratio of QA hours to Development hours. Don't listen to them. Anyone who makes that claim probably does not know what they are talking about.

Take for example the difference between implementing a new desktop application for internal use and implementing a new web app that is available to the general public.

In the first case, you have to verify that the application works on 1 machine configuration (the weakest CPU/Memory combination with the supported OS in your company), Security testing is restricted to making sure that the user roles are configured (if you even have role based security, there might not need to be any security testing on an internal app). Depending on a myriad of factors the QA hours for a project like this could range anywhere between 30% and 120% of the dev hours.

In the second case you need to verify that the app works on somewhere between 5 and 50 different OS/Browser combinations with several different Hardware configurations thrown in for good measure plus you now need to start throwing in web security testing where you have someone who knows what they are doing validate that your site is protected at least against the most common hacker attacks. Again depending on other factors for a project like this the QA/Dev hour ratio could be anywhere from 2/3 (66%) to 2.5/1 (250%).

This is just looking at the differences between 2 different types of new applications, some other factors that will influence the QA/Dev hour ratio are...


Maturity of the product,
Quality of the requirements,
Level of domain knowledge of the Dev/Business/ QA team members,
Development model followed (waterfall, agile, etc.),
Breakdown of roles and responsibilities between the teams (different companies have different expectations between Dev and QA) ,
Physical proximity of the Dev/Business/ QA teams (people are more efficient when they work together)
Level of quality needed (Banking and health care industries have much stricter quality requirements than an entertainment application)
Team Sizes (Smaller teams are more efficient than larger ones)
Effectiveness of Change Control Procedures
Scale of the project (Defects increase exponentially with the scale of the project, more defects means more QA time and therefore a higher ratio of QA hrs to dev hrs)
Quality of the Code (The single largest factor)
Quality of Documentation
Availability of test data (only applies to some projects)
Quality of the QA team (actually a much more minor factor than you'd think)


And I am sure there are several dozen more than I'm not thinking of off the top of my head. I have seen projects the ratio of QA hours to dev hours was as little as 1/10 (10%) and as high as 5/1 (500%) .


Now what you can do is over time develop a set of standard ratios for different types of projects at your company. You do this by first accurately measuring your projects for a year or two, then going back, finding reasonable groupings of similar projects and deriving average ratios between the two sets of time then when the next project comes up you can do a relatively simple analysis of which type of project it was most like and use that average in planning. If you do this you also need to keep measuring all future projects to ensure that the averages hold and do not drift too far over time

Approve or disapprove the release

The same situation mostly happens with most of us. I think the most important thing here is QA setup similar to Production environment. This is the right time to highlight the risk factors which can cause the release unsuccessful b/c of unavailability of proper testing environment. This is only one side which can be seen this time, that you can't test the email sending at the time of order placement. What about in next requirement list or wish list, if an other option is require by user which also contain this auto email sending option. What you will do next time?

If the testing environment can't be setup as production environment
Sit with your project management team (top players)...keep the situation in front of them...get the things document (not only risk factors but your suggestion also)....discuss with them....confirm more about white box testing and/or unit testing for complete order placement code...give your standing and take the mutual decision.... and document them against your (QA) response.

Its definitely better to get things document b/c this can only be your evidence if things goes worse. With my experience I always try t

Test Execution Estimates

Even though you can not estimate exactly, but you can estimate the execution time efforts depending on the Function Points in your Test Cases.

You will have to categorize your test cases in 3 different categories e.g. Critical / Major / Minor and then can check timing for 5 test cases from each group. Take average of that and multiply with number of test cases in respective groups.

This will not give you exact estimate, but it will definitely give estimate near to actual time.

I have tried this in my past projects and could successfully delviered my product within given time frame.

I will also recommend to add 10 to 15 % extra time, so that if you get stuck some where or considering some obstacles in testing, you can deliver product within your estimated time.

QA Time vs. Development Time

Well, if this was an interview question, maybe the interviewer was looking to find out how you would think through the problem. As was stated by the moderator and others, there really isn’t a good formula. Different groups have different methods/processes/ tools/models by which they arrive at their estimates. I think it’s safe to make a statement that those methods/processes/ tools/models take into consideration some of the variables that are outlined below.

Specifically speaking to the models, they should be reviewed periodically based on historical data to see if there are updates that are required. In my current organization, one of the items on our plates is to revisit our current estimation model. We’ve got our first release under our belts and are part way through our second release. We have some data to begin refining our estimation model. This will be an ongoing process where the model is periodically reviewed and updated.

Getting back to the original question, if this was an interview question, then you might want to explore an answer within the context of your current role and explain to the interviewer that this is the context in which you’re answering your question.

Just a different way to explore it

There is no formula.There are too many variables: the maturity of the product, the quality of the developers' work, the skill of the developers, the skill of the
testers, development and test environments, etc.. In the end, there is no formula based on developers' time

What’s the difference between Severity and Priority?

Severity rates a defect in terms of its impact on the software under test (SUT) and/or on its environment; Priority (a.k.a URGENCY or CRITICALITY) rates it in terms of its impact on human stakeholders (the customer, users, developers, testers …).

A common (rather subjective) scale of SEVERITY goes like this (sometimes the numbers are reversed):

1. Critical - The defect results in the termination of the complete SUT or one or more component applications of the SUT (or of associated software such as the operating system), and/or causes extensive corruption of stored data. The failed function is unusable and there is no acceptable alternative method to achieve the required results.

2. Major - The defect results in the termination of the complete SUT or one or more component applications of the SUT (or of associated software such as the operating system), or causes extensive corruption of stored data; but although the failed function is unusable, there exists an acceptable alternative method to achieve the required results.

3. Moderate - The defect does not result in a termination, but causes the system to produce incorrect, incomplete, or inconsistent results, or the defect impairs the systems usability.

4. Minor - The defect does not result in a termination, and does not impair usability, and the desired processing results are easily obtained by working around the defect.

5. Cosmetic - The defect is the result of non-conformance to a standard, is related to the aesthetics of the system, or is a request for an enhancement. Defects at this level may be deferred or even ignored.

PRIORITY will be partially based on Severity, but should take into account other factors such as the frequency with which the defect results in failure, the degree of “visibility” of failure (e.g., how many users it will affect, with what impact on their work, and whether it will be visible to external
customers), and project-related factors such as the impact on continued development or testing.

A commonly-used scale of PRIORITY reads:

1. Resolve Immediately - Further development and/or testing cannot occur until the defect has been repaired. The system cannot be used until the repair has been effected.

2. Give High Attention - The defect must be resolved as soon as possible because it is impairing development/and or testing activities. System use will be severely affected until the defect is fixed.

3. Normal Queue - The defect should be resolved in the normal course of development activities. It can wait until a new build or version is created.

4. Low Priority - The defect is an irritant which should be repaired, but repair can be deferred until after more serious defect have been fixed.

5. Suspend - The defect repair can be put off indefinitely. It can be resolved in a future major system revision or not resolved at all.

Define Test Scenario

Test Scenario: Test Scenario is the user workflow in the application.

Example: Checking Mail in Yahoo is a scenario, where user login, check the mail in inbox and then logoff. This application can have 2 different test case one for login and other one for inbox.

So Test Scenario can consist of different test cases.

How keep track of Test cases and Testing done by the testers

Believe in doing the right thing the first time...

Start creating the test wares based on the flow charts (or activity diagram) of the module you wish to test...

This flowchart must be approved by the R&D counterparts.

It is also recommended that you create an integration map of the module with other related modules. This will help you to develop the integration cases. So once you receive the release notes with a bug fix,you know which all integration cases you need to execute.

Traceability matrix is a very useful tool in case you need to keep track that none of the requirements are missed out while writing cases. The flow chart will ensure that none of the paths/scenarios are skipped out.

Maintain the cases in excel so that you can use formulas to track the cases written/executed and even the pass/fail statistics easily... Some of the useful columns for tracking purposes are date of execution,defect id (for failed cases), Requirement number, Test case id

A tipical Test Plan content

Summery
Objectives
Test Approach and Assumptions
Major Test Responsibilities
Features and Functionality to Test
Expected Result of Tests
Deliverables
Test Documentation
Test Data
Interactions with Other organizations
Testing Procedures and Walkthrough
Testing Setup
Testing Procedure
Testing Walkthrough
Tracking and Reporting Status
Test Resource Requirements
Environmental Needs
Staffing & Training Needs
Test Tool Requirements
Bug Reporting Tools and Methods
Bug Reporting Tool Strategy
Bug Classification Strategy
Triage Strategy
Bug Closure Ciretria
Schedule
Risks & Dependencies
Open Issues
Appendix
Appendix A - Test Case Specification Example

ISO Certification standards in Quality Assurance perspective

The ISO 9001:2000 Quality Management System certification enables you to demonstrate your commitment to quality and customer satisfaction, as well as continuously improving your quality systems and integrating the realities of a changing world

ISO 9001:2000 is based on eight quality management principles

customer focus
leadership
involvement of people
process approach
system approach
continual improvement
fact-based decision making
mutually beneficial supplier relationships

whats is testing effort estimation

Effort estimation can be done, based on different techniques available like Function Point Analysis, COCOMO, Use Case Point Analysis, Test Case Point Analysis, Metrics based.

Effort estimation is basically done any of the above techniques for different test activities like Test Case Preparation, Automation Script Creation and Test Execution.

Let me explain the metrics based and very commonly used effort estimation procedure in doing effort estimation for Yahoo Mail application.


Identify the requirements (Login Page, Inbox, Compose, Address)

Classify the requirements in different complexity (Simple, Average, High)

Based on the past experience, metrics will be collected on how much time it took to write test case for simple, average and high functionality. Similarly it is collected for other testing
activities.

Now multiply your complexity with the time factor which you derived from metrics to calculate the effort.

Don't report the total has your total effort, always use buffer time, it various based on the domain, tool and other factors. Example we use 20% on the previous total. This buffer will save you
on risk and other deadline factors

Now add you the buffer time and the effort calculated from metrics, and this is your Total effort for the activity.

Test Case Point Analysis: You can use this effort estimation technique
for Test Automation and Test Execution.

1. Identify the total number of test cases to be automated or executed.
2. Classify the steps into High, Medium and Low complexity based on the
business process it performs.
3. Based on the previous experience (metrics based) on how much time it
would take to execute High complexity steps. You can have the average
time.
4. Multiply this average time with total number of steps and get the
total
5. Add the buffer time with the total to get Final estimation time. This
buffer time will vary based on the application, domain and other
factors.

Most of these effort estimation techniques use Metrics Based, which you are going to calculate the actual time for all effort and then take the average.

Manual Testing or Automation Testing

Here are few points to share on Manual Testing or Automation Testing:

1) Any Automation testing scripts can cover only 60 to 80 % of entire product testing

2) There should be separate team other than testing team to develop and maintain automation test scripts Should never mix testing team and automation test script development team.

3) Automation is good for the products where software product is tending to stable condition and there are marginal feature changes in next coming releases. Here the manual testing team is moved to some other product testing assignment and test scripts are sufficient to check core features of the software product.

4) Automation and manual testing should be of right mix. Manual testing can not be avoided.

5) It is decision of project manager for budgeting the automation test script development.

6) Certainly, there are pros and cons for both manual and automation testing.

7) The decision based on type of the application, type of the resources available[software, hardware and human] , product deliverable time line and most important - the budget.

What is the Bug Free Zone?

The Bug Free Zone is a membership-based community of professionals sponsored by the International Institute for Software Testing providing a wealth of information for software test and QA professionals.

- Get acess to Articles - Software Testing Articles written by practitioners for practitioners

- Get access to Conference Presentations – Complete decks from past PSQT Conferences

- Get access to Test Process Templates - Downloadable templates for different tasks in the test process.

- Discussion Board - Communicate and Interact with other QA and Test Professionals.

- Download papers on different Testing and QA Topics

- Get access to the IIST Unified Testing Terminology

- Listen to Industry experts offering software quality and testing tips on a regular basis

- Post jobs or look at job postings

- Get answers to many questions you may have on different testing and QA topics

Membership is free at http://www.iist.org

Development hours Vs QA hours

Variable that change ratio of development hours to QA hours.

• Are the QA and Dev teams equivalently skilled between the 2 projects?

• Are the QA and Dev teams equivalently knowledgable between the 2 projects?
  

• In the business envrionment similar between the 2 projects?
  

• Is the Development model similar between the 2 projects?
  

• Is the structure of the 2 projects similar? (testing an engancement to an existing system is a vastly different animal from testing new development)
  

• Are the breakdown of responsibilities for each team the same between the 2 projects?(maybe in 1 Dev handles Configuration Management and deployment, in another QA Does for example)
  

• Do both Projects have similar data requirements? (testing a system which has a large database behind it is vastly different from tesing one with a rudimentary one)
  

• Do both Projects have similar target audiences? (Code intended for trained specialists has different requirements from code intended for the general public)
  

• Do both Projects run in the same sets of operational envrionments? (Hardware, OS, Browser, JDK, .Net Framework, etc.)
  

• Do both projects support the same number of sets of operational envrionments

And that is just off the top of my head I have come up with 10 variables that will vastly change the ratio of development time to QA time and I'm sure there are at least another 10 I've missed.

So really the discussion needs to be not on what percentage of development hours should QA take but rather how do we accurately estimate how long it will take to test a project. We also need to be vigilant and immediately challenge ANYONE who mentions a ratio between Development hours and QA hours in any context except "Well Project X is similar to Project Y and in Project X QA time was Z% of Dev time so in Project Y the Ratio should be close to Z%".

open source bug tracking systems

Only few open source bug tracking systems are known to me.

1. mantis – which is easy to use comparing to bugzilla and easy to set up.
   

2. Eventum - customizable open source bug tracking system can be used in matured QA process.
   

3. bugzilla - open source bug tracking system with web based interface and setup takes little more time comparing to mantis. Advance report generation option is great.

Define integration testing

When one piece of code, or a unit, needs to interact with another, it's an interface. When we are working on interfacing the two units, we are integrating them. Testing this integration is called an integration testing. It follows unit testing and is before system testing. Therefore it is primarily a white box technique. Please see http://en.wikipedia .org/wiki/Integration_ testing However integrating the GUI with the underlying functionality is also integration testing and is best done as a black box test

Quality Processes

In general, a quality process is one designed to manage quality by the application of quality assurance (methods, standards), quality control (measurement and correction), and quality improvement (process improvement). More specifically, a quality process is a process of conducting any of the above.

Measuring the performance of a Software Quality Assurance Engineer using Bug count

Some companies evaluate their QA resources on number of defects reported per day. I would say that this is totally irrelevant measure to judge any QA guy performance coz QA guy performance have a lot of other things beyond reported bugs. For example, effective Planning, documentation, handling issues among teams, to resolve the bugs and any kid of issues among QA department and Dev department, analytical skills etc.

Bug Severity vs Priority

Severity - refers to the effect the bug has on the functionality of the application

Priority - refers to how soon the bug has to be fixed.

If we term the priorities as P1, P2 , P3 and P4, where P1 is the High priority issue and P4 is the low priority,

we can define the Severity as follows.

P1 - Complete loss of service(of the application)-
showstopper

P2 - Severe loss of service- business logic issues

P3 - Minimal loss of service - which will not effect the core business logic but still must be resolved

P4 - Needs improvement - all the UI issues.

Most of the times, a tester assigns the severity and Leads will assign the priority

QA Engineer / Tester Vs Developer

As for the salaries. It is true that Developers make more money than an equivalently skilled an experienced Tester/QA. There are several reasons for this.

First: In Testing there are a great many tasks which can be done by someone with very little technical skills or knowledge (But I always belive every tester should be technical). Indeed in some envrionments there are manual testing tasks which can be done by those who simply lack the intelligence necessary to work as a developer. There are no equivalent development tasks.

Second: Testers do not produce revenue Developers do. Quality is an impossible concept to sell on it's own and nearly as difficult to market. Outside of QA Outsourcing firms no company has ever made money off of it's Quality department. The Developers are the ones who make the products the company sells, all the Quality department can do is act as a "Force Multiplier" we can make other departments more efficient and effective but we can not produce revenue. So when it comes to allocating money for salaries to different departments it is the departments who actually generate revenue that get the bulk of the money.

Third: Generally speaking 1 tester can test the work of more than 1 developer, there are occasional exceptions where the test team has more manpower than the development team but in the overwhelming majority of cases there are fewer testers than developers. Add to that the fact that there are more people capable of being a tester than of being a developer (it requires much less training to begin working as a tester than it does as a developer). Simple supply and demand rules will then have testers making less than developers all other things being equal.

Now the one area where testers have an advantage is that at the start of our careers it is much easier to move up in the ranks quickly, especially if you have a degree in some form of technical field and knowledge of one or more development languages which is probably why some have reported that they are "ahead" of their counterparts who started with the company at the same time but took a developmet path.

How to negotiate your salary

I found this nice site which explains salary negotiating and evaluating other aspects of job offer, really interesting article. Negotiating salary until meet the expectation is hard to done by any novice to the industry but this letter has address that issue very comprehensively.

Negotiate salary

What is the minimum time frame needed for regression testing before go-live

Seriously. We (as in test/quality/ whatever we are called where you work) do not define timelines. We do tell "The Business" what we can do with the time allocated, how much time we think we need to do the job properly, and what the risks of cutting corners are. But whether to accept that level of risk is a business decision and not a quality one. When asked to do testing such as you are it is imperative that you get actual sign off from your superiors and preferably from the customer (whoever that is in your environment) that the level of risk involved is acceptable and this means you need to give them a proper assessment of that risk. It does not mean presenting a doom and gloom scenario that everything will fail because it has not been "fully" tested (of course nothing is ever fully tested), it means honestly and clearly assessing how risky each change is to introduce defects in other areas of the code, what is the worst that could happen, what is likely to happen, and how likely is each to happen. Once you have provided that assessment and gotten signoff that the level of risk is acceptable then you have done your job and no one can fault you if serious defects are introduced into the production environment.

Automated performance and load testing of web applications written on GWT/AJAX

Resource that will be helpful for selecting automated open source performance and load testing tools that support for AJAX/GWT

http://www.testingfaqs.org/
http://www.opensourcetesting.org/
http://www.qalinks.com/Tools/Automated_ Testing/
http://www.aptest.com/resources.html
http://www.softwareqatest.com/qattls1.html

The Software Test Tool Evaluation Centre ( http://test- tools.technology evaluation. com/ ) provides numerous white papers, and a free research and evaluation service (to be found at: http://tinyurl.com/lsyfj )

Got senseless message

Normally developers put “test” message boxes to check their application behaviors. Most of the time they send their application to testing department without removing those messages (in my company). One of our testing lady has reported a bug for this kind of test message. She has written “Senseless message appeared while pressing this……” Any idiot who has development experience at lest from academic project, knows this is a message put to test the application. As testers we can report this kind of bug in more polite way rather than hearting developers. I always believe QA/testers need to be technical user. Then they can easily understand the technical and non-technical things easily. She could have report this bug more politely in such a way “Application testing message appeared while pressing this…..and this message need to be removed”

My annual Salary increment

Today I submitted my annual appraisal form to Administration/HR department. I don’t know how much salary increment I am going to be getting. But I can be satisfied with the way developers and QA guys are treated. They use self and Management rating for some career attributes. Then based on the final rating our salary increment will be decided. Most of the companies give more salary increments to their Developers than Testers. If we score high rating definitely we will get an increment than developer. I believe QA guys are important same as the developers. Because they have the more responsibility with qualifying a product. I feel satisfaction by working in company who treats same to the QAs and developers.

Application Crashing-Assumptions

Application Crashing is a Severity Level one error and there is no way of accepting any kind of explanation to it. Here are some suggestions to counter that:

QA/tester Assumptions:

1. Application should be stable at all times no matter what path or action(s) performed by the user.

2. Application should treat the user in a way that the user learns from the actions about the right processes.

3. Application should log out the user or create an exception that would lead to an error page.

Testing perspective:

1. Create Test Cases based on activity logs captured if it is a .NET project. If it is a Java/J2EE based we can do handle that in session tracking.

2. Try getting on to the application from an unknown user and see if security features are met.

3. Check the work flow of the application and identify alternate paths if identified new, handled by the development team.

Our new General Manger

Today we had a meeting with our new General Manager. It was a really interesting discussion and he gave some valuable advises to us.

1. Don’t give surprise excuses. – if you are unable to finished testing on given dead line. Don’t wait till the final date comes. If you feel that you can not finish within given time tell it to the responsible persons. That is better than hiding. Give any reason that may be your fault or some other one fault.

3. Always give your suggestions for process improvements. Based on your suggestions you can grow as well as organization can go.


5. His strategy is good as a Manager he has discussed our QA related matters with us. Most of the companies General Manager is not coming to discussions with minor staff. Where you can find the problems is from minor staff not from the staff immediately below you.

6. I was unable give any ideas but he was able to get the suggestions I thought from other QA guys. He is great. (At that time I was analyzing that person.)

QA Salary Surveys Sites

Australia QA Salary Scale:
http://www.careerone.com.au/webdav/site/jobs/shared/Salary%20Survey06/InformaionTechnology.pdf

Every region has its own, and usually HR managers have access to them
http://www.stickymi nds.com/sitewide .asp?freetext= Salary+Survey

Differant Estimation Techniques

Here are some ways you can estimate the testing effort

-You could compare the project you are estimating to a completed project that is of similar size and complexity, then base your estimate on some percentage of the effort it took to test that previous project

-You could estimate the number of test cases (or function points) in the feature you are testing, then multiple that number times the average effort hours per test case (or function point) establish. The average effort hours are determine from metrics gathered from previous
test projects.

You could also base your estimate on number of requirements, using either of the above methods

Testing summary report

Thing should be included in validation summary report

1. short description of the scope of the test effort for the product
2. test environment
3. personnel
4. hardware
5. failure count
6. severity of the failures
7. script error count
8. assessment of outstanding issues
9. assessment of products readiness for release

Web applications QA effort estimation

Somewhere between 1 hour and 10,000 hours.

An example where it is 1 hour......

Requirement: Take input file provided by client in EDI format and transform it to a standardized XML format

How do I test this in 1 hour? I build a couple of EDI Documents to serve as input and then a couple of more XML Documents the are the expected output of the conversion. Code is complete and delivered, I run each of the created input files through the app and compare the output with the expected output document I created. The creation of the input and expected output data is not part of test execution but rather of test planning so I can complete all testing of this 1000 man hours of development in 1 hour with about another 200 - 300 hours of test planning.

An example where it takes 10,000 hours

Requirement: Convert existing PHP based Web application to a Java based Web Application.

Why does this take 10 times as long to test as it does to code? Because the developers only need to write it once, I need to test it on many different machine configurations. If this is application is in a consumer facing website then I would realistically have to test it on some combination of ...

Operating Systems: Win 98 First Edition, Win 98 Second Edition, Win ME, Win 2K Pro, Win 2K Home, Win XP Home, Win XP Pro, Win XP Media Center, Vista, Mac OS 7.6.1, Mac OS 8, Mac OS 9, Mac OS 10

Browsers: Netscape 7, Netscape 8, IE 5, IE 6, IE 7, Firefox, Mozilla, Opera

That alone comes to about 62 possible configurations and I havn't even touched different service packs, different security patch levels, different Java versions, etc. realistically I could NEVER test on all of the possible meaningful different machine configurations. With this project I could easily spend 10,000 man hours testing and another 4,000 - 5,000 man hours test planning for only 1000 hours of development.

So as you can see the very concept of an "average" amount of time to test something is irrelivant, every project and product and envrionment is different and average times from other projects, products, and envrionments are irrelivant

The QA Catchall: How Quality Conscious is Your Organization

Better Software magazine April Feature Article

by Alan S. Koch

Years ago, I gave a presentation that I called "A is for Assurance: A Broad View of SQA." It explored the variety of activities that are necessary to actually assure quality. During the question-and- answer period, I had this conversation:

Audience Member (AM): I would love to do those sorts of things, but I can't see how I can get them to happen in my QA department.
Me: What is your role?
AM: QA analyst
Me: And what do you do?
AM: I develop test plans and test cases, and I test our product.
Me: And what are the other quality roles in your company?
AM: The QA manager manages the testing.
Me: Sigh! (the heavy variety)

I didn't have the heart to tell that audience member what I would say today: "You don't work in a QA department, your manager isn't in charge of QA, and what you do has nothing to do with quality assurance."

My talk resulted in frustration for people like that audience member because the software industry has made achieving quality a frustrating job. The QA people I have met and worked with are excited about quality, but they are usually in positions where they have little opportunity to assure quality.

Read more: http://www.stickymi nds.com/BetterSo ftware/magazine.asp?fn=cifea

Average Time to Test

if it takes 1000 hours to code to one requirement - that's a heck of a requirement - probably needs to be broken up into several requirements, so it makes it really hard to estimate based on
number of requirements (unless they are all that big).

Also, estimating based on the estimated number of development hours is sometimes the only (somewhat) good method to estimate if you still don't have a good idea of the functionality being coded. I've used the 30% rule before, but made sure that EVERYONE was aware that that is an INCREDIBLY rough estimation method. As soon as I get more information on the functionality and can start estimating the total number of test cases needed for new functionality and regression, a new estimate would be provided. When you can estimate the number of test cases, then you can estimate how long it will take to create and execute those test cases. And don't forget if you have to execute test cases more than once for bug fixes, etc needs to be part of the estimate too.

But, an important point to any estimation process is you can only base the estimate on the information that you have to date. As you get better information, the estimate for test SHOULD ALWAYS be updated.

Business requirements analysis

For Black Box Testing you need to read & Analyze the requirement documents.

Let's get a few things straight here. Black box testing has nothing specifically to do with reading requirements or writing tests based on them. In fact, neither development nor black box testing need formal requirements to proceed. With that said, if you are using a development model that relies on requirements, and then yes, read them, and yes, use them to create your tests.

If you go Analyze the requirement documents you would get a clear picture of the Project & the Test Coverage would be good.

2. According to me, Master Test Plan should be written based on the Project Plan and it could be revised as and when needed.

Again, no test plans need to be written. In fact test plans, and written test cases do not have to write any formal testing documents. I have seen perfectly good "test suites" written on sticky notes or a white board.

As I have said many times before, don't waste time on documents that are not

demanded by your management, clients, or development model.

3. To determine whether a requirement is Testable or not is only by participating in the Requirement review / discussion meeting. The Reason is that after getting signoff from the Customer its not good to go back & say this requirement is not testable etc... Example: Scenario 1. Data for a Transactions should come from an external application like SAP / HL7 messaging etc. So, what we need to do is that create a Text file or XML file in that format and feed it as data for Transactions as we don't have SAP / HL7 messaging.

Test Planning

The Testing process follows in the company differs from company to company. The management will adopt an process which will suitable to their company.

And the most common Process used most widely in the market is :

1.Planning stage.

2.Preparation Stage.

3.Eecution stage

4.Completion stage.

And for the Black Box testing , we need to follow the base line documents to carry the testing. without these docs we can carry the testing like Adhoc , Comparison and Exploratory Testing.

As a Test Engineer we will not involved in the preparation of Master Test Plan. This will be prepared by the Managers. Based on this Master Plan, we will get the Test Plan for our Module which will be prepared by the Leads will be the one of the Base Line Docs for our testing activities.

If it is understandable by the reader or the user, then the requirements are testable.

Reporting bug in more polite way

Today I had an argument with two developers in our lunch table. The argument was related to Bug reporting. I am always tried to see a bug in developer’s perspective. So my development friends are always keen to discuss there QA related matters with me.They were argued about the polite bug reporting ways.

One of our QA team members has reported a bug and in the last line of the bug description she has said “This conclude that the system is not stable”. I will never agree with this kind of comments in bug description. The development guy told me that’s why QA department is there. If the system is stable in the initial step then we don’t have to worry about the client releases. He is correct. Unfortunately I don’t have a way to advise this team member.

I believe developers are working hard to make the things working. We need to respect to there thinking power and effort. But I don’t accept doing careless mistakes in project. Finally I agree with my development friends regarding this matter. Get the bug fixes done from developer is an art that should be practiced by QA. We need to find bug but need to report those in very polite way without hearting developers.

Some Useful links for Software Testing Automation

Links to find resources for automated testing

Test Automation Snake Oil - James Bach
http://www.satisfice.com/articles/test_automation_snake_oil.pdf

Improving the Maintainability of Automated Test Suites - Cem Kaner,
Quality Week ’97.
http://www.kaner.com/pdfs/autosqa.pdf

Techniques and ideas for automating software testing - Bret Pettichord
http://www.io.com/~wazmo/qa/#test_automation

Architectures of Test Automation - Cem Kaner
http://www.kaner.com/testarch.html

Totally Data-Driven Automated Testing - Keith Zambelich, SQA Testing
http://www.sqa-test.com/w_paper1.html

Seven Steps to Test Automation Success - Bret Pettichord, June 2001
http://www.io.com/~wazmo/papers/seven_steps.html

Test Automation Frameworks - Carl Nagle
http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFrameworks

Using Silk Test as a Automation tool

In general I am not a huge fan of the Prepackaged Test Automation Suites, however if you are going to use one I find that with a few exceptions Silk is the best one to have.

As far the best approach, Use a Keyword driven Test Methodology.

This is one step beyond a data driven approach and truly minimizes the maintenance overhead of the test automation infrastructure.

With Data Driven testing you have a test case which is a series of steps which are set by the code, but the data entered at each step is parameterized so that you can run the same test multiple times with different data.

Keyword driven testing takes the sequence of the steps themselves and removes them from the code the same way that data driven tests removes the data from the code.

In Silk you do this by controlling the names of the objects in the frame file. Come up with a naming convention so that you can predict the name of any object based on the window it is in, it's object class, and the user understandable name of the object.

You then "teach" your code how to associate keywords to specific objects, then you build a database or spreadsheet format that lists keywords that instruct the script what steps to take.

So for example, a simple login screen, that has a Username field, a Password Field, and a Submit Button, the objects would be named

Login_TextField_Username
Login_TextField_Password
Login_Button_Submit

One "row" of data in the test data would define one step in the testcase, it requires 5 pieces of information, The screen name, the object class, the name of the object in the screen, the action you which to take (generall this will correspond to a method available to that class, however it can be something custom too), and the data to apply.

The data for logging into the system would be


Row1

Screen - Login
Class - TextField
ObjectName - Username
Action - SetText
Data - ABC123

Row2

Screen - Login
Class - TextField
ObjectName - Password
Action - SetText
Data - "Passw0rd"


Row3

Screen - Login
Class - Button
ObjectName - Submit
Action - Click
Data -


The Automation code will then read your test data repository to not only get the data to use but also the sequence of steps to follow.

GUI Testing Checklist

GUI Testing Checklist

Here are some sites that give comprehensive checklist for GUI Standards. Hope this will be helpful to all newbies in testing world.

http://www.sqatester.com/documentation/GUIchecklist.htm

http://www.methodsandtools.com/archive/archive.php

http://www.methodsandtools.com/archive/archive.php?id=37

http://usability.gov/pdfs/guidelines.html

http://usableweb.com/#Keith%20Instone'sSite

http://www.humanfactors.com/downloads/guistandards.pdf

http://www.maskery.ca/services/ui_design_standards.htm

http://www.iie.org.mx/Monitor/v01n03/ar_ihc2.htm

Measuring of Adhoc Testing

One of the possible ways to measure Adhoc testing is SessionBased Test Management. It means that you should define sessions; each of them should contain suggestions about what should be tested. Session looks like a mission, not like a Test Case. Then you split sessions between testers and they do their work. After testing done he prepare report which should contain information like this:

Session Name
Tester Name
Start Time
Duration
Test Design and Execution
Bugs investigating and Reporting
Session Setup
Test Notes
Test Areas
Risks
Bugs list
Issues list

After you have collected reports from tester you can measure their productivity, because you have enough data.

Also you can perform test planning because you can define all needed sessions (e.g 50), estimate how many sessions can be executed per day (e.g. 5), analyze how many perfect sessions you have (e.g. 70%) and you know how many testers will do this job (e.g. 7)
So, it is easy for now to estimate overall time

50 / (5 * 7 * 0.70) = 2 days

Automated or Manual Testing

"Better" implies a value analysis. Some folks are good at manual testing andpoor at automated testing simply due to their make-up. Trying to trainsomeone who really is not cut out to be an automated tester will yield poorresults. A good automated test engineer needs the predisposition of atester, but also the mental composition and interest of a developer.Finally, in the U.S. at least, automation test engineers are normally paid apremium, averaging 20% more in salary, than manual testers. On average,remember, so there will be exceptions in industry segments. The reason? Theyhave to know some programming and development.Regarding the relative stability of employment you think manual testersenjoy compared to automation types, I assert that is domain specific to adegree, but more probably company specific. If you work for a company whodiscards testers proficient in Tool A when they bring on board Tool B, thenyou need to look for a job elsewhere. The management of such a company is ingrave need of a clue. The specific techniques for Tool A and Tool B will bedifferent, but the approach to test automation, the design and architecture,the discipline and engineering of it, is directly applicable regardless ofthe tool used. And that experience is more difficult to find in employees. Ican teach just about anyone with half a brain (which, from my experience,excludes some half of the candidates) how to use any tool to a basic anduseful degree of proficiency. Only a few will ever have what it takes toprogress to intermediate proficiency. Most companies choose one solution andas that improves or changes, they make sure their employees improve andchange too. It is an investment that is in the company's best interests. Ifthe underlying technology of the product radically changes, then the testapproach -- and tools used to implement that approach -- probably will haveto change as well. Client-server vs. web browser-based would be a commonexample. Moving from C++ to Java is another example.Regarding domain specific cases for stability, it is the knowledge of thatdomain and the prior product versions that makes the tester valuable morethan the fact that the tester is a manual tester or not. I worked for acompany where the QA testers pretty much had to have the equivalentknowledge base in geology and petrophysics of at least a Master's degree.Several were PhD level. THAT knowledge and experience of product, or how theuser community used the product is what made them employable at thatcompany. How they tested the product was less important.One last thought: many companies are sold a proverbial "bill of goods" bysaavy sales reps from the automation tools vendors and are convinced Tool Ais the complete and end-to-end solution to all their testing challenges.When it doesn't pan out, due to lack of training, lack of organization andarchitecture, etc. etc., the company management tends to want to jettisonthe tool that let them down. The tool users often are jettisoned as well, aspart of a failed project. This happens all too often. Then the next "silverbullet" be-all and end-all Tool B sales rep comes along and convinces thecompany that IT'S product was what the company should have had all along.New tool and new tool users come into play. Normally, the same resultoccurs, because it is the test management of the company that doesn't knowhow to implement test automation, and the same guys are there time aftertime after time, making the same mistakes over and over.

Relation between KLOC & Test cases

KLOC means Kilo Lines of Code means 1000 Lines of Code

so KLOC = 1000 LOC
10 LOC = 1 FP ( Functional Point)
we can design 3 test cases from each Functional Point

1 FP = 3 TC ( Test Case )

1000
------- = 100 FP
10

so 1 KLOC = 100 * 3 = 300 Test Cases ( we can write approximately ....this is according to my comp test metrics )

Getting the first break in Quality Control

Please let me begin with a clear note that I am writing these for candidate who are well deserved as QC professionals. They have systematically learnt QA process, and are prepared for this break. Please don’t misunderstand that I am writing this for people who think QC is a stop gap. A good QC professional is best in his breed and has selected this career not as the last option, but as a career of his choice.

I have found that approaching directly to a company with no experience has often been fatal. If you are extremely lucky, you may get pass the HR scanning but the QA manager will have a high probability of not selecting you as he also is answerable to someone. He would not like to gamble when there are other more experienced candidates.

I have found that if you want to get a jump start in your career, please approach the small consulting firms who are second level vendors. They often have more time and are more willing to listen and interview you.

Once you get on to their list, they often can you get a good break.

This is how it works: When IT departments or companies need people in rush for short medium or long term projects, they often approach their top vendors to produce skilled staff for the same. They often rate these vendors efficiency on a quick turn around not to mention also pay them a butt load of money. They assume that skilled man powers could be bought in supper markets.

So these vendors to prove point, as they also have to compete with others, resort to these second level vendors. These are the times when your consulting firm whom I am referring as second level vendors, pushes your resume to the direct vendors. In these cases resume scanning less stringent and sometime even an informal interview can get you in.

Once you are in it is up to you to prove your worth.

I am not saying this works always but I have seen many people getting their first break in QC in similar fashion.

I know many of you may not agree to this. But if you have other suggestions, please go ahead so that the candidates who are looking for their first break can use your help in this regards.

Of course networking, forums, groups etc helps too.

Worst resort, get into a company as an intern. Trust me, if you are good in your job, it won’t take time for you to be recognized and rewarded appropriately. There are always shortages of good QC personnel.

How to be a Good software Tester/QA Engineer

• Be a self learner about new technologies and new QA methodologies and keep an eye of development techniques as well.

• You should look at the products or projects that you have recently released and find out if you missed any defects (you will learn more about testing)

• You should be learning new techniques and studying new tools that are applicable.

• You should get off your butt and show some initiative.

• Dealing with developers to get the bug fix of other kind of QA activity is a skill. Try to understand the developer attributes and work according to that.

• Learn how to write effective test cases, Minimum number of test cases with maximum test coverage

• Never judge you team mates or rank them with you. Think that all the team members are equal since all of them working for same company and same company goals.

• Try to learn some programming languages and to some simple code, which will help you one day to be a good QA manager who understands Greek of developers?

• You should be experimenting with back-up and restore tools for your test lab
  such as Ghost, Drive Image, and Frisbee