How We Estimate Quality Assurance Works at EGO

This post unveils how we approach testing estimations and demonstrate it using one of our existing projects.

June 22, 2021
Liliia Finenkova

Test estimation is an approximation of the effort required to perform various kinds of testing.  

In our QA company, we talk about estimation efforts instead of hours as it allows for more predictability. Here is why.

Let’s say we need to do performance testing, and we estimate that we need approximately 5 hours for that: two hours to write the test scripts, two hours to run them, and one hour to make a record of the results of the run.

It seems like this task can be done within a day.

Suddenly, we find out the production-like environment for testing is not ready yet. So we can’t run the script. Or a new critical bug appears that requires urgent research. Or a fix that needs to be tested. And it might appear on this project or another since the assigned QA experts might be involved in more than one project at once.  

In this case, the priorities will be changed, as well as the duration of the task. We’ll talk about handling such situations later, but the main thing is that it’s easier to estimate the efforts required to perform a testing task rather than the time.

Yet when we talk with clients about actual works, we then do our best to convert those estimated efforts into hours as precisely as possible, as clients need to plan their budget and generally prefer the Time and Material pricing model.

EGO’s QA Estimation Table

To estimate the amount of effort required to test the product, our quality assurance/testing company created a table listing typical QA tasks. It helps us organize our work, track our progress and make sure we don’t forget anything.

Test Documentation Estimation. Part 1 out of 3 of our QA Estimation Table
Test Documentation Estimation. Part 1 out of 3 of our QA Estimation Table. Source: EGO

In the long run, the table allows us to involve both the leading QA expert and executors in the estimation process and standardize the task management process and collect historical data.

Historical data is used to make more precise estimates for the current project and current QA engineer and avoid liberal planning. So while filling the table up may require a couple of extra working hours, it helps you plan your budget, lets us assign the most fitting engineers for the testing tasks, and overall ensures high efficiency of quality assurance works.

We prefer to avoid too-liberal estimations because we don’t want our QA engineers to sit on the bench for too long. We also don’t want our clients to have unnecessary doubts about the quality of our estimates.

Testing Works. Part 2 out of 3 of our QA Estimation Table
Testing Works. Part 2 out of 3 of our QA Estimation Table. Source: EGO

Also, suppose the quality assurance department has all sprint tasks done too early. In that case, the managers often don’t have a possibility and enough free time to reprioritize and assign the next tasks to them immediately. That affects the team’s velocity in a negative way.

On the contrary, the improved estimate precision achieved with this table increases clients’ trust and loyalty and allows us to come with isolated QA plans (web and mobile) that offer time- and budget-limited testing services for your digital product.

One important part of the table we’d like to focus on is the Risks buffer. This is where our software & QA company experts can include additional time for possible difficulties during the testing activities, such as task complexity, expected bugs, task switching, and executor experience.

Risks buffer. Part 3 out of 3 of our QA Estimation Table
Risks buffer. Part 3 out of 3 of our QA Estimation Table. Source: EGO

Time spent addressing expected bugs depends on the known peculiarity of the project. For instance, code migration to a new platform will require retesting the product on multiple devices or browsers. Or, because of the project requirements, we expect to have many bugs, each requiring to be documented with a screen recording. That consumes a lot of time.  

Task switching is estimated based on the approach taken on the project. If we support the existing project and further update it at the same time, we can expect critical hotfix tasks to appear often, which increases task switching risks.

Executor experience. Quality assurance engineers having little experience in the current domain with the selected technology or testing approach will require more calls with the team and more time for knowledge sharing. This is where we can take it into account. Sometimes, we also consider the time required to get access to all the necessary environments, devices, and databases here.

EGO’s Example of QA Estimation

So we have a development partner, and together with them, we work on a client’s project. Here we follow the partners’ internal development practices, but we need to consider clients’ processes as well.

The project is so massive it involves many different teams with various (though fairly low) levels of cross-dependency. The client follows the SAFe framework and has 8-10 teams united into Agile Release Train (ART).  

Due to the project’s complexity, there are product increment (PI) planning meetings and feature planning meetings for ART. These happen every three months and require up to three days.

Once such high-level planning is over, every feature is decomposed into user stories planned for implementation on sprint planning meetings.  

Here’s our sprint planning sheet that shows – among other things – our vacation schedules. Vacations and long holidays are always considered before the PI meetings for better planning and to avoid dependency bottlenecks.

EGO's sprint planning sheet
Source: EGO the QA/testing services company

Here you can figure out that:

Gross Capacity = Hours/Day * Days in Sprint

Net Capacity = Gross CapacityTotal Ceremonies

80% Planning Target = 80% * Net Capacity

In this table, total ceremonies take more time than planned SCRUM ceremonies because throughout the day, unexpected things like tech syncs, system demos, and other meetings might happen.

Finally, why we count only 80% of net capacity? Because we allocate 20% for the already existing functionality support on this project and knowledge sharing. Previously we had cases when people on the client’s side were abandoning the project without leaving enough documentation on the features they were working on. As of now, the client shuffles feature development among different teams to avoid that.

If 20% were not used, the quality assurance/testing company team might take additional tasks in the sprint marked as “stretch.” However, it doesn’t mean we become responsible for finishing them within the current sprint.

Once a user story is added to the sprint, development and quality assurance tasks can be calculated and planned. Below is an example of a high-level estimate for QA tasks on the project.

A real-world example of a filled estimation table. Part 1 out of 2
A real-world example of a filled estimation table. Part 1 out of 2. Source: EGO

As said in the beginning, initially we use story-pointing to evaluate the team’s effort. The team plays planning poker to evaluate user story complexity. Then the tasks are planned using hours, so this example describes how typical sub-tasks can be estimated and what’s included there.

The listed tasks are specific to the project but can be still taken as an example of making estimates. The hours can be rougher or you can apply estimating assumptions or insurance for testing (e.g. dependency on another team, access to the test environment, etc.)  

A real-world example of a filled estimation table. Part 2 out of 2
A real-world example of a filled estimation table. Part 2 out of 2. Source: EGO

Anyway, when the estimation table is used on the project, its advantages are clearly visible. But what might happen if it is skipped?

We had a project long ago. Everything started with simple testing of a website having a responsive design. Then, at some point, the website started growing too fast. Back then, we didn’t have back-end testing. It just wasn’t initially required, though, over time, it was getting more and more necessary.

Developers had been rolling out so many new updates that 20 hours initially allocated for testing purposes per week were getting critically insufficient. Due to that, we stopped updating the documentation and smoke tests. We also didn’t have enough resources to check the product on all the required devices.

And when the only QA engineer on the project got sick for two weeks, getting back on track for her was extremely cumbersome.

If only the EGO software tester company team had the properly made QA estimation table initially, we could avoid all those troubles.

Bottom line

The explained estimation table is one of the first QA documents created on the project since it outlines the typical tasks and the list of all the other possible artifacts that might be needed on the project.

Creating this table early allows to check and document project requirements, especially if initially they are too vague.

The estimation table is also a great way to gather historical data needed to assess the QA team’s productivity over time and the project estimation effectiveness. And for sure, it’s an essential part of risk management and mitigation efforts.

devops ninja animation
devops ninja animation
devops ninja animation
devops ninja animation

More Articles

Back to blog