Why to automate testing

During last spring I have had lot of discussions and studies about automated testing. Question has been that should we start to automate regression tests to more than 10 year old code base and related functionality. Discussions were done with stakeholders including customers, shareholders, management, scrum masters, dev teams and testers. Studies were done by using web and also books from Lisa Crispin and from Google and I participated to testing days to listen how Spotify and F-Secure handle things. Should we start to automate in our context? Answer is both yes and no. In this article I will keep focus on the added value for our product.

How it helps people?

Let’s first check from the people point of view. Good team is more likely going to build product which will deliver needed value.

For the team which wants to improve, it’s important to have fast feedback. With having automated tests integrated to builds we have possibility to have feedback. It’s usually faster than having feedback from manual tests. And definitely faster, and cheaper, than having feedback from not working functionality from customer.

Other point is that team is changing. Some people goes out, some comes in. When product lives longer than the team, you need to manage knowledge how to build the product. One of the ways to store product correct behaviour is to store it to automated tests. This way new people can know, how things needs to work. It works as a document for developers.

For example: you end up coding of new rounding functionality for invoice payments. You tested your new code and it works nicely. It’s time to go home. At the morning you find out that nightly build throwed error. It shows that you broke invoice closing when it is targeted to credit note. It’s not exactly invoice payment, but it works with same code base. Your focus was too much in your story testing, not in side-effects testing.

But you got fast feedback, still remembered your yesterday coding, understood the problem, learned about it and had a clue where issue is. Also you saved money because issue didn’t reached end customers. Who added the case to test suite? It was added together with customer, because team didn’t understood business domain completely.

If issue would have been reported by end customer, it could be month or two later when you would know about it. Would it be easy to fix then? How many similar issues you have done during those months? If you would have known about issue would it have steered you to other direction with architecture?

One of the cons for humans is that we can’t rely discipline of them. I assume that everybody has heard sentences: “This small change won’t break anything. We don’t need to test it.” Computers don’t say that. They test and test if we have told them to do so. If test suite is good then computers will test even small change side effects.

Back to example case. Let’s say we would have found our payment issue in manual regression testing near the sprint end and we wouldn’t have automated tests. After small fixes would someone be running all regression tests manually again?

What about product?

People are changing and so does the environment. Either we understand reality differently or customer business is changing. When we do changes then we might have need for refactoring. When we have good testing suite for product we are able to do changes more easily and with good quality. We are more capable to change product to model real world and to serve customer needs.

When we automate tests we are also saving time from manual regression testing. This is important for big systems. Saved time can be used for testing more tricky case and scenarios which are more difficult to automate. For those cases there might not have been time before. By focusing to corner cases we are able to increase product quality.

In our example case we could assume that we have automated tests in place. Now we have new change request from customer. It doesn’t fit to our current architecture. We know that we will have more similar requests later so team decides to change class structure. That way it will model new understanding of the environment. Team wouldn’t do this change with reasonable cost without automated regression tests. After changes code base is easier to maintain, it’s easier to understand and it’s faster to do upcoming changes.

Conclusion

My point is that either you check automation from people or product point of view it has many good things. Its investment for product success in long run. Good test suite can:

  • give fast feedback and therefore increase learning possibility
  • work as a document
  • always watch your back even for small changes
  • give possibility to create better architectures with less technical debt
  • give possibility to catch more bugs before deployment

Some of the items are more important to other stakeholders than something else for other stakeholder. So, there is good reasons to automate, but there is good reasons to not automate. I will write about those in my next post.

Why technical debt removing is painful?

Technical debt exists in code architecture and in code base. Debt appears when product requirements changes and new features are built in without taking care needed, small or big, architectural changes. Your code base starts to contain more and more shortcuts which will make it more difficult to develop system. Development speed will slow down. Finally you are so slow that developing the system does not make any sense. That sucks.

Why it’s difficult to remove technical debt? In this post I will introduce some common obstacles. I will show that usually debt comes because short term goals drives over long term goals and that without craftsmanship of the whole company or department product will have more and more technical debt. We need to start fight against debt.

Short term goals are driving

It’s easy to be blind for long term goals because of short term goals. You do what you were asked to do and deliver it fast. Everybody is happy. You have taken some shortcuts but who cares. Feedback is positive and mood is high. Some guys from the team are praised for finding clever shortcuts, like using database fields for totally different things than their name says, or just controlling same functionality with more and more input parameters and if -branches.  Short term speed tastes good.

I think this short term goal achieving is ok. As far as it’s understood that by doing these shortcuts you take debt which you need to pay back. If only short term goals are driving the development and short cuts are taken iteration after iteration and year after year then your product will be screwed up. There has to be balance between short and long term goals. Product owner has a major role in goal setting.

Technical debt is difficult to measure

If you don’t have any measurements for technical debt then it’s difficult to know that it exists. For example: As product owner, do you know how many shortcuts were taken in last sprint? Do you know their effect on upcoming sprints?  It might be that development team is the only one who knows current technical debt amount. When you don’t know about technical debt in your product then you are not ready to eliminate it. Most likely paying debt back  is not in your backlog, or is it? After all, reducing technical debt will only bring indirect value for customer.

What happens if we combine these two first problems? Only developers know about technical debt and you are rewarding them to take more debt.  You might see that velocity will drop and what you will do? You put more pressure to get to the deadlines. And again more short cuts are made. It’s like product has fallen to ground and you are still kicking it. Lucky us, product doesn’t feel pain (yet).

Legacy code

Ok, we know we need to know if technical debt exists and some way to measure it. Then we look those 1000 classes with 20 000 LOC each and without any automated tests. Critical business processes use the existing build binaries every second. What do we do? Maybe some sensitive guy starts to cry? Somebody proposes to refactor it all (product owner opens bottle of Jack). Well yes, this is serious situation. Then we need the leading developer to stand and say: “I have some ideas so let’s not make it any worse than this, let us know where it’s most valuable to start to remove technical debt and when we will start?”

Conclusion

Usually technical debt is compared to financial debt. I have an another example in my mind.  Technical debt is like a unhealthy food. Like candies and cakes. It tastes good but if you eat those candies all the time you will start to get fat, you will end up being in not a good shape. Same way as product architecture starts to be in bad shape. If you don’t continuously measure your weight you might not notice effect from candies. Fat comes slowly.  At the beginning you might not even notice it. At some point you will find out that you are too fat. You will notice that you can’t run anymore. Maybe then you will start to think differently about eating candies and start to do some walking exercises.

Be as serious with technical debt as you are with your own health. As a product owner try

  • Balancing short and long term goals for product development
  • Studying and understanding your product and technical debt in it
  • Small steps to get rid of debt

Otherwise principles of technical debt will kill your product.