What is Smoke Testing?
Smoke tests, also called build verification or acceptance tests, are minimalistic procedures designed to check the basic functions of a new software. In other words, smoke testing ensures that software is stable enough to perform its intended task.
The named “smoke” test was borrowed from a similar test that is run on computer hardware. The tester plugs in the machine to see if it will catch fire. While software cannot literally go in up flames, it can shut down or malfunction as quickly, as if it had ignited.
Similarly, build verification testing, or BTI, employs black box protocol, at least in part. Also known as a behavior test, a black box test requires a tester who does not know the design of the unit he or she is assessing. This helps the tester to remain objective during both the operations and feedback sections of the procedure.
In this article, we will cover in more detail what smoke testing is, what it entails, and how it fits into other testing methods.
The When, Why, and How of Smoke Testing
Acceptance testing checks the basic functioning of most components within a new software. However, it doesn’t give feedback on the depth of functioning. Smoke tests are more like a pass/fail exam. For example, let’s say that a programmer wants to test whether a software allows users to add items to the library as intended. To perform a smoke test, a tester will take all the steps one would to add an item to a library: select item, click and drag, and then save. If the item is added, then the test is a success.
On the other hand, applications that have severe malfunctions or shut down altogether may be reworked or redesigned. The great thing about smoke tests is that they quickly and inexpensively weed out basic malfunctions before you must move on to more time consuming and costly assessments.
That being said, a smoke test can be performed more than once to help solve malfunction problems. In fact, there are three different levels to this phase of development:
1. Integration Testing
The integration testing level is used to discover deficiencies between different components within a software. Subcategories of this level include component and systems testing. A component integration test is aimed at finding bugs or hiccups between the integration and cooperation of different components of a software.
System integration tests are more like whole package procedures. They still check the interactions between components, but often this is as a reaction to other applications.
Both forms of integration testing employ black box protocol, but they can also use white box (in which the tester knows the design), and grey box (a combination of black and white box testing.)
The following are example of the various types of integration testing and when you would use them:
- Big Bang Testing: This type can be used when most or all of the components are in place. They are run as a complete unit to see what happens. While this may seem to mimic systems testing (below) the purpose here is only to determine how well the components interface.
- Top-Down Testing: Top-down testing can help you assess interactions between units, one step at a time, according to the software’s order of operations.
- Bottom to Top Testing: Unit interactions are tested one step at a time to reverse software’s order of operations. This is very helpful for reducing time when there is speculation that the end of a process is the weakest link.
2. Systems Testing
This level of smoke testing assumes passing a general integration test so that the bigger bugs short-stopping function between components have been worked out.
A systems test is where the software is operated and checked as a whole to make sure that is generally functioning as intended. Systems testing usually requires black box protocol.
Here are some of examples of the more common systems testing approaches and how they are useful:
- Functional Testing: Functional testing is useful for assessing a system’s ability to accomplish what it was designed to do. However, this type of testing does not take into consideration the software’s performance level or usability.
- Regression Testing: Fixing one defect in a program can reveal or create another. Regression testing can uncover changed behavior once software has been amended.
- Usability Testing: To determine ease of functions, end-user groups are employed to test software. This is a non-functional test. Think allowed sessions often to follow in which users discuss their experience in terms of easy and clarity of operations. This method is also used at the acceptance testing level (below).
- Performance Testing: Additional work loads are added in this non-functional testing procedure. This helps determine levels of reactivity and constantly under different conditions.
- Security Testing: This Is another non-functional test It helps you discover potential attack points in your system, so that you can set up defenses against viruses and malware.
- Compliance Testing: This is also referred to as conformance testing. This non-functional form of assessment makes sure that software is being developed according to regulations and standards.
3. Acceptance Testing
The highest level of smoke testing is concerned with the end user’s needs. The word “acceptance” here refers to the customer’s point of view. Will users see this product as meeting or exceeding their wants and needs? Will it fit into the business or operations that it is expected to serve?
Testing procedures at this level are traditionally black box. However, they are also often creative in their attempts to measure utility in real-world applications.
Because acceptance tests can be more inventive, they tend to fit into looser categories. However, the examples below are still definable in terms of their functions:
- Alpha Acceptance Testing: This is inhouse testing that is done by software developers and sales personnel who are not assigned to the project. This can provide you with a private yet professional peer review.
- Open-Sourced Acceptance Testing: These are groups that have been secured outside the company for objective testing purposes. This allows you to check functionality within a wide range of end users.
- Customer Acceptance Testing (CAT): If software is commissioned by an external company or organization, it is often a good idea to let them test it while it is in development. This creates a shortcut to customer satisfaction and prevents developers from having to go back to the drawing board to keep a contract.
- User Acceptance Testing (UAT): Similar to CAT, this let’s end users have their say during the development process. Unlike customer acceptance or open-sourced testing, UAT can directly target a specific public demographic.
What is Sanity Testing
At the acceptance level, smoke testing may look a lot like sanity testing. After all, sanity tests are run on altered software builds to root out bugs. However, there is a difference.
Sanity tests are usually reserved for slighter manipulations of function or code. They are used to tighten logical applications. In general, a smoke test is more concerned with how stable a software is, and a sanity test aims to make sure that an application makes sense design-wise.
The following comparative examples can further illustrate the gap that separates smoke and sanity testing:
- Test Cases: Smoke testing will utilize test cases. Sanity testing will not.
- Goals: Smoke tests determine pivotal factors about a build’s functioning stability. Sanity tests are more about the rational use of function from the perspective of the user.
- Testers: Smoke tests will allow for project developers to test and give feedback. Sanity tests use testers who are separated from the design model.
- Components: Smoke tests will eventually involve the entire application. Sanity tests focus primarily on critical components.
- Presumed Stability: Stability problems are expected during a smoke test. Sanity tests begin with an expectation of general stability.
Bottomline: Smoke Testing Allows For Greater Efficiency
Smoke and sanity testing make development quicker, cheaper, and ultimately more available. While at times these procedures can seem a little chaotic, they are the straightest line towards development success.