ISTQB 3- Static Testing

Preparation Course

ISTQB Foundation Level

Chapter 3- Static Testing

START

by Katty Ode

INDEX

2. Review Process

1. Static Testing Basics

Chapter 03

Static Testing

Static Testing

Objetive: This chapter describes static test techniques, including reviews, and provides an overview of how they are conducted.

Static testing

• Dynamic testing: Testing excecuting code
• Static testing: Testing without executing code

Static and Dynamic Testing

Static Testing is a type of a Software Testing method which is performed to check the defects in software without actually executing the code of the software application. Static testing is performed in early stage of development to avoid errors as it is easier to find sources of failures and it can be fixed easily. The errors that can’t not be found using Dynamic Testing, can be easily found by Static Under Dynamic Testing, a code is executed. It checks for functional behavior of software system, memory/cpu usage and overall performance of the system. Hence the name "Dynamic"

Static Testing

Dynamic Testing

Static and Dynamic Testing

Static testing

• In contrast to dynamic testing, which requires the execution of the software being tested, static testing relies on the manual examination of work products (i.e., reviews) or tool-driven evaluation of the code or other work products (i.e., static analysis). Both types of static testing assess the code or other work product being tested without actually executing the code or work product being tested.

3.1. Static Testing

Work Products that Can Be Examined by Static Testing

• Specifications, including business requirements, functional requirements, and security requirements
• Epics, user stories, and acceptance criteria
• Architecture and design specifications
• Code
• Testware, including test plans, test cases, test procedures, and automated test scripts
• User guides
• Web pages
• Contracts, project plans, schedules, and budget planning
• Configuration set up and infrastructure set up
• Models, such as activity diagrams, which may be used for Model-Based testing

3.1.2 Benefits of Static Testing

Additional benefits of static testing may include:

• Detecting and correcting defects more efficiently, and prior to dynamic test execution
• Identifying defects which are not easily found by dynamic testing
• Preventing defects in design or coding by uncovering inconsistencies, ambiguities, contradictions, omissions, inaccuracies, and redundancies in requirements
• Increasing development productivity (e.g., due to improved design, more maintainable code)
• Reducing development cost and time
• Reducing testing cost and time
• Reducing total cost of quality over the software’s lifetime, due to fewer failures later in the lifecycle or after delivery into operation
• Improving communication between team members in the course of participating in reviews

3.1.3 Differences between Static and Dynamic Testing

• Static testing and dynamic testing can have the same objectives. Static and dynamic testing complement each other by finding different types of defects.
• Another distinction is that static testing can be used to improve the consistency and internal quality of work products, while dynamic testing typically focuses on externally visible behaviors.
• One main distinction is that static testing finds defects in work products directly rather than identifying failures caused by defects when the software is run.

3.1.3 Differences between Static and Dynamic Testing

Compared with dynamic testing, typical defects that are easier and cheaper to find and fix through static testing include:

• Requirement defects (e.g., inconsistencies, ambiguities, contradictions, omissions, inaccuracies, and redundancies)
• Design defects (e.g., inefficient algorithms or database structures, high coupling, low cohesion)
• Coding defects (e.g., variables with undefined values, variables that are declared but never used, unreachable code, duplicate code)

3.1.3 Differences between Static and Dynamic Testing

Compared with dynamic testing, typical defects that are easier and cheaper to find and fix through static testing include:

• Deviations from standards (e.g., lack of adherence to coding standards)
• Incorrect interface specifications (e.g., different units of measurement used by the calling system
• than by the called system)
• Security vulnerabilities (e.g., susceptibility to buffer overflows)
• Gaps or inaccuracies in test basis traceability or coverage (e.g., missing tests for an acceptance criteria)

3.2 REVIEW PROCESS

Reviews vary from informal to formal.

• Informal reviews are characterized by not following a defined process and not having formal documented output.
• Formal reviews are characterized by team participation, documented results of the review, and documented procedures for conducting the review.
• The formality of a review process is related to factors such as the software development lifecycle model, the maturity of the development process, the complexity of the work product to be reviewed, any legal or regulatory requirements, and/or the need for an audit trail.

3.2 REVIEW PROCESS

• The focus of a review depends on the agreed objectives of the review (e.g., finding defects, gaining understanding, educating participants such as testers and new team members, or discussing and deciding by consensus).

3.2 REVIEW PROCESS

3.2.1 Work Product Review Process

The review process comprises the following main activities:

1. Planning
2. Initiate review
3. Individual review (individual preparation)
4. Issue communication and analysis
5. Fixing and reporting

3.2.1 WORK PRODUCT REVIEW PROCESS

• Defining the scope, which includes the purpose of the review, what documents or parts of documents to review, and the quality characteristics to be evaluated
• Estimating effort and timeframe
• Identifying review characteristics such as the review type with roles, activities, and checklists
• Selecting the people to participate in the review and allocating roles
• Defining the entry and exit criteria for more formal review types (e.g., inspections)
• Checking that entry criteria are met (for more formal review types)

PLANNING

3.2.1 WORK PRODUCT REVIEW PROCESS

• Distributing the work product (physically or by electronic means) and other material, such as issue log forms, checklists, and related work products
• Explaining the scope, objectives, process, roles, and work products to the participants
• Answering any questions that participants may have about the review

INITIATE REVIEW

3.2.1 WORK PRODUCT REVIEW PROCESS

INDIVIDUAL REVIEW (Individual Preparation)

• Reviewing all or part of the work product
• Noting potential defects, recommendations, and questions

3.2.1 WORK PRODUCT REVIEW PROCESS

ISSUE COMMUNICATION AND ANALYSIS

• Communicating identified potential defects (e.g., in a review meeting)
• Analyzing potential defects, assigning ownership and status to them
• Evaluating and documenting quality characteristics
• Evaluating the review findings against the exit criteria to make a review decision (reject; major changes needed; accept, possibly with minor changes)

3.2.1 WORK PRODUCT REVIEW PROCESS

• Creating defect reports for those findings that require changes to a work product
• Fixing defects found (typically done by the author) in the work product reviewed
• Communicating defects to the appropriate person or team (when found in a work product related to the work product reviewed)
• Recording updated status of defects (in formal reviews), potentially including the agreement of the comment originator
• Gathering metrics (for more formal review types)
• Checking that exit criteria are met (for more formal review types)
• Accepting the work product when the exit criteria are reached

FIXING AND REPORTING

Question 1 of 3

Which of the following statements CORRECTLY reflects the value of static testing? a) By introducing reviews, we have found that both the quality of specifications and the time required for development and testing have increased b) Using static testing means we have better control and cheaper defect management due to the ease of detecting defects later in the lifecycle c) Now that we require the use of static analysis, missed requirements have decreased and communication between testers and developers has improved d) Since we started using static analysis, we find coding defects that might have not been found by performing only dynamic testing Select ONE option

Rightanswer

Question 2 of 3

You are reading a user story in the product backlog to prepare for a meeting with the product owner and a developer, noting potential defects as you go. Which of the following statements is true about this activity? a) It is not a static test, because static testing involves execution of the test object b) It is not a static test, because static testing is always performed using a tool c) It is a static test, because any defects you find could be found cheaper during dynamic testing d) It is a static test, because static testing does not involve execution of the test object Select ONE option.

Rightanswer

Question 3 of 3

You are working as a tester on an Agile team and have participated in over two dozen user story refinement sessions with the product owner and the developers on the team at the start of eachiteration. As the reviews have gotten more effective at detecting defects in user stories and the product owner more adept at correcting those defects, you and the team notice that the team’s velocity, as shown in your burndown charts, has started to increase. Which of the following is a benefit of static testing that MOST DIRECTLY applies to increased velocity? a) Increasing total cost of quality b) Reducing testing cost c) Increasing development productivity d) Reducing total cost of quality Select ONE option.

Rightanswer

3.2.2 Roles and responsibilities in a formal review

The participants in any type of formal review should have adequate knowledge of the review process. The best, and most efficient, review situation occurs when the participants gain some kind of advantage for their own work during review-ing

• Author
• Management
• Facilitator / Moderator
• Review Leader
• Reviewers
• Scribe (or recorder)

3.2.2 Roles and responsibilities in a formal review

• Creates the work product under review
• Fixes defects in the work product under review (if necessary)

AUTHOR

3.2.2 ROLES AND RESPONSIBILITIES IN A FORMAL REVIEW

• Is responsible for review planning
• Decides on the execution of reviews
• Assigns staff, budget, and time
• Monitors ongoing cost-effectiveness
• Executes control decisions in the event of inadequate outcomes

MANAGEMENT

3.2.2 ROLES AND RESPONSIBILITIES IN A FORMAL REVIEW

• Ensures effective running of review meetings (when held)
• Mediates, if necessary, between the various points of view
• Is often the person upon whom the success of the review depends

FACILITATOR (often called moderator)

3.2.2 ROLES AND RESPONSIBILITIES IN A FORMAL REVIEW

• Takes overall responsibility for the review
• Decides who will be involved and organizes when and where it will take place

REVIEW LEADER

3.2.2 ROLES AND RESPONSIBILITIES IN A FORMAL REVIEW

• May be subject matter experts, persons working on the project, stakeholders with an interest in the work product, and/or individuals with specific technical or business backgrounds
• Identify potential defects in the work product under review
• May represent different perspectives (e.g., tester, developer, user, operator, business analyst, usability expert, etc.)

REVIEWERS (checkers or inspectors)

3.2.2 ROLES AND RESPONSIBILITIES IN A FORMAL REVIEW

• Collates potential defects found during the individual review activity
• Records new potential defects, open points, and decisions from the review meeting (when held)

In some review types, one person may play more than one role, and the actions associated with each role may also vary based on review type. In addition, with the advent of tools to support the review process, especially the logging of defects, open points, and decisions, there is often no need for a scribe. Further, more detailed roles are possible, as described in ISO standard (ISO/IEC 20246).

SCRIBE (or Recorder)

3.2.3 Review Types

Although reviews can be used for various purposes, one of the main objectives is to uncover defects. All review types can aid in defect detection, and the selected review type should be based on the needs ofthe project, available resources, product type and risks, business domain, and company culture, among other selection criteria.

A single work product may be the subject of more than one type of review. If more than one type of review is used, the order may vary. For example, an informal review may be carried out before a technical review, to ensure the work product is ready for a technical review.

3.2.3 Review Types

The types of reviews described below can be done as peer reviews, i.e., done by colleagues qualified to do the same work.

The types of defects found in a review vary, depending especially on the work product being reviewed. Reviews can be classified according to various attributes.

3.2.3 REVIEW TYPES

• Main purpose: detecting potential defects
• Possible additional purposes: generating new ideas or solutions, quickly solving minor problems
• Not based on a formal (documented) process
• May not involve a review meeting
• May be performed by a colleague of the author (buddy check) or by more people
• Results may be documented
• Varies in usefulness depending on the reviewers
• Use of checklists is optional
• Very commonly used in Agile development

Informal review (e.g., buddy check, pairing, pair review)

• Main purposes: find defects, improve the software product, consider alternative implementations, evaluate conformance to standards and specifications
• Possible additional purposes: exchanging ideas about techniques or style variations, training of participants, achieving consensus
• Individual preparation before the review meeting is optional
• Review meeting is typically led by the author of the work product
• Scribe is mandatory
• Use of checklists is optional
• May take the form of scenarios, dry runs, or simulations
• Potential defect logs and review reports are produced
• May vary in practice from quite informal to very formal

3.2.3 REVIEW TYPES

Walkthrough

• Main purposes: gaining consensus, detecting potential defects
• Possible further purposes: evaluating quality and building confidence in the work product, generating new ideas, motivating and enabling authors to improve future work products, considering alternative implementations
• Reviewers should be technical peers of the author, and technical experts in the same or other disciplines
• Individual preparation before the review meeting is required
• Review meeting is optional, ideally led by a trained facilitator (typically not the author)
• Scribe is mandatory, ideally not the author
• Use of checklists is optional
• Potential defect logs and review reports are produced

3.2.3 REVIEW TYPES

Technical review

3.2.3 REVIEW TYPES

• Main purposes: detecting potential defects, evaluating quality and building confidence in the work product, preventing future similar defects through author learning and root cause analysis
• Possible further purposes: motivating and enabling authors to improve future work products and the software development process, achieving consensus
• Follows a defined process with formal documented outputs, based on rules and checklists
• Uses clearly defined roles, such as those specified in section 3.2.2 which are mandatory, and may include a dedicated reader (who reads the work product aloud often paraphrase, i.e. describes it in own words, during the review meeting)
• Individual preparation before the review meeting is required

Inspection

3.2.3 REVIEW TYPES

• Reviewers are either peers of the author or experts in other disciplines that are relevant to the work product
• Specified entry and exit criteria are used
• Scribe is mandatory
• Review meeting is led by a trained facilitator (not the author)
• Author cannot act as the review leader, reader, or scribe
• Potential defect logs and review report are produced
• Metrics are collected and used to improve the entire software development process, including the inspection process

Inspection

3.2.4 Applying Review Techniques

There are a number of review techniques that can be applied during the individual review (i.e., individual preparation) activity to uncover defects. These techniques can be used across the review types The effectiveness of the techniques may differ depending on the type of review used.

• Ad Hoc
• Checklist-based
• Scenarios and dry runs
• Perspective-based
• Role-based

3.2.4 APPLYING REVIEW TECHNIQUES

In an ad hoc review, reviewers are provided with little or no guidance on how this task should be performed. Reviewers often read the work product sequentially, identifying and documenting issues as they encounter them. Ad hoc reviewing is a commonly used technique needing little preparation. This technique is highly dependent on reviewer skills and may lead to many duplicate issues being reported by different reviewers.

AD HOC

3.2.4 APPLYING REVIEW TECHNIQUES

A checklist-based review is a systematic technique, whereby the reviewers detect issues based on checklists that are distributed at review initiation (e.g., by the facilitator). A review checklist consists of a set of questions based on potential defects, which may be derived from experience. Checklists should be specific to the type of work product under review and should be maintained regularly to cover issue types missed in previous reviews.

Checklist-based

3.2.4 APPLYING REVIEW TECHNIQUES

Checklist-based

The main advantage of the checklist-based technique is a systematic coverage of typical defect types. Care should be taken not to simply follow the checklist in individual reviewing, but also to look for defects outside the checklist

3.2.4 APPLYING REVIEW TECHNIQUES

In a scenario-based review, reviewers are provided with structured guidelines on how to read through the work product. A scenario-based review supports reviewers in performing “dry runs” on the work product based on expected usage of the work product (if the work product is documented in a suitable format such as use cases). These scenarios provide reviewers with better guidelines on how to identify specific defect types than simple checklist entries. As with checklist-based reviews, in order not to miss other defect types (e.g., missing features), reviewers should not be constrained to the documented scenarios.

Scenarios and dry runs

3.2.4 APPLYING REVIEW TECHNIQUES

In perspective-based reading, similar to a role-based review, reviewers take on different stakeholder viewpoints in individual reviewing. Typical stakeholder viewpoints include end user, marketing, designer,tester, or operations. Using different stakeholder viewpoints leads to more depth in individual reviewing with less duplication of issues across reviewers. In addition, perspective-based reading also requires the reviewers to attempt to use the work product under review to generate the product they would derive from it.

Perspective-based

3.2.4 APPLYING REVIEW TECHNIQUES

A role-based review is a technique in which the reviewers evaluate the work product from the perspective of individual stakeholder roles. Typical roles include specific end user types (experienced, inexperienced, senior, child, etc.), and specific roles in the organization (user administrator, system administrator, performance tester, etc.). The same principles apply as in perspective-based reading because the roles are similar.

Role-based

3.2.5 Success Factors for Reviews

In order to have a successful review, the appropriate type of review and the techniques used must be considered. In addition, there are a number of other factors that will affect the outcome of the review. Organizational success factors for reviews include:

• Each review has clear objectives, defined during review planning, and used as measurable exit criteria
• Review types are applied which are suitable to achieve the objectives and are appropriate to the type and level of software work products and participants

3.2.5 Success Factors for Reviews

• Any review techniques used, such as checklist-based or role-based reviewing, are suitable for effective defect identification in the work product to be reviewed
• Any checklists used address the main risks and are up to date
• Large documents are written and reviewed in small chunks, so that quality control is exercised by
• providing authors early and frequent feedback on defects Participants have adequate time to prepare
• Reviews are scheduled with adequate notice
• Management supports the review process (e.g., by incorporating adequate time for review activities in project schedules)
• Reviews are integrated in the company's quality and/or test policies.

3.2.5 Success Factors for Reviews

People-related success factors for reviews include:

• The right people are involved to meet the review objectives, for example, people with different skill sets or perspectives, who may use the document as a work input
• Testers are seen as valued reviewers who contribute to the review and learn about the work product, which enables them to prepare more effective tests, and to prepare those tests earlier
• Participants dedicate adequate time and attention to detail
• Reviews are conducted on small chunks, so that reviewers do not lose concentration during individual review and/or the review meeting (when held)

Question 1 of 5

Which of the following options are roles in a formal review? a) Developer, Moderator, Review leader, Reviewer, Tester b) Author, Moderator, Manager, Reviewer, Developer c) Author, Manager, Review leader, Reviewer, Designer d) Author, Moderator, Review leader, Reviewer, Scribe Select ONE option.

Rightanswer

Question 2 of 5

Which of the review types below is the BEST option to choose when the review must follow a formal process based on rules and checklists? a) Informal Review b) Technical Review c) Inspection d) Walkthrough Select ONE option

Rightanswer

Question 3 of 5

Which of the following statements on the use of checklists in a formal review is CORRECT? a) As part of the review planning, the reviewers create the checklists needed for the review b) As part of the issue communication, the reviewers fill in the checklists provided for the review c) As part of the review meeting, the reviewers create defect reports based on the checklists provided for the review d) As part of the review initiation, the reviewers receive the checklists needed for the review Select ONE option

Rightanswer

Question 4 of 5

Which of the following CORRECTLY matches the roles and responsibilities in a formal review? a) Manager – Decides on the execution of reviews b) Review Leader - Ensures effective running of review meetings c) Scribe – Fixes defects in the work product under review d) Moderator – Monitors ongoing cost-effectiveness

Rightanswer

Question 5 of 5

The reviews being used in your organization have the following attributes: • There is a role of a scribe • The purpose is to detect potential defects • The review meeting is led by the author • Reviewers find potential defects by individual review • A review report is produced Which of the following review types is MOST likely being used? a) Informal Review b) Walkthrough c) Technical Review d) Inspection

Rightanswer