Testing Cheat Sheet

Project Life Cycle

This is where the project plan is created. This phase begins by setting SMART specific, measurable, attainable, realistic, timely goals. The scope of the project is defined and a project management plan is created, identifying cost, quality, resources and a timetable. Some of the features of this phase include a scope statement, setting of milestones, communication, risk management plans and a work breakdown structure.

During the execution phase, the project team execute the tasks, deliverables and milestones defined in the project scope. Some tasks that make up this phase include developing the team and assigning resources using key performance indicators, executing the project plan, procurement management and tracking and monitoring progress.

Project Planning

Execution

Project Initiation

The project team will start to define the project and start to sell the project to stakeholders to demonstrate its value for the business.

Monitoring

The monitoring phase keeps track of the overall progress of the project. It is often done by setting up project controls and key performance metrics to measure the effectiveness of the project execution. This includes Test Case pogress, defect tracking and remaining test effort among other things. Keeping on top of these items ensures that any potential issues such as delayed schedules, scope creep and over-budget can be identified as early as possible.

Closing

Closing phase is the formal closure of a project, relevant sign offs are needed by the customer, project stakeholders and sponsors. For testing, this includes the sign offs of test completion reports, referring back to the initial test plan/strategy and ensuring all deliverables have been achieved.

Fundamentals Of Testing

Why Is Testing Necessary

Testing ensures software quality by identifying and fixing issues before release, reducing risks, enhancing user satisfaction, and validating performance. The extent of testing before release varies based on software complexity, criticality, and resources. It should sufficiently validate functionality, stability, and minimize risks, considering time constraints and project goals.

The objectives of testing are to ensure software quality, validate functionality, identify defects, assess performance, verify compliance with requirements, enhance user experience, reduce risks, and ensure a reliable and successful software release. Testing will also gain confidence about the level of quality and will provide information to aid decision-making.

Test Planning and Control:

• Define test objectives, scope, and strategies.
• Create a detailed test plan outlining resources, schedule, and deliverables.
• Establish test management and communication processes.

Test Analysis and Design:

• Review requirements and specifications to understand what needs to be tested.
• Create test cases, test scenarios, and test data based on identified requirements.
• Design test environments and necessary test tools.

Test Implementation and Execution:

• Develop and set up test cases, ensuring proper test data and conditions.
• Execute test cases, record results, and identify defects.
• Compare actual outcomes with expected outcomes to assess pass/fail status.

Evaluating Exit Criteria and Reporting:

• Determine when testing should be stopped based on predefined exit criteria.
• Generate test summary reports, detailing testing activities, results, and any issues encountered.
• Communicate testing progress and outcomes to stakeholders.

Test Closure Activities:

• Complete remaining test documentation and wrap up test environments.
• Perform lessons learned sessions to identify improvements for future testing efforts.
• Assess the success of testing against the defined objectives and deliverables.

Testing Objectives

Fundamental Test Process

1. Testing Shows the Presence of Defects: The primary purpose of testing is to uncover defects, confirming that the software isn't entirely free of issues.
2. Exhaustive Testing is Impossible: Testing every possible input and scenario is impractical due to time and resource constraints. Testing is about risk assessment and mitigation.
3. Early Testing: Begin testing activities as early as possible in the software development life cycle to catch defects sooner and reduce costs.
4. Defect Clustering: A small number of modules or functionalities typically contain most defects. Focus testing efforts on high-risk areas.
5. Pesticide Paradox: Repeated tests using the same techniques can become ineffective at finding new defects. Test cases must evolve and adapt.
6. Testing is Context-Dependent: The effectiveness of testing is influenced by the context of the project, including its objectives, risks, and constraints.
7. Absence-of-Errors Fallacy: The mere absence of defects doesn't guarantee the software's correctness or quality. Testing provides insight into the software's behavior but can't prove its perfection.

Seven Testing Principles

Testing Code of Ethics

The Phycology of Testing

• Integrity and Accuracy: Testers uphold honesty and accuracy in their work, ensuring that testing activities and results are truthful and reliable.
• Impartiality: Testers remain objective and unbiased, avoiding conflicts of interest and treating all stakeholders fairly.
• Responsibility: Testers take responsibility for the quality of their work, striving to improve testing processes and outcomes.
• Transparency: Testers communicate openly and transparently, providing clear and accurate information to stakeholders.
• Public Interest: Testers prioritize the safety, health, and well-being of the public, considering the potential impact of their testing on society.
• Professionalism: Testers demonstrate professionalism in their interactions and conduct, upholding the reputation and credibility of the software testing field.
• Continuous Improvement: Testers commit to ongoing learning and development, staying current with industry practices and contributing to their own professional growth.
• Respect: Testers treat colleagues, stakeholders, and other professionals with respect and courtesy, fostering a positive and collaborative testing environment.

The psychology of testing explores the cognitive and behavioral aspects of testers and their interactions with the testing process. It delves into how human factors influence testing effectiveness, decision-making, and collaboration. Testers' psychological traits, biases, perceptions, and motivations impact their approach to testing and the quality of their work. Understanding the psychology of testing helps optimize testing strategies, improve communication, mitigate biases, and enhance overall testing outcomes.

Software Life Cycle

Test Cycle Closure

Test Plan is executed and the prepared test cases are run. All test results are recorded and documeted, all defects are logged for failed tests. Defect reports are created to track issues. Once defects are fixed, they are retested.

Test Cycle Closure phase is the completion of test execution. This includes test completion reporting, collection of test completion matrices and test results. The testing team will discuss and analyze the testing artifacts and complete a lessons learned activity. This will improve the future test cycles.

Execution

The Senior Test Manager will create the test plan/strategy and give estimates for test effort and cost for the project. Other things such as test environment, limitations, risks, scope and schedule are all determined at this stage. The testing tools are determined, roles and responsibilites are outlined for each member of the test team.

Test case development and Environment Setup

Test Planning

In test Case development phase,the creation, verification and rework of test cases & scripts is completed. The Test environment setup decides the software and hardware conditions under which a product is tested. The test team may not always be involved in the creation of the test environments as the development team may do it. Test data is also created if the environment is ready..

Requirement Analysis

The test team will review the requirements from a testing point of view to identify all testable requirements and will collaborate with the rest of the time to elaborate on any vague requirements. As part of this stage, testing priorities are outlined, and a requirement traceability matrix is created.

Test Support

5) Potential Benefits & Risks for Test Management Tools: Potential Benefits: Centralized test planning, organization, and execution. Efficient test case management and traceability. Comprehensive reporting and progress tracking. Collaboration and communication among testing teams and stakeholders. Risks: Initial setup and training for tool adoption. Overhead in maintaining and updating test cases and plans. Potential complexity in tool usage, leading to reduced efficiency.

4)Potential Benefits & Risks for Static Analysis Tools: Potential Benefits: Early detection of code defects and vulnerabilities. Consistent coding standards and best practices enforcement. Improved code quality and reduced technical debt. Enhanced security by identifying potential security vulnerabilities. Risks: False positives or false negatives leading to wasted time. Over-reliance on automated tools may lead to overlooking nuanced issues. Tool setup and configuration may require expertise and time investment.

3) Potential Benefits & Risks for Test Execution Tools: Potential Benefits: Efficiently execute a large number of test cases, saving time and effort. Accurate and repeatable tests, reducing human errors. Enhanced test coverage and quicker feedback on code changes. Integration with continuous integration/continuous deployment (CI/CD) pipelines for seamless testing. Risks: Initial setup and learning curve for tool usage. May not cover all testing scenarios, leading to potential blind spots. Maintenance overhead, especially when application changes or tool updates are needed.

Effective Use of Tools

Introducing a Tool into an Organisation

Types of Test Tools

Management of Testing & Tests - Test Management Tools: JIRA, TestRail, Zephyr Performance & Monitoring - Performance Testing Tools: JMeter, LoadRunner, Gatling Monitoring Tools: New Relic, AppDynamics, Prometheus Specific Testing Needs - Security Testing Tools: OWASP ZAP, Burp Suite Usability Testing Tools: UserTesting, Optimal Workshop Compatibility Testing Tools: BrowserStack, Sauce Labs Test Execution & Logging - Test Automation Tools: Selenium, Appium, Cypress Defect Tracking Tools: Bugzilla, JIRA, Redmine Test Specification - Requirements Management Tools: IBM Engineering Requirements Management DOORS, Jama Connect Test Design Tools: Enterprise Architect, Visual Paradigm Static Testing: Code Review Tools: Crucible, Gerrit Static Analysis Tools: SonarQube, ESLint (for JavaScript)

6) Main Considerations: When introducing a new testing tool to our organization, it's essential to carefully evaluate its potential impact and benefits. We need to consider factors such as the tool's compatibility with our existing processes, the needs of our teams, and the overall objectives of our testing efforts. Key considerations include:

• Alignment with our testing goals and methodologies.
• Integration capabilities with our existing toolchain and systems.
• Cost-effectiveness and return on investment.
• User-friendliness and ease of adoption.
• Training and support requirements for our teams.

7) Start with a Pilot Project: To ensure a smooth transition and evaluate the tool's effectiveness, we recommend starting with a pilot project. This involves selecting a well-defined and manageable project where the tool can be applied. The pilot project will allow us to:

• Assess the tool's functionality and suitability in a real-world scenario.
• Identify any challenges or obstacles specific to our organization's context.
• Gain practical experience and feedback from teams using the tool.
• Determine the tool's impact on efficiency, quality, and collaboration.

Management of Testing & Tests - Involves planning, organizing, and overseeing all aspects of testing activities to ensure quality and alignment with project goals. Performance & monitoring - Focuses on assessing software's speed, responsiveness, and resource utilization while continuously tracking and analyzing system behavior. Specific testing needs - Tailors testing processes to address unique requirements, such as security, usability, compatibility, or regulatory compliance. Test Execution & Logging - Executing test cases and recording outcomes systematically, ensuring efficient testing and providing a record of results. Test Specification - Document outlining test objectives, scope, resources, and procedures to guide testing efforts effectively. Static Testing - Reviewing documents and code to identify defects early in the development process, enhancing software quality and reducing costs.

8) Success Factors for Deployment: Successful deployment of the new testing tool hinges on several key factors:

• Clear Objectives: Define specific goals and objectives for using the tool to guide the implementation process.
• Effective Training: Provide comprehensive training and resources to ensure all team members can use the tool effectively.
• Engagement and Communication: Keep teams engaged, informed, and aligned throughout the deployment process.
• Continuous Improvement: Encourage ongoing feedback and iterative improvements based on user experiences.
• Support and Assistance: Offer dedicated support and assistance to address any challenges or questions.
• Measurable Results: Establish metrics to measure the tool's impact on testing efficiency, quality, and project outcomes.

Test Design Techniques

Experienced Based Testing

• Exploratory testing - Simultaneous learning, test design and test execution.
• Error Guessing - A tester will use their experience to guess problematic areas of the application.

Choosing the right testing techniques is very important. Many factors must be considered, such as;

• The risks and objectives of the project
• The type of system & the dev cycle
• Regulatory standards
• The time and budget of the project.
• Knowledge & experience of the test team

Structure Based or White Box Techniques

• Decision Testing & Coverage - All possible branches from each decision point are executed at least once. This implies that all the edges of the control flow graph are traversed
• Statement testing & Coverage - Each statement within the source code is executed and tested.
• Condition Testing/ Multiple Condition Testing
• All Path Testing

Choosing Test Techniques

Test Development Process

The test design specification will define test conditions, test approach and high level test cases associated with a test item. It will also determine which test suites and cases to run and which to skip. The test case specification gives a detailed summary of what test scenarios will be tested and how they will be tested. It specifies the tests purpose, identifies the step-by-step procedure, input, desired output and outline the pass/fail criteria for determining acceptance. The test procedure specification is a document that describes the specific sequence of actions for the execution of a test. The test procedure will test the implementation of a requirement.

• Equivalence Partitioning - Equivalence classes are evaluated for given input conditions
• Boundary Value Analysis - The extreme boundaries between partitions of the input values are tested.
• Decision Tables - A table that specifies which actions to perform depending on given conditions.
• State Transition Diagrams - A Diagram that describes all of the states an object can have, dependant on the conditions set upon it.
• Use Case Testing - Identifies test cases to cover an entire system from start to end. Helps identify gaps in an application that might not be found within unit testing.

Specification-based or Black-box Techniques

Test Management

Incident Management - Process of identifying, analyzing, and resolving issues during testing to maintain project progress and quality.Incident Logging - Systematically recording identified issues during testing, including details like description, severity, and reproduction steps. Test incident report - Formal document detailing a discovered issue's nature, impact, and steps to reproduce, aiding efficient issue resolution between testing and development teams.

Test Monitoring - Continuous oversight of testing activities to ensure alignment with plans and goals, making adjustments as needed for effective and efficient testing. Test Reporting & Control - Collecting, analyzing, and communicating testing information to guide decisions, manage risks, and maintain testing quality. Test Summary Report - Comprehensive document summarizing testing process, scope, results, and metrics, aiding stakeholders in assessing software quality and release readiness.

Independent Testing - An independent team who undertake testing activities that excludes the developer to avoid author bias. Tasks of a Test Lead - The test lead will plan, monitor and control the testing activities and tasks. They will create the test plan and all other activities involved within. Task of a Tester - Testers will help contribute to the test plan, they will create and execute test cases, report defects and gather metrics on the project.

Risks and Testing

Test Progress Monitoring & Control

Configuration Management

Test Organisation

Test Planning & Estimation

Establish and maintain the integreity of the products and ensure all items of test-ware are identified, version controlled, tracked for changes, related to each other.

Test Approaches - Analytics - Selecting and defining the approaches based on the analysis of some factors or conditions associated with the product. Model Based - The run time behaviour of a software is checked against predictions made by a model. Methodical - A pre-determined set of conditions such as quality standard or a checklist which may relate to a particular type of testing. Standards or process compliant strategy - A panel or committee for standards establishes guidelines or processes for tests to identify test conditions and test cases. Reactive - Tests are designed and implemented after the real software is delivered, tests are based on defects found within a system. Consultative Strategy - Consultations from key stakeholders to define scope of test conditions. Regressions Averse - Strategies focus on reducing regressions risks for functional and non functional parts

Test Planning -Test Policy explains the goals the project wishes to achieve through testing activities.Test Strategy details the general testing methods used by the project. Test Estimation is an activity which approximates how long a task takes to complete. Test Plan is a detailed document that details test strategies, objectives, schedule, estimations and resources for a project.

Risks and Testing

Risk: Probability / Likelihoos & Impacts - Risk involves assessing the likelihood of a potential issue occurring (probability) and the potential consequences if it does (impacts). This helps prioritize and manage risks effectively. Project & Product Risks - Project risks pertain to factors that might affect project success, such as timeline delays. Product risks relate to issues that could affect the quality or functionality of the final product. Risk-Based testing approach - A strategic testing approach that focuses testing efforts on areas with higher risks. This maximizes testing efficiency by addressing the most critical potential issues first.

Entry and Exit Criteria - Entry Criteria gives the prerequisite items that must be completed before testing can begin.Exit Criteria defines the items that must be completed before testing can be concluded.

Testing Road Map

Project Life Cycle

Fundamentals of Testing

Software Life Cycle

Test Support

Test Design Techniques

Test Management