Infra QA_24-Q1

Agenda

• Metrics and Team Performance
• Business and Infra Organization Growth's Impact to the Team
• PRBs into QA Opportunities
• Demo - Network Regression Unit Testing
• Roadmap
• Multi-Stage Testing Model in Detail
• Priority Items
• Q&A

Testing Coverage

Test Automation ROI

Team Resources

Infra-QA Dashboard

92+%

~$10,000/yr

Testing Coverage

Metrics

Up Next...

Enabled QA Test Automation

Tooling Innovations and Standardization

TARGET

servicES

App Support - SSL Certificate Testing

FEB 2022

• Cloud Services
• Database Services
• Core Infra Services
• Global IT Ops

• Application Support
• Cybersecurity
• Unified Communications

Multi-stage QA Testing

App Support - PGP Keys Testing

2024-Q2

APR 2022

2024-Q3

SCCM Compliance Test for Endpoints & Servers

QA Laboratory Expansion

APR 2023

2024-Q3

AI POV

Health Check Tool and Power BI Reporting

MAY 2023

TARGET

TOOLING

App Support - PROD Application Sanity

JUN 2023

Continuous

Automation Framework Enhancements

Network Engineering and Security - Regression

OCT 2023

TARGET

Core Infrastructure and SQL Services - Regression

UPSKILL

2024-Q2

2024

Global IT Infra Ops and APS Workflow Integration

SCCM

2024-Q4

2024

Citrix

Business Growth

Organization Growth

Operational Applications

Weekends Supported

Infra QA Resources

4 team members rotating the support

200+ IT Professionals across multiple timezones

~500 Applications and Services

60+ weekend days worked on 2023

5 M&As since 2021

90% in support of Global IT Ops, Network, and Applications

Growing Business + New Applications + New Team Members == Infra QA Team Continuous Improvement Ares and Ares IT have significantly grown in the last 3 years with Infra QA maintaining and optimizing deliverables.

Problems into opportunities

Issues and Challenges

Infrastructure and Network Unavailability and Instability

URL blocked/not accessible

Server not responding

Applications not operationally ready after change implementation

App functions not working

Application intermittent issues

Goal

Infrastructure and Application Stability

L1 Testing

L2 Testing

L3 Testing

Hardware, Ports, Policy Compliance

App Services and User-Facing Tests

Network Services and Site Certificates

QA Solution

Demo - Network Regression Testing

Infra QA Roadmap

Adapting, Scaling, and Standardizing QA Capabilities.

2023-Q4

2023-Q2

2024-Q4

2024-Q2

L2 Network Services and Certificate Testing

QA Lab Expansion and Modernization

Infra and App Process Integration

App/Service Owners Collaboration, Upskilling

2024-Q3

2024-Q1

2023-Q3

L3 Application Services Availability Testing

L1 Infra Hardware and Policy Compliance

Health Check Tool and PowerBI Reporting

Multi-Stage Infra Testing

"Get-Service" test for all required/dependent app services

"Test-Connection" test of all App, Web, and SQL Servers

L1

L3

L4

L2

Selenium Application Sanity Tests

Infrastructure Maintenance

Application Layer is ready to be tested

"Web-Request" test of all application URLs

ApplicationSupport Escalation

Global IT Ops Escalation

Application Onboarding and Infrastructure Requirements

Include Infra QA testing requirements when onboarding apps/services. Standardizing and scaling infrastructure/hardware requirements.

01

Application Services, Dependencies, and Priorities

IN PROGRESS -- Priotize the top/critical applications and services to identify dependencies for operation readiness.

02

IN PROGRESS -- Develop a Server Regression Test Pack that checks assets for availability, accessibility, and compliance.

Server Baseline Configurations and Tests

03

COMPLETED -- Enhanced Automated Network Regression Test Pack.

Network Baseline Configurations and Tests

04

top 5

Endpoint Baseline Configurations and Tests

COMPLETED -- Enhanced Workstation Regression Test Pack.

05

Priority Items

Next steps:

• Continuous improvements
• Partnership with the APS to determine critical applications
• Collaboration with the Application Owners to establish sanity testing requirements

• Process Maturity
• Estate review and analysis with industry standards/best practices
• Standardization of QA Metrics
• Change Management workflow enhancements
• Application Catalog information update
• Service Now QA Dashboard
• Power BI QA Dashboard

• Technology Advancements
• Apply AI best practices across QA processes
• Discovery and onboarding of new tools and solutions

How we communicate?

Perform verifications on networking components that dictate the flow of information.

Create accessibility and validity layers acting as "gate-checks" before full Application testing.

• Is the application URL accessible?
• Is the URL's certificate valid?
• Do we have specific or custom rules to route the traffic?
• These prerequisites will be reusable for Networking-specific changes.

Can't access...

URLs and resources are not accessible due to outages or changes.

Issues are not proactively caught/reported until a user reports it.

• Application URL is not accessible in a location.
• Service performance degradation.

All backend stuff is ready. Now what?

Execute identified application tests.

Utilize the existing QA Test Automation pack to run the application smoke and sanity verifications.

• Execute with confidence and stability.
• Infrastructure or Application issues are isolated and segregated.
• Efficient and precise test results

Got an idea?

Bring it to life with an interactive window

Create a new layer with all the Genially features.

• Generate experiences with your content.
• It’s got the Wow effect. Very Wow.
• Make sure your audience remembers the message.
• Activate and surprise your audience.

What users do not see?

Perform verifications on assets that are the backbone of the application.

Create efficiency and compliance verification layers acting as "gate-checks" before full Application testing.

• Is the server available?
• Is the server compliant?
• Are the needed ports and configurations accurate?
• These prerequisites will ensure the optimal utilization of testing runtime.

QA Lab Expansion

Increasing the footprint and reach of Infra QA testing.

Provision endpoints in all Ares Offices/Branches.

• Increased Network Engineering/Security Testing
• Asset to be readily available for troubleshooting
• Use the assets for the Health Check Tool initiative

L1 Component Testing

Establish an asset compliance test prior to executing user-facing validations.

Verify that application/services components are:

• Available
• Accessible
• Compliant with Industry Standards and internal Cybersecurity controls

Why is it not opening?

Application dependency on Infrastructure may not be fully ready yet.

Create a new layer with all the Genially features.

• False failures
• Inefficient utilization of runtime
• Induce delays in signing off

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

L2 Component Testing

Provide a scalable and reliable test approach to verify Network health.

Utilize PowerShell to:

• Check URL/Website certificate validity
• Invoke traffic to verify networking rules (whitelisting and blacklisting)

Stability and Security

Consistent, repeatable, and secure process to have services operationally-ready.

Ensure the following:

• Applications and Core Services are ready to use.
• Underlying infrastructure and technology are secure.
• Repeatable and consistent verifications.
• Lessen and augment problems and issues.

Got an idea?

Bring it to life with an interactive window

Create a new layer with all the Genially features.

• Generate experiences with your content.
• It’s got the Wow effect. Very Wow.
• Make sure your audience remembers the message.
• Activate and surprise your audience.

Why is it not working?

Applications may be served but not all functionalities are operational.

This impacts testing by:

• Unknown service dependencies
• Limitations and delineation of QA ownership

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

It was working before...

Rules may have been missed or have been inadvertently impacted by a recent change.

This impacts testing by:

• Incomplete, unfamiliar, or unknown testing requirement
• Induce delays in signing off

L2 Components

Verify that all networking components and rules are implemented.

QA Testing Process will include:

• Application URL certificate validity check.
• Application accessibility.
• Cloud applications whitelisting test.
• Restricted content blacklisting test.
• All passed? Then proceed to the next testing stage.

Process Integration

Seamlessly introduce the QA Test Automation Workflow into existing processes.

Infra and App Processes:

• Service Now - Service Catalog
• Jenkins
• Cybersecurity
• Global IT Infrastructure Ops
• Application Support
• Application Onboarding

L3 Component Testing

Review the L1 and L2 test results before executing user-facing tests.

Create a gate-check to optimize test execution runtime.

• Optimal use of execution runtime
• Infra issues are isolated and caught earlier in the test cycle
• Leverage the same test approach for Disaster Recovery scenarios

Not responding...

Server might not be fully operational just yet.

This impacts testing by:

• False failures
• Inefficient utilization of runtime
• Induce delays in signing off

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

L1 Components

Ensure that all corresponding hardware components are available and accurate.

QA Testing Process will include:

• Hardware/Asset availability.
• Port accesibility.
• GPO and Security Controls compliance.
• All passed? Then proceed to the next testing stage.

Collaboration

Partner and work with stakeholders.

Discover the application/service pre-requisites for operational stability.

• List of servers
• List of services
• Other dependencies
• Review existing test cases and create new ones as needed.

L3 Components

Validate that all previous components are working and proving the expected end-user functionality.

QA Testing Process will include:

• Application service/s availability.
• Application smoke and sanity test.
• Everything looking good? Then the application is operationally ready for use.

Health Check Tool

Regularly test and scan the environment for application and services stability.

Execute the Infra QA tests to catch potential issues.

• Proactively detect issues
• Establish stability and health trends
• Capitalize on constant unattended testing for pre and post change implementation
• Leverage data for AI/ML

It is up but not ready.

Integration points are not functioning as expected due to dependencies.

Inter-dependency among applications and services causing:

• Parts of the application to not work.
• User interruption.