Infra QA_24-Q1

Agenda

• Metrics and Team Performance
• Business and Infra Organization Growth's Impact to the Team
• PRBs into QA Opportunities
• Demo - Network Regression Unit Testing
• Roadmap
• Multi-Stage Testing Model in Detail
• Priority Items
• Q&A

Testing Coverage

Infra-QA Dashboard

92+%

4

~$10,000/yr

Testing Coverage

Test Automation ROI

Metrics

Up Next...

Enabled QA Test Automation

App Support - SSL Certificate Testing

App Support - PGP Keys Testing

QA Laboratory Expansion

Health Check Tool and Power BI Reporting

Tooling Innovations and Standardization

FEB 2022

APR 2022

APR 2023

MAY 2023

servicES

TARGET

Multi-stage QA Testing

SCCM Compliance Test for Endpoints & Servers

2024-Q2

2024-Q3

App Support - PROD Application Sanity

Network Engineering and Security - Regression

JUN 2023

OCT 2023

• Cloud Services
• Database Services
• Core Infra Services
• Global IT Ops

• Application Support
• Cybersecurity
• Unified Communications

TOOLING

TARGET

Automation Framework Enhancements

2024-Q2

2024-Q4

Core Infrastructure and SQL Services - Regression

Global IT Infra Ops and APS Workflow Integration

Continuous

Team Resources

AI POV

2024-Q3

TARGET

UPSKILL

SCCM

2024

Citrix

2024

Growing Business + New Applications + New Team Members == Infra QA Team Continuous ImprovementAres and Ares IT have significantly grown in the last 3 years with Infra QA maintaining and optimizing deliverables.

Business Growth

5 M&As since 2021

Organization Growth

200+ IT Professionals across multiple timezones

Operational Applications

~500 Applications and Services

Weekends Supported

60+ weekend days worked on 2023

Infra QA Resources

4 team members rotating the support

90% in support of Global IT Ops, Network, and Applications

Problems into opportunities

L1 Testing

Infrastructure and Application Stability

QA Solution

Issues and Challenges

Goal

Hardware, Ports, Policy Compliance

L2 Testing

Network Services and Site Certificates

L3 Testing

App Services and User-Facing Tests

Infrastructure and Network Unavailability and Instability

Application intermittent issues

App functions not working

URL blocked/not accessible

Server not responding

Applications not operationally ready after change implementation

Demo - Network Regression Testing

Infra QA Roadmap

2024-Q1

L1 Infra Hardware and Policy Compliance

2023-Q3

Health Check Tool and PowerBI Reporting

2024-Q3

L3 Application Services Availability Testing

2023-Q4

L2 Network Services and Certificate Testing

2023-Q2

QA Lab Expansion and Modernization

2024-Q2

App/Service Owners Collaboration, Upskilling

Adapting, Scaling, and Standardizing QA Capabilities.

2024-Q4

Infra and App Process Integration

L1

L3

L4

L2

Multi-Stage Infra Testing

"Web-Request" test of all application URLs

Application Layer is ready to be tested

Global IT Ops Escalation

Infrastructure Maintenance

Selenium Application Sanity Tests

"Test-Connection" test of all App, Web, and SQL Servers

"Get-Service" test for all required/dependent app services

ApplicationSupport Escalation

Priority Items

top 5

Network Baseline Configurations and Tests

Endpoint Baseline Configurations and Tests

Include Infra QA testing requirements when onboarding apps/services.Standardizing and scaling infrastructure/hardware requirements.

Application Onboarding and Infrastructure Requirements

IN PROGRESS -- Priotize the top/critical applications and services to identify dependencies for operation readiness.

Application Services, Dependencies, and Priorities

Server Baseline Configurations and Tests

04

05

01

02

03

IN PROGRESS -- Develop a Server Regression Test Pack that checks assets for availability, accessibility, and compliance.

COMPLETED -- Enhanced Automated Network Regression Test Pack.

COMPLETED -- Enhanced Workstation Regression Test Pack.

Next steps:

• Continuous improvements
• Partnership with the APS to determine critical applications
• Collaboration with the Application Owners to establish sanity testing requirements

• Process Maturity
• Estate review and analysis with industry standards/best practices
• Standardization of QA Metrics
• Change Management workflow enhancements
• Application Catalog information update
• Service Now QA Dashboard
• Power BI QA Dashboard

• Technology Advancements
• Apply AI best practices across QA processes
• Discovery and onboarding of new tools and solutions

• Is the application URL accessible?
• Is the URL's certificate valid?
• Do we have specific or custom rules to route the traffic?
• These prerequisites will be reusable for Networking-specific changes.

Create accessibility and validity layers acting as "gate-checks" before full Application testing.

Perform verifications on networking components that dictate the flow of information.

How we communicate?

• Application URL is not accessible in a location.
• Service performance degradation.

Issues are not proactively caught/reported until a user reports it.

URLs and resources are not accessible due to outages or changes.

Can't access...

• Execute with confidence and stability.
• Infrastructure or Application issues are isolated and segregated.
• Efficient and precise test results

Utilize the existing QA Test Automation pack to run the application smoke and sanity verifications.

Execute identified application tests.

All backend stuff is ready. Now what?

• Generate experiences with your content.
• It’s got the Wow effect. Very Wow.
• Make sure your audience remembers the message.
• Activate and surprise your audience.

Create a new layer with all the Genially features.

Bring it to life with an interactive window

Got an idea?

• Is the server available?
• Is the server compliant?
• Are the needed ports and configurations accurate?
• These prerequisites will ensure the optimal utilization of testing runtime.

Create efficiency and compliance verification layers acting as "gate-checks" before full Application testing.

Perform verifications on assets that are the backbone of the application.

What users do not see?

• Increased Network Engineering/Security Testing
• Asset to be readily available for troubleshooting
• Use the assets for the Health Check Tool initiative

Provision endpoints in all Ares Offices/Branches.

Increasing the footprint and reach of Infra QA testing.

QA Lab Expansion

• Available
• Accessible
• Compliant with Industry Standards and internal Cybersecurity controls

Verify that application/services components are:

Establish an asset compliance test prior to executing user-facing validations.

L1 Component Testing

• False failures
• Inefficient utilization of runtime
• Induce delays in signing off

Create a new layer with all the Genially features.

Application dependency on Infrastructure may not be fully ready yet.

Why is it not opening?

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions!Premium tip: Get information on how your audience interacts with your creation:

Got an idea?

Here you can include a relevant fact to highlight

Here you can include a relevant fact to highlight

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions!Premium tip: Get information on how your audience interacts with your creation:

Got an idea?

Here you can include a relevant fact to highlight

Here you can include a relevant fact to highlight

• Check URL/Website certificate validity
• Invoke traffic to verify networking rules (whitelisting and blacklisting)

Utilize PowerShell to:

Provide a scalable and reliable test approach to verify Network health.

L2 Component Testing

• Applications and Core Services are ready to use.
• Underlying infrastructure and technology are secure.
• Repeatable and consistent verifications.
• Lessen and augment problems and issues.

Ensure the following:

Consistent, repeatable, and secure process to have services operationally-ready.

Stability and Security

• Generate experiences with your content.
• It’s got the Wow effect. Very Wow.
• Make sure your audience remembers the message.
• Activate and surprise your audience.

Create a new layer with all the Genially features.

Bring it to life with an interactive window

Got an idea?

• Unknown service dependencies
• Limitations and delineation of QA ownership

This impacts testing by:

Applications may be served but not all functionalities are operational.

Why is it not working?

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions!Premium tip: Get information on how your audience interacts with your creation:

Got an idea?

Here you can include a relevant fact to highlight

Here you can include a relevant fact to highlight

• Incomplete, unfamiliar, or unknown testing requirement
• Induce delays in signing off

This impacts testing by:

Rules may have been missed or have been inadvertently impacted by a recent change.

It was working before...

• Application URL certificate validity check.
• Application accessibility.
• Cloud applications whitelisting test.
• Restricted content blacklisting test.
• All passed? Then proceed to the next testing stage.

QA Testing Process will include:

Verify that all networking components and rules are implemented.

L2 Components

• Service Now - Service Catalog
• Jenkins
• Cybersecurity
• Global IT Infrastructure Ops
• Application Support
• Application Onboarding

Infra and App Processes:

Seamlessly introduce the QA Test Automation Workflow into existing processes.

Process Integration

Review the L1 and L2 test results before executing user-facing tests.

L3 Component Testing

• Optimal use of execution runtime
• Infra issues are isolated and caught earlier in the test cycle
• Leverage the same test approach for Disaster Recovery scenarios

Create a gate-check to optimize test execution runtime.

• False failures
• Inefficient utilization of runtime
• Induce delays in signing off

This impacts testing by:

Server might not be fully operational just yet.

Not responding...

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions!Premium tip: Get information on how your audience interacts with your creation:

Got an idea?

Here you can include a relevant fact to highlight

Here you can include a relevant fact to highlight

• Hardware/Asset availability.
• Port accesibility.
• GPO and Security Controls compliance.
• All passed? Then proceed to the next testing stage.

QA Testing Process will include:

Ensure that all corresponding hardware components are available and accurate.

L1 Components

• List of servers
• List of services
• Other dependencies
• Review existing test cases and create new ones as needed.

Discover the application/service pre-requisites for operational stability.

Partner and work with stakeholders.

Collaboration

• Application service/s availability.
• Application smoke and sanity test.
• Everything looking good? Then the application is operationally ready for use.

QA Testing Process will include:

Validate that all previous components are working and proving the expected end-user functionality.

L3 Components

• Proactively detect issues
• Establish stability and health trends
• Capitalize on constant unattended testing for pre and post change implementation
• Leverage data for AI/ML

Execute the Infra QA tests to catch potential issues.

Regularly test and scan the environment for application and services stability.

Health Check Tool

• Parts of the application to not work.
• User interruption.

Inter-dependency among applications and services causing:

Integration points are not functioning as expected due to dependencies.

It is up but not ready.