Infra QA_24-Q1

• Metrics and Team Performance
• Business and Infra Organization Growth's Impact to the Team
• PRBs into QA Opportunities
• Demo - Network Regression Unit Testing
• Roadmap
• Multi-Stage Testing Model in Detail
• Priority Items
• Q&A

Agenda

2024

Citrix

2024

SCCM

UPSKILL

TARGET

2024-Q3

AI POV

Team Resources

Continuous

Global IT Infra Ops and APS Workflow Integration

Core Infrastructure and SQL Services - Regression

2024-Q4

2024-Q2

Automation Framework Enhancements

TARGET

TOOLING

• Cloud Services
• Database Services
• Core Infra Services
• Global IT Ops

• Application Support
• Cybersecurity
• Unified Communications

OCT 2023

JUN 2023

Network Engineering and Security - Regression

App Support - PROD Application Sanity

2024-Q3

2024-Q2

SCCM Compliance Test for Endpoints & Servers

Multi-stage QA Testing

TARGET

servicES

MAY 2023

APR 2023

APR 2022

FEB 2022

Tooling Innovations and Standardization

Health Check Tool and Power BI Reporting

QA Laboratory Expansion

App Support - PGP Keys Testing

App Support - SSL Certificate Testing

Enabled QA Test Automation

Up Next...

Metrics

Test Automation ROI

Testing Coverage

~$10,000/yr

92+%

Infra-QA Dashboard

Testing Coverage

90% in support of Global IT Ops, Network, and Applications

4 team members rotating the support

Infra QA Resources

60+ weekend days worked on 2023

Weekends Supported

~500 Applications and Services

Operational Applications

200+ IT Professionals across multiple timezones

Organization Growth

5 M&As since 2021

Business Growth

Growing Business + New Applications + New Team Members == Infra QA Team Continuous Improvement Ares and Ares IT have significantly grown in the last 3 years with Infra QA maintaining and optimizing deliverables.

Applications not operationally ready after change implementation

Server not responding

URL blocked/not accessible

App functions not working

Application intermittent issues

Infrastructure and Network Unavailability and Instability

App Services and User-Facing Tests

L3 Testing

Network Services and Site Certificates

L2 Testing

Hardware, Ports, Policy Compliance

Goal

Issues and Challenges

QA Solution

Infrastructure and Application Stability

L1 Testing

Problems into opportunities

Demo - Network Regression Testing

Infra and App Process Integration

2024-Q4

Adapting, Scaling, and Standardizing QA Capabilities.

App/Service Owners Collaboration, Upskilling

2024-Q2

QA Lab Expansion and Modernization

2023-Q2

L2 Network Services and Certificate Testing

2023-Q4

L3 Application Services Availability Testing

2024-Q3

Health Check Tool and PowerBI Reporting

2023-Q3

L1 Infra Hardware and Policy Compliance

2024-Q1

Infra QA Roadmap

ApplicationSupport Escalation

"Get-Service" test for all required/dependent app services

"Test-Connection" test of all App, Web, and SQL Servers

Selenium Application Sanity Tests

Infrastructure Maintenance

Global IT Ops Escalation

Application Layer is ready to be tested

"Web-Request" test of all application URLs

Multi-Stage Infra Testing

L2

L4

L3

L1

COMPLETED -- Enhanced Workstation Regression Test Pack.

COMPLETED -- Enhanced Automated Network Regression Test Pack.

IN PROGRESS -- Develop a Server Regression Test Pack that checks assets for availability, accessibility, and compliance.

03

02

01

05

04

Server Baseline Configurations and Tests

Application Services, Dependencies, and Priorities

IN PROGRESS -- Priotize the top/critical applications and services to identify dependencies for operation readiness.

Application Onboarding and Infrastructure Requirements

Include Infra QA testing requirements when onboarding apps/services. Standardizing and scaling infrastructure/hardware requirements.

Endpoint Baseline Configurations and Tests

Network Baseline Configurations and Tests

top 5

Priority Items

• Continuous improvements
• Partnership with the APS to determine critical applications
• Collaboration with the Application Owners to establish sanity testing requirements

• Process Maturity
• Estate review and analysis with industry standards/best practices
• Standardization of QA Metrics
• Change Management workflow enhancements
• Application Catalog information update
• Service Now QA Dashboard
• Power BI QA Dashboard

• Technology Advancements
• Apply AI best practices across QA processes
• Discovery and onboarding of new tools and solutions

Next steps:

How we communicate?

Perform verifications on networking components that dictate the flow of information.

Create accessibility and validity layers acting as "gate-checks" before full Application testing.

• Is the application URL accessible?
• Is the URL's certificate valid?
• Do we have specific or custom rules to route the traffic?
• These prerequisites will be reusable for Networking-specific changes.

Can't access...

URLs and resources are not accessible due to outages or changes.

Issues are not proactively caught/reported until a user reports it.

• Application URL is not accessible in a location.
• Service performance degradation.

All backend stuff is ready. Now what?

Execute identified application tests.

Utilize the existing QA Test Automation pack to run the application smoke and sanity verifications.

• Execute with confidence and stability.
• Infrastructure or Application issues are isolated and segregated.
• Efficient and precise test results

Got an idea?

Bring it to life with an interactive window

Create a new layer with all the Genially features.

• Generate experiences with your content.
• It’s got the Wow effect. Very Wow.
• Make sure your audience remembers the message.
• Activate and surprise your audience.

What users do not see?

Perform verifications on assets that are the backbone of the application.

Create efficiency and compliance verification layers acting as "gate-checks" before full Application testing.

• Is the server available?
• Is the server compliant?
• Are the needed ports and configurations accurate?
• These prerequisites will ensure the optimal utilization of testing runtime.

QA Lab Expansion

Increasing the footprint and reach of Infra QA testing.

Provision endpoints in all Ares Offices/Branches.

• Increased Network Engineering/Security Testing
• Asset to be readily available for troubleshooting
• Use the assets for the Health Check Tool initiative

L1 Component Testing

Establish an asset compliance test prior to executing user-facing validations.

Verify that application/services components are:

• Available
• Accessible
• Compliant with Industry Standards and internal Cybersecurity controls

Why is it not opening?

Application dependency on Infrastructure may not be fully ready yet.

Create a new layer with all the Genially features.

• False failures
• Inefficient utilization of runtime
• Induce delays in signing off

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

L2 Component Testing

Provide a scalable and reliable test approach to verify Network health.

Utilize PowerShell to:

• Check URL/Website certificate validity
• Invoke traffic to verify networking rules (whitelisting and blacklisting)

Stability and Security

Consistent, repeatable, and secure process to have services operationally-ready.

Ensure the following:

• Applications and Core Services are ready to use.
• Underlying infrastructure and technology are secure.
• Repeatable and consistent verifications.
• Lessen and augment problems and issues.

Got an idea?

Bring it to life with an interactive window

Create a new layer with all the Genially features.

• Generate experiences with your content.
• It’s got the Wow effect. Very Wow.
• Make sure your audience remembers the message.
• Activate and surprise your audience.

Why is it not working?

Applications may be served but not all functionalities are operational.

This impacts testing by:

• Unknown service dependencies
• Limitations and delineation of QA ownership

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

It was working before...

Rules may have been missed or have been inadvertently impacted by a recent change.

This impacts testing by:

• Incomplete, unfamiliar, or unknown testing requirement
• Induce delays in signing off

L2 Components

Verify that all networking components and rules are implemented.

QA Testing Process will include:

• Application URL certificate validity check.
• Application accessibility.
• Cloud applications whitelisting test.
• Restricted content blacklisting test.
• All passed? Then proceed to the next testing stage.

Process Integration

Seamlessly introduce the QA Test Automation Workflow into existing processes.

Infra and App Processes:

• Service Now - Service Catalog
• Jenkins
• Cybersecurity
• Global IT Infrastructure Ops
• Application Support
• Application Onboarding

Create a gate-check to optimize test execution runtime.

• Optimal use of execution runtime
• Infra issues are isolated and caught earlier in the test cycle
• Leverage the same test approach for Disaster Recovery scenarios

L3 Component Testing

Review the L1 and L2 test results before executing user-facing tests.

Not responding...

Server might not be fully operational just yet.

This impacts testing by:

• False failures
• Inefficient utilization of runtime
• Induce delays in signing off

Got an idea?

Use this space to add awesome interactivity. Include text, images, videos, tables, PDFs... even interactive questions! Premium tip: Get information on how your audience interacts with your creation:

• Visit the Analytics settings;
• Activate user tracking;
• Let the communication flow!

L1 Components

Ensure that all corresponding hardware components are available and accurate.

QA Testing Process will include:

• Hardware/Asset availability.
• Port accesibility.
• GPO and Security Controls compliance.
• All passed? Then proceed to the next testing stage.

Collaboration

Partner and work with stakeholders.

Discover the application/service pre-requisites for operational stability.

• List of servers
• List of services
• Other dependencies
• Review existing test cases and create new ones as needed.

L3 Components

Validate that all previous components are working and proving the expected end-user functionality.

QA Testing Process will include:

• Application service/s availability.
• Application smoke and sanity test.
• Everything looking good? Then the application is operationally ready for use.

Health Check Tool

Regularly test and scan the environment for application and services stability.

Execute the Infra QA tests to catch potential issues.

• Proactively detect issues
• Establish stability and health trends
• Capitalize on constant unattended testing for pre and post change implementation
• Leverage data for AI/ML

It is up but not ready.

Integration points are not functioning as expected due to dependencies.

Inter-dependency among applications and services causing:

• Parts of the application to not work.
• User interruption.