SACA Prototype Evaluation

Security Audit Compliance Agent — FGD Questionnaire
Technology Acceptance Model (TAM) + Human Agency Scale (HAS)

This questionnaire has three stages. Please complete each stage at the time indicated by the facilitator. There are no right or wrong answers — your candid professional opinion is what matters.

1

Stage 1 — Before the demo (10–15 min)

Background, current practices, and prior expectations of AI in audit

2

Stage 2 — During the demo (20–30 min)

Immediate reactions to each prototype task, then facilitated group discussion

3

Stage 3 — After the demo (15–20 min)

TAM constructs, Human Agency Scale, zone mapping, and reflections

Your Participant ID (from facilitator)

Strictly confidential · For research purposes only · Participation is voluntary

Stage 1 of 3 — Pre-FGD

Before the Demo

Complete this section before the prototype demonstration begins. Estimated time: 10–15 minutes.

Section A

Demographic & Professional Background

A1Years of Experience in Audit / IT Audit / Information Security

A2Primary Role

A3Familiarity with PCAP / Network Log Analysis

A4Current Tools Used for Network Security Analysis (select all that apply)

A5Experience with AI / Machine Learning Tools in Audit Work

Section B

Current Audit Practices & Pain Points

Rate each statement: 1 = Strongly Disagree → 5 = Strongly Agree

Section C

Prior Expectations of AI-Assisted Audit Tools

Rate each statement before you see the demonstration.

Stage 2 of 3 — During FGD

During the Demo

Record your immediate reactions after each demonstration step. The facilitator will guide the timing.

⚡ Complete each task block immediately after watching that step — before moving to the next one. Keep responses brief; depth comes in Stage 3.

Section D

Task-Level Reactions

Section E

Group Discussion — Key Points

The facilitator will lead discussion on each prompt. Jot down key points or quotes you want to remember — these are for your own reference and will be captured separately by the facilitator.

Stage 3 of 3 — Post-FGD

After the Demo

Complete this section after the demonstration and group discussion. Estimated time: 15–20 minutes. Scale: 1 = Strongly Disagree → 5 = Strongly Agree unless stated otherwise.

Section F — TAM

Perceived Usefulness

Section G — TAM

Perceived Ease of Use

Section H — TAM

Trust & Explainability

Section I — HAS

Human Agency Scale

The HAS (Shao et al., Stanford 2025) measures the degree of human involvement needed when AI assists with a task.

LevelTeam dynamicHuman involvementAI role

H1AI drives taskNone requiredAutomation

H2AI drives taskMinimal — key checkpointsAutomation

H3Equal partnershipCollaborate throughoutAugmentation

H4Human drives taskAI needs your inputAugmentation

H5Human drives taskContinuous involvementAugmentation

I-AHAS Level per Audit Task — circle H1–H5 for each task

I-BAutomation Desire per Task (1 = Not at all → 5 = Entirely)

Consider both the benefit of freeing up your time AND concerns about professional control or job displacement.

I-CTask Characteristics — Drivers of Your HAS Preference

Rate how strongly each dimension applies to IT audit compliance work using this prototype.

I-DReasons for Wanting Collaboration (if you chose H3–H5 for any task — select all that apply)

I-EReasons for Wanting Full Automation (if you chose H1–H2 for any task — select all that apply)

I-FOverall HAS Preference for This Prototype

Considering ALL the audit tasks as a whole, which single HAS level best describes your overall preferred level of collaboration?

I-F.2Briefly explain why you chose this level:

Section J — TAM

Behavioral Intention to Adopt

Section K — HAS

Desire–Capability Zone Mapping

For each audit task, assign it to one of four zones based on your assessment of AI capability and your desire for automation today. 🟢 Green Light = High desire + High capability (deploy now) · 🔴 Red Light = Capable but I resist · 🟡 R&D Opportunity = I want it but not ready · ⚪ Low Priority = Low on both

Section L

Open-Ended Reflections

L1What is the single most compelling feature of this prototype for your audit work?

L2What is the most significant barrier to adoption of this tool in your organization?

L3Any additional comments, suggestions, or concerns about the prototype?

✓

Thank you!

Your responses have been submitted and saved successfully.

Your professional insight is invaluable for improving AI-assisted IT audit tools that genuinely serve the practitioner community.

If you have any questions about this study, please contact the facilitator.

Participant ID: · Session complete