SysEngineering

Harry — Helpdesk Agent

Harry is the natural-language Slack front end for IT requests. Employees DM him (“can I get Claude?”, “I’m locked out of Okta”, “how do I set up the VPN?”) and Harry either answers from the IT team’s own documentation with a citation, or files the right JSM request — where the helpdesk team’s workflows and automation take over (approval routing, Okta group assignment, provisioning).

Built EXACTLY like the live GW Admin Assistant (Greg): same FastAPI/Cloud Run skeleton, same Claude Opus 5 agentic tool loop, no intent lists or phrase matching anywhere. Harry decides nothing by keyword — the model reads the request and calls tools.

Nothing is trained or embedded. Harry can only state something written in a document it just read, and it links that document. Every answer is traceable, and a wrong answer is fixed by editing a page rather than retraining a model.

Status: live in the Hillspire production Slack. Files both JSM flows, relays ticket updates back, walks people through setup with the guides’ own screenshots, and — since 2026-09-24 — checks conference-room availability across every Workspace tenant (read-only). Filing is real when HARRY_DRY_RUN=false; the default is true. See HANDOFF.md for live deployment state and CLAUDE.md for design invariants and the decision log.

Division of labor (confirmed with the helpdesk team, 2026-08-21)

Piece Owner
Slack agent (this folder): NL understanding, catalog, entity stamping, policy checks, creating the IO ticket with the correct request type, requester auto-filled Systems Engineering (Adam)
Everything after the ticket exists: manager lookup via Okta (their OIDC app), approver assignment, approval notifications, Okta app assignment Helpdesk team (automations trigger off the request type)

Harry deliberately contains zero Okta code: manager resolution was considered and cut because their automation already does it the moment a 1780 ticket lands. A guardrail test fails the build if anyone adds an Okta client here. One exception since 2026-10-04 (ITPROJ-1372): okta_profile.py READS the requester’s own Okta record (GET only, no writes) for the my_profile tool, widened 2026-10-21 to the requester’s own assignment to catalog-listed apps (okta_app_ids). The same guardrail test pins it to that, and still fails for an Okta client anywhere else.

The two flows

The helpdesk team runs two JSM request types on the IO service desk (serviceDeskId 67):

Flow Request type What happens after Harry files
manager_approval 1780 “Approved access request” (every app except the five below) Their automation pulls the requester’s manager from Okta, adds them as approver, fires the approval notification; app assigned after approval
auto_assign 1760 “Access request” (Claude, ChatGPT, Zoom Workspace Reservation, Adobe Acrobat DC, Microsoft 365) Their automation assigns the app in Okta immediately (observed live: seconds)

Both are verified end to end in prod. On the approval path (IO-73276) the manager was resolved from Okta and set as approver 4 seconds after creation, and Product Type plus the Okta group add landed on approval.

Which flow an app takes lives in apps.py — two hardcoded tables, overridable via HARRY_APPS_JSON without a deploy. Since 2026-10-21 (Jason) the layout is AI first: Claude and ChatGPT are auto-assign (AI training still required), Zoom Workspace Reservation, Adobe Acrobat DC and Microsoft 365 stay automatic, and every other app needs manager approval. 22 entries, 21 per person (Adobe Acrobat Studio for For-Profit, Pro for Non-Profit); 5 auto-assign, 16 per person on approval. Harry suggests AI before filing, and for Lucidchart, Miro, Smartsheet, Airtable and Vena files only once the person says why AI (or, for Vena, OfficeConnect) won’t do. For Claude and ChatGPT themselves, the justification page accepts any work-related reason. HANDOFF, “AI-first rollout”. The model only ever sees catalog keys and flow names; request type IDs stay server-side. Every confirmed request = one IO ticket, created by a dedicated service account with raiseOnBehalfOf carrying the requester.

apps.py is only the apps Harry sets up himself. The company’s approved apps list is the SaaS Library (apps.APPROVED_APPS_URL), which includes them and many more. Any other app goes through request_unlisted_app to a person, after Harry checks where it stands on the SaaS Library: never if the company did not approve it, and one on neither list goes to the VSAQ form instead (temporary). HANDOFF, “SaaS Library”.

Their automation matches on the ticket SUMMARY — both the app name and the hyphenated entity label — so Harry’s catalog label is the join key. check_group_match.py replays their matcher offline; run it after any change on either side.

Entity status (For-Profit vs Non-Profit)

License type depends on the requester’s legal entity. entity_status.py ports the DOMAIN_TO_STATUS table from integrations/okta_entity_status/sync-entity-status.js: someone @theschmidt.org is Non-Profit no matter what anyone types; @hillspire.com is For-Profit. Derived from the verified Slack profile email, stamped into the JSM request, never overridable by the user or the model. Unmapped domain = Harry refuses to file and points at the regular helpdesk.

Knowledge layer — answering from documentation

28 whitelisted ITOPS guides, of which 5 support a step-by-step walkthrough with screenshots and 3 are office pages: Wi-Fi, printers, rooms, and who to ask on site, such as the office manager or where to get a mouse, where the page has it. Harry asks which office before answering from one. The whitelist IS the security boundary: internal IHI pages are excluded because some contain agent-directed text, and a test pins every page id to the ITOPS space. The one policy document Harry reads from IHI — the per-app justification standard — lives in a separate structure that read_help_guide cannot reach, and Harry is forbidden from quoting or linking it.

A year of IO tickets showed 12.2% were about topics already documented in the user-facing ITOPS Confluence space. knowledge.py lets Harry answer those from the real guides, with a link.

Deliberately small and curated:

Vendor help sites — product how-to the guides do not cover

Our guides cover how the company runs an app; they were never meant to say how to merge two PDFs in Acrobat or freeze a row in Excel. vendor_docs.py lets Harry answer those from the vendor’s own help site, through Anthropic’s server-side web search and web fetch. On in production; once on, the Confluence page below is the switch.

Conference rooms — availability across every Workspace (read-only)

Ask Harry things like:

Harry lists the office’s conference rooms and marks each free, partly (with the free windows), busy or unknown for the window asked about. Times are always the office’s local time and Harry names the zone. If no office is named, Harry asks — it never guesses one, and a bare “Newport” is refused as ambiguous (two sites).

What it will not do, on purpose:

Rooms live in several Google Workspace tenants (each tenant’s support@ created its own), so a single office is answered by fanning out to every tenant with that tenant’s own credentials and merging. Two read-only scopes, pinned by test; the per-tenant scope to append and the /diag/rooms inventory endpoint are in HANDOFF.md.

First live inventory (2026-09-24): 135 rooms, 16 offices, 29 tenants. Next up, in order: Zoom Room health on the answer (“free until 3, but its Zoom Room is down”); booking as the person via delegated access with an in-Slack chooser as the commit step; then a Zoom meeting on the event. Each is its own phase and its own scope decision.

Policy enforcement — KnowBe4 training

Paid AI seats require completed AI Awareness training — Claude, and ChatGPT as of 2026-09-21. training.py verifies it against KnowBe4’s Reporting API (read-only by construction — no write endpoints).

Not a model tool: the check runs inside submit_app_request, keyed off requires_training on the catalog entry, so the model can neither skip nor spoof it. The failure posture is asymmetric on purpose:

KnowBe4 says Harry does
explicitly not completed blocks, links the training dashboard
definitely not enrolled (not among its users, or the course exists and they have no enrollment) blocks the same way (since 2026-10-21)
explicitly completed files, ticket stamped verified complete via KnowBe4
anything ambiguous (no token, API down, a read that did not finish, no course matching the name) files as normal, stamped self-attested

Blocking on ambiguity would mean a drifted course name locks everyone out of Claude. With no KNOWBE4_API_TOKEN the gate is entirely inert.

Architecture

Employee ──DM/mention──▶ main.py (FastAPI, Slack Events API)
                           │  resolves requester email from Slack profile
                           │  injects their own open requests (read from Jira)
                           ▼
                         agent.py (Claude Opus 5 tool loop — Greg's, verbatim)
                           │  tools (helpdesk.py → active_tools()):
                           ├─ list_apps              the 21 apps Harry sets up + flows,
                           │                            and the SaaS Library link
                           ├─ read_help_guide        one whitelisted ITOPS page
                           │                            (knowledge.py, flag-gated)
                           ├─ web_search / web_fetch Anthropic server tools, locked by
                           │                            allowed_domains to the vendor help
                           │                            sites on a Confluence page
                           │                            (vendor_docs.py, flag-gated)
                           ├─ submit_app_request     → POST /rest/servicedeskapi/request
                           │                            duplicate guard → training gate
                           │                            → request type by flow, entity
                           │                            stamped, DRY-RUN by default
                           ├─ check_saas_library     → where any other app stands on the
                           │                            SaaS Library, read-only BigQuery
                           │                            (flag-gated)
                           ├─ request_unlisted_app   → any other app: an approved
                           │                            access request (1780) with a
                           │                            why-not-AI reason and no Okta
                           │                            group, so a person decides
                           ├─ check_request_status   → GET, requester's own tickets only
                           ├─ cancel_request         → customer-facing Cancel transition,
                           │                            their own PENDING requests only
                           ├─ start_walkthrough      → step-by-step setup, for the 5
                           │                            guides that carry real steps
                           ├─ show_screenshot        → the guide's OWN screenshots,
                           │                            uploaded into the conversation
                           ├─ which_platform         → Mac/Windows from JSM Assets,
                           │                            silently, or ask
                           ├─ my_profile             → the requester's own Okta record,
                           │                            read-only (flag-gated)
                           ├─ check_my_training      → their own KnowBe4 status, no filing
                           ├─ justification_standard → the helpdesk team's per-app bar,
                           │                            for JUDGING, never to recite
                           ├─ my_tickets             → every ticket they reported, any
                           │                            channel, latest public update
                           ├─ find_rooms             → room availability per office,
                           │                            busy/free only (flag-gated)
                           ├─ change_my_meeting      → a room or Zoom link on a meeting
                           │                            they organize, by button click
                           └─ file_help_ticket       → escalate to a human with the
                                                        diagnosis attached (type 1762)

Their JSM automation ──POST /jsm/notify──▶ main.py (notify.py)
   {"issue_key": "IO-123"}                   │  reads THAT ticket from Jira
   X-Harry-Secret: <shared>                  │  derives recipient + outcome
                                             ▼
                                           DM to the requester: approved /
                                           declined / withdrawn / provisioned /
                                           already had it / cancelled (reason unknown)

Guardrails live in code and the test_*.py files, not prompts: the model can’t pass identity, entity status, or request-type IDs (reserved args are stripped); an app exists in exactly one flow table; dry-run is the default and missing JSM config forces it; status lookups only return the requester’s own tickets; knowledge.py, training.py and monthly_report.py are asserted to contain no write path at all; and every web tool carries allowed_domains, while vendor_docs.py is asserted to make exactly one HTTP call, a GET of its own Confluence page.

Behaviours worth knowing. All but the last were added after live use found the need:

The improvement loop

monthly_report.py is a read-only monthly job that joins the month’s Jira tickets with Harry’s own harry_audit_v1 events and emits four lists. Tickets are the demand signal — they are never fed to Harry as knowledge.

List Signal Response
FIX a guide was read and the person filed a ticket anyway rewrite the guide
REACH Harry already files this, but it arrived by email/portal channel, not content
ADD genuinely uncovered demand write a guide / add an action
RETIRE guide never read obsolete, or its covers blurb is wrong

FIX is the performance loop — a per-document quality score. So the improvement mechanism is explicit: Harry doesn’t get smarter; the documented knowledge gets more complete and Harry’s reach over it gets wider, via a ranked backlog a human approves. Harry never edits Confluence or extends its own whitelist — the whitelist is the security boundary.

Running locally

pip install -r requirements.txt
pytest                                        # guardrail tests, no credentials needed

# Talk to Harry from the terminal — fastest way to tune the prompt (no Slack needed):
python try_harry.py "can i get claude?"
python try_harry.py --email someone@theschmidt.org "i need lucidchart"
python try_harry.py                           # interactive, keeps history
# Vendor help sites: set HARRY_VENDOR_DOCS_ENABLED=true first. A how-to question then
# runs real web searches ($10 per 1,000) and reads the sites page.

# Read-only recon:
python discover_jsm.py --check                # service account's JSM access
python discover_knowbe4.py --check --campaigns # KnowBe4 token + real course name
python check_group_match.py --poison          # replay their Okta-group matcher

# The monthly improvement report (read-only):
python monthly_report.py                      # FIX + RETIRE + metrics, no LLM cost
python monthly_report.py --days 30 --classify --out report.md

# File exactly one ticket, deliberately (prints the payload, needs --apply):
python file_test_ticket.py --app lucidchart --requester someone@hillspire.com

# Run the Slack service:
uvicorn main:app --port 8080

Say harry test to Harry in Slack for a live config/connectivity readout, or hit /diag — which reports the deployed commit, so “is my fix live?” is one look.

Deploy with .\deploy.ps1 (see HANDOFF.md). Flags are not sticky: pass -DryRunFlag false -KnowledgeFlag true every time you want both on.

Secrets

Google Secret Manager is the single machine-readable source, for both Cloud Run and local CLI runs. .env holds sm://<secret-name> pointers, which local_env.py resolves via the already-authenticated gcloud. The 1Password CLI is not installed here, so op:// references are skipped rather than resolved.

deploy.ps1 refuses to sync a pointer as a payload — doing so would overwrite the real credential with the literal reference string — and throws if a pointer’s target disagrees with the secret it mounts.

Configuration

Conference rooms (all optional; the tool is absent unless the flag AND the route list are set):

Variable Meaning
HARRY_ROOMS_ENABLED true to offer room availability (deploy.ps1 -RoomsFlag, default true).
GW_ROUTES_SECRET Secret Manager path of the shared tenant route list (a path, not a secret).
GOOGLE_CLOUD_PROJECT Project that short gw-sa-* secret names resolve in.
DIAG_TOKEN Guards /diag/rooms and /diag/llm; without it they 404. Mounted from Secret Manager (harry-diag-token); .env holds only the sm:// pointer.
HARRY_ROOMS_EXCLUDED_TENANTS Comma-separated route keys to skip.
HARRY_CSAT_ENABLED Default false. Five-star rating after a ticket resolves (csat.py, POST /slack/interactive). Needs the Slack Interactivity URL and the BigQuery sink first; see HANDOFF “CSAT ratings”
Env var Notes
ANTHROPIC_API_KEY Same key as the GW Admin Assistant. LLM_PROVIDER / ANTHROPIC_MODEL (claude-opus-5) / ANTHROPIC_EFFORT (low) as in Greg; Vertex + OpenAI fallbacks intact
SLACK_BOT_TOKEN / SLACK_SIGNING_SECRET From the Harry Slack app. Live app is in the Hillspire production workspace (TP8A8CZ7H)
SLACK_ALLOWED_USER_IDS / SLACK_ALLOWED_TEAM_IDS / SLACK_ALLOWLIST_REQUIRED Same fail-closed posture as Greg. Slack member IDs are per workspace
ATLASSIAN_BASE_URL / ATLASSIAN_EMAIL / ATLASSIAN_API_TOKEN The dedicated service account that files tickets and reads Confluence guides and the vendor sites page. Needs agent/on-behalf permission on IO for raiseOnBehalfOf
JSM_SERVICE_DESK_ID / JSM_REQUEST_TYPE_APPROVAL_ID / JSM_REQUEST_TYPE_AUTO_ID Prod: 67 / 1780 / 1760. Sandbox IDs differ — re-discover
HARRY_DRY_RUN Default true. Filing returns a receipt; nothing is created. Only the literal false disables it
HARRY_KNOWLEDGE_ENABLED Default false. Turns on the whitelisted ITOPS documentation guides
HARRY_VENDOR_DOCS_ENABLED Default false. Turns on vendor help sites (on in production); once on, the Confluence page is the switch. Vendor help sites through Anthropic web search and fetch, locked to the sites on the Confluence page (vendor_docs.VENDOR_SITES_PAGE_ID). First-party API only (not Vertex or OpenAI)
HARRY_OKTA_PROFILE_ENABLED Default false. Turns on my_profile, the requester’s own Okta record (okta_profile.py); see HANDOFF “Okta profile”
HARRY_OKTA_TOKEN_SECRET Secret Manager path (not a secret) to harryreadonly@’s SSWS token, read at runtime. harry test checks it whenever it is set
KNOWBE4_API_TOKEN Optional. Unset = the training gate is inert (everything self-attested). KNOWBE4_BASE_URL for non-US accounts; KNOWBE4_TRAINING_MATCH if the course is renamed
HARRY_DUPLICATE_WINDOW_DAYS Default 1. 0 disables the duplicate guard — needed to demo or re-test the same app twice in a day
HARRY_RAISE_ON_BEHALF Default on: file with raiseOnBehalfOf=<requester>; auto-retries without it if the service account lacks the permission
HARRY_APPS_JSON / HARRY_GUIDES_JSON Optional catalog / guide-table overrides without a deploy; the catalog override passes every field through except okta_app_ids, which lives in code only so widening the Okta read needs a reviewed change
JSM_PROJECT_KEY Default IO. Scopes request-key linkification so ordinary prose isn’t rewritten
JSM_ENTITY_STATUS_FIELD_ID Optional: customfield id to carry entity status once the helpdesk team confirms one
HARRY_BUILD Stamped by deploy.ps1; surfaced in /diag
JSM_REQUEST_TYPE_HELP_ID Escalation target, 1762 (“Questions”). Unset = Harry cannot raise a help ticket and says so plainly. 1762 accepts a summary ONLY — the details attach as a comment
HARRY_NOTIFY_SECRET Optional. Unset = /jsm/notify returns 503 and the callback is dark. Set = their automation can call it with X-Harry-Secret. Compared in constant time
HARRY_PREFS_ENABLED / HARRY_PREFS_TABLE / HARRY_PREFS_KEY_SECRET Default off. All three, or notification_settings does not exist. The key secret is a Secret Manager path (not a secret) to harry-prefs-key, which signs each saved setting. The table is the harry-prefs sink’s, project.dataset.table; harry test reads it whenever it is set. See HANDOFF “Notification settings”
DIAG_TOKEN Gates /diag/llm