Bay automation runbook

How the automation stack actually works — what commands fire when, what to check when something fails, who to call.

Back
Operator reference · authored Markdown rendered to HTML
Bay automation stack — what fires when, what to check when it doesn't
When something breaks at 6:55 PM and there's a booking at 7, this page is what you read instead of paging vendor support.
v 4.2 · current Last updated Jun 2 by Mike Smith · 12 prior revisions ›

1 · Architecture overview

Five layers in the bay-automation stack. A failure at layer N usually surfaces as a symptom at layer N+1, so triage top-to-bottom.

1
Smart plug · Kasa HS300 Wi-Fi-controlled outlet feeding the bay PC + monitor. Polled every 30s. If unreachable → bay reads "Out of order". Likely cause: router or Wi-Fi credentials rotated.
2
Monitor · 4K display Powers from the smart plug. Wakes on HDMI handshake. If powered but black → HDMI handshake stalled. Pull-and-reseat the HDMI usually resolves.
3
Bay PC · Windows 11 Always-on at the OS level. Powers via smart plug. If unresponsive → confirm via remote-desktop helper before truck-rolling. The PC also runs the local sync agent.
4
Simulator software · Trackman or Foresight FSX Auto-launched on PC boot. Talks to vendor license server on cold-launch. If splash hangs at "Connecting…" → vendor license server is slow. See per-vendor quirks.
5
Ball-tracking · radar or camera array Calibrates on first shot of each session. Recalibration triggered by tap. If shots read flat/short → wipe lens, re-calibrate. If still bad → temperature drift > 8°C since last calibration.

The local sync agent on each bay PC pushes telemetry to POST /operator/bay-automations/sync every 30s. The "Last sync 18s ago" tile on the dashboard reads this endpoint's last_received_at field — see §4.

2 · Booking lifecycle

Five scheduled commands fire per booking. Times are relative to the booking start; "min" = minutes.

  • T − 10 min
    Power on bay PC + monitor Sends POST to smart plug · expects {status: "on"} within 10s · retry 3× on timeout
  • T − 5 min
    Wake monitor · HDMI handshake PC sends DDC "wake" command · waits for monitor EDID response · 60s timeout
  • T − 1 min
    Launch simulator software Spawns the FSX or Trackman process · waits for license-server handshake · 90s timeout · falls through to alert if hung
  • T = start
    Session active · door code valid SMS door code goes out · ball-tracking idle until first shot · session timer starts
  • T + end
    Graceful shutdown Saves session data · closes sim software · monitor sleep · smart plug stays on for 30 min idle
  • T + 30 min idle
    Power off if no new booking Cuts smart plug if no booking within 30 min · skipped if a booking starts in the next 15 min (avoids on/off thrash)

Lifecycle is configurable per club — defaults shown above. To change timings see Bay automation configuration ›

3 · Common failures

Four issues account for ~85% of automation failures in the past 90 days. Triage in order.

Power-on timeout

Smart plug doesn't respond within 10s ~50% of failures
What to check Confirm the bay's smart plug is reachable: dashboard → bay tile reads "Out of order" if 3 retries failed. Check the smart-plug LED — solid = networked, blinking = Wi-Fi lost.
What to try
  1. Tap Force on in the bay tile to manually trigger the smart plug.
  2. If still no response, walk to the bay and toggle the smart plug button physically.
  3. If physical toggle works but app doesn't → router reboot. See vendor contacts if router has shipped a firmware update overnight.

Monitor wake fails (HDMI handshake)

PC powered but monitor stays black ~22% of failures
What to check Wiggle the HDMI cable at both ends. Most often the cable end at the PC has worked loose.
What to try
  1. Pull and reseat the HDMI cable at both ends.
  2. If still black → power-cycle the monitor (button on back, 10 sec).
  3. If still black → swap the HDMI cable from a spare bay; if that fixes it, the cable is bad.

Simulator launch hangs at "Connecting…"

Sim software stuck on license-server splash ~15% of failures
What to check Trackman or Foresight license server status at status.trackman.com / status.foresightsports.com respectively.
What to try
  1. Wait 2 minutes — vendor license servers are usually back within 90s.
  2. If status page reports green but splash still hangs → close the sim and re-launch from the desktop shortcut.
  3. If sim launches in offline mode (no license check) → bay is usable but no shot validation. Switch to manual scoring.

Ball-tracking miscalibrated

Shots read flat, short, or sideways ~8% of failures
What to check First shot of each session triggers auto-calibration. If air temperature shifted > 8°C since last calibration (open vs closed door, AC cycle), recal is required.
What to try
  1. Wipe the radar / camera lens with a microfiber cloth.
  2. Tap Recalibrate in the bay's sim software (Trackman: Tools → Recalibrate; Foresight: settings gear → Recalibrate).
  3. If still off → take 3 known shots (driver, 7-iron, putter) and compare to expected values. Off by > 15% → call vendor support.

4 · Hardware sync explained

What the "Last sync 18s ago" tile means and what to do when it goes stale.

Each bay PC runs a small local sync agent that posts telemetry (power state, monitor state, sim software heartbeat, current session, last shot timestamp) to POST /operator/bay-automations/sync every 30 seconds. The last_received_at column on the dashboard is when the cloud last got a packet.

What stale means

  • 0–60s: healthy. Tile shows "X seconds ago" in green.
  • 60–180s: warning. One missed beat is fine; two is a hint.
  • 180s+ ("stale"): the sync agent is down. Bay state on the dashboard is now last-known, not current.

How to manually re-sync

  1. From the bay's detail page, tap Force sync now. This triggers an immediate POST from the local agent.
  2. If that succeeds, the tile flips green within 5s. If it doesn't, the agent process is dead.
  3. To restart the agent: remote-desktop into the bay PC and run net stop snagatime-sync && net start snagatime-sync.
  4. If the agent won't start, check Event Viewer for the snagatime-sync service. Usually a TLS cert refresh or a Windows update is the culprit.

5 · Per-vendor quirks

Vendor-specific behavior that isn't documented well by the vendors themselves.

Trackman

  • License-server outages — Trackman's status page lags actual recovery by 5–10 min. If splash hangs, try a re-launch even if status is still red.
  • Update behavior — auto-updates run on the first launch of the day. A morning booking can hit a 4-minute update window. Schedule updates manually overnight if morning bookings are critical.
  • Recalibration — Tools → Recalibrate. Walk-up calibration takes 30s.
  • Support phone — see contacts. The vendor's first-line support reads from a script; if you can describe the issue in their language ("license handshake timeout", "DDC wake failure"), you skip 2 layers.

Foresight

  • FSX Play vs FSX 2020 — different config files; check %PROGRAMDATA%\Foresight\fsx-version.txt for which is on each bay.
  • Camera dust — Foresight's overhead camera array is more dust-sensitive than radar. Daily wipe-down recommended for evening leagues.
  • USB drift — the GCQuad's USB connection occasionally drops on Windows update reboots. If a bay reads "no device" after a reboot, replug USB once.
  • Recalibration — Settings → Calibration → Quick recal. 45s.

6 · Emergency contacts

Escalation order: bay operator → club owner → vendor support → SnagATime ops.

Club owner
Mike Smith
(425) 555-0118
Available 8am–11pm · text-first preferred
Trackman support
Trackman customer success
+1 (888) 487-2562
24/7 phone · ask for "venue operations"
Foresight support
Foresight commercial team
+1 (888) 213-0035
M–F 8am–6pm PT · ticket via support.foresightsports.com outside hours
SnagATime ops
SnagATime on-call
+1 (206) 555-0143
For platform issues only · vendor hardware → vendor first

7 · Recent failure history

Last 7 automation failures across all 4 bays. Look for patterns — same bay 3× in a week, same time of day, same vendor.

  • Jun 2 · 7:14 PM
    Bay 1 · Sim launch hang · Trackman license server returned 503 for ~90s · operator re-launched at 7:16 PM
    2 min
  • May 31 · 6:55 PM
    Bay 3 · HDMI handshake · monitor stayed black at scheduled wake · operator reseated cable
    4 min
  • May 29 · 11:42 AM
    Bay 1 · Power-on timeout · smart plug unreachable · Wi-Fi credentials rotated by router update overnight
    22 min
  • May 27 · 5:01 PM
    Bay 4 · Ball-track miscal · temperature drift +12°C from morning · operator recalibrated mid-session
    3 min
  • May 25 · 7:02 PM
    Bay 2 · Sim launch hang · Foresight FSX update in progress · 4-min auto-update completed
    4 min
  • May 22 · 3:30 PM
    Bay 3 · HDMI handshake · second occurrence this week — cable replaced
    8 min
  • May 19 · 8:30 AM
    Bay 1 · Sync agent dead · Windows update reboot · agent service didn't auto-restart
    12 min

Pattern check: Bay 3 had 2 HDMI failures inside 9 days; the cable was replaced on May 22 and the pattern stopped. Worth watching Bay 1 for sync-agent restarts after Patch Tuesday.