Optical Jig | AI Vision Testing System
A GPU-accelerated visual intelligence system that trains on every screen state of an HMI, cluster, or display - then validates live frames from a calibrated camera against expected states, automatically, inside every TestBot sequence.
CAN and UDS Agents can verify that the right message was sent at the right time. What they cannot verify is whether the instrument cluster actually showed the correct warning icon, whether the speedometer needle reached the right angle, or whether the correct screen transitioned after a mode change. That gap - between what the ECU transmitted and what the display rendered - is where field escapes and recall campaigns originate.
Traditional approaches to closing this gap require either manual visual inspection or pixel-level screenshot comparison. The TestBot AI Vision Testing System closes the gap differently: it learns every legitimate screen state and element state from trained reference images, then makes confident predictions from a live camera feed.
The system is a two-component hardware and software platform. The AI Vision Processor - a web application running on an NVIDIA Jetson AGX board - hosts the trained AI models, performs GPU-accelerated inference on live camera frames, and communicates results to TestBot over MQTT. The Vision Camera is a calibrated camera unit mounted to view the HMI, cluster, or display under test and streams frames continuously to the processor.
Together they give TestBot a real-time visual sense: at any point in a test sequence, TestBot can ask what screen is currently displayed and what the state of each element on it is - and receive a structured JSON answer within milliseconds.
Capture and label every legitimate screen and widget state during setup so the trained model knows the full visual vocabulary of the device under test.
The Vision Camera observes the display externally and continuously streams frames to the Jetson processor without display output taps or firmware hooks.
Each frame is classified on the Jetson GPU, then reduced to a screen name and a per-element state map for TestBot.
The Jetson processor publishes classified results to TestBot in JSON, allowing visual checks to run inside the same sequence as CAN, UDS, DRB, and IO steps.
The system does not do pixel comparison. It classifies screens and element states using AI models trained on all the states you define during the training phase. This means it tolerates display brightness variation, minor rendering differences, and real-world camera conditions - producing the same confident classification an engineer would make by looking at the screen.
Book a DemoCapture and label every screen state and widget state, then train the AI model on the Jetson processor. Retrain incrementally when firmware updates add or change states.
The calibrated Vision Camera streams live frames from the HMI or cluster display exactly as a human tester would observe it.
Jetson classifies the active screen and every trained widget state - including dial position, icon presence, and text values - into structured JSON.
TestBot compares the actual visual state to the expected value and writes the result into the same HTML, PDF, or Excel report as the rest of the bench sequence.
Visual checks can verify screen transitions, fault indicators, and widget state changes in the same sequence as CAN and DRB events.
Full electrical, interface, and operating characteristics for bench integration.
| Specification | Value |
|---|---|
| AI inference hardware | NVIDIA Jetson AGX, GPU-accelerated |
| Software environment | Python-based web application on Jetson OS |
| Inference mode | Live camera stream, continuous frame processing |
| Widget types validated | Dial, image/icon, text |
| TestBot interface protocol | MQTT (receive test context; publish results) |
| TestBot result format | JSON - screen name + per-element state map |
| CAN interface | DB-9 connector, up to 5 Mbps |
| CAN database support | DBC file import |
| CAN repetition rate | Configurable per signal |
| Example CAN IDs | Fuel gauge: 0x2BD; Telltale: 0x2BC (configurable) |
| Training workflow | Capture → label → train via Jetson web application |
| Retraining | Incremental - add new screens or states without full retrain |
| Enclosure | IP40 rated ABS plastic (camera unit) |
| Operating environment | Bench use, 0°C to +50°C |
The AI Vision system recognises and validates three widget types on any trained screen, while also tracking screen identities and transitions.
Speedometers, tachometers, fuel gauges, and temperature gauges are classified by needle position or fill state against trained angular ranges.
Warning lights, indicator icons, status images, and symbol overlays are identified by visibility and colour state.
Odometer values, trip counters, mode labels, and status strings are extracted as text for exact or pattern-match comparison.
Before widget evaluation, the system identifies the active screen so checks run in the correct visual context.
Train Capture and label all screen and element states in the Jetson web application, then run the training pipeline to build the AI models.
Mount and connect Position the camera, connect CAN and MQTT, and add the AI Vision Agent to the TestBot station configuration.
Define Add visual check steps to the TestBot sequence alongside CAN, UDS, DRB, and IO steps.
Execute and report Run the sequence; visual results appear in the report with full timestamps and no separate manual review step.
The system is designed for production-grade bench automation - trained on real device states, robust to display variation, and natively integrated with TestBot.
| Capability | TestBot AI Vision System | Manual visual check | Pixel / screenshot comparison | OCR-only tools |
|---|---|---|---|---|
| TestBot integration | Native MQTT + JSON | No | Custom script | Custom script |
| Dial / gauge validation | AI-classified | Manual | Brittle | No |
| Icon / warning validation | AI-classified | Manual | Brittle | No |
| Text / value extraction | AI + OCR | Manual | No | Yes |
| Screen identification | Trained classifier | Manual | Fragile | No |
| Tolerates rendering variation | Yes | Yes | No | Partial |
| Repeatable and unattended | Yes | No | Partially | Partially |
| Report evidence | In TestBot report | Manual notes | Separate log | Separate log |
| Retraining on firmware change | Incremental | Just look | Full recapture | Partial |
The AI Vision Testing System is available as a hardware bundle. The AI Vision Agent licence is renewed annually per station. Contact us for a quote based on the number of display variants to be trained and the number of test stations.
Answers to the questions teams usually ask before adding visual validation to a TestBot sequence.
It is a two-part system - an NVIDIA Jetson AGX AI processor running a trained image classification application, and a calibrated Vision Camera - that gives TestBot the ability to read and validate what is visually displayed on an HMI, instrument cluster, or any display under test, without human observation.
Use the AI Vision Testing System to validate what the display actually shows - not just what the ECU sent - inside the same TestBot sequence.