Validation and Evaluation
After deploying a model, you need to confirm that it completes the task reliably in the real environment. NexCore offers two ways to do this: on-site validation (inference tests and direct observation) and NexBench evaluation (automatic scoring by execution node and step).
On-Site Validation
Define the Test Conditions
Set the conditions before you validate, so the results are meaningful:
| Condition | Example |
|---|---|
| Task definition | Move a cube from the tray into the box; success means the cube ends up inside the box |
| Scene range | Cube placed randomly within a 30 cm × 20 cm area of the tray |
| Number of trials | 20 trials per model version |
| Environment | Normal indoor lighting, unchanged background |
Run the Validation
- Click Inference Test on the deployment instance and enter the test scenario and number of trials.
- Complete the safety confirmation and start the test.
- On-site staff observe each trial and use the emergency stop if needed.
- When the test ends, review the execution records in the inference job details.
Record the Results
At a minimum, record the test conditions, number of trials, number of successful trials, failure observations, and the model version. Failure segments are kept automatically in inference feedback for review.
NexBench Evaluation
NexBench is evaluation built into each inference job. It collects data and scores the run by execution node and execution step, so you can pinpoint where failures occur and compare model versions.
Turning It On
- Turn on NexBench when creating a deployment or inference job. It's off by default.
- The setting is saved in the deployment record.
- With NexBench off, execution data still flows back to the platform; it just isn't scored.
| Controlled by | |
|---|---|
| Whether a run is scored | The NexBench toggle |
| Whether data flows back | Always on; no toggle needed |
Metrics
When enabled, results are reported by node and execution step:
| Metric | Meaning |
|---|---|
| Success rate | The share of trials in which the node succeeded |
| Cycle time | The node's average execution time |
| Accuracy | The node's positioning or placement error |
- Missing nodes are marked Not Evaluated and are not scored as zero.
- Low-scoring nodes are added to the bad case list automatically and can be returned to curation after review.
Cross-Job Summary
Model Evaluation → NexBench in the left navigation provides a summary across inference jobs, where you can:
- compare node success rates and cycle times across model versions;
- track how scores for the same task change over time;
- jump from summary results to the corresponding inference jobs and bad cases.
Using the Results
- Evaluation results are viewed in the platform and used to guide quality improvements.
- Success rate, accuracy, and cycle time depend on the task definition, data volume, and test conditions; results from different conditions aren't directly comparable.
- Formal acceptance follows the test conditions agreed by both parties; evaluation results are one input to it.
Recommendation
Turn on NexBench during model validation and use node scores to find where failures occur. On cycle-time-sensitive production lines, enable it in a validation environment first before deciding whether to use it in production.
Common Issues
| Issue | Resolution |
|---|---|
| Inference results have no scores | NexBench wasn't turned on for that inference job |
| A node shows Not Evaluated | The node didn't run or no data was captured in that trial; it isn't scored as zero |
| Two model versions have very different success rates | Confirm the test conditions were the same, then review the bad cases for the failing nodes |

