Evaluation sets for audit, QA/QC and safety, written by licensed auditors, engineers and inspectors. Every answer is traced to the clause.
ASME VIII Div 1 vessel, full radiography. Shell plate 12 mm. One elongated slag inclusion, 8 mm long. Accept or reject?
Reject. The limit is t/3 = 4 mm, and 8 mm exceeds it.
Right verdict, wrong limit. t/3 applies only from 3/4 in to 2-1/4 in.
Reject. At 12 mm the 1/4 in (6.4 mm) limit applies; 8 mm exceeds it. Repair and re-radiograph.
Correct thickness band, limit, verdict and action.
Read the scenario, decide, then reveal the reference answer and the graded model responses.
A 30-person fabrication shop has no written procedure for hazard identification. Supervisors complete a standard risk assessment form before each new job. The auditor sampled 15 forms: all complete, consistent, and matching the controls observed on the floor. The auditor asked for a written hazard-identification procedure and was told the shop has never had one.
Should a nonconformity be raised for the missing written procedure?
No nonconformity. The standard requires a process, with documented information only to the extent needed for confidence it is carried out as planned. Fifteen consistent forms that match site controls give that confidence.
Ref: ISO 45001:2018, 6.1.2.1 and 6.1.1Nonconformity against 6.1.2.1: there is no documented procedure for hazard identification, which ISO 45001 requires.
Over-calls. Invents a mandatory procedure requirement the standard does not contain.
No nonconformity. 6.1.2.1 requires a process, and documented information only to the extent needed. The sampled forms give that confidence. An optional improvement could suggest a one-page method statement.
Correct, well reasoned, and keeps the improvement in proportion.
Compliance answers are right or wrong against a standard. Getting them right takes people who have raised the nonconformance, rejected the weld and signed the report.
A model that cites the wrong thickness band, or skips the exception in a table, gets the verdict wrong. Our tasks test that precision.
Good auditors know when there is no finding. We write no-finding tasks so models learn not to invent requirements.
Lead auditor, NATA, AWS and API qualifications are verifiable, so vetting rests on evidence, not a chat interview.
HSE-Bench, from UNSW and CSIRO Data61, found models above 90% on a single step and about 68% when the answer had to carry the issue, the rule, the application and the conclusion. That is the judgment these tasks grade.
Clause-traced test sets for audit, QA/QC and safety. The 10-task sample is free. A 50-task pack is the paid step for RL and evaluation teams. A reusable benchmark is licensed from USD 100,000, with tasks held back from the download.
For teams proving a safety or compliance model
Request the 10-task sampleLicence and pricesOur delivery process is built to ISO 9001 by a founder who audits to it.
For data platforms hiring into safety, construction and engineering. The pod is already through credential, domain and calibration gates.
For expert-data platformsRequest a podTasks with a gradable answer, from the evidence to the nonconformance and the clause. Built to train and test, not only to read.
For RL and evaluation teamsRequest the packWe run your agent on the 10 tasks. Each answer is marked from 0 to 5 against the clause. A right verdict with the wrong rule scores 3, and that is not a pass.
For safety and compliance softwareRequest a scorecardThe sample and the scorecard are free. Expert grading is USD 140–220 an hour. The reusable benchmark is the licence.
Emailed, with the reference answers. Every sales conversation starts here.
USD 0
Request the sampleYour agent on the 10-task sample. Each answer is scored from 0 to 5 against the clause. We send the misses.
USD 0
Request a scorecardVerifiable tasks for RL and evaluation teams. One discipline, clause-traced, with a reference answer on every task.
USD 500–2,000 a task
Request a quoteA reusable set, licensed to one legal entity. Held-out tasks stay off the download.
From USD 100,000
Request a licenceThe door-opener: credentialed auditors and inspectors scoring your own outputs, with a written rationale.
USD 140–220 an hour
Request gradingA pre-vetted group of auditors, engineers and inspectors for a data platform that cannot fill the work with a general interview.
Quoted
Request a podPayment is by invoice. A licensed pack downloads from Client login after it is released. The first packs are workplace health and safety, and welding and NDT, 50 tasks each, with 10 held back.
Licence or certificate verified with the issuing body.
A written paper for that discipline, untimed, marked against the gold answer. Under 70% stops there.
Grade five AI answers against a rubric and catch the planted errors.
Explain each grade so a non-expert can follow it.
At least 85% agreement with our gold-standard answers.
Licence checked with the issuer. No timed chatbot interview. At least 85% agreement with the gold set.
Paul has spent two decades on the inspection side of compliance.Read the record
Or email hello@kanonlabs.co.