New study finds today’s leading AI modelsfall short in dangerous goods knowledge, particularly in stowage, segregation, and regulatory recall tasks where errors carry safety risks.

NEW YORK, NY — 8 September 2026 — In a first-of-its-kind study, NCB set out to determine how well artificial intelligence (AI) understands and can respond to dangerous goods (DG) compliance questions. Developed by NCB’s Hazcheck team in collaboration with Durham University, the research introduces DGEval, the first benchmark built specifically to evaluate large language models’ (LLMs’) knowledge of the International Maritime Dangerous Goods (IMDG) Code. 

DGEval tested 13 AI models from six providers with 1,678 questions spanning multiple-choice recall, open-ended questions, structured lookups from the Dangerous Goods List (DGL), and identification of the specific IMDG Code behind answers. 

Key findings: 

• The top performing model, Google Gemini 3.1 Pro, scored 95.8% on multiple-choice questions, exceeding a corrected human practitioner baseline of 83.8%. OpenAI’s GPT-5.5 was the only other model to exceed the human baseline. 

• Stowage and segregation, the two IMDG Code areas most directly tied to on-vessel safety, were the weakest-performing areas across every model tested, including the top performer. Models scored an average of 35 percentage points lower on these questions (roughly 49–50%) than on classification and documentation questions (82–85%), a gap that reflects a limitation in operational knowledge rather than a weakness tied to any single AI system. 

• Regulatory recall, testing whether a model can identify which section of the IMDG Code supports its answer, averaged just 14.2% across all models, far below performance on the underlying knowledge questions.  

• LLaMarine, a model built with maritime regulatory text, performed no better than its unmodified base model on DG questions, indicating that general maritime training does not substitute for DG-specific insight.  

• Enabling web search improved structured DGL lookup accuracy by an average of 25 percentage points but had little effect on multiple-choice reasoning performance. 

No independent research had previously tested whether AI chatbots can be relied on for DG-specific question. The stakes are significant: non-compliance with the IMDG Code can result in fire, explosion, or the loss of life, vessel, or cargo. As AI tools become a common first stop for compliance questions, an incorrect or incomplete answer can carry those same consequences. 

NCB recommends that any AI-assisted lookup be verified manually against the published IMDG Code (available to purchase through International Maritime Organisation resellers), or in combination with automated compliance systems, such as Hazcheck Validate. 

“The results demonstrate the considerable potential of AI,” shared Mike Durkin, CIO, NCB. “They also show that human oversight remains essential when dangerous goods regulations are interpreted and applied.”

DGEval is designed to be applied on an ongoing basis as models continue to evolve.  

The research paper, Evaluating Large Language Model Performance on International Maritime Dangerous Goods Code Compliance, is currently accessible in pre-print format here: https://arxiv.org/abs/2608.21036. 

Media Inquiries & Contact Information 

For more information about our research or NCB, please email services@hazcheck.com, or call our New York Headquarters at +1 (212) 785-8300. 

About NCB 

NCB (National Cargo Bureau) is an independent not-for-profit organization dedicated to the mission of safeguarding life and cargo through safer, smarter supply chains globally. NCB provides trusted inspection, compliance, and certification services. Through the integration of NCB Hazcheck digital platforms and services, NCB also delivers industry-leading digital solutions for dangerous goods management. 

NCB’s Hazcheck Systems supports dangerous goods data documentation and validation, cargo screening, container inspections, and e-learning. NCB was selected by the World Shipping Council as the independent provider of the Cargo Safety Program digital platform. 

With operational hubs in the United States, United Kingdom, and Asia-Pacific, and a global distribution network, NCB enables safer, smarter, and more efficient maritime transport worldwide. 

Learn more at https://www.ncb.org and https://www.hazcheck.com.