The Naval Undersea Warfare Center Division, Keyport (NUWC Keyport) and the Naval Sea Logistics Center (NSLC) are collaborating with Virginia Polytechnic Institute and State University to explore the application of artificial intelligence (AI) in enhancing the efficiency and reliability of Navy supply chains. The initiative, known as the Sentiment and Topic Analysis for Reliable Supply (STARS) project, aims to leverage large language models (LLMs) to improve the accuracy and consistency of contractor performance evaluations.
The STARS project addresses the current challenges associated with contractor assessments, which often include numerical ratings accompanied by written narratives that can exhibit inconsistencies. These discrepancies can hinder accurate performance evaluation and informed decision-making regarding future contracts. The three-year project, initiated in 2022 and funded by the Naval Engineering Education Consortium (NEEC), managed by NUWC Division, Newport, is currently in the research and development phase at Virginia Tech.
At the core of the STARS project is sentiment analysis, a technique powered by LLMs. These AI models are trained on extensive text datasets to identify patterns and nuances in language indicative of positive, negative, or neutral sentiment. When applied to contractor assessments, the models will flag instances where the sentiment expressed in the written narrative appears to contradict the assigned numerical scores.
“The goal is to utilize AI-powered sentiment analysis to gain a deeper understanding of the narrative text within contractor assessments,” explained Brett Davis, Department Head of Supplier Quality and Logistics IT Systems at NSLC and the project’s programmatic subject matter expert. “Improving the quality of narrative text will significantly enhance its value for critical applications such as supply chain risk management.”
Inconsistencies in assessments can arise due to various factors, including differing writing styles and experience levels among reviewers, a lack of clarity or specificity in narrative descriptions, and the inherent difficulty in conveying tone effectively in written communication.
John Greener, NSLC’s Chief Technology Officer and Product Owner for the STARS project, highlighted the broader potential of this technology. “In our two years of collaboration with Virginia Tech, we’ve identified potential applications for this technology beyond contractor assessments,” stated Greener. “This project provides a valuable opportunity to explore cutting-edge AI technologies with benefits that could extend across the Navy, the Department of Defense, and the wider federal government.”
Greener believes that AI-powered analysis could be applied to various acquisition-related tasks, including developing more precise statements of work and improving the accuracy of contractor performance predictions.
NUWC Division, Keyport, headquartered in Washington State, plays a crucial role in developing, maintaining, and sustaining undersea warfare superiority for the United States Navy. The organization maintains a network of detachments and operating sites to provide ready support to Fleet operational forces across the Pacific.
