Is an Apple an Orange? A Large Language Model Benchmark for Candidate Term Extraction and Subclass Decisions Against Upper Ontologies in Engineering and Materials Science

Engineering and materials-science projects increasingly depend on interoperable data, yet building the ontologies that enable this interoperability remains manual, slow, and expensive. We benchmark large language models (LLMs) on two early-stage taxonomy-creation tasks: candidate term extraction and binary subclass decisions against the basic formal ontology (BFO) and common core ontologies (CCO). Thirteen LLMs (10 proprietary, three open-weight) and six non-LLM baselines (spaCy, KeyBERT, WordNet, NLI) are evaluated on two engineering articles covering additive manufacturing and carbon-fiber-reinforced polymer (CFRP)–metal hybrid bonding. For extraction, F1 follows an inverted-U in chunk length, peaking at 750–1,000 characters, and Claude-Opus-4.7 reaches about 73% versus at most 28% for the best baseline. For subclass decisions, a Set-Inclusion prompt (“Is every x a y?”) is most balanced, and Claude-Opus-4.7 attains about 94% combined accuracy on low-level terms, with GPT-5.4 at comparable accuracy and the best baseline (WordNet) at 64%. Moderate hierarchy depth aids acceptance, whereas free-text examples reduce it. GPT-4o-mini delivers 5,940 F1-points per dollar versus 19 for GPT-4-32k. Six workflow paths map these results to model and review-policy choices. Absolute F1, TPR, TNR, and model rankings are expected to shift with corpus, ontology family, and model release date.