Get answers to common questions about METRAI, our evaluation process, and how we help you select the right AI vendor.
A standard AI evaluation typically measures statistical performance — forecast accuracy, error rates, benchmark scores. A behavioral assessment evaluates how the system makes decisions under real operational conditions: how it responds to demand volatility, whether its recommendations respect your constraints, how it handles missing or late data, and whether its outputs are stable enough for planners to trust. Behavioral assessment answers the question your procurement process cannot: will this system actually work in my operations?
METRAI evaluates planning AI across four domains: demand planning and forecasting, store replenishment and computer-assisted ordering, allocation (including fashion and seasonal inventory), and DC-level inventory planning. If you are evaluating a system that spans multiple domains — for example, an integrated planning platform — we structure the assessment to cover each relevant decision type.
Yes. Our Current System Diagnostic stress-tests the planning system you run today against the same scenarios we use for vendor evaluations, using historical data already available to your team. You’ll learn where it performs well, where its decisions break down, and whether to tune it or replace it.
If you are actively choosing a new planning system, start with a New Vendor Evaluation. If you are unsure whether your current system is the problem, start with a Current System Diagnostic. The diagnostic results can become the baseline for any later vendor evaluation.
Not always. Many diagnostics can be conducted using historical inputs and decisions already available to your team. Live system access or vendor cooperation is not always required. For vendor evaluations, we scope the data and system access needed for the specific scenarios and vendors under consideration.
A Current System Diagnostic typically takes four weeks from data scoping to final readout. A New Vendor Evaluation report is delivered within five to seven business days after scenario execution is complete. The overall engagement timeline depends on the number of vendors, planning domains, and availability of representative data.
We work with your team to define the scenarios that matter most for your business. You provide access to representative historical data — demand history, order history, constraint parameters — and the vendor provides system access or output data for the scenarios we define. We do not require production system access; most assessments run on a defined scenario set with vendor-provided outputs.
Yes. METRAI has no commercial relationship with any planning AI vendor. We do not receive referral fees, partnership revenue, or any other compensation from vendors whose systems we evaluate. Our only commercial relationship is with the companies that commission assessments. This independence is the foundation of the assessment's value — if we had vendor relationships, the results would not be credible.
The final report includes a behavioral certification for each system evaluated, stress-test results across the scenarios we ran, a decision profile describing how each system makes choices under different conditions, a planner trust score based on explainability and recommendation stability, and a fit recommendation that maps each system's behavioral profile to your specific operational requirements. The report is designed to be presented to a procurement committee or board.
Reach out to our team to discuss your specific evaluation needs.