Google DeepMind has piloted a double-blind evaluation framework to prevent benchmark contamination during frontier AI testing.
Google DeepMind announced on Thursday that it successfully completed a secure evaluation of the Gemini 2.5 Flash Lite model using a cryptographically isolated environment. The project was executed in collaboration with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons to ensure that neither the model developer nor the benchmark maintainers could access confidential assets during the testing phase.
During the pilot, AVERI evaluated Gemini 2.5 Flash Lite using reserved prompts sourced from the MLCommons AILuminate safety benchmark. These assessments targeted high-risk categories including cyberattacks, chemical and biological hazards, hate speech, self-harm, and violent-crime elicitation. Simultaneously, the Singapore AI Safety Institute independently tested the model against confidential prompts tailored to evaluate harmful content within regional regulatory contexts.
Industry experts have increasingly raised concerns regarding benchmark contamination, wherein frontier models achieve artificially high scores simply because test questions were inadvertently ingested during pre-training or fine-tuning phases. Google noted that while traditional safeguards like contractual restrictions and zero-logging policies provide baseline security, robust cryptographic mechanisms are now required to guarantee uncompromised performance metrics.
To establish this secure boundary, the evaluation infrastructure relied on Google Cloud Confidential Computing technology. Specifically, the pilot utilized a Google Cloud A3 Confidential VM equipped with Intel TDX host-memory encryption and NVIDIA H100 Confidential GPU hardware. Hardware-level encryption alongside remote attestation ensured strict isolation for both proprietary model weights and sensitive evaluation prompts.
Google stated that this architecture ensures evaluation integrity without exposing foundational intellectual property. “Neither side gets to peek,” the company noted, emphasizing that the technical framework allows independent evaluators to accurately measure real-world capabilities without risking the exposure of proprietary parameters.