Skip to main content
Applies to:
  • Plan:
  • Deployment:

​
Summary

Issue: After resolving the aead::Error decryption failure and enabling time-based retention, Brainstore logs show Failed to execute static group by, Unable to pushdown filter, and Deleted index documents with xact_id < ... messages. Cause: These are query optimization fallbacks and retention confirmation logs, not errors. They appear during normal Brainstore operation. Resolution: No action required. All three log types indicate expected behavior.

​
Log message reference

​
Failed to execute static group by, falling back to dynamic

This is a non-critical warning. It fires when a field (e.g., error) has inconsistent types across span documents — sometimes a string, sometimes a nested object. Brainstore attempts a fast columnar aggregation first. When schema inconsistency prevents it, Brainstore retries using a dynamic aggregation path that handles mixed types. Queries return correct results either way. The dynamic path is slightly slower but functionally equivalent.

​
Unable to pushdown filter: Comparison not a field and literal

This is a non-critical query optimization warning. Brainstore’s index layer can only accelerate simple field = literal comparisons. Filters involving coalesce() expressions generally cannot be pushed down to the index: 
coalesce(metadata.tool_choice.function.name, '') = 'select_choice'
When this occurs, the optimizer evaluates the filter at the query layer instead. Filters still apply correctly — just without index acceleration. These warnings originate from Braintrust’s own UI queries. Exception: coalesce(field, 'literal1') != 'literal2' patterns are automatically rewritten to the null-safe not-equal operator (field <!=> 'literal2'), which can be pushed down to the index. If you see this warning for a != comparison that uses coalesce(), confirm that both literals are the same type.

​
Deleted index documents with xact_id < ...

This is an info-level confirmation that the retention worker ran successfully. xact_id is an internal transaction ID used to identify records outside the retention window. Seeing this log means your retention policy is working correctly. No data loss occurs — only documents older than your configured retention threshold are removed.

​
Fixing the underlying aead::Error (if retention is still not running)

If retention has not yet recovered and the original decryption error persists, the most common remaining cause is a stale bt_data_plane_service_token encrypted under an old key.

​
Step 1: Verify key consistency

Confirm FUNCTION_SECRET_KEY is identical on both the API host and the Brainstore host. If SERVICE_TOKEN_SECRET_KEY is unset, the API falls back to FUNCTION_SECRET_KEY. Both services must use the same value.

​
Step 2: Restart both services

Restart the API and Brainstore hosts after confirming the keys are correct.

​
Step 3: Refresh the service token

In Settings → Service tokens, click Refresh on bt_data_plane_service_token. This re-encrypts the token under the current key. If the token was refreshed before the keys were corrected, it must be refreshed again after.