Skip to main content

Active Metadata Engine (AME) – v3 Release Notes

Version: 3.2 Supersedes: release/v25.10.1

The 3.2 line makes AME the source of warehouse governance: constraints, comments and classification tags are now generated from the same metadata that builds the tables. It also adds two build targets — Databricks and PostgreSQL — and closes SQL-injection exposure across the logging paths. See What changes in 3.2 for the build-target, model, loading and governance detail.

🚀 Features

  • Automated warehouse governance. SQL generation now emits unique constraints, primary and foreign keys, PII and category tags, column comments, and business-key domain tag inheritance on normalised child tables. Constraint generation defaults to on.
  • Data product and quality surfaces. Delivered Model v4 contract surfaces, asynchronous worker-based QPI quality-check dispatch, versioned source file structures (SourceFileV3b) and Field Categorisation management.
  • System observability and hardening. Exposed System Health telemetry APIs, added multi-channel QPI report fan-out, adopted scheme-neutral object store paths (OBJECTSTOREURL), and eliminated SQL injection across logging paths.

🐛 Bug Fixes

  • SQL injection and tracing security. SQL injection vulnerabilities across all logging paths were eliminated by transitioning to bound parameters, and queue worker crashes were resolved by adding automatic reconnection logic and exponential retry backoff.
  • SQL generation and schema integrity. Parent-path matching in hierarchical normalisation was corrected, technical metadata columns (__*_HID, __*_CHECKSUM) are protected from aliasing rules, description escaping (' → '') was fixed, and overwrite methods now drop existing tables before recreating them.
  • API data consistency. Duplicate relationship query results were resolved, inner aggregate filtering in scheduler queries was fixed, NaN/NaT values are converted to null for clean JSON serialisation, and primary key requirements are enforced for append and versioned imports.
  • Log classifier precision. Severity classification now matches keywords on exact word boundaries, and configuration payload blocks are excluded from false-positive error triggers.

Upgrade notes​

  1. Bring the backend to v3 first. The 3.2 line requires /api/v3, with /api/v3.2 for object-level generation. Nothing else in the upgrade can move until it does. The sourcefile data contract is negotiated — tried at v3.2, then v3.1, then v3, with a 404 falling through and a 422 stopping immediately.
  2. Re-generate definitions and diff them before running a load. Published table shapes change for hierarchical sources, technical columns are __-prefixed with a single trailing __checksum, and column ordering changed.
  3. Decide on constraints and classification. Constraint generation is on by default — set ENABLE_CORE_UNIQUE, ENABLE_FOREIGN_KEY and ENABLE_PUBL_PRIMARY_KEY to 0 to keep the previous output. Set TAG_NAME, SENSITIVE_TAG_NAME and SENSITIVE_TAG_VALUE to your own scheme rather than living with the defaults.
  4. Child publish definitions carry additional tag statements, because domain tags now include business keys inherited from ancestor levels.
  5. Re-baseline any test comparing generated DDL byte-for-byte — Snowflake and Databricks CREATE TABLE now carry COMMENT, and the rundate default is resolved per request rather than frozen.
  6. Field categorisations cannot yet be deleted from the UI.