Config UI – v3 Release Notes

Component: Config UI (released under the name User Interface Toolkit)

A large release. The Config UI was moved onto a current platform, repainted in the Simplitics brand palette, and given a new visual mapping experience, a landing dashboard, and a per-source-system model of what the INGEST agent actually reads. It also completes the migration off the v2 backend API.

Highlights

INGEST is now described per operator, not per connection kind

The connection and export forms previously showed the union of every field for a connection kind, which told you nothing about the source you were actually configuring. Both pages are now driven by a registry of operator classes — the sourcetype the agent runs — so each form asks only for the fields that operator reads, marks the ones it requires, and flags the ones it ignores rather than hiding them.

• Operator picker on both the Connections and Exports pages. FILE connections store the choice in StorageType and queues in the queue_type property; DB and API have nowhere in the contract to record it, so the choice is detected from the port or the API URL and remembered per browser — and the picker says which of the three it was instead of presenting a guess as fact.
• Connection presets per operator, derived from connections that actually run — ports, delimiters, grant types, control headers, scopes. Applying one only fills empty fields; "reset to these values" is a separate action. A preset never carries a credential and never invents a value, and it lists what it deliberately left for you.
• Message queues (AWS SQS, Azure Service Bus, Redis) are configurable as CUSTOM connections, with coordinates, batch bounds and per-export overrides.
• File systems (local, S3, Azure Data Lake Gen2, SFTP) gained their full field, property and filter-condition surface, including the DuckDB reader options and the file-selection window.
• API authentication method now uses the values the agent compares against — OAuth 2.0, Token, Basic. The dropdown previously offered oauth / basic / bearer / apikey / none, none of which the runtime matches, so no Authorization header was ever built for a connection saved through it. Connections carrying a legacy value are flagged and rewritten on save.
• Auth properties are required conditionally, from the auth method and grant type rather than from the operator, and choosing a method seeds the keys that configuration needs into the property list — a form that visibly asked for grant_type used to save a payload without it.
• Provider guidance is offered, never enforced. Graph's /.default scope, Talkdesk's scope and Heartpace's static token are surfaced with their reason and a one-click apply. When the token URL stops belonging to the provider — the signature of a gateway in front of the source — the suggestions withdraw themselves and the UI says why.
• DBEncryptedConnection and DBTrustCertificate are ordinary database connection fields. They were previously modelled as SQL Server connection properties, which could never be saved: the DB contract has no property list.

Credentials are never replayed

The API encrypts every value it receives, so posting a stored credential back encrypts it a second time and the connection fails at the next export with nothing in the UI to show why.

• Stored passwords are never rendered; the inputs are hard-wired empty and the loaded connection is stripped of DBUsrPwd / StorageUsrPwd on the way in.
• Encrypted values inside a property list are blanked on load, and the keys that were cleared are reported so you know what to retype. Detection is by value, so a plaintext secret is left alone.
• The JSON preview always carries the password key so the payload stays postable, but never the stored value. A downloaded preview therefore contains a live credential whenever one was typed.

Dynamic time values

Export definitions carry values the runtime resolves at run time — the output file suffix, a partitioned source's file path, the ends of a file-selection window, request parameters, where clauses.

• A picker that asks which moment and how far back from it, rather than asking you to spell a token, showing both the assembled value and a plain-English reading.
• The full [__<TIME>[_MINUS_<N>_<UNIT>][_AS_<FORMAT>]__] grammar, including HIVE and HIVE_PADDED as separate choices — month=8 versus month=08 is not interchangeable, and which one works depends on how the source wrote its partitions.
• Format inheritance is shown as its own option, naming the format a token with no _AS_ will inherit.
• Legacy spellings are flagged, never silently rewritten — [__LAST_N_DAYS__] and [__LASTEXECUTIONTIME__], the latter being a name the grammar does not list, so exports carrying it may never have resolved.
• dateTimeFormat now has a form. It had been in the contract from the start with nothing rendering it.
• User-defined parameters are switchable between Text, Number and Date & time, with the kind inferred from the value.

Visual mapping

• New React Flow diagram view for mappings, the default view mode, with drag-and-drop mapping creation and deletion, grouped nodes, hover highlighting, search, subject-area filtering, a "show only mapped" toggle and image export.
• Mapping lines overlay in the tree views, with breadcrumb navigation linking source to target.
• The key mapping dialog fixes the problem in place. It previously listed your mappings, stated a rule, and offered only a destructive button. It now names what is missing, explains what a business key is, and lets you pick the source field and press Map and save. Removal drops to a text link naming its consequence and count.
• Orphaned mappings are surfaced — mappings whose target attribute or model was renamed or removed were hidden from the tree and could be neither seen nor cleaned up. A banner lists them with a Remove button.
• Mapping properties save. The tree node carried only {fileName, modelName}, so the dispatched payload lacked target, source and mappingNumber and the save silently skipped — no request, no error, no change.
• Filters, optional calculations, sort order and sort direction are editable per attribute in the UI. They previously had to be applied through the API or a source-code view.

Dashboard

• New landing dashboard with counts of systems, source files, fields, mappings and models; an end-to-end data lineage graph built from live API data; an architecture overview; and a drill-down panel that navigates to a source with its system preselected.
• Widgets fail visibly instead of hanging — a failed top-level fetch previously left the stat cards pulsing forever.

Additional Tasks

• New Additional Tasks page for managing pre- and post-processing tasks, with sidebar navigation, validation and a dependency multi-select.

Settings

• Split into focused sections: Data Lake settings, Data Warehouse settings and Field Classifications, with a collapsible sidebar and preview.
• Postgres added as a data platform option.

Simplitics brand palette

The app is rendered in Simplitics' own colours, and each colour carries one meaning: turquoise for mapped/connected/success, lilac for relations, amber for keys, zinc for neutral and off-states, red reserved for errors and destructive actions. The green/amber/red coverage ramp is deliberately kept — a traffic light reads better than brand tones.

Performance

Every endpoint behind these views takes 8–15 seconds, and this release attacks the waits that stack on top of that.

• The dashboard makes no uncached reads, and its data is held for 15 minutes and cached for 30, with a Refresh button as the way out.
• The mappings view fetches in parallel — opening a source file went from roughly five sequential waits to two.
• The source tree paints without waiting for the target models, roughly halving the wait, with a skeleton naming what is still missing.
• Models no longer load twice on mount, and "system has no exports" is no longer treated as an error.

Save, feedback and dialogs

• One rule for save confirmations. Errors persist until dealt with; plain success is transient and placed at the action; success carrying a next step persists; a toast is only for feedback that outlives the view that produced it.
• Every save button shows it is working and blocks a second submit.
• Save is disabled when there is nothing to save, with the button stating why.
• Unsaved work warns before the tab closes.
• Every dialog closes with Escape and a backdrop click, both suppressed while an action is in flight.
• Browser confirm() / alert() replaced by an in-app confirmation dialog.
• Success is only shown when the save actually succeeded — import, model save and relationship save all previously reported success on failure paths.

Other changes

Area	Change
Exports	never is selectable as a schedule cycle on all four export types
Exports	Exports stored with schedule cycle "never" loaded with an empty dropdown and then silently failed to save
Fields	Custom data type via an Other option on the data type dropdown, with auto-detection of non-standard saved values
Models	Apostrophes are allowed in model descriptions
Models	The Models page no longer crashes when the list fetch fails
Source files	targetNormalization is normalised for both display and API compliance; a deactivation confirmation dialog was added
Tree	Inline field editing, advanced filters, drag-and-drop reordering in read-only mode, and a relationship editing view
Navigation	Redesigned navigation with per-view header tabs; each page has its own browser tab title
UX	Guidance boxes across views, empty-state guides with an action slot, and shared spinner, skeleton, toast and field-error components
Accessibility	Form labels associated with their inputs; stale validation errors clear; success messages auto-hide after 5 s while errors persist; icon-only buttons use real tooltips

Upgrade notes

1. Node 24 is now required. All Dockerfile stages run node:24-alpine.
2. The backend must serve /api/v3/*. All proxy routes now target v3; a backend still on v2 will break systems, source files, model relations, extra processors and string encryption.
3. New environment variables. SIMPLEAPI_API_URL, DATAAPI_API_URL, API_USERNAME, API_PASSWORD, MOUNT_PATH, PDQ_CHAT_API_URL.
4. Install with npm ci. yarn.lock has been removed.
5. Dashboard data is up to 15 minutes old by design. Press Refresh for current figures.
6. Field categorisations cannot be deleted from the UI until the backend supports DELETE on that endpoint.

Known limitations

• The in-app chat widget ships but is not reachable from the UI.
• Batch edit / selection mode in the tree view is implemented but temporarily disabled.
• In-app navigation is not guarded against unsaved changes — only tab close and reload are.
• Save-disabled-when-clean covers data import and settings only.
• Export-less systems still answer HTTP 400 on a cold load; the client handles it, but the endpoint should answer 200 with an empty list.