Tonic Structural release information
File connector - For Parquet files, fixed the support for applying generators to Decimal128 and Decimal256 columns.
MongoDB - Fixed an issue where applying the Categorical generator to a document database column could result in an error.
Custom generator presets and sensitivity rules on Structural Cloud - On Structural Cloud, you can now configure custom generator presets and custom sensitivity rules. The presets and rules that you configure are only available to your organization. The Account Admin permission set is now granted permission to configure custom generator presets and custom sensitivity rules.
Salesforce - Added logic to set BusinessHours reference field values to valid destination values for scratch orgs.
When Azure SSO is configured incorrectly, Structural now logs a warning instead of exiting.
MongoDB - Fixed an issue where applying the Categorical generator to a document database column could result in an error.
Updates to Table View - On Table View, the column headings now display icons to identify the protection status and the detection confidence level for each column. It also indicates when there is a recommended generator, and allows you to apply the recommendation to the column.
Salesforce - Added additional destination constraint lifting logic to prevent FIELD_FILTER_VALIDATION_EXCEPTION and DUPLICATES_DETECTED errors.
The UUID Key generator now supports UUIDs that are stored in strings without hyphens. When the input value contains hyphens, the output contain hyphens. When the input value does not contain hyphens, the output does not contain hyphens.
MySQL - Improved resilience to certain issues while processing tables.
Salesforce - Added additional destination constraint lifting logic to prevent FIELD_FILTER_VALIDATION_EXCEPTION and DUPLICATES_DETECTED errors.
Bug fixes and other internal updates.
Structural no longer supports Docker engines that are older than 20.10.9.
Bug fixes and other internal updates.
MySQL - Fixed an issue where data generation would hang after the destination database crashed.
Google BigQuery - Added basic support for the BigQuery RECORD type.
Sharing Structural logs with Tonic.ai is changed from opt-in to opt-out. Log sharing is now enabled by default. To disable log sharing, set the environment setting ENABLE_LOG_COLLECTION to false on all Structural containers.
Added an optional parameter, preserveExistingTags, to the workspace import API endpoint. When true, the tags in the imported workspace file do not overwrite the existing tags in the target workspace for the import. The default behavior is to overwrite the existing tags with the tags from the imported file.
Before an upsert job, Structural now performs constraint validation on the destination table. This avoids potential upsert failures because of missing uniqueness or primary key constraints as well as violations of foreign key constraints for downstream tables.
Fixed an issue where some columns found by custom sensitivity rules could not be marked as not sensitive.
MySQL - Improved resilience to certain issues during constraint application and processing tables.
MySQL - Fixed an issue with long-lived destination database connections.
Using secrets managers for database passwords - With the new secrets manager feature, Enterprise users can now opt to retrieve database passwords from AWS Secrets Manager. On Structural Settings view, you configure the available secrets managers. On the workspace details, to use a secrets manager for the database password, you select an available secrets manager and the secret to use from that secrets manager.
Bug fixes and other internal updates.
Snowflake - Fixed an issue where temporary files in the cloud storage provider were not deleted when a job was canceled.
SQL Server - Fixed an issue where Structural processed SPROC object re-creation when TONIC_SQL_SERVER_SKIP_CREATE_DB=true.
Bug fixes and other internal updates.
SQL Server - Fixed an issue on older versions of SQL Server where the error UnknownPropertyException (NetName) was thrown.
For upsert, improved the error handling. Instead of failing the upsert job when a single table fails, do a try/catch of issues with warnings.
Bug fixes and other internal updates.
Fixed an issue where the Database View filters initially displayed no content.
Oracle - Fixed an issue where automatic sensitivity scans on workspaces that exceeded the job timeout were reported as failed instead of canceled.
Salesforce - Resolved FIELD_INTEGRITY_EXCEPTIONs on OpportunityLineItem creation.
The generator configuration for the HIPAA Address generator no longer suggests linking address types that cannot be linked. You can no longer link the Address and HIPAA Address generators.
PostgreSQL - Fixed an issue where upsert failed with a bad insert statement.
Bug fixes and other internal updates.
To prevent upsert failures, improved the schema check to include comparisons of primary keys, unique columns, foreign key constraints, and indexed columns in the source and destination databases.
Fixed an issue introduced in v1332 where the dropdown for selecting a user when transferring a workspace or assigning a global permission set would not close upon selection.
Added a new environment setting TONIC_ENABLE_JAVA_SDK_DOWNLOAD that controls whether to allow downloads of the Java SDK. The default value is true.
Fixed an issue where users who had permission to view destination data, but did not have permission to view source data, were able to use composite generators to view unmasked data.
Bug fixes and other internal updates.
Databricks - Fixed an issue where the Struct Mask generator failed on array columns in Databricks 14.3+.
PostgreSQL - Fixed an issue where upsert failed for generated columns.
When you output destination data to Ephemeral or to a container repository, you can now configure the workspace to override the statistics seed.
Bug fixes and other internal updates.
Bug fixes and other internal updates.
From the Java SDK, you can now enable diagnostic logging when you generate statistics as part of processing a DataFrame.
Fixed an issue that caused the preview generated values to display incorrectly in the Database View generator recommendations.
Bug fixes and other internal updates.
Fixed Jobs hang when database connections (source or destination) are not available.
SQL Server - Added support for propagating non-system SQL CLR assemblies from the source to the destination.
Bug fixes and other internal updates.
Fixed an issue that caused the preview generated values to display incorrectly in the Database View generator recommendations.
MongoDB - Fixed an issue where specifying an unscanned collection as a subset target caused data generation to fail.
Salesforce - The Date Truncation generator now works correctly.
PostgreSQL - Fixed an issue where upsert operations failed on tables that contained custom types and unique constraints.
Bug fixes and other internal updates.
Bug fixes and other internal updates.
For Data Pipeline V2, fixed a rare issue where a job was not marked as failed after a fatal error, and the worker would not process other jobs.
Sensitivity confidence information in Privacy Hub - On Privacy Hub, the column boxes in the At-Risk Columns panel now reflect the confidence level in the sensitivity detection for each column. The sensitivity confidence is also displayed for the columns on the Recommended Generators by Sensitivity Type panel.
Bug fixes and other internal updates.
Bug fixes and other internal updates.
Fixed an issue where connection type limit in the Connection Type section of workspace settings was displayed incorrectly.
Bug fixes and other internal updates.
Fixed an issue where Microsoft Entra ID SSO groups were not retrieved if the membership in them was transitive.
Bug fixes and other internal updates.
Fixed an issue where PostgreSQL container artifacts failed to generate.
You can now specify the port that dotnet-monitor listens on within a worker. To set the port, use the environment setting TONIC_WORKER_DOTNET_MONITOR_PORT. Workers that are colocated on the same instance should be assigned different port bindings.
File connector - For custom Amazon S3 URLs, you can now indicate to trust the server certificate. To configure this, use the TONIC_AWS_S3_TRUST_SERVER_CERT environment setting.
MongoDB - Fixed an issue that caused layout problems for Privacy Hub.
Snowflake - Fixed an issue where a worker would sometimes become unresponsive after cancelling a data generation job.
Google BigQuery - Improved progress tracking for tearing down and recreating entities in the destination dataset.
Databricks - For new workspaces, changed the default Databricks runtime to 14.3.x-scala2.12.
When you configure a workspace to write output to a container repository, you can now provide a custom password for the destination database superuser.
File connector - The file upload now restricts accepted files to those that have supported file extensions.
MySQL - Improved resilience to transient issues when applying table constraints.
Fixed an issue where the first recommendation in the Recommended Generators by Sensitivity Type panel was cut off.
Fixed an issue where the generator recommendation preview sometimes failed with the message No Workspaces have been created yet.
Added API endpoints to enable and disable subsetting for a workspace.
Database View redesign and sensitivity confidence - On Database View, the Applied Generator column is now split into two columns. The Status column shows the current sensitivity status (Not Sensitive, At Risk, or Protected). From the Status column, you can display information about the detected sensitivity type and configure whether the column is sensitive. For at risk columns, the Status column also indicates the confidence level in the sensitivity detection. The Applied Generator column indicates the current generator assignment and allows you update the generator configuration.
Improved the detection accuracy for values in defined categories, such as gender.
Added the option to collect performance metrics when starting a data generation job that uses the Data Pipeline V2 processing.
When you write output to a container repository, you can now use assumed roles for the AWS credentials for Amazon ECR.
Snowflake - Improved performance when loading workspaces that connect to large databases.
Bug fixes and other internal updates.
Fixed an issue that caused the job details view to crash for some jobs that had warnings.
Added a new configuration setting TONIC_SENSITIVITY_SCAN_IGNORE_PII_TYPES. Accepts a comma-separated list of sensitivity type identifiers that the sensitivity scan should ignore.
Fixed an issue where missing data could cause the application to crash.
Bug fixes and other internal updates.
On Database View, when you create a sensitivity rule from the bulk edit function, Structural now automatically populates the data type.
Bug fixes and other internal updates.
Fixed an issue where generator application errors appeared to be encrypted.
Bug fixes and other internal updates.
Fixed an issue where generator application errors appeared to be encrypted.
Fixed an issue where generator suggestions caused the sensitivity scan to fail.
Fixed an issue that caused an error to sometimes occur when configuring a workspace to write output to Tonic Ephemeral.
Fixed an issue that caused display issues for Collection View rows that had applied generators.
Fixed an issue that prevented the application from redirecting after the workspace settings were saved.
Docker Compose files that are generated for output to a container repository now no longer contain an obsolete version specification.
Removed data science mode from Structural. The option to enable data science mode no longer appears in workspace settings.
International Address generator - The new International Address Generator can generate street addresses and postal codes for Canada, and postal codes for the United Kingdom.
Snowflake - Improved performance when loading workspaces that connect to large databases.
For new workspaces that support it, Data Pipeline V2 is now the default data generation option.
Structural now provides suggested generators for additional sensitivity types such as marital status.
The Protection Audit Trail now includes sensitivity rule events, including when a sensitivity rule is created, edited, or deleted.
Bug fixes and other internal updates.
Fixed an issue that caused a horizontal scrollbar to appear beneath the Privacy Hub sensitive column cards.
Bug fixes and other internal updates.
Structural no longer requires an external mount to monitor workers.
The new environment setting TONIC_DB_SCHEMA configures the Tonic Structural application database schema. Use the new environment setting TONIC_MIGRATION_ENABLE_LOGGING to enable or disable logging when applying migrations to the Tonic Structural application database. Note that if TONIC_DB_SCHEMA is a non-public value, then you must restart the API container. If there was existing data in the Tonic Structural application database, that data is not migrated. After the API is restarted and the migrations are applied, you can import a backup of the existing data.
When you create a custom sensitivity rule from the Bulk Edit panel of Database View, you can now choose to apply the generator preset to all of the workspace columns that match the new rule. The Save and Apply option replaces the previous Save and Scan option.
Bug fixes and other internal updates.
Oracle - When TONIC_ORACLE_SKIP_CREATE_DB is true, tables in the destination database are preserved. If a table filter is provided in TONIC_TABLE_WHITELIST, then destination tables that are not in the whitelist are preserved. Before this change, those tables were truncated.
Bug fixes and other internal updates.
MongoDB - Added the environment setting TONIC_MONGO_MASK_FIELD_IDS. When set to true, Structural assigns the Mongo ObjectId Key generator to ObjectId field names. Consistency is enabled. The default value is false.
Bug fixes and other internal updates.
Improved error handling for PII detectors.
Google BigQuery - Fixed an issue where some BigQuery workspaces did not release rows from memory in a timely manner.
MySQL - The new environment setting TONIC_MYSQL_USE_NATIVE_DUMP_TOOL, when set true, prefers the mysql native tools over the existing default of mariadb.
MySQL - You can now configure the following environment settings to override the Structural default behavior when a connection opens and a session is established:
TONIC_MYSQL_NETWORK_READ_TIMEOUTTONIC_MYSQL_NETWORK_WRITE_TIMEOUTTONIC_MYSQL_WAIT_TIMEOUTTONIC_MYSQL_LOCK_WAIT_TIMEOUTTONIC_MYSQL_INNODB_LOCK_WAIT_TIMEOUT
Azure SSO - Added support for authenticating application service principals using the EntraID client-credentials flow. Service principals can access the Structural API. For configuration requirements, refer to the Azure/EntraID SSO configuration information in the Structural User Guide.
SQL Server - Fixed the copy of stored procedures that reference full-text indexes.
Structural now supports diagnostic logging for upsert data generation.
Snowflake - Fixed an issue where, when using the connection string and key-pair authentication options, the test connection button was disabled unless a password was entered.
Updated the legal text that is displayed for new account creation on self-hosted Structural instances.
For custom sensitivity rules, column matching rules are now always case insensitive. Previously, the column matching rules were always case sensitive.
SQL Server - Added support for:
- Full-Text Search catalogs and indexes
- Change tracking for database and table scopes
Bug fixes and other internal updates.
Improved the free trial onboarding flow. Added to the available data connectors. Fixed an issue with creating a Google BigQuery workspace. Fixed a duplicate display issue.
File connector - Fixed an issue that a caused authorization failures when using Assume Role to authorize access to Amazon S3 from Structural Cloud.
Fixed an issue where after an import from a JSON file, Subsetting view did not immediately reflect the state of the workspace.
Spark - Removed support for Livy on Hive.
PostgreSQL - Fixed an issue where Structural failed to process tables that contained a generated column that referenced a user-defined type.
Scheduling data generation - From the Jobs view (renamed from Job History) for a workspace, you can now configure the data generation to run automatically on a schedule. The schedule consists of one or more cron expressions, along with the time zone to use for the schedule. The Structural API includes new endpoints to manage the job schedule.
MongoDB - Fixed an issue that caused connection errors for connections to a Mongo Atlas cluster from Tonic Cloud.