Tonic Textual release information

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

On the Home page, in addition to typing or pasting text to redact, you can now upload a file.

Improved the detection of URLs and email addresses.

The URL for the generated Textual SDK documentation is changed to https://tonic-textual-sdk.readthedocs-hosted.com/en/latest/index.html.

Upgraded libraries to address security vulnerabilities.

Bug fixes and other internal updates.

Improved detection of signature blocks in PDFs.

On self-hosted instances, when a dataset file fails to upload, you can now download the associated logs.

For .docx dataset files, the image handling configuration is now also applied to SVGs.

The new Getting Started checklist guides users through the initial tasks to preview redaction, set up API access, install the SDK, and send a redaction request.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Custom entity types - You can now define custom entity types, to identify sets of values that are unique to your industry or organization. For each custom entity type, you provide one or more regular expressions to define the matching values. You can then enable or disable the entity type for each dataset and pipeline.

Added additional language support.

Fixed an issue that caused slow load times for customers with large datasets. Calls to GET /api/dataset and GET /api/dataset/{datasetid} no longer return entity information for the dataset files. Instead, the new GET /api/dataset/{datasetid}/pii_info endpoint returns the entity information for a dataset’s files.

A new dataset settings option controls the output in .docx tables. By default, table content goes through the regular scan and redaction process, and detected entity values are handled based on the dataset’s entity type handling configuration. You can also choose to completely block out all table cells, in which case each table cell is covered by a black box.

Bug fixes and other internal updates.

File statistics for pipelines - The pipeline details page now displays a summary of information about the pipelines files, including the number of files, the number of words in the files, the number of detected entity types, and the number of detected topics. For entity types, the display includes the number of detected values for each type. For topics, the display includes the number of files that involve each topic.

On the dataset details page, the preview count for each entity type now reflects the count of values that are assigned that type in the output files. Previously, values that matched multiple entity types were included in the preview count for all of the matching types.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

New Textual Home page - The Textual Home page now contains an updated version of the Playground, where you can see how Textual detects and replaces entity values in text. There is no longer a separate Playground page, and there is no LLM Synthesis option. For each entity type, you can configure handling options and added or excluded values. Textual generates Python and cURL versions of the request that you can copy.

Improved handling of fillable PDF forms.

From the Request Explorer, in addition to testing added and excluded values, you can now also select the handling type for each entity type. The Unified toggle is replaced with options to view either the original values with their corresponding types (Identification) or the actual output with the replacement values (Replacement).

Regular expression-based email address detection no longer validates the domain name, which makes the detection more general.

Edit and replay recorded requests - When you use the Request Explorer to preview a recorded redaction request from the SDK, you can now edit the request to add and exclude entity values. You can then re-run the redaction and view the differences between the original request and the edited request.

When you configure excluded values for an entity type, you can block detection of a specific type within a matching phrase. For example, if you add the phrase "one moment, please" to an excluded value for numeric values, the word “one” is not detected as a numeric value in that specific context.

For images in dataset .docx files, added a replace option that replaces the images with black rectangles instead of scanning the images for sensitive values.

Fixed an issue that prevented users from saving the dataset configuration for .docx comments.

Record and view redaction requests - When you make a redact call to redact a plain text string, you can now record the request. You specify the amount of time to keep the recording, and any tags to assign to the request. From the new API Explorer page, you can then view and analyze the recorded redaction requests, to assess the quality of the redaction.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Improved the accuracy and performance of NER models.

New redaction options for datasets - The settings panel for datasets now includes additional configuration options:

• You can configure whether to redact or remove images in .docx files.
• You can choose to remove comments from .docx files.
• For PDF files, you can choose to detect and redact scanned signatures.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Added the Tonic NER model version to the model information. The API endpoint /api/environment/models reports version strings for NER models.

Entity manager for entity types - The new entity manager allows you to view all of the occurrences of each entity type in a dataset. it displays the original value, the context in the original file, and the context in the transformed file. To view the entities manager, from the entity value preview list, click Open Entities Manager. Note that by default, for the NUMERIC_VALUE entity type, Textual only provides context information for the first 20 occurrences. To change this, set the SOLAR_NER_OCCURRENCE_IGNORE_NUMERIC_VALUE environment variable to false.

Bug fixes and other internal updates.

Improved detection of names, particularly in ASR transcripts.

Added an optional jsonpath_allow_lists to redact_json. You use jsonpath_allow_lists to override NER results at specific JSON Path expressions.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Textual can now redact images in .docx files.

Fixed a rare issue where Azure OCR returned a400 response when the file upload stream contained corrupted data.

Improved synthesis on days of the week and ordinal numbers that are flagged as DATE_TIME.

Textual now only disables a numeric span when it overlaps one of the following disabled types: DATE_TIME, DOB, LOCATION, LOCATION_ADDRESS, LOCATION_ZIP, MONEY, CREDIT_CARD, PHONE_NUMBER.

Textual now allows you to parse EML and MSG files.

You can now use the Python SDK to configure Azure pipelines.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

You can now use the Python SDK to configure Amazon S3 pipelines.

Amazon Textract can now be used to process dataset files.

On the Python SDK, added parameters for pipeline creation, including the file location, the connection credentials, and whether to also generate redacted files.

Improved the Textual NER model throughput on long strings that contain a large number of numeric characters.

Added the redact_html function to the SDK, which allows you redact sensitive values from HTML strings.

Improved detection of names and organizations.

Disabled auxiliary model detection of WORK_OF_ART.

Improved the Textual NER model throughput on long strings that contain a large number of detected entities.

Added support to store dataset files in a specified S3 bucket, instead of in the Textual application database.

When Textual replaces first name values, it now attempts to use a name with the same gender.

For the DOB (date of birth) entity type, you can now configure synthesis options. You can set how to shift the date.

Bug fixes and other internal updates.

Improved the synthesized values for the PERSON_AGE entity type.

You can now configure the entity type handling for a dataset before you upload the dataset files.

You can now provide added and excluded entity values when you use the SDK to redact individual strings and files.

Added a new method to the SDK. redact_xml works similarly to redact_json. To produce a redacted output, you pass in a redact_xml string.

Improve Pipeline UI to include better Python SDK code snippets

Improved the user experience when you load a large number of files to a dataset.

Updated the UI for adding and excluding values for entity types. Changed the tab labels to Add to detection and Exclude from detection, and removed the requirement to click the edit icon for the entries.

Added support in the SDK to create dataset include lists to define additional values for an entity type.

Removed support for en_core_web_trf and en_core_web_lg auxiliary models. Disabled model inference for ORGANIZATION, PERSON, LOCATION, and MONEY entity types. Updated the auxiliary model configuration environmental variables to have new default values:
TEXTUAL_AUX_MODEL_GPU: false
TEXTUAL_AUX_MODEL: en_core_web_sm

Fixed a redaction issue that was caused by a regression from v140.

Improved the Textual NER model, specifically for datetime values and and electronic health records.

Fix for correctly re-synthesizing files as part of pipelines

When you call the dataset.add_file method in the Textual SDK, you can now pass in IO bytes.

You can now specify a list of additional values to include for each entity in a datasets. This allows Textual to identify values that it might not identify because they are specific to your organization or industry. The list can contain both specific values and regular expressions.

Improved the file list display for datasets to better accommodate longer file names.

For an uploaded file pipeline, added a Back to Files breadcrumb to return the user to the main file list.

On the dataset details page, the bulk edit function for entity type handling is now a dropdown instead of separate buttons.

You can now use the Python SDK to delete files from a dataset.

To improve performance, enabled date synthesis inference on GPU. Added the environment variable TEXTUAL_DATE_ SYNTH_ GPU to manage whether to use it.

Renamed the following environment variables:

• SOLAR_PREFER_GPU to TEXTUAL_AUX_MODEL_GPU
• SOLAR_AUX_MODEL to TEXTUAL_AUX_MODEL

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

You can now create pipelines that use files from Azure Blob Storage.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

Improved performance for previewing PDF files.

Bug fixes and other internal updates.

Bug fixes and other internal updates.

The responses for textual.redact and textual.llm_synthesis now include:
br<

• The language for each entity value.
• The start location (new_start) of each entity value in the redacted content.
• The end location (new_end) of each entity value in the redacted string.

For uploaded file pipelines that also generate redacted files, you can now configure the handling option for each entity type.

Bug fixes and other internal updates.

Added a HEALTHCARE_ID entity type for identifiers associated with health care.

Removed the right-hand panels from the Home page. Added the API Keys panel to the dataset details page to accompany the code snippets.

Bug fixes and other internal updates.

On the Playground, LLM Synthesis is now turned off by default.

Improved synthesis for DATE_TIME entities by recognizing non-standard date formats.

Added a forgot password option to allow Textual Cloud users to reset their password.

Improved our detection of date values.

Removed the US_DRIVER_LICENSE entity type.

Introduced a new model that improves performance for non-English languages.

The responses for textual.redact and textual.llm_synthesis now include the redacted or synthesized value.

Bug fixes and other internal updates.

When Textual synthesizes values, it now matches the capitalization of the original value.

You can now set up a pipeline to connect to a Databricks workspace. You can also configure a Databricks pipeline to generate redacted files in addition to the JSON output.

Textual can now detect values in multiple languages. Textual Cloud supports a set of non-English languages. For a self-hosted instance, you must enable multi-language support and provide the language models for Textual to use.

For PDF and image files, the pipeline file details now include any tables and key-value pairs that are in the file.

The Pipelines and Datasets pages now display lists instead of cards.

Bug fixes and other internal updates.

Textual now supports a pay-as-you-go option for Textual Cloud. When you set up a pay-as-you-go subscription, you provide a credit card that is automatically billed each month for your Textual usage.

Textual no longer includes the option to create custom models to use for datasets.

Redesigned the structure of the JSON output.

Bug fixes and other internal updates.

Bug fixes and other internal updates.