General Availability of Swirl 3.2.0

[erikspears]

Team Swirl is excited to announce General Availability of Swirl 3.2!

This release includes the first version of the AutomaticPayloadMapperResultProcessor, RequireQueryStringInTitleResultProcessor, DropIrrelevantPostResultProcessor and NoModQueryProcessor, along with new SearchProviders for OpenSanctions.org and LittleSis.org. We've also added support for running OpenAI in Azure, storing query templates in JSON, plus updated Python, Django, and OpenAI API versions, and resolved issues found in previous releases. Finally, the Galaxy UI has been updated to ⭐ results above a particular relevancy score.

We're seeking anyone interested in testing new Connectors to Collibra, Thoughtspot and Pinecone DB. Please contact Swirl via email or Slack for access.

Highlights of Release 3.2.0:

• New AutomaticPayloadMapperResultProcessor which profiles response data to find good strings for Swirl's title, body, and date_published fields. It is intended for SearchProviders that would otherwise have few (or no) good result_mappings options.
• New NoModQueryProcessor that only removes leading SearchProvider Tags and does not modify the query terms in any way. It is intended for repositories that allow non-search characters (such as brackets).
• New RequireQueryStringInTitleResultProcessor that drops results that do not contain the query_string_to_provider in the result title field. It is now included by default in the "LinkedIn - Google PSE" SearchProvider.
• New DropIrrelevantPostResultProcessor drops results with swirl_score < settings.MIN_SWIRL_SCORE (set to 500 by default) and results with no swirl_score.
• Swirl now supports using Azure OpenAI models for RAG processing in addition to OpenAI's ChatGPT models.
• The Galaxy UI now highlghts results with a swirl_score above a configurable threshold with a star in the results list.
• New SearchProvider.query_template_json field which stores the query_template as JSON. The MongoDB SearchProvider has been updated to use this feature.
• Support for lowercasing of URLs is now available via a new LC_URL option in SearchProvider result_mappings.
• Added support for list-of-list responses from source APIs, where the first list element is the column names.
• Search the LittleSis free database of "who-knows-who at the heights of business and government" with the new LittleSis.org SearchProvider.
• Find sanctions targets and persons of interest with the new OpenSanctions.org SearchProvider.