openHistorian v2.8

This is the official release for openHistorian v2.8

This is version 2.8.132 of the openHistorian, released on November 17, 2021

NOTE: If you are upgrading from a prior version of the openHistorian, older than v2.8, you must migrate your existing configuration to use the latest database schema when the Configuration Setup Utility is run after the installation completes - this version includes schema updates.

Updates Since v2.8 (RC4) May Release

• Improvements for parsing / generating spanned IEEE C37.118-2011 Configuration 3 Frames
• Fixes for STTP Reverse Proxy subscribing server auto-reconnection
• Optimizations to PDC initialization for non-historian associated devices
• CSU updates to adjust default value of max thread pool values to reduce context switch issues on larger machines
• Added option to allow disconnection event for file-based inputs when end-of-file is encountered
• Improved Grafana metadata select operations for variable queries
• Added slice-based expression Eval function to GSF Grafana functions
• Improved access locking around legacy OH1.0 metadata files
• Added option to OSI-PI output adapter to only synchronize the AlternateTag field
• Fixed manager statistic overflow issues related to average latency calculations
• Improved synchrophasor phase guessing to include high confidence options
• Updated phasor measurement mapper data stream monitor to allow restarts for passive connections
• Improved operation of the GEP Data Extraction tool
• Added extra security to loading tag template files with calculation operations
• Improved SynchronizeLocalClock adapter device time validation operations
• Updated usages of CreatePointTag to apply analog and digital label parameters where applicable
• Added SEL PDC Import tool to installation
• Added APP DFR Import tool to installation
• Updated included PMUConnectionTester installer to v4.6.7
• Updated included StreamSplitter installer to v1.1.3
• Improved default tag-naming expressions when applied from UpdateTagNames tool:

{CompanyAcronym}_{DeviceAcronym}[?{SignalType.Source}=Phasor[:eval{'{PhasorLabel}'.Trim().ToUpper().Replace(' ','_')}_eval{'{SignalType.Abbreviation}'.Substring(0,1)}eval{'{Phase}'=='+'?'1':('{Phase}'=='-'?'2':'{Phase}')}[?{BaseKV}>0[_{BaseKV}]][?{SignalType.Suffix}=PA[.ANG]][?{SignalType.Suffix}=PM[.MAG]]]][?{SignalType.Acronym}=ALOG[:eval{('{Label}'.Trim().Length>0?'{Label}'.Trim().ToUpper().Replace(' ','_'):'ALOG'+((int){SignalIndex}).ToString().PadLeft(2,(char)48))}]][?{SignalType.Source}!=Phasor[?{SignalType.Acronym}!=ALOG[:{SignalType.Acronym}[?{SignalIndex}!=-1[eval{((int){SignalIndex}).ToString().PadLeft(2,(char)48)}]]]]]

See latest updates since previous release candidate.

Security Updates:

This version of the openHistorian corrects two vulnerabilities, both related to the web interface in the version 2.7 release of the openHistorian – note that the main branch and nightly builds have been patched since September 2020. The vulnerabilities were (1) CVE-2020-11023, related to the version of the jQuery JavaScript library embedded with openHistorian, and (2) CVE-2019-10768, related to the version of the Angular JavaScript library embedded with openHistorian.

The jQuery JavaScript library is heavily used by the openHistorian web interface, including the login page. The Angular JavaScript library is only used on the “Real-time Measurements” screen and a user must be authenticated to access the screen. Both reported vulnerabilities require vulnerable JavaScript code to be exploited by an attacker and would require an authenticated user execute the modified code.

Note that none of the GPA web-based code actually used any of the affected functions identified in the JavaScript library vulnerabilities, as such, the noted vulnerabilities could only be exploited by manually modifying an existing or creating a new web page in the openHistorian. Updating a web page in openHistorian requires physical file access to the “wwwroot” folder in the openHistorian. Generally, GPA does not consider either of these vulnerabilities as exploitable within an unmodified openHistorian deployment.
FYI, also including the following note on the recently discovered Grafana vulnerability:

Grafana Vulnerability Notice

The version of Grafana distributed with the latest openHistorian is 7.3.4. Deployed standalone versions of Grafana prior to versions 8.3.2 and 7.5.12 are affected by the following vulnerability: CVE-2021-43813. This is a directory traversal vulnerability for fully lowercase or fully uppercase .md files which is limited in scope and allows access to files with the extension .md to authenticated users only.

Importantly, however, this vulnerability does not affect Grafana instances hosted by the openHistorian. The Grafana instance hosted by the openHistorian runs behind a reverse proxy in the openHistorian web interface. The openHistorian web interface normalizes the PATH requests going to Grafana which mitigates the vulnerability. The only access allowed to the openHistorian hosted Grafana instance is through the openHistorian web interface that requires user authentication.

New Tools / Features:

• AdapterExplorer Tool - analyzes the configuration of a Time-Series Library system validating inputs, outputs and current values
• BulkCalculationState Tool - allows bulk state management of action adapters by searching and filtering
• SEL PDC Importer Tool - directly imports SEL 3373 or 3573 PDC configuration files
• Bulk Sequence Calculator Adapter - creates and calculate sequence values (+/-/0), in bulk, based on current system configuration
• openHistorian Python API - used for reading and writing openHistorian data using Python
• Exports from the web UI now include a progress bar and estimated time remaining
• Dyanmic Iaon Force Graph Web Display - visualizes Time-Series Library inputs, actions, outputs and measurements that bind them
• New system level statistics in Manager application with global lead/lag time adjustment option

Major Updates:

• Added full support for IEEE C37.118 Std2011 optional Configuration Frame 3
• Added web option for COMTRADE CFF export support (ASCII, binary, binary32 and float32)
• Added tag removal options for OSIsoft PI adapter metadata synchronizations
• Improved phase guess operations for synchrophasor device wizard
• Updated DNP3 adapters to latest OpenDNP3 code base with SSL support
• Added "SystemName" for system level deployment level identification provided at install time
• Added SNMP reporting for system statistics
• Improved operation with PostgeSQL schema
• Updated Grafana version to 7.3.4

Notable Updates to openHistorian for version 2.8

• Added AdapterExplorer tool to setup package
• Added auto-sync ability for service / manager database configuration mismatch. Also added system name to manager title
• Added BaseKV to PhasorDetail model for export
• Added batch file and instructions for initializing openhistorian-python submodule
• Added better error handling for Grafana auth proxy controller during user synchronization steps
• Added better exception handling and reporting to operation that attaches to existing archives.
• Added bulk calculation state tool refresh to UpdateDependencies script.
• Added Cancellation Token to Cancel Hadoop connection if Adapter is stopped
• Added COMTRADE export support from Trend/Export web page
• Added code to correct any non-unique SQL Server Device unique IDs before database migration.
• Added command line switch "-NoMutex" to skip single instance mutex check and allow multiple instances to run on the same machine.
• Added Command to ScheduledEmail to Send email immediately for testing
• Added CreateOutputStream tool to installer.
• Added default Javascript minification exclusion for force-graph.js
• Added delete option for bulk calculation states
• Added error page to default application configuration
• Added extra logging and better status message to hadoop Adapter
• Added extra logging to Scheduled Email
• Added file ignores for Python and virtual environment configuration
• Added flag for auto-removal of oldest files before full
• Added full phasor export button (in addition to per-device functionality)
• Added Iaon graph to main menu
• Added initial implementation of a native Python API
• Added initial implementation of Grafana ad-hoc filters feature.
• Added initial ScheduledEmail Adapter
• Added initial web-based Iaon force graph
• Added initial version of Hadoop to openHistorian input Adapter
• Added limit on number of rows affected by SQL statement in SNR summary and fixed SQL Syntax
• Added minimum Signal threshold similar to Unbalance and improved handling of SQL Failures
• Added Moving Window to standard deviation for improved data visibility
• Added new dependent DLLs to installer for updated PostgreSQL database assembly
• Added new tag templates.
• Added Noise Threshold to unbalance Adapter
• Added OnNewMeasurment call to Unbalanced Adapter
• Added OpenHistorian.TrenDAPController.dll to wix script
• Added option to save Daily SNR summaries to SQL DB
• Added phasor CSV export to device details (CHNAM export)
• Added PythonAPIModule as a submodule to new openhistorian-python repo
• Added sample application animation
• Added system name text box to company info screen in installer
• Added system name to HTML page titles
• Added SystemName to global settings
• Adjusted default desired remaining space to 100GB
• Adjusted Unbalance and SNR Report UI to only display Days for selection
• Dropped check for CurrentAdapterIndex against Count – devices haven’t been added when this property gets called.
• Fixed build issue with application of assembly bindings
• Fixed CSU failure messages for SQL server
• Fixed Cutoff in scheduled email SQL execution
• Fixed default literals to use C# 7.0 syntax
• Fixed Error in Alarm counting in SNR reports
• Fixed Exception were parameter list does not match sql Query
• Fixed installer deployment names for Npgsql dependencies
• Fixed issue with measurement being overwritten on every new timestamp in Hadoop DataLoader to allow reading multiple Timestamps in one update
• Fixed issues with parameter values updates
• Fixed OH HistorianView installed shortcut on start menu
• Fixed PostgreSQL operation for assigning existing database
• Fixed security warning in SnapStreamClient Also simple code refresh
• Improved error handling in Grafana auth proxy controller for admin synchronizations
• Improved error page operation.
• Improved Grafana API to handle graphing of min/max extremes over a publication interval and improved interval operations handling
• Improved IaonGraph link labels, legend creation and positioning and window responsiveness to system messages.
• Improved phasor input validation of example MAS adapters
• Improved status display list of attached archive files in openHistorian output adapter.
• Improved trend API to handle exposing min/max extremes over a publication interval
• Improved update of error template name from CSU
• Increased output buffer, reduced wait timeout and applied consistent lock to console output buffer for bulk calculation state tool.
• Increased system name label length for setup dialog
• Migrated Tools/SampleFunctions to SDK format
• Moved client connected state check into critical region in bulk calculation state tool
• Moved Reporting for SNR and Unbalance from OH to SQL DB
• Moved SNR SQL Summary writes to synchronous operation
• Performance improvements to SNRCalculation.PublishFrame().
• Removed auto-addition of error template name setting from CSU (auto added by GSF now)
• Removed BulkCalculationState tool from repo since it now lives in GSF
• Removed positive sequence power calculations from custom tag template.
• Removed unused DataHub and Model definitions
• Renamed archive integrity checker tool and namespace to have no spaces.
• Renamed folder path to ArchiveIntegrityChecker (spaces causing sign tool to fail)
• Renamed full calculations tag template
• Restart SQL Writer on Failure and quit aging summaries after 10 consecutive failures to avoid Queue building up
• Separated phase guess logic into high and low confidence operations for improved quality of operation
• Separated SNR and Unbalance Report into 3 pages for SNR, I Unbalance and V Unbalance to avoid confusion
• Set bulk calculation state tool with an ngen priority of 1
• Switched Hadoop DataLoader to use ODBCConnection instead of ADOdataConnection
• Updated AdapterExplorer configuration
• Updated Alarm logic in Unabalance adapter to work properly and obtain Percent in Alarm Values
• Updated all database connection tests and failure messages to be consistent
• Updated all openHistorian project files to support C# 8 language features
• Updated assembly info in CSU
• Updated attribute operation in ValidateErrorTemplateConfiguration
• Updated both beta and nightly scripts to properly echo and execute with 2019 MSBuild
• Updated build script to use VS 2019
• Updated bulk calculation state tool to better handle console based reload config operations
• Updated bulk calculation state tool to target 64-bit deployment
• Updated bulk calculation state tool with better filtering options.
• Updated BulkCalculationState tool to ignore grouped device names that are suffixed with 'CALC'
• Updated CSU to auto-include setting for error template.
• Updated CSU to execute ValidateAssemblyBindings with proper working directory.
• Updated CSU to force openHistorian output adapters into available historian types regardless of whether or not LoadImplementations function loads types.
• Updated CSU to minimize before attempting to start service to prevent unresponsive UI
• Updated CSU to wrap database rights access migration in try/catch in case old database cannot be accessed.
• Updated d3 and force-graph dependencies to their non-minimized versions. GSF web hosting auto-minimizes and force-graph was failing in production deployments after minimizing an already minimized file.
• Updated Default Values for SOCO Hadoop Loader Adapter
• Updated existing binding redirects for Netwonsoft.Json to point to version 12
• Updated export data handler to accept selected point ID values as a post This allows for very large point ID selection that could otherwise exceed URI parameter string limits
• Updated force-graph to version 1.39.2
• Updated full checkout calculations to comment out bulk sequence calculators
• Updated full checkout calculations to include time_sync
• Updated Grafana controller to better respond to opt-in "include peaks" filter expression option
• Updated Grafana home page to better update titles with system name When possible from home page script.
• Updated Grafana version to 7.3.4
• Updated Hadoop DataLoader to not use Hadoop Keywords in Query
• Updated installer to add Utilities folder with CLI shortcuts / also now remembers last install path.
• Updated installer to reference bulk calculation state from installer project directory.
• Updated installer to retrieve last service account during installation.
• Updated jquery ajax calls to use "method" parameter instead of "type"
• Updated LocalOutputAdapter server dispose to check for null reference, in case snap server failed to start.
• Updated MAS adapter examples to associate output measurements with the same device used for input measurements.
• Updated MAS adapter examples to be less sensitive to changes in configuration while child adapters are initializing.
• Updated MAS adapters example to read current device ID by acronym.
• Updated MAS adapters sample to properly query physical database device ID for provided runtime device ID
• Updated MaximumArchiveDays to an auto-property.
• Updated OH local output adapter file listing to be on a new line
• Updated openHistorian Manager to update config file with validated assembly bindings on build.
• Updated phase guess operation: since positive sequence is the default and always treated with accuracy suspicion, it's value is tested first and moved zero sequence phase to second check since values should be very unique.
• Updated Postgres assembly version to use 4.0.11
• Updated remaining project files to apply post-build signing - especially for openHistorian.Adapters.dll so it will not have issues loading in CSU.
• Updated Reporting of SNR and Unbalance to avoid session based SQLlite files
• Updated ScheduledEmail to work with SQL Query and File based xml data
• Updated service account reg search to be more generic
• Updated setup complete screen in CSU to only attempt service start if service is not already running. Naming convention tool, when run, also starts service.
• Updated setup to include DeviceStatAdapters
• Updated SNR and Unabalance Reporting HubClient to include new Alarm Counts
• Updated solution reference to sub-module based openhistorian-python repo project
• Updated Status of SNR and Unbalance adapter to catch null condition if initialization failed
• Updated system name property value in installer to default to a single space instead of an empty string which is not allowed.
• Updated tag templates to accept static singleton declared custom action adapter instances.
• Updated Unbalance Adapter to include standard Alarming with Delay and Hysteresis
• Updated Unbalanced Computation to write Summary to SQL and use updated Parallel initialization
• Updated update dependencies script to pull in CreateOutputStream exe

See full release notes since v2.7 release.

File Hashes (SHA256)

openHistorian.Binaries.zip => AE59857AB4FC17E2E9700478D6572E84452E8B290AB040D9E31E1736282A83A2
openHistorian.Installs.zip => 847C5A613EF1A07B8EE6E6A35E62F6787C5C8F4C1CF3DB52B5353D729F0EA9A6
openHistorian.Scripts.zip => 8F71BEDAB98D871C71F32CB85F349BDFA1673E9C14610EF679A1F705D551C4DF