5492
GSF 2.3 Release Notes
GSF 2.3 Release Notes
The Geospatial Services Framework (GSF) provides geospatial processing as a web service. It is a component-based system that runs on Node.js(R) and uses a Redis(R) datastore.
Node.js(R) is a JavaScript runtime built on Chrome's V8 JavaScript engine. Node.js(R) uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.
Redis(R) is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker.
This help article contains release notes for GSF 2.1. It includes features that were added since the last release. See the following sections:
System Requirements
| Platform |
Hardware |
Operating System |
Supported Versions |
| Windows |
Intel/AMD 64-bit |
Windows |
7 SP1, 8, 10, Server 2012, Server 2016 |
| Linux |
Intel/AMD 64-bit |
Linux |
Kernel 2.6.32, glibc 2.12 |
Supported versions indicate that GSF was tested on that version. You can install and run GSF on other versions that are binary-compatible with those listed.
Additional Requirements:
New Features in ENVI
HTTP/S file download support
A new gsf-http-download-parameter-mapper is now available. This parameter mapper makes it easy to get task input data directly from any HTTP or HTTPS url.
See the help topic on gsf-http-download-parameter-mapper for more details: http://localhost:9191/help/gsf-http-download-parameter-mapper/#/help/gsf-http-download-parameter-mapper/home.html
Azure Block Blob Workspace Manager
A new module called gsf-azure-blockblob-workspace-manager has been added. It allows users to specify an Azure Block Blob container as the workspace for GSF processing results.
See the help topic on gsf-azure-blockblob-workspace-manager for more details: http://localhost:9191/help/gsf-azure-blockblob-workspace-manager/#/help/gsf-azure-blockblob-workspace-manager/home.html
Google Cloud Parameter Mapper
The gsf-google-cloud-parameter-mapper module has been updated to support the gs: protocol for bucket URLs. Support was also added for storage.cloud.google.com, in addition to storage.googleapis.com.
Configuration via Environment Variable Substitution
It is now possible to use environment variables within module configuration values. This allows configuration values to be specified externally, rather than being hardcoded in the GSF configuration file. See the help topic on gsf-config-loader for more details: http://localhost:9191/help/gsf-config-loader/#/help/gsf-config-loader/home.html
Redis configuration
The ability to configure Redis connection information through the environment variable GSF_KUE_REDISCONNECTION is now available.
The new environment variable provides additional capability for providing connection information over GSF_REDISPORT and GSF_REDISHOST environment variables. GSF_REDISPORT and GSF_REDISHOST have been deprecated. Please transition to GSF_KUE_REDISCONNECTION.
See the help topic on gsf-kue-job-manager for more details: http://localhost:9191/help/gsf-kue-job-manager/#/help/gsf-kue-job-manager/home.html
The preferred method
Redis Task Catalog
A new module called gsf-redis-task-catalog has been added. This module queries all of the configured engines for their service names and available tasks and stores that information in a Redis Cache. This improves the performance of GSF when retrieving service and task information.
See the help topic on gsf-redis-task-catalog for more details: http://localhost:9191/help/gsf-redis-task-catalog/#/help/gsf-redis-task-catalog/home.html
Refresh Task Catalog Request Handler
A new module called gsf-refresh-catalog-request-handler has been added. This module provides a REST endpoint to refresh the Task Catalog. This allows the catalog data to be refreshed if a service or task is removed from a running GSF system.
See the help topic on gsf-refresh-catalog-request-handler for more details: http://localhost:9191/help/gsf-refresh-catalog-request-handler/#/help/gsf-refresh-catalog-request-handler/home.html
Updated gsf-info-request-handler
The gsf-info-request-handler has been modified to return only a sanitized set of the server configuration values by default. The module can be configured to return all server configuration values if desired. The sanitized output consists of a list of the currently configured module types.
The purpose of this modification is to prevent any sensitive configuration values, such as user credentials for accessing external resources, from being displayed to users of GSF.
See the help topic on gsf-info-request-handler for more details: http://localhost:9191/help/gsf-info-request-handler/#/help/gsf-info-request-handler/home.html
Python Engine
The gsf-python-engine has been updated to support default values set in the task file to be passed through to the python function.
See the tutorial for an example on how to implement default values: http://localhost:9191/help/tutorials/python-engine.html
Add waitForJob Method to EngineAPI
A new method, called waitForJob, has been added to the EngineAPI object. This method provides a callback that notifies the caller when a specific job is complete. This makes it easier to write JavaScript tasks that submit jobs using the EngineAPI object.
Updated Node.js
Updated Node.js to version 8.11.3.
Updated JavaScript Client SDK
Updated gsf-js-client-sdk to version 2.2.1.
New features:
- Added headers object to the ServerArgs object to allow custom headers to be used in requests.
- Added Server.jobInfoList() for fetching an array of the current JobInfo objects.
This is only applicable if you are creating custom modules and want to use the new jobOptions feature.
Preview version of gsf-request-handler
A new module called gsf-request-handler has been added. This module implements a new RESTful API to GSF, and is available as a preview of the next generation API. This new API is being introduced to allow us to expand the features of GSF going forward.
We will be deprecating the current API (ese-request-handler) in the next major release of GSF.
If you use the GSF JavaScript Client SDK, you will need to use the 3.0 version of the SDK to communicate with a GSF server configured with the gsf-request-handler. For help with migrating a client application to version 3 of the SDK, see the Migration Guide: https://geospatial-services-framework.github.io/sdk-docs/manual/migration-guide-v3.html
See the help topic on gsf-request-handler for more details: http://localhost:9191/help/gsf-request-handler/#/help/gsf-request-handler/home.html
Job Logging
It is now possible to control job logging for the IDL and ENVI service engines on a per-request basis using the jobOptions property in the request body (requires use of gsf-request-handler). Configurations specified in the jobOptions object will override the server and engine level configuration. For more details, see the engine's documented configuration options.
Job Results File
A new server configuration, called jobResultsFile, controls the writing of a file in the workspace that contains the job results object. This configuration may also be set during a request by adding it to the jobOptions object (requires use of gsf-request-handler). This makes it possible to write job results to a file.
See the help topic on gsf-server-core for more details: http://localhost:9191/help/gsf-server-core/#/help/gsf-server-core/home.html
Add new gsf-workspace-parameter-mapper
A new module called gsf-workspace-parameter-mapper has been added. Designed to work with the new gsf-request-handler, this parameter mapper translates between http://[server]/jobs//workspace/ urls and a workspace directory on the server.
See the help topic on gsf-workspace-parameter-mapper for more details: http://localhost:9191/help/gsf-workspace-parameter-mapper/#/help/gsf-workspace-parameter-mapper/home.html
Task filtering for idl-service-engine and envi-service-engine
A new optional configuration called taskInclusionFilter allows filtering of tasks available to an engine. Without this filter, all built-in IDL and ENVI tasks are available for use in GSF. To control which built-in tasks can be run, configure the idl-service-engine or envi-service-engine with a taskInclusionFilter which specifies a task name pattern.
See the help topic on envi-service-engine or idl-service-engine for more details: http://localhost:9191/help/envi-service-engine/#/help/envi-service-engine/home.html
Service name override
Engines can now be optionally configured with a custom ServiceDefinition object which may be used to override the service name and description. See engine configuration documentation for more information.
postJobCleanup for ParameterMappers
ParameterMappers have a new lifecycle method called postJobCleanup. This allows a Parameter Mapper to perform cleanup after a job is completed.
See the help topic on ParameterMapperAPI for more details: http://localhost:9191/help/gsf-basic-parameter-mapper/#/help/gsf-basic-parameter-mapper/docs/ParameterMapperAPI.html
Update to gsf-server-core@4
The new server core API provides a new jobOptions value in JobInfo. This can be set when a job is submitted and provides a location to specify additional processing options that may be used by various modules. These are different from inputParameters in that they may not be relevant to the analytic but might specify directives to individual modules. For instance, the job route is now specified as an option instead of a positional parameter. Job Managers should be updated to use this new API. Task Catalogs also have an updated API that allows for more control over managing tasks. It is now possible to specify filters on taskList as well as publish and remove tasks.
Fixed Issues
- GSF-31 Unix installer requires user to hit Enter twice when entering path to IDL for ENVI
- GSF-60 The service script fails when using the "exists" command
- GSF-61 Using an input value that gets translated by a parameter mapper as input to a metatask doesn't allow the original value to be passed to the subtask
- GSF-64 Jobs fail when gsf-expiring-workspace-manager is enabled in certiain situations
- GSF-65 Files in workspace subdirectories are not downloadable
- GSF-67 ese-job-parameter-mapper does not allow custom api root
- GSF-72 File download incorrectly checks authorization for taskList action instead of getFile action
- GSF-75 ese-request-handler should use 'best' parameter mapper result
- GSF-85 GSF should ship with node binaries for RedHat and Centos at a minimum
12/6/18 - mm