Software

software-417880_1920.jpg

All the source code developed under this project is freely available at https://github.com/EOEPCA

A selection of building blocks are in development, addressing the three key domains identified in the architecture:

  1. Resource Management - catalogue and discovery of data and applications
  2. Processing and Chaining - deployment and execution of applications
  3. User Management - identity, security, access control and accounting


Releases

We are working towards the following major release milestones:

  • 1.0 August 2021
  • 1.1 October 2021
  • 1.2 December 2021

Meanwhile Release 0.9 includes beta versions of the following building blocks:

  • Login Service
  • User Profile
  • Policy Enforcement Point (PEP)
  • Policy Decision Point (PDP)
  • Application Deployment & Execution Service (ADES)
  • Processor Development Environment (PDE)
  • Resource Catalogue
  • Data Access Services
  • Workspace

For more details see the Release Note and this description.

What's new

  • User Workspace
    Providing management and access to user owned resources, including full resource catalogue and data access services.
    Workspace resources are access protected to the owning user.
  • Integration of ADES with Protected User Workspace
    Protected access to user's workspace, authorized according to policy.
    User's processing results are pushed to their Workspace bucket, and registered in their Resource Catalogue and Data Access Services.
  • Commercial Operator Identity Hub (COIH)
    Integration of ESA's Commercial Operator Identity Hub (COIH) as an identity provider.
  • OGC API Records
    Support added to Resource Catalogue.
  • EOEPCA Helm Chart Repositories
  • Normalized Hotspot Indices
    New sample application for ADES deployment.


Building Blocks

Login Service
User Management

The Login Service building block provides an OIDC and UMA compliant solution enabling authentication and authorization mechanisms within an Exploitation Platform. Other building blocks, such as the Policy Decision Point, Policy Enforcement Point and User Profile rely on this Building Block to provide several standard interfaces.

User Profile
User Management

In order to support the Billing Service, the User Profile building block allows to identify users (leaving a reference to their home IdP), to assign them Billing Identities, Service API keys, License keys and to record Terms and Conditions acceptance. It’s a persistence service with interfaces that will be queried by other building blocks (License Manager, Billing Service, Policy Decision Point) and modified by both the License Manager and the Login Service (during creation of a new user profile or assignment of new Licenses).

Policy Enforcement Point (PEP)
User Management

The main functionality of the PEP is to be able to stand between a client and the client’s desired resource. By creating this setup, where only the PEP is visible to an external request, we effectively secure whatever is behind the PEP. The PEP will enforce any policy for a resource configured in the Authorization Server, following the UMA 2.0 standard.

Policy Decision Point (PDP)
User Management

The main functionality of the PDP is to be able to perform complex Policy Decisions based on Policy Documents. In order to do so, several functionality blocks are identified:

  • Policy Check Endpoint. A XACML-compliant endpoint that allows to submit JSON XACML requests and receive the corresponding responses.
  • Policy Repository Management. An exposed API allowing to register and/or query policies assigned to specific resources.
Application Deployment & Execution Service (ADES)
Processing and Chaining

The Processing & Chaining domain area provides an extensible repository of processing functions, tools and applications that can be discovered by search query, invoked individually, and utilised in workflows. ADES is responsible for the execution of the processing service through both a OGC WPS 1.0 & 2.0 OWS service and an OGC Processes REST API. The processing request are executed within the target Exploitation Platform (i.e., the one that is close to the data).

Processor Development Environment (PDE)
Processing and Chaining

The Processor Development Environment provides a rich, interactive environment in which processing algorithms and services are developed, tested, debugged and ultimately packaged so that they can be deployed to the platform and published via the marketplace. It includes a JupyterLab instance and the Cloud Theia IDE.

Resource Catalogue
Resource Management

The purpose of the resource catalogue service is to register metadata from either data sources or processing results where they are indexed for discovery and search by downstream applications.

This service uses the COTS https://pycsw.org product with custom configuration.

Data Access Services
Resource Management

There are actually two kinds of deployments of the same software stack in two slightly different roles.

The first role is sort of system global instances, registering and serving upstream data products whereas the second one is used within the context of a user workspace. In this second context, only user provided data will be registered and served.

Workspace
Resource Management

The Workspace is responsible to coordinate a user’s resources within the platform, and make them available through Catalogue and Data Access services. In doing so it provides an abstraction of the underlying infrastructure. The Workspace provisions storage within the underlying infrastructure, typically buckets, on behalf of the user. Components needing access to this user storage, such as the ADES staging out processing results, interrogate the Workspace to obtain details of the storage.