Agora Event - Introduction and Open Discussion about EO Exploitation Platform Common Architecture
On Thursday 16th of May at 12 o’clock the team presented the EOEPCA initiative in the Agora Metallica room.
Richard Conway, the EOEPCA Senior Software Architect, presented to the attendees the following points:
- EOEPCA context
- Use Cases
- System Architecture
- Processing and changing
- Resource Management
- User Management
If you would like to see the full presentation, please click in the link below:
The presentation went well and the number of attendees was over 30 people. Different companies participated in the event and in the QA session afterwards such as AirBus, Deimos, Solinex, Terradue, Atos, GMV, JNCC, Capgemini, Telespazio France, SISTEMA GmbH, T-Systems.
The presentation was followed by a QA session where the attendees had the opportunity to interact with the EOEPCA team and debate about the different aspects/challenges of the project. Please see below some of the very interesting questions asked:
Q1. Could the Common Architecture address other aspects of an exploitation platform such as high-performance computing?
A1. Ref. workshop slide 26 (Parallel Processing): The work of the OGC testbeds 13/14 only considers serial processing jobs running in a single Docker container. However, within the existing design there is a possible approach in which the Dockerized processor makes subordinate invocations to exploit some specific data processing compute cluster within the platform, e.g. processor is some python code that invokes a dask or SLURM cluster to perform its work. This would only work if the Platform to which the processor was deployed provided the expected compute service. Thus there would need to be a ‘contract’ through which the Platform declares the services it ‘Provides’, and the Processor declares the services it ‘Requires’. The orchestrator would rely on these declarations to ensure that processors are only deployed/executed on platforms that can support them.
Q2. It was stated that EO data must be FAIR (Findable, Accessible, Interoperable, Reproducible) and will not stay within the EO community
A2. The concept of ‘Reusable Research Objects’ is known to the EOEPCA team, and are seen as an approach in support of the FAIR principals. The Common Architecture will take the FAIR needs into account – see use case ‘Consumer performs Open Science’ (https://eoepca.github.io/use-case-analysis).
Q3. Confirm that UCs are open to community recommendations
A3. Yes. The Use Case document (draft) is published on GitHub (https://eoepca.github.io/use-case-analysis) and we encourage/welcome review and feedback – plus additional use cases that are not currently covered.
Q4. Note CEOS best practice, be wary of emerging standards that are not mature eg JSON LDs as deter take up
A4. The recommendation to follow the CEOS Best Practise is well noted. The JSON-LD approaches are being explored through current Testbed-15 – which we follow with interest. We are committed to follow a standards-based approach – including the evolution of standards where necessary. Our approach will be incremental to take account of such evolving standards.
Q5. Is the Common Architecture closed to companies that are not directly involved in its conception?
A5. No, quite the opposite. All technical documentation and implementation will be open source and available on GitHub. The Domain Experts will be ‘procured’ – specifically from companies across industry external to the system team. And we will continue to engage with the wider community to ensure a consensus-based approach is adopted.
Q6. Will data encryption to supported, particularly across platforms. Change of ownership of data?
A6. End-to-end data encryption has not previously been considered, but will be added as a use case for future consideration.
Q7. Are the OGC standards for interfaces at a level of maturity to be adopted?
A7. Standards will need to be evolved in partnership with OGC. WPS is a standard and trusted. WPS-T is not yet a standard – it is being explored through the OGC Innovation Programme and is the subject of an OGC Software Working Group. Where the functionality required of the Common Architecture is not yet met by a standard, then it is necessary to evolve the standards. This is the approach being taken by working closely with the OGC.
Q8. Request for terminology clarification between standards and proposed standards, as proposed standards are not standards.
A8. Any suggestions?
Q9. Who are the stakeholders of this project?
A9. ESA through its technical officer Cristiano Lopes in the sponsor. There are many stakeholders in that this is intended to be an open architecture freely available to the EO community. Thus, stakeholders include existing and future platform providers, as well as their end-users.
Q10. What does the Common Architecture mean for existing deployment, such as TEPs?
A10. EOEPCA is not intended to immediately replace current EPs, but could offer an attractive evolutionary path where value is seen to be had. The idea is to try to create a digital market for interoperability, and lower the barrier to users coming to the market to access EO data.
Q11. Could you say more about Billing and Accounting features
A11. Based upon the ‘Platform Architectures’ session (Weds morning) – it is clear from the questions/discussion that billing is an important topic – and also a complex one. The current Common Architecture design has not yet considered the area of Accounting & Billing – except to recognise that it is a crucial problem to solve in the architecture. This topic may be left to the Domain Experts to advance. Some work on this topic has been made in the OGC testbed – the engineering reports will provide a useful input to the EOEPCA design.
Q12. Are block chains under consideration?
A12. Not specifically – but no technology is yet ruled out. In particular, if they can offer an approach to distributed accounting/billing, then this would be of interest.
Q13. Where do DIAS fit in to accounting and billing?
A13. DIAS are platforms a lower layers of the reference model, they are free to adopt the standards and interfaces offered by EOEPCA for their own market places.
Q14. Timeline for ITT and architecture implementation
A14. Domain expects ITT expected June, architecture will begin by refining architecture for 2-3 months, then begin incremental implementation, whilst continually reviewing it evolution
Q15. Architecture useful as well as reusable reference implementation components?
A15. Yes. There is re-use at (at least) two levels:- the architecture can be re-used as building-blocks/interfaces against which 3rd-party implementations can be made; the reference implementation can be re-used as ready-made components to be adapted for platform deployments. All docs/software will be available on GitHub. In addition, we will provide support to developers in the re-use of these components.
Q16. How do you define success of this project?
A16. Success is related to achieving consensus and measured according to uptake by existing and future platforms.
Q17. When does funding project end?
A17. Funding for phase 1 ends Nov 2020. There is provision in the contract for a follow on phase (2).