Documentation

ADC SAC meeting 20 July 2020 11am-12pm

Present

Tamara Davis, Simon Driver, Andrew Hopkins (Chair), Natasha Hurley-Walker, Amanda Karakas, Fiona Panther, Christian Wolf

In attendance

Simon O’Toole (ADC)

Apologies

N/A

Agenda

11:00 Welcome and introduction (Andrew)
11:05 ADC Overview and status update (Simon O’Toole)
Items for discussion
11:20 Links with NCI through ODC (Optical Data Centre)
11:30 Role of ADC in ADACS
11:40 Links to Gravitational Wave Data Centre, Australian SKA Regional Centre, and other data centres
Summing up
11:55 Review of provisional recommendations arising from discussion
12:00 Close

Minutes

Simon O’Toole provided an overview and status update of ADC. The committee asked a few questions to clarify some items, in particular around the process for ingesting new datasets, and incremental updates of existing datasets:
Surveys need longevity if they’re hoping for long-term stability of the data hosting. ADC has a plan to establish more secure long-term funding. It also has very strong support from AAL and is perceived as an important Australian resource to maintain long-term.
Community feedback on the performance, functionality and utility of ADC has already been requested through a recent ASVO survey, and there is an ADC-specific questionnaire to be released in the next month.

The committee had a discussion of where the interface should lie between NCI and ADC in the context of the national ODC. Points noted during the discussion:

Scope of SkyMapper at NCI is quite limited compared to ADC scope (team management, broader functionality, etc).
Regarding what lives at NCI vs elsewhere: Data, data access nodes, portals, functionality, all could live in different locations.
Having NCI as a host is also related to facility longevity and ongoing support.
Question about access to SkyMapper from systems outside NCI. Answer is that either queries are forwarded, or accessed programatically, and NCI has access restrictions defining who can access what kind of data. Currently IP filtering, but eventually through sign-on-access. Security is still a concern, depending on what is being accessed. Data is a simple thing to manage from that perspective, while broader functionality might be more problematic. A concern was raised about downtime at major data centres, potentially limiting the functionality of ODC.
ACTION (Chris Wolf): Look into the degree to which access to SkyMapper has not been available over the past two years due to NCI downtime.


There is a goal to avoid duplication of resources. NCI has data storage and expertise in maintaining datasets, while ADC has expertise in developing community-driven tools and functionality as well as strong buy-in from the astronomical community. Each should continue to provide these services, and NCI and ADC need to coordinate to establish seamless integration between services and data access.


Recommendation: NCI should serve as a host for data, and ADC should continue to provide its broad range of functionality for interacting with that data.


The committee had a discussion of the role of ADC with ADACS. Examples were given of at least three cases where ADACS had developed software but that there was no place for long-term hosting of the software, or resources for maintenance. The point was made that ADC could provide this resource, and that ADC software engineers would be a good complement to the existing ADACS team bringing expertise with optical datasets and pipelines, as well as the ADC infrastructure.


Recommendation: Closer integration between ADACS and ADC, with ADC to be recognised as an ADACS node for delivering software engineering related to optical datasets and pipelines. ADC to provide hosting, maintenance, and longevity of relevant ADACS deliverables, where they are aligned with other ADC functionality.


The committee had a discussion of links to other data centres.
The functionality and data products of the GW DC were discussed, with a suggestion that having those either hosted at ADC, or able to be accessed/queried through an interoperable functionality to link ADC holdings with GW DC holdings, would enhance the value of both centres.
ACTION (Simon O’Toole) Discuss with Fiona about linking ADC to GW DC.


The point was made that interoperability between data centres is the key. There is a need to move away from user-interface to programatic functionality and links between data centres.
AusSRC was discussed in its role supporting some EMU (ASKAP) development work, and noting its interest in engaging more closely with ADC. Avoid duplication of outward-facing interfaces, given that AusSRC will be heavily focused on SKA data processing.
ACTION (AMH) put Simon and Slava in touch to develop AusSRC engagement with ADC.


Meeting ended at 12:05.