PSDI Data Lakehouse

The PSDI Data Lakehouse is a centralised platform that offers a programmatic access to various datasets that are hosted in the cloud environment. It is built to ingest large volumes of data from multiple sources and make them accessible remotely through a uniform machine interface without the need to download full datasets locally. All data in the lakehouse can be queried using the same syntax thus avoiding a lock-in with a specific API for a specific dataset. From the FAIR (Findable, Accessible, Interoperable and Reusable) data principles perspective, what the data lakehouse enables compared to a traditional research data repository is 'I': data interoperability. To achieve it, the original data sources are transformed in a common internal representation, so that a reasonable data search, data filtering and data aggregation performance can be achieved over an individual large dataset or a combination of them. The PSDI data lakehouse can support queries in users' own computational notebooks, scripts, executable workflows or AI jobs. We offer computational notebooks with examples of queries; they can be a good starting point to explore the lakehouse content and functionality.