Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
proxy.golang.org : github.com/m-lab/etl-embargo
Package embargo performs embargo for all sidestream data. For all data that are more than one year old, or server IP in the list of M-Lab server IP list except the samknow sites, the sidestream test will be published. Otherwise the test will be embargoed and saved in a private bucket. It will published later when it is more than one year old. Package embargo implemented site IP loading from public URL or local file and check whether an IP is in the whitelist which is the list of all sites exceot the samknows sites. Parse filename and return componants like log-time, IP, etc. Filename example: 20170315T01:00:00Z_173.205.3.39_0.web100 Package gcs implements a simple library for basic operations given bucket names and file name/prefix, such as ls, cp, rm, etc. on Google Cloud Storage. Implement the umembargo process when the previously embargoed files are more than one year old.
Registry
-
Source
- Documentation
- JSON
purl: pkg:golang/github.com/m-lab/etl-embargo
Keywords: etl, pipeline
License: Apache-2.0
Latest release: over 4 years ago
First release: over 4 years ago
Namespace: github.com/m-lab
Stars: 0 on GitHub
Forks: 1 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 1 day ago