proxy.golang.org : github.com/calebcase/triegen
Package triegen implements a code generator for a trie for associating unsigned integer values with UTF-8 encoded runes. Many of the go.text packages use tries for storing per-rune information. A trie is especially useful if many of the runes have the same value. If this is the case, many blocks can be expected to be shared allowing for information on many runes to be stored in little space. As most of the lookups are done directly on []byte slices, the tries use the UTF-8 bytes directly for the lookup. This saves a conversion from UTF-8 to runes and contributes a little bit to better performance. It also naturally provides a fast path for ASCII. Space is also an issue. There are many code points defined in Unicode and as a result tables can get quite large. So every byte counts. The triegen package automatically chooses the smallest integer values to represent the tables. Compacters allow further compression of the trie by allowing for alternative representations of individual trie blocks. triegen allows generating multiple tries as a single structure. This is useful when, for example, one wants to generate tries for several languages that have a lot of values in common. Some existing libraries for internationalization store all per-language data as a dynamically loadable chunk. The go.text packages are designed with the assumption that the user typically wants to compile in support for all supported languages, in line with the approach common to Go to create a single standalone binary. The multi-root trie approach can give significant storage savings in this scenario. triegen generates both tables and code. The code is optimized to use the automatically chosen data types. The following code is generated for a Trie or multiple Tries named "foo": type fooTrie The trie type. func newFooTrie(x int) *fooTrie Trie constructor, where x is the index of the trie passed to Gen. func (t *fooTrie) lookup(s []byte) (v uintX, sz int) The lookup method, where uintX is automatically chosen. func lookupString, lookupUnsafe and lookupStringUnsafe Variants of the above. var fooValues and fooIndex and any tables generated by Compacters. The core trie data. var fooTrieHandles Indexes of starter blocks in case of multiple trie roots. It is recommended that users test the generated trie by checking the returned value for every rune. Such exhaustive tests are possible as the the number of runes in Unicode is limited. Example_build shows how to build a simple trie. It assigns the value 1 to 100 random runes generated by randomRunes. Example_lookup demonstrates how to use the trie generated by Example_build.
Registry
-
Source
- Documentation
- JSON
purl: pkg:golang/github.com/calebcase/triegen
License: BSD-3-Clause
Latest release: about 9 years ago
First release: about 9 years ago
Namespace: github.com/calebcase
Stars: 0 on GitHub
Forks: 0 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: about 1 month ago