Re: Hash Type Size

[Peter Brooks <peter.h.m.brooks@gmail.com>]

On Tuesday, 3 September 2013 03:47:52 UTC+2, Randy Brukardt  wrote:
> 
> So my answer to "what's a good way to hash a private type/access 
> value/system.address?" is that no such way exists. The best you can do is 
> convert the data to a stream-element array and hash that somehow, but that 
> most likely would be a dubious hash (there is a lot of unused bits in a 
> typical record, and it would be really easy for those bits to end up being 
> most of the result). The whole point is that a hash function needs to be 
> tailored to the data that it is hashing.
> 
This looks like a good topic for somebody's Ph.D!

I'd imagine it possible to have some broad applicability for certain algorithms. In the case of triplestores, the ones used by Wikipaedia contain, mainly, English words and phrases, with an increasing number of entries in other languages. It's be useful to know if a hash function that works for Wikipaedia data in English is more or less effective when used on German or Korean.

The large triplestores of genetic data are clearly a different matter and I can see tailoring the algorithm there would be important - what, though, do you actually need to know about the data to design a hash algorithm?

With RDF, which all triplestores are made of, the question that's occupying me is how much benefit there is to encoding the different parts of the URI.

The triples are of the form:

<http://dbpedia.org/resource/Alabama> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/AdministrativeArea>

subject - predicate - object

Clearly the '<http://' and '>' don't need to be part of the hash, I'll need a mapping both ways, from subject -> predicate -> object and from object -> predicate -> subject, that's what'll be in the binary trees.

The predicates will be a much smaller set, so an associative array will probably be best - probably a three-way associative array, one with the source of the predicate 'www.w3.org/1999/02/22-rdf-syntax-ns' another with the particular predicate 'type' and the third with the whole URI - but the same hash-key for each, as they're the same relation.

This is a bit of a simplification as the triples are actually quads (subject,predicate,object, graph), but that doesn't affect the hashing question.