Comments on: Sequentially assigned IDs and giving away competitive intelligence http://www.brianp.net/2009/10/06/sequentially-assigned-ids-and-giving-away-competitive-intelligence/ Occasional Writings Sun, 28 Feb 2010 03:21:42 -0800 http://wordpress.org/?v=2.9.1 hourly 1 By: <img src='http://www.brianp.net/wp-content/plugins/rpx/images/openid.png'/> khan http://www.brianp.net/2009/10/06/sequentially-assigned-ids-and-giving-away-competitive-intelligence/comment-page-1/#comment-15 khan Sun, 22 Nov 2009 12:01:39 +0000 http://www.brianp.net/?p=192#comment-15 Brian, what flaws, if any, exist by using a hash function to generate the public IDs? Using a hash function like Keccak (http://keccak.noekeon.org/), you could limit the output key to 64 bits (more compact than UUID) and use a private integer as the input key (which could be uniquely, efficiently and conveniently generated by a database) and use a salt (a Keccak feature) to thwart brute force attacks (by computing the has of a series of integers to see which one produces a given hash). Collisions appear to be a significant issue, but if your user-creation script (perhaps even enforced by the db with a UNIQUE constraint on the ID_HASH column), when a collision is encountered, the penalty is to call the db increment trigger and move to the next integer and hope you don't get another collision. The principal downside I can see is that it's not O(1), but the authors claim "13 cycles per byte", which is reasonably efficient for modern hardware and the number of bytes being hashed in this application. Another hash function that could do the trick. Skein: http://www.skein-hash.info/about. Brian, what flaws, if any, exist by using a hash function to generate the public IDs? Using a hash function like Keccak (http://keccak.noekeon.org/), you could limit the output key to 64 bits (more compact than UUID) and use a private integer as the input key (which could be uniquely, efficiently and conveniently generated by a database) and use a salt (a Keccak feature) to thwart brute force attacks (by computing the has of a series of integers to see which one produces a given hash).

Collisions appear to be a significant issue, but if your user-creation script (perhaps even enforced by the db with a UNIQUE constraint on the ID_HASH column), when a collision is encountered, the penalty is to call the db increment trigger and move to the next integer and hope you don’t get another collision.

The principal downside I can see is that it’s not O(1), but the authors claim “13 cycles per byte”, which is reasonably efficient for modern hardware and the number of bytes being hashed in this application.

Another hash function that could do the trick. Skein: http://www.skein-hash.info/about.

]]> By: <img src='http://www.brianp.net/wp-content/plugins/rpx/images/openid.png'/> Ron http://www.brianp.net/2009/10/06/sequentially-assigned-ids-and-giving-away-competitive-intelligence/comment-page-1/#comment-10 Ron Tue, 06 Oct 2009 18:17:38 +0000 http://www.brianp.net/?p=192#comment-10 Even without signing up every day, with a reasonably small sample, you can get a pretty good estimate of the user base by applying the <a href="http://en.wikipedia.org/wiki/German_tank_problem" rel="nofollow">German Tank Problem</a>. Not really a problem when you actually want people to know how many users you have, but you probably don't want to tell competitors how many advertisers you have (by using an advertiser id in the URL or query arg). I like to combine randomness with an autoincremented value: timestamp. Use the most entropic bits (the microseconds) for the high order bits of the id and the less entropic bits (seconds, mins, hours, ...) for the low order bits (or simply reverse the timestamp bit order altogether). Then toss on some random bits at the end. Even without signing up every day, with a reasonably small sample, you can get a pretty good estimate of the user base by applying the German Tank Problem. Not really a problem when you actually want people to know how many users you have, but you probably don’t want to tell competitors how many advertisers you have (by using an advertiser id in the URL or query arg).

I like to combine randomness with an autoincremented value: timestamp. Use the most entropic bits (the microseconds) for the high order bits of the id and the less entropic bits (seconds, mins, hours, …) for the low order bits (or simply reverse the timestamp bit order altogether). Then toss on some random bits at the end.

]]>