Software Open Access

ekzhu/datasketch: Improved performance for MinHash and MinHashLSH

Eric Zhu; Vadim Markovtsev; aastafiev; Wojciech Łukasiewicz; ae-foster; Jordan Martin; Ekevoo; Kevin Mann; Keyur Joshi; Spandan Thakur; Stefano Ortolani; Titusz; Vojtech Letal; Zac Bentley; fpug


JSON-LD (schema.org) Export

{
  "description": "<ul>\n<li>Performance improvement for MinHash's update method.</li>\n<li>Make MinHash updates 4.5X faster by using <code>update_batch</code> method for bulk update on MinHash. [See API doc].(<a href=\"http://ekzhu.com/datasketch/documentation.html#datasketch.MinHash.update_batch\">http://ekzhu.com/datasketch/documentation.html#datasketch.MinHash.update_batch</a>)</li>\n<li>Further performance gain by using bulk generation of MinHash using <code>MinHash.bulk</code> or <code>MinHash.generator</code>. See <a href=\"http://ekzhu.com/datasketch/documentation.html#datasketch.MinHash.bulk\">API doc</a> and <a href=\"https://github.com/ekzhu/datasketch/pull/142\">pull request</a>.</li>\n<li>Optional compression for MinHash LSH index by hashing the bucket key produced by <code>MinHashLSH._H</code>. See <a href=\"https://github.com/ekzhu/datasketch/pull/143\">pull request</a>. This leads to saving of memory/storage space used by the index.</li>\n</ul>\n<p>Thank you @Sinusoidal36!</p>", 
  "license": "", 
  "creator": [
    {
      "@type": "Person", 
      "name": "Eric Zhu"
    }, 
    {
      "affiliation": "@athenianco", 
      "@type": "Person", 
      "name": "Vadim Markovtsev"
    }, 
    {
      "@type": "Person", 
      "name": "aastafiev"
    }, 
    {
      "@type": "Person", 
      "name": "Wojciech \u0141ukasiewicz"
    }, 
    {
      "@type": "Person", 
      "name": "ae-foster"
    }, 
    {
      "@type": "Person", 
      "name": "Jordan Martin"
    }, 
    {
      "@type": "Person", 
      "name": "Ekevoo"
    }, 
    {
      "affiliation": "Six Five Design", 
      "@type": "Person", 
      "name": "Kevin Mann"
    }, 
    {
      "affiliation": "University of Illinois, Urbana-Champaign", 
      "@type": "Person", 
      "name": "Keyur Joshi"
    }, 
    {
      "affiliation": "Adobe", 
      "@type": "Person", 
      "name": "Spandan Thakur"
    }, 
    {
      "@type": "Person", 
      "name": "Stefano Ortolani"
    }, 
    {
      "@type": "Person", 
      "name": "Titusz"
    }, 
    {
      "affiliation": "@blindspot-ai", 
      "@type": "Person", 
      "name": "Vojtech Letal"
    }, 
    {
      "affiliation": "Klaviyo", 
      "@type": "Person", 
      "name": "Zac Bentley"
    }, 
    {
      "@type": "Person", 
      "name": "fpug"
    }
  ], 
  "url": "https://zenodo.org/record/4323502", 
  "codeRepository": "https://github.com/ekzhu/datasketch/tree/1.5.2", 
  "datePublished": "2020-12-15", 
  "version": "1.5.2", 
  "@context": "https://schema.org/", 
  "identifier": "https://doi.org/10.5281/zenodo.4323502", 
  "@id": "https://doi.org/10.5281/zenodo.4323502", 
  "@type": "SoftwareSourceCode", 
  "name": "ekzhu/datasketch: Improved performance for MinHash and MinHashLSH"
}
1,453
228
views
downloads
All versions This version
Views 1,45331
Downloads 2281
Data volume 276.3 MB795.7 kB
Unique views 1,26130
Unique downloads 1111

Share

Cite as