Software Open Access

ekzhu/datasketch: Improved performance for MinHash and MinHashLSH

Eric Zhu; Vadim Markovtsev; aastafiev; Wojciech Łukasiewicz; ae-foster; Jordan Martin; Ekevoo; Kevin Mann; Keyur Joshi; Spandan Thakur; Stefano Ortolani; Titusz; Vojtech Letal; Zac Bentley; fpug


JSON Export

{
  "files": [
    {
      "links": {
        "self": "https://zenodo.org/api/files/36d93203-e8cb-4c69-a521-a5e4d845304b/ekzhu/datasketch-1.5.2.zip"
      }, 
      "checksum": "md5:a3d3bce4aa309dab4bcd0ed08870cbc8", 
      "bucket": "36d93203-e8cb-4c69-a521-a5e4d845304b", 
      "key": "ekzhu/datasketch-1.5.2.zip", 
      "type": "zip", 
      "size": 795687
    }
  ], 
  "owners": [
    28557
  ], 
  "doi": "10.5281/zenodo.4323502", 
  "stats": {
    "version_unique_downloads": 111.0, 
    "unique_views": 30.0, 
    "views": 31.0, 
    "version_views": 1453.0, 
    "unique_downloads": 1.0, 
    "version_unique_views": 1261.0, 
    "volume": 795687.0, 
    "version_downloads": 228.0, 
    "downloads": 1.0, 
    "version_volume": 276318710.0
  }, 
  "links": {
    "doi": "https://doi.org/10.5281/zenodo.4323502", 
    "conceptdoi": "https://doi.org/10.5281/zenodo.598238", 
    "bucket": "https://zenodo.org/api/files/36d93203-e8cb-4c69-a521-a5e4d845304b", 
    "conceptbadge": "https://zenodo.org/badge/doi/10.5281/zenodo.598238.svg", 
    "html": "https://zenodo.org/record/4323502", 
    "latest_html": "https://zenodo.org/record/4323502", 
    "badge": "https://zenodo.org/badge/doi/10.5281/zenodo.4323502.svg", 
    "latest": "https://zenodo.org/api/records/4323502"
  }, 
  "conceptdoi": "10.5281/zenodo.598238", 
  "created": "2020-12-15T20:58:00.644340+00:00", 
  "updated": "2020-12-16T00:27:12.391925+00:00", 
  "conceptrecid": "598238", 
  "revision": 3, 
  "id": 4323502, 
  "metadata": {
    "access_right_category": "success", 
    "doi": "10.5281/zenodo.4323502", 
    "description": "<ul>\n<li>Performance improvement for MinHash's update method.</li>\n<li>Make MinHash updates 4.5X faster by using <code>update_batch</code> method for bulk update on MinHash. [See API doc].(<a href=\"http://ekzhu.com/datasketch/documentation.html#datasketch.MinHash.update_batch\">http://ekzhu.com/datasketch/documentation.html#datasketch.MinHash.update_batch</a>)</li>\n<li>Further performance gain by using bulk generation of MinHash using <code>MinHash.bulk</code> or <code>MinHash.generator</code>. See <a href=\"http://ekzhu.com/datasketch/documentation.html#datasketch.MinHash.bulk\">API doc</a> and <a href=\"https://github.com/ekzhu/datasketch/pull/142\">pull request</a>.</li>\n<li>Optional compression for MinHash LSH index by hashing the bucket key produced by <code>MinHashLSH._H</code>. See <a href=\"https://github.com/ekzhu/datasketch/pull/143\">pull request</a>. This leads to saving of memory/storage space used by the index.</li>\n</ul>\n<p>Thank you @Sinusoidal36!</p>", 
    "license": {
      "id": "other-open"
    }, 
    "title": "ekzhu/datasketch: Improved performance for MinHash and MinHashLSH", 
    "relations": {
      "version": [
        {
          "count": 16, 
          "index": 15, 
          "parent": {
            "pid_type": "recid", 
            "pid_value": "598238"
          }, 
          "is_last": true, 
          "last_child": {
            "pid_type": "recid", 
            "pid_value": "4323502"
          }
        }
      ]
    }, 
    "version": "1.5.2", 
    "publication_date": "2020-12-15", 
    "creators": [
      {
        "name": "Eric Zhu"
      }, 
      {
        "affiliation": "@athenianco", 
        "name": "Vadim Markovtsev"
      }, 
      {
        "name": "aastafiev"
      }, 
      {
        "name": "Wojciech \u0141ukasiewicz"
      }, 
      {
        "name": "ae-foster"
      }, 
      {
        "name": "Jordan Martin"
      }, 
      {
        "name": "Ekevoo"
      }, 
      {
        "affiliation": "Six Five Design", 
        "name": "Kevin Mann"
      }, 
      {
        "affiliation": "University of Illinois, Urbana-Champaign", 
        "name": "Keyur Joshi"
      }, 
      {
        "affiliation": "Adobe", 
        "name": "Spandan Thakur"
      }, 
      {
        "name": "Stefano Ortolani"
      }, 
      {
        "name": "Titusz"
      }, 
      {
        "affiliation": "@blindspot-ai", 
        "name": "Vojtech Letal"
      }, 
      {
        "affiliation": "Klaviyo", 
        "name": "Zac Bentley"
      }, 
      {
        "name": "fpug"
      }
    ], 
    "access_right": "open", 
    "resource_type": {
      "type": "software", 
      "title": "Software"
    }, 
    "related_identifiers": [
      {
        "scheme": "url", 
        "identifier": "https://github.com/ekzhu/datasketch/tree/1.5.2", 
        "relation": "isSupplementTo"
      }, 
      {
        "scheme": "doi", 
        "identifier": "10.5281/zenodo.598238", 
        "relation": "isVersionOf"
      }
    ]
  }
}
1,453
228
views
downloads
All versions This version
Views 1,45331
Downloads 2281
Data volume 276.3 MB795.7 kB
Unique views 1,26130
Unique downloads 1111

Share

Cite as