Introducing pwned-check

by Greg Cochard

I follow Troy Hunt and he recently released a list of 306 million password hashes on his breach notification site ';--have i been pwned?.

Normally when someone does this they let you search the list, but offer you no way to analyze it yourself. Troy recognizes that it's a security risk for you to send your password to a third-party service, and includes a disclaimer against sending your actively-used passwords to (even) his service.

That's all fine and good, but what does this have to do with pwned-check?

Well, he also released all of the hashes, available for download in 7-Zip format, with the files being aggressively cached by Cloudflare, so thanks to both of them for providing this service for free to everyone.

This brings me to my part: I have written a cli tool to streamline the whole process. It will automatically download, unzip, and search the hash dumps for any passwords you pass to it. It does not use the API as that can still potentially leak some information, and it is rate-limited to one request per 1500ms per IP.

You can install it via NPM, and on first-run it will download the files and unzip them.

Usage is simple. You can run it in one of three ways.

  1. Call it directly with your password on the cli (not recommended, as this will be saved in your shell history):
    • pwned-check <password>
  2. Call it standalone on the cli, and it will prompt you for a password, running in a loop until you cancel:
    • pwned-check
  3. Use it programmatically by requiring it in your own JS code.
const {findHash, hashAndFind} = require('pwned-check');
// if plaintext
hashAndFind(plain, callback);
// if hashed
findHash(hash, callback);

You can read the sha.js file to see how it implements the interactive mode.

When you enter your password, it will compute the SHA1 hash and then perform a radix-based search of the password files to determine whether the password has been compromised. This makes it much more efficient than using grep or attempting to open the files with a text editor, as it does not load more than one line of the file into memory at any given time. In the worst case, it will take ~40 disk seeks to find whether the hash exists in the file, so it is extremely fast as well.

The only requirement is that you have the 7z (for OSX) or p7zip (for Linux) binary installed on your system and in your PATH. Oh, and 13GB of free disk space.

I have not added support for Windows, as I don't have a Windows system with which to test. You can send me a pull request if you have the time to add support for it.

Try it out and let me know how you like (or dislike) it!

UPDATE 2018-03-01:

Troy has published Version 2 of the password list and API which uses k-anonymity to keep password requests reasonably private. It computes the SHA1 hash of the password, then sends the first 5 characters to the API, receiving all hashes with those first 5 characters as a reply.

This seems like a good enough privacy solution for most people, so I updated pwned-check to consume this API, but I have published to a feature branch rather than a release on npm. I didn't want to break the strong privacy guarantees that the original pwned-check promised. I am also rewriting the search algorithm to handle variable length strings using the version 2 file, but I don't see this as a high priority. Therefore I am placing the project into maintenance mode, and deprioritizing the rewrite.

Stay tuned for my next project related to this, it's still in the making but should be ready very soon, and I am excited for its potential impact.