So I proved that this idea is somewhat feasible. There were two major problems that need to be addressed: 1. there were too many probe requests, 2. there needed to be some way to send found devices back to a server.
The “too many probe requests” problem was easy to fix. I implemented a bloom filter, and for each sniffing session, sniff requests from previously identified devices would be ignored. I also got rid of the Arduino environment since I need access to a newer SDK, and the Arduino environment ships with an older SDK. To set-up the SDK was quite a hassle. (SDK)
The other problem was (in my opinion) more important: Data need to be encrypted before sent – it is a pretty bad idea to send MAC addresses in plain text. So I built a protocol around XTEA-CCM, eXtended Tiny Encryption Algorithm in CCM (CTR + CBC MAC) mode. XTEA was chosen because it is easy to implement and has no known security-breaking flaws. The weakness of XTEA is its slowness – not a problem here since data sent is usually less than 4KB: the 120MHz ESP8266 should be able to handle that easily. Now the problem is what protocol to use for transmission of this encrypted data: the ESP8266 supports TCP and UDP. I decided to go with HTTP/1.1 (which is, at its core, TCP). Using raw TCP / UDP sockets is of course convenient, but requires a standalone IP address, and server with standalone IP address is kind of expensive (a whopping $3/month!). By using HTTP, I can use services like Heroku, Google App Engine…, which are free for small projects like this. (Cost is important)
I wrote a few wrappers around the original APIs to abstract things a bit and to make life easier. Original SDK requires you to do almost everything manually (WiFi connection management, DNS lookup, etc.), so it is nice to separate core logic from supporting code – currently user_main.c is less than 200 SLOC while the entire repository is around 1300 SLOC. Dumping 1300 lines into a single file would be a disaster.