The goal, afaik, is to stop facial detection software from learning to recognize you and put a face to a name, not to frustrate visual similarity searches. The images are supposed to be visually similar -- so similar that they're indistinguishable to a human viewer.
Surveillance software that purports to accurately identify a person across multiple images is not just looking for the same content with some visually insignificant modifications. It's reading your facial structure, attaching your name to it, and searching for it in every image received. Fawkes is working to defeat that specific use case, not all fuzzy matchers in general.
P.S. If you have a human assailant running a reverse image search for photos of you, I think you're well past the point that something like this could be expected to help.
But this suggests a way to defeat the cloak -- run your input through image similarity search, then run your facial recognition software on the hits. This won't work in full generality, not every picture is on the internet like that, but it can certainly help, I imagine.
I think that technique probably works a lot better with high-resolution professional headshots than it does with candid photos at the family reunion, for example.
However, if someone is willing to go to that level of effort, the target probably needs to aim for something a little more forceful than tricking Facebook's autotagger.
Surveillance software that purports to accurately identify a person across multiple images is not just looking for the same content with some visually insignificant modifications. It's reading your facial structure, attaching your name to it, and searching for it in every image received. Fawkes is working to defeat that specific use case, not all fuzzy matchers in general.
P.S. If you have a human assailant running a reverse image search for photos of you, I think you're well past the point that something like this could be expected to help.