Can you confirm that? I've just tried that in the bugfix branch and seems to be working for anonymous users too.
Note you have to:
* Either check "Allow users to upload avatars" or "Allow Gravatar avatars" AND
* Check "Default avatar" AND
* Upload the default avatar image