ssh-add -d/-D deletes only manually added key from gnome-keyring.
There is no way to delete mechanically total keys.
This is the original tease, and it ‘s inactive decidedly present .
so, for case, if you have two different automatically-loaded ssh identities associated with two unlike GitHub accounts — say for solve and for home — there ‘s no way to switch between them. GitHubtakes the first one which matches, so you constantly appear as your ‘home ‘ user to GitHub, with no way to upload things to work projects.

Reading: How can I remove an SSH key?

Allowing ssh-add -d to apply to automatically-loaded keys ( and ssh-add -t X to change the life of automatically-loaded keys ), would restore the behavior most users expect .

More precisely, about the return :

The perpetrator is gpg-keyring-daemon :

• It subverts the normal operation of ssh-agent, mostly just so that it can pop up a pretty box into which you can type the passphrase for an encrypted ssh key.
• And it paws through your .ssh directory, and automatically adds any keys it finds to your agent.
• And it won’t let you delete those keys.

How do we hate this ? Let ‘s not count the ways — life ‘s excessively light .
The failure is compounded because newer ssh clients automatically try all the keys in your ssh-agent when connecting to a host.
If there are excessively many, the server will reject the connection.
And since gnome-keyring-daemon has decided for itself how many keys you want your ssh-agent to have, and has autoloaded them, AND WO N’T LET YOU DELETE THEM, you ‘re goner .

This tease is still confirmed in Ubuntu 14.04.4, angstrom recently as two days ago ( August 21st, 2014 )
A possible workaround :

• Do ssh-add -D to delete all your manually added keys. This also locks
  the automatically added keys, but is not much use since gnome-keyring will ask you to unlock them anyways when you try doing a git push.
• Navigate to your ~/.ssh folder and move all your key files except the one you want to identify with into a separate folder called backup. If necessary you can also open seahorse and delete the keys from there.
• Now you should be able to do git push without a problem.

Another workaround :

What you actually want to do is to turn off gpg-keyring-daemon raw.
Go to System --> Preferences --> Startup Applications, and unselect the “ SSH Key Agent (Gnome Keyring SSH Agent) “ box — you ‘ll need to scroll down to find it .
You ‘ll still get an ssh-agent, only now it will behave sanely : no key autoloaded, you run ssh-add to add them, and if you want to delete keys, you can. Imagine that .

This comments actually suggests :

The solution is to keep gnome-keyring-manager from ever starting up, which was queerly unmanageable by finally achieved by removing the broadcast charge ‘s carry through license .

Ryan Lue adds another matter to corner case in the comments :

In case this helps anyone : I even tried deleting the id_rsa and id_rsa.pub files raw, and the key was however showing up .
Turns out gpg-agent was caching them in a ~/.gnupg/sshcontrol file ; I had to manually delete them from there .

That is the case when the keygrip has been added as in here .