In Mail, go to the Preferences.
Under General change the New messages sound to None
Now open the System Preferences and go to Notification Center.
Turn off all notifications for the Mail ap