If you want to change the default Joomla 2.5 favicon / Joomla 3.0 favicon on your web site you should replace the favicon.ico file kept in the templates/your_template_name folder with the new one.
The file can be replaced through cPanel->File Manager or by using your favourite FTP client.
You can create a favicon manually on your computer or you can use an online favicon generator.
Once you are ready clear your browser's cache and your Joomla's cache (if it is enabled) and reload your website.
Joomla is a powerful software that is easy to work with once you have the right host to support you. Before you lose any more time with the wrong host, check out the hostlantern Joomla hosting offer.