Images and graphics make the web a more interesting place, but as a web site owner you must be careful not to overdo it with heavy image files.
There are also SEO factors to consider. Search engines like Google take into account your page load times and score accordingly, with heavy graphics files being one of the worst offenders. If your site ends up with a lower ranking then your site's findability will be reduced making it less likely that a customer will be able to find you.
PNG files are perfect for transparency. Consider that using a PNG with transparent areas can not only end up looking more stylish but could save you in weight as the same dimension in JPG could weigh heavier, depending on the complexity of the background being used. JPGs are a great format, but make sure that you are optimising them in order to get the maximum "bang for your buck".