The word comes from the Latin "to dig". Originally, it just meant anything dug up. Today, it means any remains or traces of organisms.
So, "fossil fuels" are indeed fossils. Coal is the remains of plants. Oil is too, but not as directly.
We've all seen a T. Rex skeleton on TV. A lot of fossils do look like bones, although they often aren't bone. After 10,000 years or so, buried bone has most likely mineralized and effectively become rock. Luckily for us, mineralization doesn't destroy much detail.
A few fossils have been found frozen, or pickled in swamps, or dessicated and wind-buried in deserts. Sadly, these are rare and usually recent. Others were buried in an oxygen-free environment, so that no rot occurred. Unfortunately, these environments are usually underwater, so we mostly have sea creatures. We were very fortunate that several Archaeopteryx (land birds) were washed into such a place. Most of the birds that have fossilized were seabirds.
Trace fossils include things like footprints - that is, any trace that the creature was there. We sometimes find trails, tubes, borings, worm casts, burrows, leaf imprints, skin imprints, feces, and so on. With land animals, most of the traces are because they walked in a muddy area, just before the mud dried (and was buried before the next rain). Some of these mudstones even show raindrop marks, and cracks from drying in the sun.
Some trace fossils are casts, in the statue sense. The remains dissolved, leaving a cavity which filled with mineral, such as opal.
Other fossils are whole creatures that got stuck in tree resin, which became amber. Most of our fossil insects are in amber.
But far and away the most common fossils are pollen and shells. Pollen is usually only noticed under a microscope, but it's amazingly long-lasting. Shells are more obvious. The White Cliffs of Dover are made up of crushed shells, and one drill hole in North Dakota encountered 2200 feet of crinoid shells.
