Let's start this way: Bacteria are small single-celled organisms. Unlike eukaryotic cells (like the ones that make up animals, plants, fungi and many single-celled organisms such as yeasts) which package their genetic material (DNA) in membrane structures called nuclei, bacteria (which are also referred to as prokaryotes) don't have a nucleus. Bacteria also don't have other organelles which are found in eukaryotes, such as endoplasmic reticulum, mitochandria, chloroplasts, golgi bodies, etc. But like eukaryotes, bacteria do have an outer lipid membrane.
So, you can think of bacteria of little fluid filled sacs. Along with eukaryotic, bacteria are an example of "cells" which are the smallest unit of living things on earth.
Viruses, on the other hand, do not have an outer membrane. They don't the capacity to make their own proteins. They don't have the capacity to replicate or metabolize on their own -- of which bacteria (and eukaryotes) do. It's for this reason that we don't consider viruses "living" things. Viruses have to "infect" a cell -- injecting its genetic material inside -- and use the cell's metabolic machinery to reproduce.
Viruses are essentially small protein "capsules" that have a piece of genetic information packaged inside. Because viruses are required to infect a cell to propagate themselves, they are classified as obligate intra-cellular pathogens.
Viruses tend to only infect specific cell types. Some infect bacteria. Some infect plants. Others infect animals. The specificity of what a virus can infect is goes beyond whether they can infect bacteria, plants or animals. Usually viruses have to infect a very specific animal, plant or bacteria. And even in the case of viruses that infect animals and plants, not only do the infect specific animals, they are usually limited to a few cell types within that animal.
Take HIV for example: to the best of my knowledge HIV only infects humans (i.e. the "H" in HIV -- Human Immunodeficiency Virus). But it doesn't just infect any human cell. HIV only infects white blood cells -- specifically a class of white blood cells called T cells (there are many other "classes" of white blood cells including such things as B cells, NK cells, neutrophils, basophils, eosinophils and macrophages).
I hope this sheds some light on the matter.