Whats the difference between yeast and bacteria?

Bacteria and Yeast are not related in any genetic sense.

Yeast, by and large, eat sugar and produce alcohol and various esters and phenols (and CO₂). To most brewers (and vintners), though, the only yeast of any consequence is Saccharomyces, and any other microbe is to be avoided at all costs.

Brettanomyces is another yeast genus, which typically produces "wild" flavors (from pineapple and cherry to horse blanket and mouse poop) and aromas ("Bad feet or good cheese," as a friend of mine says). It is considered a flaw in many beers, but is essential to the flavor of a certain subset. Brettanomyces can also break down and consume sugars that Sacchromyces finds unfermentable.

Different bacteria eat different substances and produce different things as a result. There are several that are useful in brewing certain beers (Flanders Red/Brown, Lambic/Gueuze, Berliner Weisse, and other beers inspired by those styles).

• Acetobacter eat ethanol and produce acetic acid (think vinegar), and other esters.
• Enterobacter eat glucose and produce acetic and lactic acids, and various esters, DMS, etc.
• Lactobacillus eat sugar and produce lactic acid and ethyl lactate (the same acid in kosher pickles and yogurt)
• Pediococcus eat glucose and produce lactic acid, Diacetyl, and no CO₂

(See "Wild Brews" from Jeff Sparrow for more comprehensive explanations)

You may occasionally find that a beer has been infected by something that you didn't intend. It may or may not taste good, but it is a signal that you need to step up your cleaning/sanitation regimen and/or replace old equipment (especially anything plastic). I had an APA that got some sort of infection that gave it a bit of lactic twang, but it worked well with the citrusy hops, and the infection didn't show up until the batch was almost all gone. I just drank it.

Bacteria and Yeast exist in different kingdoms of the classification system we use for living things. The primary difference is that bacteria are Prokaryotes, while yeast (fungi) are eukaryotes.

If you really want to knock yourself out here is the difference between the pro- and the eukaryotes.

Different microorganisms vastly change the result of a fermentation. I would argue that the strain of yeast you use usually has a greater effect on the character of the beer than anything else. Bacteria that could infect your beer, especially coliforms and lactobacilli, produce acid that sour your beer (ever wonder where malt vinegar comes from?).

I didn't know this until pulling up Wikipedia, but yeast is part of the Fungi Kingdom, whereas bacteria are part of the (go figure) Bacteria Kingdom.

I know it's not a terribly informative answer, but it indicates that even though they're both single-celled organisms they are very significantly different.