What's a good first equipment buy after a basic starter kit?

How do you feel about making the switch to all grain? If you're up for it, it's cheap and easy to convert a water cooler into mash tun. All-grain is a bit tricky at first, but once you're used to it you'll get a superior beer.

I've upvoted a 'real' answer, but if you want to save some money on a chiller, you can make one pretty easily. You would need a propane torch and solder, but even if you have to buy those items, it's still pretty cheap. This video shows you how you can make one. It will shorten your brew day, which is something that some brewers value.

I bought a starter kit from Midwest brewing and expanded it a little at a time. It was a groupon and a good deal. I have made some very good beer with this.

For extract, the variables you really need control over (and thus the equipment) are

• yeast growth (part one). For beers with character of your choosing, it is so much nicer if you can grow the wonderful varieties of yeast from wyeast and whitelabs and do fun things like cultivate the yeast from bottles of chimay. This is basically free. You just need a few jars.
• fermentation temperature. I agree, get a thermostat (stc-1000 or similar). Ideally get a fridge. As a (slightly) cheaper option, you can control a fan over a dustbin with water in. As long as you can keep a steady, say, 18C (64F) then you can make quality ale.
• cooling post boil. Yes, get a cooler. Make it a plate cooler. (Though immersion and plate coolers take about the same time to cool a whole batch, the plate cooler chills the actual wort going through it in just a couple of seconds, which is hundreds of times faster than the immersion cooler).
• yeast (part two). Something very handy is a stirplate. It consists of a computer fan, old mobile/cell phone charger and two strongish magnets. Stir plates guarantee you enough yeast to finish off fermentations properly, which lets you make dryer, cleaner beers.

Auto-siphons and carbuoys are good to have, although you can make do without them. Plastic buckets with taps at the bottom make siphoning un-necessary. There is one shortcoming: siphoning can be better if you want to make really clear beers. Beer that has been left to clear is better siphoned off the top. Oxygen permeability of plastic is not a problem over ale fermentation times. Lager I am not so sure, but fermenting lager is an advanced topic for all sorts of reasons and probably comes at the very end of the long list you are just starting. Good luck!

I recently went from a basic brew kit/extracts and now to starting all grain.
I believe the wort chiller was the best initial investment in my case, followed by the auto siphon.

+1 each for the autosyphon and temperature control. Glass carboy? Guess I'd say carboy in general, without getting into the plastic v glass argument (each has its fanboys and never will the two camps agree, my small carboy is glass and my large ones are certified nalgene and polycarbonate because of weight handling issues). Something simple that I wish I had early on is a "wine thief" to make taking intermediate SG's easier after I moved to carboys. The faucet attachment to spray out bottles is good and can be used to rinse out carboys if you are stuck inside and can't use a garden hose. Getting some Erlenmeyer flasks to culture yeast can be handy. And a clip-on tube clamp for the syphon tubing is something that my local homebrew supplier turned me onto and has proven to be a great help, if you don't get the autosyphon.

Here's a really good thermometer...

http://www.homebrewfinds.com/2011/10/themoworks-rt600c-minmax-super-fast.html

Neither of those... The next purchase to making better beer is a thermostat controller and a fridge. The controller you can get at your LHBS or at an online shop. Then cruise craigslist for the fridge. You can get a fine chill in the sink with ice, like you already are doing (right!). A secondary isn't necessary really anyway. And several people make great beer just using buckets which are cheap and easy to clean.

Temp control!

I would say a wort chiller and and auto-siphon both seem like excellent ideas. The chiller will save you a TON of time in the process, and the auto-siphon is good for sanitization and ease of use. Auto siphons are pretty cheap too, but the chiller might cost you a fair amount. As for the carboy, I usually don't rack to secondary, I just leave it in the bucket until it's done, so I wouldn't see it as necessary but that's up to you. I really only use my carboy for cider now since it doesn't foam over during fermentation.