When is get_template_part() preferable to simply using the template.php files?

I don't feel like there is ever a valid reason NOT to use get_template part. Before it existed, WordPress themes were filled with repeated code and were very time-consuming + confusing to make larger modifications. Having one loop file that is always used to display posts is a godsend both for child themes and those who wish to "fork" themes. If you look at the code for the Twenty Eleven theme (currently in development) you will see that template parts have been extended into post formats as well. You will also notice that "The Loop" has been removed from the template and placed in the calling template. I think this is the best ways to use template parts for post content and is much more flexible than Twenty Ten.

Did you know that other template behave almost identically to get_template_part()?

IMHO the more flexible a parent theme is the better!

Building on Kaiser's answer, get_template_part() is useful for repetitive code. For example, most of your main template parts probably have a very similar loop. Putting that loop into a file called loop.php and calling get_template_part('loop'); means that, the next time you want to change the code in your site's loop, you do it once in one place, rather than the dozen or so times in the dozen or so template files.

That's one huge advantage of get_template_part(). The other one, and the more important one, in my opinion, is that it works very well with parent/child themes. Using the loop.php example again, if you have a child theme and wanted to modify the loop code in your child theme, but didn't want to change the rest of the template, you would only need to override the one loop.php file in your child theme, rather than the dozen or so other template files.

A recommended approach for using get_template_part would be for including bits of code that would otherwise be repeated frequently in all your templates. Like if you had conditionals defined within your loop that you wanted to include in archive.php, search.php, single.php etc.

It also allows child themes to override that file and include additional more specific files. For example if you used get_template_part( 'loop', 'single' ) and your theme only has a file named loop.php then a child theme could include a file named loop-single.php that would override your loop.php.

It's basically a custom includes tag for template files other than header, sidebar, or footer.

get_template_part() should (imho) be used if you use that code in more than one template. For ex. a header, a nav menu or a sidebar.

Templates should (imho - again) be used if a template is different in more than just some lines from the rest of the templates.

I personally only use a simple index.php that includes conditional tags to load my query specific template parts in my small base theme. In the larger theme i got only hooks in my index.php file and hook in my template parts conditionally (but the setup is far more complex - mostly real cms stuff).