I know many of you faced the goal explained above. There are tools of different levels of involvement and there is of course no ready-baked answer to this question, but here is my best take so far for the current two active versions and Drupal 8 in development.
The three areas of Drupal language support
(A) First off, you can run a single language foreign language website without a need for content or configuration translation. Because the Drupal user interface comes in a flavor of English, you'll need to translate that. But all your content and configuration can be entered in your language, so you are fine there.
(B) Second, if you need to mark your content with language information, such as if you are running a multilingual blog, where you post in different languages, but will not translate your posts, you need language assignment with multiple language options.
(C) Finally, if you need to have the same content translated, the same navigation replicated or similar navigation produced for different languages, you need translation for your content and configuration.
The three types of data for Drupal translation
When it comes to translation, Drupal data can be separated to three buckets: (1) User interface (2) Content (3) Configuration. Drupal has very extensive support for user interface translation, I'd say too much support for content translation and usually not so bright support for configuration translation. Let's enumerate what Drupal has on offer for each piece.
1. Interface translation
Your user interface is whatever text is included in the module/theme that is shipped with the module and is not user editable.
Drupal assumes the user interface produced by module/theme source code is written in a flavor of English. The Drupal Localiztion API documentation on drupal.org has extensive coverage with cheat sheets for using t() and friends for interface translation. This stands for Drupal 6, 7 and there are no drastic changes planned as of now for Drupal 8 either. Some possible improvements are discussed in the A modern t() groups discussion and the A modern t() drupal.org issue as well as Move interface strings entirely out of Drupal code files? on groups.drupal.org.
2. Translatable content
The Drupal 7 mantra is to use entities and fields for content. Whatever is not using this system we consider configuration for the purpose of this discussion.
Fields thankfully have language support built-in, so its a user choice if a field is multiplied per language and can store different values or shared among different languages. If your module deals with content, consider if using a field for it would make sense. If it handles relations to content, then consider if making the relation a field would make sense, so the translators can change relations per language. See the entity_translation module to test out how your module behaves in a multilingual setting.
Unfortunately Drupal 6 does not have such flexibility. You can work with the i18n module API and more specifically with the i18n_sync functionality if you want to tie into synced fields. CCK fields are supported.
Both Drupal 6 and 7 have a built-in module called content translation that maintains copies of nodes as translations. This caused several problems in the past, and the plan for Drupal 8 is to phase this functionality out and instead support translatability of all fields/properties on entities which can be used to replicate this functionality, but would also work for all entity types. Work is ongoing at http://drupal.org/node/1269842.
3. Translatable configuration
For this discussion, whatever is not hardcode text in your modules and is not stored in fields/entities is considered configuration. This has a defining characteristic of having a user interface for editing the values.
Both Drupal 6 and 7 have translation for configuration supported via the contributed i18n module, although the user experience in the Drupal 7 version is way better. You can register your translatable configuration pieces with the i18n_string module and proxy your add/update/delete/view operations via that module in essence. The module will always look up the right language version for your configuration value and can generate a translation user interface on your editing screens as a separate tab. Look at the i18n_block or i18n_contact submodules in i18n which should provide simple examples for how this is structured.
Yes, the system is much more complex compared to just using t(), but it needs to handle adding new items, removing items, changing items, permissions, text formats, avoiding cross-site-scripting, etc. All of which t() does not need to care about. See my previous post on the topic. Still, you don't need to depend on i18n_string module for your work, just abstract out the API functions on your objects, and call i18n_string there when/if available.
The i18n_string system currently uses the Drupal core built-in textgroup support in the user interface translation functionality, however, i18n module is moving away from that in the 7.x-2.x version and Drupal 8 removed that support. If you use the i18n_string facilities, you'll at least have an easier time migrating to the new interface and might even have a backwards compatibility layer.
For Drupal 8, the Configuration Management Initiative is reworking how all configuration is stored and while that is still deeply in the works, we don't know where configuration translation is going. It is certainly a top goal to fill this feature hole directly in Drupal core for Drupal 8.
Interface translation should be front-end and admin-end
While we talk about interface I think we should care about 2 implementations - front end (content) and back-end (admin interface). the most trouble we get in menu system when 'Add content' and others are works in both areas. Now textgroup support is dropped and I see no ability to separate this.
textgroups are not to divide UI translation
Textgroups were designed not to divide UI translation but to provide space for configuration translation (which turned out to be the wrong approach to do it, and therefore removed). Have you successfully used textgroups for dividing UI translation? How?
If you want to use a specific language for the administration interface, look at the http://drupal.org/project/admin_language module. The issue for Drupal 8 core inclusion is at http://drupal.org/node/322995