⚡️ Ideas
David David Tana Navigator Aug 4, 2024

Data Portability and Long-Term Accessibility

Explain the problem as you see it

The app lacks robust data portability features, making it difficult to ensure long-term access to accumulated knowledge. Current export options are limited to manual Markdown exports or a single JSON file, which don't provide an easily accessible or future-proof solution.

Why is this a problem for you?

  • Data lock-in: Risk of losing access to years of knowledge if service becomes unavailable.
  • Time-consuming exports: Manual exports are impractical for maintaining an up-to-date archive.
  • Limited portability: Current formats don't preserve all metadata and relationships.
  • Future accessibility concerns: Difficulty in retrieving specific old information.
  • Lack of data ownership: Users don't have full control over their data.
  • Hindered workflow evolution: Difficulty in transferring data to new systems as work methods change.
  • Compliance issues: Challenges in meeting data retention and backup requirements.

Suggest a solution

  1. Automated, scheduled exports
  2. Comprehensive export formats preserving all data and relationships
  3. Selective exports for specific projects or date ranges
  4. Full-featured API for custom integrations and backups
  5. Import capabilities for data migration and restoration
  6. Export preview and verification features
  7. Clear documentation on data structures for long-term usability

⁨3⁩ ⁨Comments⁩

I think targeting a reasonably open standard (like Obsidian-flavored Markdown or LogSeq-flavored Markdown) might make sense.

Especially when paired with the ability to setup a regular backup schedule to be sent to an offsite location like Dropbox, Google Drive, or S3 (of the user’s choosing).

  • LogSeq in particular is open source and an outliner, like Tana. If the backup were to omit any Tana-specific nodes like search nodes, perhaps it’ll be good enough to retain at least the raw data in outliner form where the nodes’ bidirectional links are preserved.

  • Obsidian-compatible Markdown files where each top-level node is a Markdown file and children are bullets, could work too. Referenced nodes become block references, and inline references become [[wikilinks]], and search nodes stripped out.

I could live with either of those for data portability.

For images, the relative links should be preserved and all media should be exported to a folder. Obsidian should be able to handle this. I’m unsure about LogSeq, but there should be a way they handle images too.

Well known and explored issue in community/team. Improved user friendly exports and a more powerful API will both come in future. Current export options of full JSON as well as markdown export of any node and children offer a lot of capacity for users to manually complete their own exports, but more to come.