Information Managers and Social Science Research

I first realized the advantages of Personal Information Manager (PIM) software while working as a researcher and writer at the University of Regina. I began using Microsoft OneNote to assist with the preparation of reports and encyclopedia articles. OneNote was very convenient for compiling lists of internet pages on topics ranging from important historical figures to the development of indicators of social cohesion. It was also excellent for collecting text, pictures, associated URL and adding my own notes and annotations. There were many advantages of using OneNote over creating these files in WORD:

  • clipping information from a web browser and pasting it into Word was simple and resulted in a nearly exact replication of the webpage’s formatting.
  • the URL of the internet page was automatically included in OneNote directly beneath the pasted webclipping
  • easier to structure the layout of content in OneNote.

Despite being packaged with MS Office, Onenote was largely overlooked by many mainstream users. Yet it built up a base of enthusiastic fans, including myself. However, after switching to Linux as my primary OS in 2007, I had to look for an alternative PIM that would run in Linux. I was also looking for a way to avoid proprietary file formats or software that did not offer methods of exporting data into formats that other software could open and read.

I began to look for an alternative that would run in Linux and not lock my data into a proprietary format. Although Linux users today have several decent open-source PIM applications available to them such as Keepnote and Cherrytree, back in 2007, the PIM applications available for Linux were weak indeed.
My search for a reliable PIM led me on a crazed search for a replacement and the consensus among the serious bunch over at http://www.outlinersoftware.com/ led to this list of major contenders some of which I tested quite thoroughly, and others which simply intrigued me, but which I did not test.

How can a PIM help with social science research?

Anyone doing a Master’s or PhD degree will soon find themselves acquiring truly huge amounts of texual data in the form of PDF files, Word document, emails, and pertinent articles and documents from a myriad of sources including the internet. This is both the boon and the plague of this information era– we can access vast amounts of information. But unless we have systems in place for organizing the data and its content, the information is not really of much use.

I discussed this very same problem related to organizing photo and video files in an earlier post. Photographers need finely tuned methodologies to keep their collections in order and thankfully several good DAM (digital asset management tools) exist to help with that process.

For organizing textual data and documents, a number of software applications are available. These programs are referred to by different names including PIM (Personal Information Managers), Knowledge Managers, Freeform Personal Organizers. In general, they use per-structured user interfaces that constrain how the user enters information into the system. The difference lies in how successfully they implement user interfaces and in their organizational and search capabilities.

Every user’s personal preference will be different, and different tools will appeal to people differently. Also, the type of data we are trying to organize will affect which software application is most appropriate. I’ll introduce my situation. Perhaps it is similar to yours.

Files on my computer were stored in a nested hierarchic of folders.

Q-Dir 6.04 folders directory

Within each folder are stored dozens or perhaps hundreds of documents. This is well-organized overall, but the trouble starts when I began to accumulate hundreds of .DOC or ODT text documents and found I often to open several documents scattered throughout my folders. This was an awkward way of dealing with files. More importantly, it was nearly impossible to find specific files.

Undoubtedly we all have times when we recall “I have a file on heavy-metal soil pollution from fertilizer” but then are unable to find it. One solution is to use a file search tool like X1 Search or Copernic.

Another solution is to store the majority of documents within a PIM application that can search the entire database in seconds according to a wide range of criteria. The PIMs I considered all had the ability to organize data in several ways and in several stages. There are many advantages to such a system:

  1. Organizing notes in hierarchical trees in various tabs forces us to engage in a cognitive exercise in organizing data. Choosing the for higher level nodes allows for easier navigation and provides context for the notes that it contains.
  2. The process of adding keywords to the notes requires us to begin with a literal and basic interpretation of the notes and then try to see if the notes contain latent themes not-so-obvious categories. The meaningful and descriptive keywords that we select for notes function to underpin broader categories or themes.
  3. Once the notes are categorized in hierarchical trees and organized with keywords, we might be theorize on broader themes that might have remained invisible.

It might be useful to use the same type of hierarchical controlled vocabulary of keyword for both photos, videos and for use in a PIM. The discussion here http://onlineqda.hud.ac.uk/Intro_QDA/how_what_to_code.php about the process of coding texual data in QDA software helps explain the benefits of this time consuming process.

Overview:
After examining online reviews and testing several PIM applications, I decided on the capabilities I wanted:

  • Easy and intuitive to use and not require extensive training before I can use it effectively
  • Allows categorization of data in a tree hierarchy of notes
  • Powerful search capabilities so that I can find what I want quickly and see the search terms highlighted in context in the search results
  • Offers a portable version that can be installed on a USB memory stick
  • Includes a powerful web-clipping tool to import web-page content with images and formatting intact. The transitory nature of the internet means that information found online today may not necessarily be there tomorrow. Web clipping tools allow us to add online content to our own databases.

Applications that I have tried

I tested the following applications.

MyBase
License: proprietary
Price: $79.00 USD
Website: http://www.wjjsoft.com/

Mybase also has a plethora of advanced features and a very good reputation. The software’s fan base in North America seems somewhat small, but clearly dedicated. The developer, Wjjsoft, based in Chengdu, China implemented Unicode coding standards and so MyBase is able to handle English and Chinese text without any problems whatsoever. Some note-taking applications that I tested were not able to support Chinese characters leading to the corruption of text in documents. If you live in China, the program can be purchased from wjjsoft.cn for only 50 rmb. For people living outside of China, its costs $75 unless you can find it on sale. The programmer must be an absolute genius because the MyBase installer file is only 2.5 megs in size, yet offers similar functionality as Microsoft OneNote which requires a 200 mb installer. Though the program suffers from a weird menu system, fact that version 7 released in 2014 is cross-platform is enough to push MyBase to my highest ranking

Strengths:

  • As of 2015 version 7 is cross platform and available for Linux, Mac and Windows. This is outstanding because if you spend hundreds of hours organizing data in MyBase, you can now easily switch to another operating systems and continue to use MyBase. Now that Linux has a truly high-end information organizer available, perhaps more people will be able to switch to Linux. The lack of a good information organizer was a prime reason why, after using Linux for 4 years, I returned to using Windows 7 in 2011.
  • The web-clipping tool in MyBase is particularly good. Users can highlight a portion of a webpage, right-click and choose “Save to MyBase”. The result is a nearly identical copy of the selection in your MyBase database

Weaknesses:

  • Import feature was unable to sequentially import the RTF files exported from TreeDBnotes.
  • Tree menu did not offer the ability to format, highlight etc text in the tree menu.
    the menu is non-standard and not very intuitive and unnecessarily complicates the learning process
    the developer almost never responds


RightNote

License: proprietary
Price: $29.95 – 59.95 USD
Website: http://bauerapps.com/rightnote/

When I tested RightNote in 2012, it had good reputation and similar layout and design to TreeDB and MyInfo. As of 2014, the latest versions of RightNote look impressive indeed.
Similar to other information organizers, notes in RightNote can be organized by of any number of pages with each page containing its own hierarchical tree of notes. This type of format gives users considerable flexibility and power to organize notes according to need.

One of RightNote’s powerful features missing in some of the other PIM applications, including MyInfo, is that each page and note can be either a word processing format, spreadsheet format, and source code note format. Incredibly convenient, this allows users to seamlessly include different types of information in a single notebook.

Strengths:

  • Very good support and constant updates
  • Excellent web-clipping tool,
  • Syncs with Evernote which enables it to sync notes from your phone’s Evernote app. Quite convenient!

Weaknesses:

  • Interface looks a rather dated.
  • It syncs with Evernote which is a note-taking app that I try to avoid. I’d prefer if it offered its own phone app.


UltraRecall

License: proprietary
Price: $99 or $49
Website: http://www.kinook.com/UltraRecall/

UltraRecall has many highly advanced features, a broad fan base and an outstanding reputation as stable, powerful and professional software that can be used easily with very large databases.

I installed the trial version in 2011, but found its directory tree did not offer the ability to format, highlight etc text in the tree menu. Being able to format the directory tree with bold, italics, different coloured text and highlighting provides a useful method for quickly locating information.

Strengths:

  • Good layout. Data can be organized by pages (or “tabs”) and by hierarchical note tree within each page.
  • Excellent keywording- each note can be assigned multiple keywords.
  • Excellent search capabilities: users can search by hierarchical note tree organization, by keyword, by full text searches.
  • Unparalleled web-clipping tool allows users to highlight a portion of a webpage, right-click and choose “Save to UltraRecall”. The result is a nearly identical copy of the selection in your database that can also be edited in UltraRecall to remove unwanted content.

Weaknesses:

  • Unable to apply formatting to tree hierarchy
  • Relatively weak keywording system

InfoQube
License: proprietary
Price: $49.95
Website: http://www.infoqube.biz/

Created in Canada, InfoQube is a PIM developed on many of the concepts employed in 1990s era PIM application called Ecco Pro that found a huge fan base. While Ecco Pro is still used by die-hard fans, InfoQube is inspired by many of the Ecco Pro features.

I’ve tried InfoQube when it was still in development many times, but honestly was never been able to figure it out, or the program crashed so much that I gave up. Nevertheless all reports indicate InfoQube is astonishingly powerful once you come to understand its logic and interface. The program, undoubtedly designed by a software genius, has been in a beta development form for years, but an official stable release is apparently due soon.

InfoQube remains a mystery to me. It is certainly powerful and widely praised for its advanced capabilities that seem to exceed those of other PIMs. But I’ve just never spent enough time with it to figure it out. I have included it here on this list and encourage PIM fans to try it.

TreeDBNotes
License: proprietary
Price: $24.95 – $34.95 USD
Website: http://www.mytreedb.com/purchase.html

In my search for a note-taking application to replace OneNote, I purchased TreeDBNotes Pro in 2007. During the three years that I used it daily, it crashed frequently, seized up and lost plenty of data. The software’s user forum had so many bug and stability related complaints, the developer simply deleted the entire forum where I too had lodged many bug-reports and complaints.

TreeDBnotes had great potential, the developer seemed content with selling new licenses and never fixing anything. I’ve watched it occasionally since, but very few bugfixes and updates have taken place and the software seems as unreliable in 2015 as it was when I stopped using it in 2011.

Strengths:

  • Good layout. Data can be organized by pages (or “tabs”) and by hierarchical note tree within each page.
  • Installable in Linux, good search capabilities.
  • The word count tool, remove formatting tool, and remove spaces tool were particularly useful for my work as an editor, writer and researcher.

Weaknesses:

  • Anemic web-clipping tool was introduced in 2012 but performed poorly in my tests. Nearly useful for serious work.
  • Software is unreliable, unstable, bug-filled,
  • Poorly supported, very infrequent and insignificant bug updates.

Open Source Alternatives

Keepnote
License: open source
Price: free but consider donating to open source projects
Website: http://keepnote.org/

Keepnote is an open source, cross platform hierarchical note-taking application featuring rich text and stores data in HTML and XML files. It runs in Windows, Linux and on MacOS X.

In 2010, I attempted to migrate my notes from TreeDB to Keepnote, but it was never able to displayed Chinese text and images without leading to some form of corruption of the characters. The html/text editor in Keepnote was too simple and would never display items cut and pasted from the internet in their original form. I continued to experiment with Keepnote until 2012 when development halted but the newer releases still did not add the functionality I needed.

When the project was active, the developer was responsive and clearly dedicated to building an excellent open-source note-taking application. I imagine he would really appreciate help from anyone who could assist with coding. Keepnote could be an outstanding note program and even replace expensive proprietary programs if some significant improvements were made.

Strengths:

  • Open source, cross-platform project
  • Data is stored in user-accessible HTML and XML files to ensure data is easily accessible if you want to switch to another program,
  • Tabbed interface to display several notes conveniently.
    Weaknesses: problems with Chinese text, overly simple html/text editor, no webclipping capability. With no updates since 2012, the project appears to be dead

CherryTree
License: open source
Price: free but consider donating to open source projects
Website: http://www.giuspen.com/cherrytree/

CherryTree is a hierarchical note-taking application, featuring rich text and syntax highlighting, and stores data in a single xml or sqlite file. It runs in Windows, Linux and on MacOS X and many language are supported.

Since 2011, I occasionally download new versions of CherryTree to test. Prior to 2012, Keepnote had a superior interface, was more intuitive and easier to use. Since 2012, however, CherryTree has continued to develop and new versions appear regularly. CherryTree also has a relatively weak file editor that is unable to maintain the formatting of text pasted from a web browser and lacks a web-clipping tool. Cherrytree could eventually challenge proprietary programs if some significant improvements were made.

Strengths:  open source, cross-platform project, enthusiastic developer
Weaknesses: simple html/text editor, no webclipping capability, weak tagging system.

MyTetra
License: open source
Price: free but consider donating to open source projects
Website: http://webhamster.ru/site/page/index/articles/projectcode/138
Also very similar to Keepnote. I have included it on this list because it seems to be actively developed and has the potential to become a powerful application. I have not used it but would be curious to see what you think.

PiggyDB
License: open source
Price: free but consider donating to open source projects
Website: http://piggydb.net/
I have never used PiggyDB, but have read many good reviews about its capabilities. I want to try it simply because of its hilarious name.

Other software that I have not used:

IdeaMason (defunct)
I have not tried it, but it appears to be quite capable and worth investigating. Development on the program stopped for a time before IdeaMason sprang back to life and was re-released as version 4. It was widely praised by its users and by users at Outlinersoftware. IdeaMason was relatively inexpensive and had reams of features including a webclipper. The company and product appear to have disappeared.

WebResearch
License: proprietary
Price: $29 or $49 USD
Website: http://www.macropool.com/en/

WebResearch was one of the earliest and most powerful PIMs apparently. It seems that it still has a very strong following.

TreeProjects
License: proprietary
Price: $29 or $49 USD
Website: http://personaldatabase.org/
This is widely praised for being well-designed. I’ve never tried it though it seems to be lagging behind the capabilities of the top contenders on my list.

AM-Notebook
License: proprietary
Price: 29.95 Euros
Website: http://aignes.com/notebook.htm

I used AM-Notebook for only a short time because of its cool ability to draw flow-charts. I stopped using it because I had already committed my data to TreeDBNotes and was too far along to switch applications. Development of AM-Notebook appears to have stopped in 2012.

NoteCase Pro
License: proprietary
Price: varying prices
Website: http://www.notecasepro.com/

Notecase Pro started out as a highly-regarded open source program called Notecase and was included with most Linux distributions. At some point, the developer began developing it into a more full-featured proprietary application that runs on nearly every operating system that exists.

When I tried it in 2013, it was very similar to Keepnote in its layout and capabilities. However, it was so similar to Keepnote that the $70 NoteCase Pro did not appear to have any features or functionality that might convince me to buy it. Why pay $70 for it when I could donate to KeepNote and keep my data in a totally non-proprietary open format?

Strengths:

  • Cross-platform and well designed.
  • Keyword tagging system

Weaknesses:

  • Does not seem to offer any more utility than the open source Keepnote
  • Lacks a web-clipping tool
  • Poor ability to maintain formatting of text and images pasted in from a browser

Conclusion

After examining the various PIN information managers in 2011, I chose MyInfo 6. It had outstanding reviews from other users, was the most intuitive, had powerful search capabilities and a nice interface. In the last two years, RightNote has added many powerful improvements to their product.

If I were to buy a PIM today, I’d be very tempted to choose MyBase 7. Why the change of heart after abandoning MyBase in favour of MyInfo? The ability to run MyInfo 7 on Linux, Mac and Windows is a feature that will enable me to overlook the program’s idiosyncrasies.

Regarding open source tools- currently none of the tools as they stand in 2015 is capable of meeting my requirements as outlined at the beginning of this article. If Keepnote were to implement significant changes to its HTML editor, and include a web-clipping tool, it could match the utility of many of the proprietary PIMs.

I’ve introduced both proprietary and open-source PIMs to many graduate and PhD students, colleges and friends. Invariably, they had never heard of such tools before and soon find them an invaluable part of organizing their research materials.