Research Skills

Tools for Information Gathering

Which tools do you recommend for collecting scientific information?

An important part of research is gathering existing information from papers and books. Thanks to the Internet, most of that information is now within easy reach. However, the number of information services on the web is overwhelming, and therefore it is worthwhile selecting only the best.

A good information collection tool is as exhaustive as possible in its covering of sources, but provides a good ordering so that you immediately see which results are the most important. Moreover a good tool is maximally simple and transparent in its use, avoiding any cluttering with irrelevant options, pictures, advertisements, etc. A strict adherence to this philosophy explains the phenomenal success of Google and Wikipedia—probably the two most important web services overall.

The following is a selection of tools based on the extensive (but of course still limited) experience of ECCO researchers. Using such tools may save you days if not months in the preparation of your projects, papers and dissertations. 


Web Search

Google is the best tool to quickly search the web as a whole, thanks to its very extensive coverage, clean, user-friendly interface, and smart algorithms to order the results so that the top entries indeed tend to be the most important ones.

Dogpile is a meta-search engine: it combines the results from several existing search engines, including Google, Yahoo and MSN. Because these search engines together cover a larger number of pages, this is useful when you are looking for something very specific that is hard to find. Otherwise, Google will give you a better ordering and presentation of results.


Scientific Papers

There are three large search engines for the scientific literature, with databases that only partially overlap. This means that if you are looking either for a comprehensive picture, or for something difficult to find, it may be necessary to use all three. Unfortunately, there is no clear method to find out what is included in each database, as these commercial services don't provide full information about their sources.

  1. Google Scholar is the first choice, as it covers a very wide range of material, and orders the result according to the number of citations and other relevant indicators of quality. Most usefully, it provides immediate access to the papers it finds, either freely for self-archived papers or preprints, with restricted access via your university library for the journals that the library subscribes to, or for payment to the publisher. Another useful feature is that each paper is linked to the papers that cite it, so that it is easy to find related material.
  2. Web of Science (including the Science Citation Index) is the most authoritative database in the scientific world, especially for its citation count for papers and authors. Compared to Scholar, however, its coverage is much more limited; it does not give direct access to the papers; and it only searches titles and keywords, while ignoring the content. You must access it via an institution that subscribes to the service, which is the case for the VUB.
  3. Scirus was set up by Elsevier publishers as a competitor to the two previous databases. While it claims to be the most comprehensive scientific search engine, it seems to find much few papers than Scholar, though it is more comprehensive and userfriendly than Web of Science. Its subscription counterpart, Scopus, also includes citation data, but the VUB does not offer access to this database.

The VUB library allows you to search for papers in its holdings and has links to a number of more specialized databases. However, in general papers in journals held by the VUB can be found and accessed more quickly via Google Scholar. is a very useful database of articles published by Springer. It allows you to finds all papers that contain certain words. Moreover, it classifies those papers according to geographic location (on a world map), country, year, journal, author, institution, etc. Like that you can immediately see in which countries, journals or years terms like "cybernetics" or "memes"  are used most frequently.


Google Books searches inside books, and allows you to read a few relevant pages of most books. It also provides links to libraries or booksellers (such as Amazon) where you can order the book., the largest bookseller in the world, has a very extensive catalog, and now also allows search of book content (not just titles and authors like other booksellers). 

The VUB library lets you search title words or authors for books available at the VUB, or within the network of libraries in Belgium and the neighboring countries with which it has established agreements for "interlibrary lending". This means that if the book is not available at the VUB, you can order it via the database and collect it at the VUB after a short waiting period, provided you pay a small fee.


The following databases allow the rapid identification of scientific journals to publish your research in, or to browse relevant articles from.

Journal Info is an extensive database collected by Lund University, including open-access journals and journals listed in the Journal Citation Reports below. It lists (or provides links to) basic characteristics of the journal: home page, self-archiving policy, impact factor and other quality indicators, price, number of issues, etc. You can search for (combinations of) title words, including parts of words using "*"as wildcard operator (e.g. "complex* systems" will find both "Systems and Complexity" and "Journal of Complex Systems"), but not for more general keywords.

JournalSeek claims to be the largest database of freely available journal information, including description (aims and scope),  home page, and subject category. The advantage compared to Journal Info is that it also searches for keywords within the "aims and scope" and thus finds more relevant journals. The disadvantage is that you get less characteristics at a glance.

Journal Citation Reports is the database used by the Web of Science, which means that the journals it includes are more authoritative (so-called "A1" publications), but much more restricted. It is most famous for its calculation of the "Impact Factor", which is traditionally seen as THE measure of the importance of the journal, in terms of the average number of citations it gets.


Zotero is an extremely useful "plug-in" for the free web browser Firefox. It can extract bibiographical data from practically all the important publication sites, such as Amazon, Web of Science and Google Scholar. The data are kept in its own database, which you can export in all the traditional bibliographical formats. Especially when combined with a Zotero plug-in for your word processor, this is extremely useful for creating bibliographies for papers and dissertations:

  1. using Firefox/Zotero search various websites for relevant publications
  2. mark the publications you want and have them imported automatically to your Zotero database
  3. when you need to make a list of references, select the right entries from the database, and paste them straight into your text in the required format!

CiteULike is a website where you can collaboratively create, store, search and publish bibiographies. It uses similar methods like Zotero to extract bibliographical data from common website formats.


OneLook searches hundreds of general and specialized dictionaries (such as our own Web Dictionary of Cybernetics and Systems) at once, and provides links to their results. This is particularly useful for rare, technical terms, such as "stigmergy", that may not appear in common dictionaries.

  Find definitions similarly checks a (much smaller) number of dictionaries, but immediately shows you the most important results. This is  useful for more common words where you want to compare the subtleties of the different meanings.



Wikipedia, the free encyclopedia to which everyone can contribute, is the best and most comprehensive way to get a good introduction to about any subject you can think of, including advanced scientific topics such as stigmergy, self-organization, or the Theory of Constraints.

Encyclopaedia Brittannica (freely searchable from the VUB) is the most comprehensive of the traditional, commercially published encyclopedias. It is smaller and less up-to-date than Wikipedia (e.g. it does not find "stigmergy"), but its results are more authoritative.


Google Translate is the best of several translation engines we tried out. It provides translation of webpages or inserted text between the main languages you are likely to encounter, such as English, Dutch, French, Spanish, etc. The main competitors are Babelfish and Systran. Anyway, you will still have quite some work to manually "clean up" the translations if you want to use them, but the work is least with Google Translate.



Idea Processing

How can use I an outline to better organize my ideas?

What is an outline?

The most important advice I give to all my students who have to write down their ideas for a project, paper or thesis, is: make an outline! A finished outline looks like a hierarchical "table of contents" of the text you are going to write with all the main headings and subheadings in the most logical order. E.g.


  • 1. Introduction
    • 1.1 motivation
    • 1.2 the problem
    • 1.3 previous work
  • 2. My approach
    • 2.1conceptual framework
    • 2.2 methodology
      • 2.2.1 data gathering
      • 2.2.2 data processing
      • ...
  • ...
  • 6. Conclusion

It could also look like a simple checklist or "to do" list, e.g.

Travel to Italy

  • prepare luggage
    • clothes
    • documents
    • ...
  • get plane ticket
  • go to airport
    • check bus schedule
  • ....

In practice a developing outline is a very loose and flexible structure which constantly evolves as your ideas become clearer and more detalied. Thanks to outlining software (built into most word processors), outlines are very easy to manipulate and change, so that they can always reflect your most recent thoughts on the subject.

In my experience, when writing a paper, it's always much better to start with an outline than with a draft. An outline is easier to make because you start by first putting in various, as yet unstructured, ideas, in the order that you remember them, without real constraints. At this stage you are merely playing with ideas, using them as building blocks that can be arranged in various combinations to see what structures they produce. That is why the original outlining applications on the computer, such as "ThinkTank", were called idea processors.

Overcoming hurdles

This complete freedom of writing anything at any place in any style makes it psychologically much easier for you to start the difficult undertaking of writing a paper. They help you to overcome any "fear for the empty page" or "writer's block" you may have. You don't have to worry about writing things that are incoherent, undeveloped, poorly formulated, or insufficiently supported: these shortcomings are easily corrected later.

As your view of the issue become clearer, you will nicely order, add, edit and reorganize the outline, until it looks like a more or less complete and coherent plan of action. Once you have an outline that looks satisfactory, writing the actual text becomes much easier, because you now know what to write when and where.

This helps you to avoid one of the most common hurdles when writing scientific texts: after having spent several pages building up your argument, you suddenly discover that an essential step in the reasoning is still missing and you don't know how to continue. This may get you entangled into various tricky situations:

  • you get blocked and postpone finishing the paper;
  • you continue writing about related issues hoping that inspiration will come, making the paper ever longer and more convoluted;
  • you decide to skip or side-step the issue, making you vulnerable to serious criticism from the later readers/referees;
  • you conclude that you started out on the wrong track, and will have to completely rewrite the pages that came before

Of course, using an outline will not make you immune to these problems: what looks like a coherent line of reasoning in short, outline form, may still turn out to have an essential gap when filling in the details. But the probability of this happening is much smaller and the consequences are less severe, as your writing until then will be more structured and to the point, and therefore easier to reorganize.

Using outline software

Even while writing, you can still decide to change the order of ideas, add things or leave out things. An outline is not a rigid structure that you have to adhere to, but a checklist or guide helping to remind you of the most important things you have to cover, without losing a view of the whole. Flexibility is the core principle!

I personally work best best with a dedicated outline software (the no longer updated More for the Mac), but all word processing programs, such as MS Word, have built-in outlining facilities. The important thing is that the program should make it easy to drag and drop text lines with ideas, so as to change their order and their hierarchical level: promoting or demoting headers, e.g from the style "heading 2" to "heading 3". Demoting typically is represented visually by a right indentation (an additional tab or space in front of the line). For example:

  • topic a
  • topic b
  • topic c

If you feel that the order is not quite right, this can be transformed to:

  • topic a
  • topic c
  • topic b

On the other hand, you may feel that topic c should be seen as a part of topic b, and then you "demote" it:

  • topic a
  • topic b
    • topic c

If your program offers a good visual representation of such an ordered hierarchy, you don't need all the fancy numbering conventions (like IV.3. a. or that are used in printed section headings. If necessary, such hierarchical numbering can always be automatically added when you prepare the actual paper/thesis.
A recent new online outliner, LooseStitch is a very nice collaborative tool, with OPML import/export features.

Keeping it short

The essence of an outline is that it is much shorter than a paper. Thus, outline paragraphs or "headings" are normally less than one line long. They do not form full sentences, but list key ideas in "telegraph style". For example, don't write in your outline:

Complexity is a subtle and difficult to define concept, that has been used differently in different domains



  • difficult to define
  • different uses in different domains

Therefore, when preparing a paper you can save a lot of typing by first organizing your ideas as an outline. Only when you are satisfied that everything you need to say, but nothing more, is there, in a direct and logical order, you should start writing the actual sentences of the paper.

In my experience, a paper that was first outlined tends to be much shorter than one where the ideas where developed while writing the full text. That is because you can avoid most repetitions, vague formulations, cross-references, material that is of little relevance, and other forms of "literary procrastination". This is particularly important for papers submitted to journals and proceedings, where there are typically strict limits on the length, and shorter papers have higher chances of being accepted for publication.

Outline as a thinking aid

I start an outline typically as a checklist of ideas that seem relevant, in abbreviated form. I then spend a lot of time reading and rereading the things that are there, moving some up or down, or putting things that belong together under a new header, and now and then adding what seems to be missing. I regularly look again at the same outline with intervals of days, weeks or sometimes even months and years. Each time I look at the same ideas with a fresh mind, new ideas are triggered, and things I learned or understood in the meantime find their place in what I noted down before.

The outline can be seen as an extended memory, keeping track of your ideas on a particular subject over the weeks and years, so that you are unlikely to forget the fruit of earlier study and thinking. By examining the outline you make your extended memory interact with your internal memory, thus triggering your mind to fill the gaps, notice the incoherences, elicit new ideas, etc.

The precise words or sentences used are less important for outlines than for papers, where the emphasis is more on the visual structure produced by the arrangement of headings. That is why an outline should not contain too much text: when you cannot see everything on the screen, you lose the advantages of visual juxtaposition of the core elements. You may remember that there is a "magical number" (7 plus or minus 2) that denotes the maximal number of items you can hold at once in your working memory. With an outline you can significantly increase that number, by having all items close together on the screen, and move them around to try out different combinations. So an outline is not only a tool that helps you to write, but one that actually helps you to think and remember what you thought...

More useful advice on making outlines:

An example outline

Below you'll find an example outline for a paper on cybernetics I wrote (note that the final paper has a different outline, since things always change while you are writing):

Cybernetics and second order-cybernetics

  • Historical Development
    • Wiener & Rosenblueth
    • Macy meetings
    • self-organization, autonomy
    • second-order cybernetics
    • modelling
    • relations with sciences of complexity
    • robotics, cyberspace
  • Relational Concepts
    • Isomorphisms between domains
      • biology
      • technology
      • society
      • psychology
    • substrate independent modelling
    • systems, organization
    • networks
    • variety
    • information and entropy
  • Circular Processes
    • self-application, recursion
    • closure
    • fixpoints, eigenvalues, attractors
    • self-organization
    • autopoiesis
    • reproduction
    • circular causality
    • negative feedback
    • positive feedback
    • self-reference, reflexivity
      • the liar's paradox
  • Control systems
    • the control relation
    • goal or reference level
    • perception
    • action
    • amplification
    • the law of requisite variety
  • Control hierarchies
    • control of the goal
    • the law of requisite hierarchy
    • multilevel systems
    • metasystem transitions
  • Models
    • feedforward
    • prediction
    • homomorphisms
    • the law of regulatory models
  • Model-building and the role of the observer
    • constructivism
    • distinctions
    • observer-observed interaction
    • Epistemological issues
  • References


Links to this Page


Improving mental skills

How can I improve my mental skills?

Research and writing is an activity that demands the utmost of your brain, so it is worth investing in anything that will increase your intellectual capabilities. Happily, scientists have discovered a whole range of factors that affect IQ, concentration, memory, etc., and that you can to some degree control.

The most general observation is that the brain is a part of your body, requiring the same supply of blood, oxygen, nutrients, minerals, etc. as other organs. Therefore, anything that improves the functioning of your body will in general also improve the functioning of your brain. Moreover, brain activity is very energy intensive, using up to 25% of the calories, while making up only about 5% of the body. This is confirmed by fMRI, EEG and infrared scanning methods to observe brain activity, which show how whole sections of the brain "light up" with activation while you are performing various cognitive tasks, such as speaking, perceiving or problem-solving. Thus, the brain is in particular need of a constant energy supply, and anything that supports this can improve your intellectual capacity.

Here are some of the best methods to improve brain functioning:

  • physical exercise: increases blood circulation, not only to muscles but also to the brain. Experiments with rats have moreover shown that in the longer term exercise stimulates the growth of neurons, and the rats' ability to solve problems. Particularly recommended are not too intensive, aerobic exercises that do not demand specific concentration, such as walking, cycling or swimming. During these activities you often get your best ideas... 
  • healthy eating: provides a regular supply of calories (fuel to burn), antioxidants (to neutralize the free radicals produced by the "burning"), and essential fatty acids (to build the insulation of the neurons in the brain), with a special emphasis on Omega 3 (fish oil), which has been proven to combat a whole range of mental problem, such as ADHD, dyslexia, depression, ...
    Avoid foods with a high glycemic index (mostly sweets, bread and potatoes), which make the glucose and insulin levels in the blood increase and then decrease too quickly: they provide energy in the very short term, but produce fatigue in the middle term, and serious health problems in the long term. Better eat foods that digest more slowly (e.g. meat, fish, vegetables, nuts) and thus provide energy over an extended period. This is the simplest method to avoid the post-lunch dip.
  • nutritional supplements: can provide additional antioxidant protection, and facilitate bloodflow and energy production. Particularly recommended are: B-vitamins, Ginkgo, OPC (Pycnogenol), Acetyl-L-Carnitine, Alpha Lipoic acid, and Co-factor Q10. For a short-term boost, the caffeine found in coffee, tea or Guarana supplements is still most reliable; moderate use does not seem to be harmful.
  • minimizing stress: get regular, sufficiently long sleep, and avoid disturbances and interruptions, such as constantly incoming phone calls or email messages. Poor sleep, interruptions, and being busy with several things at the same time strongly reduce your ability to concentrate, and thus your ability to tackle difficult problems. A good method to reduce distractions and worries during your work is GTD
  • mental exercise: like the muscles, the brain becomes fitter by being used. Particularly useful are "flow" producing activities, such as certain types of computer games, where you get continuous feedback about how well you are doing, and where the level of difficulty gradually increases with your increasing skills. However, rather than spending your time with cross-word puzzles or games, you can exercise your brain more productively by engaging in intellectual discussions with colleagues. This is one of the best flow-producing ways to stimulate your thinking. 


More details: New Scientist article: 11 steps to a better brain



The GTD Method

"Getting Things Done" - a summary of the method

The personal productivity consultant David Allen (2001) has proposed a simple and effective approach for managing  your day-to-day tasks or activities. Based on years of experience in teaching knowledge workers how to deal with their backlog of unprocessed issues, the method is known as “Getting Things Done”, or GTD for short. GTD is intended to minimize stress and anxiety while maximizing productivity—in the sense of maximizing the number of useful tasks performed, not in the sense of maximally achieving a given objective.

The intended result of applying GTD is being able to keep up with a high workflow in a relaxed manner. The main principle is to get everything that is nagging you out of your mind and into a trusted external memory (file system), so that you can stay focused on what you actually have to do now, rather than on various ideas, plans and commitments for later. GTD uses the following stages:


Collect (1)

The first phase is to collect everything that catches your attention as potentially relevant to your activities, whatever its subject, importance or degree of urgency. This includes incoming letters, emails, phone calls, reports, articles from magazines, agenda items, suggestions and requests from other people, and personal ideas and memories. For the collecting process, you need one or more collection tools, which can be physical (trays, folders, notebook, etc.), or electronic (email application, outliner, or word processor, on a computer or a PDA). These together define your “in-basket”.

Collection is just the first step: to gain control over the collected materials, you need to empty the in-basket regularly. Emptying means deciding what to do with—not actually doing—the items in the collection. This happens by processing and organizing the items one by one.


Process (2) & Organize (3)

 The first question to ask is: “What is this stuff?” Note that “stuff” is a catchall word, which can refer to an email, something at the back of your mind, a note, a voice-mail, a scrap from a newspaper, etc., i.e. any item that has been collected. More precisely, the crucial question is: “Is it actionable?”, i.e. does it require you to perform an action?

  if it is not actionable, there are three possibilities:

o      eliminate the item if you really will not use it (i.e. throw it in the trash bin);

o      incubate the item for possible implementation later (i.e. store it in a Someday/maybe file that you will review at a later time);

o      reference the item if it does not require action but may need to be consulted later (i.e. store it in a Reference file, which is organized so that items are easy to classify and retrieve).

if it is actionable, then you have to decide, “What is the next physical action?”

o      if there is more than one action required, store it in your Projects list.

o      if the action requires less than two minutes, it is not worth the effort of entering it into the system: better perform it immediately.

o      if you are not the best person to do it, delegate the action to a more qualified person/organization, and keep track of whether you get back the desired result by entering a note in the Waiting for file.

o      if the action is to be done on a particular day and time, defer it to this moment, and note it on your Calendar.

o      if the only time constraint is that you should do it as soon as you can, put it in your Next actions file.

When you review your Projects list, for each project you should start developing a Project plan. This in general does not mean a formal scheme with milestones, deadlines and objectives, but a formulation of the overall goal or desired outcome, with a focus on the list of next actions required to move towards this goal. Once these actions are defined, they need to be reviewed, which means that they should follow the part of the flowchart that describes the decision tree for actionable items. There is in general no need to plan actions far ahead: once the first “next action” is done, the next “next action” will probably become obvious. 


Review (4)

The reviewing phase is crucial to remind you of what you still have to do. The daily review includes reviewing first your Calendar (which are the things that you have to do imperatively on this day?), and then your Next actions list (which are the things that you should do as soon as practicable?). The weekly review is a more in-depth review of all your (potentially) actionable files (In-basket, Calendar, Next actions, Projects, Project plans, Waiting for, Calendar and Someday/maybe). It is essential to get an overview of what has to be done in the coming period, and thus get the feeling of being in control. Concretely, it means that you make sure that the different files in your external memory are kept up-to-date. This will include a complete cleaning of your desk, email, and other collection places, and thus some further processing and organizing according to the flowchart.

 A regular review is important in order to develop and maintain genuine trust in your system. Most people who are not applying GTD do this kind of review a few times a year, for example at the beginning of a new year. This gives them a great feeling of clarity, control, and purpose. These good intentions, however, quickly dissipate when new, unprocessed things start to accumulate, and previous plans become outdated because of changing circumstances or lack of follow-up. If they would do such a review systematically every week, this feeling of control and goal-directedness could become permanent.


Do (5)

Having all your lists of to-dos up-to-date, what should you do right now? Allen proposes three models for deciding which action to perform. The first is the “four-criteria model for choosing actions in the moment”, which advises that you consider the following factors in the listed order:

1. Context: What can you do here and now? You cannot do the same actions when you are at your desk as when you are walking in the street. The context limits your choices to the tasks you can (practically) perform. If you have a large number of “next actions”, it is recommended to classify them by context (“office”, “home”, “errant”, etc.), so that actions requiring the same context can easily be performed together.

2. Time available: How much time do you have now? Fit the duration of the next action you choose to the amount of time available: if the time is limited, do only short actions.

3. Energy available: How much energy do you have at this moment? Adapt your choice of action to your level of physical and mental energy: when you are tired do only routine actions, and keep difficult actions for when you feel more energetic.

4. Priority: What are your priorities? Given the context, the time and energy available, what action should be done first? The two following models help you to answer that question.

The “threefold model for evaluating daily work” proposes the following possible strategies:

(1)       Do work as it shows up

(2)       Do predefined work

(3)       Define your work

Is the work that shows up (1) the most urgent thing you have to do? When you accomplish tasks as they appear (answering a phone call, chatting with a colleague passing by, replying to an email that just arrived, etc.) by default it means that you are deciding that these tasks are the most important ones at this moment. Alternatively, you can decide, if possible, to postpone the work that shows up, in order to focus on your predefined work (2). This means that you systematically go through your Next actions list. If you do not have any next actions listed, or if you do not feel confident that the listed “next action” is the best thing to do, you have to define your work (3). This is similar to the reviewing phase, where you clear your mind by updating your system of to-dos.

Still, to be confident that what you are doing is truly important, you need a deeper insight in your general goals and values. The “six-level model for reviewing your own work” can support such clarification. Allen uses an airplane analogy to define the levels (Allen, 2001, p. 51):

-                50, 000+ feet: Life

-                40, 000 feet: Three- to five-year vision

-                30, 000 feet: One- or two-year goals

-                20, 000 feet: Areas of responsibility

-                10, 000 feet: Current projects

-                Runway: Current actions

You can define goals for different terms or time-spans, from tasks to undertake immediately (Runway) to missions that extend over the rest of your life span (Life). The latter require you to answer almost philosophical questions, like “What is my purpose in life?” It is important to engage from time to time in this “vertical thinking” (Allen, 2001, p.  20-21), and write down and review those lists of goals, so as not to be constantly chasing priorities at the runway level. 



Allen, D. (2001) Getting Things Done: The Art of Stress-Free Productivity, (Penguin).

Heylighen F. & Vidal C. (2008) Getting Things Done: the science behind stress-free productivity (ECCO working paper 2007-08), to appear in Long Range Planning

Wikipedia contributors. (2007) Getting Things Done. Wikipedia, The Free Encyclopedia.