Effective Data Collection Methods: Understanding Informal and Formal Sources

Every good piece of research, report, or technical document begins with one essential task: gathering reliable data. Before you can analyse trends, draw conclusions, or present findings, you need raw information to work with. But data does not arrive neatly packaged. It exists in many forms, from a casual conversation with an expert to a peer-reviewed study in a national database. Knowing where to look, and how to collect information from each kind of source, is a core skill for anyone working in research, library science, or technical writing. This post breaks down the informal and formal sources of data, the techniques used to gather them, and the practical methods for organising what you collect so it remains useful long after the project ends.

Understanding data and its sources 🔗

Data is simply information gathered from various sources that can be used for a specific purpose, often expressed in a form that supports analysis and decision-making. The quality of any research depends heavily on the reliability of the data behind it, which is why selecting the right source matters so much. A common way to classify data is by origin. Primary data is original information you collect yourself for a specific question, while secondary data is information that already exists and was gathered by someone else. Both have their place, and both can come from sources that range from highly structured to entirely informal.

In library and information science, sources are also grouped into documentary and non-documentary categories. Documentary sources are recorded and preserved in physical or digital form, such as books and journals. Non-documentary sources depend on direct interaction or observation and are not always written down. Understanding these categories helps you decide where to invest your time and effort during a project.

Informal sources of data 🔗

Informal sources are often overlooked, yet they can supply current, localised, and context-specific insights that formal records miss. These sources are dynamic, meaning they rely on communication, observation, or experience rather than fixed documents. They are especially valuable in the early stages of research, when you are still shaping your questions and exploring a topic.

Oral sources 🔗

Oral sources include interviews, conversations, and expert testimonies. They are typically used in qualitative research, where personal experiences, opinions, and insights carry weight. Oral sources allow for deep exploration of a subject and are particularly useful in social sciences, anthropology, and history. For example, a researcher studying changes in a village craft tradition might learn far more from speaking with elderly artisans than from any printed report. Oral histories also preserve knowledge that has never been formally documented, making them a unique window into lived experience.

Ephemeral sources 🔗

Ephemeral sources refer to information that is transient and may not have an enduring presence. Blog posts, social media updates, and podcasts fall into this group. Although such content may not last long, it can offer timely, current, and often subjective views on a topic. Ephemeral material is challenging to manage precisely because it is short-lived, which is why information professionals study ways to capture and describe it. Research published in the Proceedings of the Association for Information Science and Technology shows how even fleeting forms, such as recordings of live performances, can be made searchable through careful description and metadata. For a technical writer, monitoring ephemeral sources can reveal emerging issues or public sentiment before they appear in formal literature.

Non-scientific sources 🔗

Non-scientific sources include everyday experiences and anecdotal information. Personal observations, diaries, and casual conversations with community members or experts can offer genuine insight into complex topics. While these sources are not considered rigorous by academic standards, they can enrich your understanding and surface perspectives that structured data leaves out. The key is to treat them as a starting point. Anecdotes can suggest a pattern worth investigating, but they should be verified through more reliable methods before they inform any conclusion.

Formal sources of data 🔗

Formal sources are systematically created, organised, and stored so that information can be easily accessed, verified, and reused. In library and information science, these documentary sources are seen as highly reliable because they provide permanent records of knowledge, in contrast to oral or informal communication that may be temporary or hard to verify. When accuracy and authority are essential, formal sources should be your foundation.

Academic journals and articles 🔗

Peer-reviewed journals are among the most dependable formal sources for scientific and academic data. They present detailed, well-researched, and evidence-backed information, having passed review by other experts before publication. For students and researchers, journals are ideal for projects that demand depth and credibility. The Library of Congress online resource guide points to several open repositories, including the Directory of Open Access Journals, which offers free access to hundreds of full-text periodicals in the field of library and information science. Tools such as the Education Resources Information Center, sponsored by the U.S. Department of Education, further widen access to scholarly literature.

Books and published reports 🔗

Books written by reputable authors, along with industry reports, white papers, and official documents, supply data that is usually backed by rigorous research or established policy. Books provide comprehensive treatment of a subject, while reports often contain the latest statistics and analysis on a focused issue. These sources are valuable when you need both breadth and authority, such as understanding the historical development of a topic alongside its present state.

Government and institutional databases 🔗

Government and institutional databases are among the most reliable formal sources, offering statistical data, public records, research papers, and policy documents. In the Indian context, publications such as the census, economic surveys, and the Statistical Abstract supply structured data for countless studies. International bodies like the International Monetary Fund, the World Bank, and the International Labour Organization also publish statistical information widely used as secondary data. Research institutions, including the Indian Statistical Institute, release findings that researchers draw on regularly. Because these sources are maintained by accountable organisations, they offer a strong balance of reliability and accessibility.

Techniques for gathering information 🔗

Knowing your sources is only half the work. You also need practical techniques to collect data from them efficiently. The method you choose should align with your research aim, your questions, and the resources available. Poor alignment between objectives and collection methods can lead to irrelevant findings and weaker results.

Fieldwork 🔗

Fieldwork means collecting data directly from the real-world setting where it occurs. It commonly involves observations, interviews, and focus groups conducted on site. Fieldwork captures the complexity and nuance that numbers alone might miss, but it brings logistical challenges. Travelling to locations, coordinating teams, locating participants, and managing schedules all demand strong organisation. Researchers often work with local contacts or community leaders who can grant access and build trust, a step that takes time and planning. A useful practice in field research is to monitor data quality while collection is still underway. Reviewing submissions daily and flagging inconsistencies early allows errors to be corrected before they spread, which is far better than discovering a systematic problem after fieldwork has ended.

Databases and online systems 🔗

Modern data collection relies heavily on databases and online systems. A database management system lets you enter, select, and organise data, then retrieve it quickly when needed. Online survey tools allow field teams to capture responses through digital forms, which can work offline in remote areas and sync automatically with central systems once a connection is available. Other techniques include data warehousing, which consolidates diverse data from many sources into a single repository where it is cleansed and made readily accessible. As datasets grow, indexing methods help speed up retrieval, a concern explored in technical guidance from the National Academies Press. For a researcher, these systems turn scattered information into an organised resource ready for analysis.

Practical approaches for organising data 🔗

Collecting data is wasted effort if you cannot find and use it later. Once data is gathered, the next step is to organise and process it carefully so it supports your analysis. This stage acts as the backbone of the entire project, and treating it as an afterthought often leads to confusion and lost work.

Systematic entry and processing 🔗

After collection, data must be entered, cleaned, and checked for errors. If a survey was conducted on paper, the responses usually need to be transferred into a spreadsheet. If data was captured on a mobile device, it may need to be exported from cloud storage. This initial processing turns raw responses into a usable dataset. Building a clear structure at this point, such as consistent column headings and standard formats for dates and categories, saves considerable time during analysis.

Storage and retrieval 🔗

The main purpose of any storage system is to store and retrieve data efficiently. When choosing one, consider factors such as data safety, ease of use, storage capacity, cost, and reliability. According to guidance from ScienceDirect, storage can range from simple paper records to spreadsheets for small datasets and dedicated databases for large ones. For most student and professional projects, a well-structured spreadsheet or a small database is enough, provided the data is backed up and clearly labelled. Documenting your methods alongside the data is equally important, because future users, including your future self, need to understand how the information was collected before relying on it.

Cataloguing for future use 🔗

Organising data is similar to cataloguing findings systematically for ease of reference, a principle at the heart of library science. Giving files clear names, recording the source and date of each dataset, and keeping a simple index of what you have collected all make retrieval faster. When data is properly catalogued, you can return to it months later and quickly locate exactly what you need, whether for a follow-up study, a review, or a new question that builds on earlier work. This discipline transforms a one-time collection effort into a lasting, reusable resource.

What do you think? Which type of source, informal or formal, do you find more trustworthy for the topics you study, and why? And if you were starting a small research project today, how would you organise your data so that you could still make sense of it a year from now?

References

1. https://research-methodology.net/research-methods/data-collection/
2. https://en.wikipedia.org/wiki/Library_and_information_science
3. https://asistdl.onlinelibrary.wiley.com/doi/abs/10.1002/pra2.1049
4. https://guides.loc.gov/library-science/external-resources
5. https://www.geeksforgeeks.org/data-analysis/methods-of-data-collection/
6. https://www.surveycto.com/data-collection-quality/field-research-methods/
7. https://nap.nationalacademies.org/read/21958/chapter/6
8. https://www.sciencedirect.com/topics/computer-science/field-data-collection