.

[Abstract][Introduction] [Part 1: the nature of identity in a community and a computer network] [Part 2: taxonomies of networking]; [Part 3: parsimonious evaluation for community networking] [Conculusions ] [References]

Community Networks: Identities, Taxonomies and Evaluations

A refereed paper for the Electronic Networking 2002 - Building Community Conference, July, 2002

Home: webstylus.net

Tom Denison, Gary Hardy, Graeme Johanson, Larry Stillman, Don Schauder -- Centre for Community Networking Research School of Information Management Systems Monash University

http://www.ccnr.net

Abstract

'Community networking'is a troublesome catchphrase, packed as it is with many assumptions and unspoken expectations about how electronic tools can be used by community groups and organisations. It is our view that community networking, as it is currently constituted, will not progress without its stakeholders having a better understanding of the boundaries and relationships between the elements that constitute this emerging area of social and technological practice. Clarification of these different understandings will lead to better conceptual development, coherent cross-disciplinary integration and improved outcomes for community networking activity.

This paper reviews a number of key issues that currently concern researchers in the Centre for Community Networking Research, including:

Introduction

There are three easily accessed sources of knowledge about community networks: observations derived from direct involvement in their daily activities, descriptions of them by committed participants and stakeholders, and disciplinary conceptualisations of their collective values and impact. This paper tries to make use of all three. It also aims to clarify some key assumptions that are part of the current discourses within each source. It is not surprising that discussions around 'community networking' can sometimes generate a sense of frustration and confusion, resulting in strongly polarised opinions amongst its proponents. This paper is intended to alleviate some of that confusion and inform debate. In it we suggest that considering the potential for transformation of community identity in the transition from real to virtual spaces, delineating a schema to describe the major strands of the diverse experience and practice of 'community networking', and adopting evaluative ways of thinking from the formative stages of any community venture are all useful means to achieve greater rigour and clarity for policy developers, researchers, commentators and practitioners alike.

Our main contentions are as follows: online networks reflect the essential features of real-life communities; five different types of network engagement can be described from analysis of their primary functions; and evaluation of the validity of the conceptualisation behind the foregoing propositions, and of the ongoing activity of community networks, is dependent on methodical but parsimonious use of clearly understood evaluative aims and well-targeted evaluative techniques.

Part 1: the nature of identity in a community and a computer network

Thereis currently a wide range of terms upon which commentators and policy-writers draw when they discuss community engagement and involvement with information and communications technologies (ICT). Anyone familiar with publications in this area will have encountered terms such as 'community networks', 'community information networks', 'civic networking', 'freenets', 'computer supported social networks', 'community computer networks', 'online neighbourhood network', 'virtual community', 'online community' and 'community informatics'. It is unhelpful that such terms are used more or less interchangeably and usually without definition. (Cohill and Kavanaugh 2000; O'Neil 2001; Wellman 2001)

Scanning this literature, it is easy to become confused by much of the discussion. For example, 'community networking' is itself a troublesome catchphrase. 'Network' is as ubiquitous as it is indeterminate. 'Community' covers a spectrum from groups located in small specific geographic locations to widely distributed individuals with a common interest. The compound term applies at once to the architecture of the underlying ICT as well as to the most utopian community-building aspirations of some visionary participants. Furthermore, 'community network' is applied to undertakings ranging from wide-scale, inclusive infrastructure, training, education and content undertakings of the sorts listed on the Community Networking Clearing House site (http://lone-eagles.com/community.html)--for example, organisations such as www.prairinet.org, associated with a library school in the American agricultural mid-West, that has made community networking part of its university extension activity--to quite small-scale ICT-related activities within a single community organisation. The contrasts are great.

To begin to try to clarify some of the confusion, a core characteristic of both 'community' and 'network' needs to be identified. It seems uncontroversial to assert that social relationships lie at the heart of every community and network. (Wallace 1999: 14-24) If so, we can assume that the personal identity of participants, transferred to the broader identity of the community members and networkers as a group, are likely to define the types of communicative interactions within a community and network in large part. We expect a close connection between the main features (or identity) of members of the community and of the network. The communications analyst Morley, and cultural geographer Robins see group identity and community as inseparable; they write that identity implies "cohesion, community, unity, integration, security, idealised wholeness and plenitude". (Morley and Robins 1995: 23)

Of concern to a number of community participants and network designers and managers is the possibility that identity and the fundamental nature of relationships might be radically altered by the very fact of computer use. Fifty years ago, the spread of television gave rise to similar anxieties. In other words, do computer networks change the identity of community, or vice versa? If there is change, is it for better or worse? These are hard questions, yet there is no lack of adventurous answers.

Some argue that online community is the only form of community of real interest today. For Jordan, a computer network is the main place to experience community: "the circuit of cyberpower" is community. "The words flowing past individuals' screens around the world create communities. And this is true for all the parts of cyberspace, whether it is email used for decision making or MUDs [Multi-User Dimensions] for collective sexual exploration". Online community derives from "the three connected principles of identity fluidity, renovated hierarchies and informational spaces". (Jordan 1999) Although network contributors can adopt many personae, social controls may vary a little from the real world when online, and content perhaps takes on a range of formats, these differences do not affect commonality between the two spheres according to Jordan. Communities and computer networks merge identity harmoniously. Others are not so bold. For some, real community can only exist if face-to-face communications--that is, meetings in physical spaces, based on shared, expressed needs--are sustained. Thus researchers into the nature of work inside the World Bank argue that they "do not know of any true communities in which a portion of the members do not …see each other face-to-face, look each other in the eye, sniff each other out" frequently. Yet at the same time, computer networks are an essential layer of the daily functions of that organisation. They provide a "professional raison d'etre". They are so close to the real community that they are described in its own typical language. Xerox calls the work teams online "family groups" while Hewlett Packard calls them "learning communities". (Denning, Pommier and Shneier 2002) In these examples, the physical community and computer network are treated separately for some purposes, but are melded for joint outcomes.

In surveying such literature, it quickly becomes clear that each commentator attributes great importance to satisfying human needs as a fundamental part of community identity, whatever the locus. The social interactions, the communicative transactions, are fundamental. From the point of view of the community sector, it is necessary to identify the range of these shared needs. They can be classified as normative (defined by an authority--e.g., the determination of what constitutes reduction of the 'digital divide'); felt (experienced--e.g., perceptions of loneliness by online communicators); expressed (shown by an act--e.g., sharing resources or emotional support among members of a charitable community); and comparative (inferred from comparisons--e.g., the number of times that one community meets in comparison with another like community). (Black and Hughes 2001: 28)

In this summary of the abstract concepts of community identity, computer network and social interaction, we need to concede that some communities and networks have identity thrust upon them. They have no choice as they are defined by external forces, often by their actual or perceived external enemies. Religious cults rely heavily on enforced exclusivity to force members to follow the party line. (Black and Hughes 2001: 104) Communities of computer hackers are an example of peer identification that is sometimes imposed uninvited. Political persecution can lead to the creation of online communities, where homeless refugees in flight from oppression manage to maintain a modicum of group solidarity from their diaspora. In this case the online identity is the primary cohesive social interaction, physical opportunities having been removed by hostile forces. Thus exiled Kurds use websites to sustain their links to art, music, cuisine, flags and anthems, these sites orginating in active physical groupings in Australia, Sweden, Denmark, France, Spain, Germany, Russia, the USA and Canada. In this example, "the Internet [is] very important in constructing … national identity" among a dispersed physical cohort threatened with extinction. (Bakker 1999).

Another group of theorists argue that the online version of community is but a weak shadow of the real. This may be so in some cases. Online networks tend to be more fragile, superficial and shallow than real-life communities, and members are under-committed and value their belonging less. In this view, offline organisation and regulation of space, time and movement are replicated online, but this can never happen as powerfully or effectively as in the real world. Cyberspace is a place for play, for rehearsing for the serious real world, for simulating responsibility, but never assuming it seriously. "Cyberspaces and the rules of engagement within them [might] bear a remarkable resemblance to real-world spaces and protocols", but they can never be the same. (Kitchin 2000) As depicted, the divide is unbridgeable.

There is widespread disagreement about the extent of overlap between identity in physical communities and online networks, and about the significance of the divergent uses of social interactions offline and online. It is important to note, however, that there is a common view that communicative transactions form the corpus of group identity wherever it is. Such transactions allow for appropriate self-expression and satisfy a range of social needs. It can be argued that online networks enhance the ability to exploit this central feature. Maybe one social advantage of online communities is that they invite plurality in identification. Maybe the social cues for articulating identity are sharpened and simplified when transferred to online communities. Maybe the ability to speedily move from one community online to another is an advantage for some purposes.

It is hard to generalise. Thanks to computer networks, it is easier to belong to multiple communities simultaneously. Whereas social mobility traditionally required a struggle, or a rigorous rite of passage, to migrate from one location to another, the networked world endorses involvement in several social milieux at once. Agre names these easy groupings "caucuses", for example, groups of Latino engineers, radical teachers, conservative lawyers, peace-activist physicians or Christian students. Presumably it would be possible to recombine some of these variables without too much effort: radical Christian lawyers have been known to exist. "The Internet is one great laboratory of hybrid identity-formation … and we should embrace it for this". Communities, in this view, do not need to adhere to strict boundaries and meanings; they no longer have to correlate with languages and national identities. Their novelty, online, is that at one and the same time they can be sites of collective cognition and solitary action. (Agre 1999)

Identity helps characterise a constructive aspect of community networks, communicative transactions, and ways to enhance social capital. The physical community and computer network both allow for the expression of similar identities and the satisfaction of similar social needs, and online networks provide an opportunity to enhance the spread and speed of community engagement. Is it possible to categorise these forms of identity for the purpose of analysing f ICT policy? The following section of the paper is an attempt to provide a practical taxonomy of current forms of networking, based not on internal social interactions but rather on main activities, and it is followed in Part 3 by a discussion of the types of evaluation techniques used to understand the value and merit of these different types of ICT activity.

Part 2: taxonomies of networking

If the focus of analysis is shifted from the relationship between identity and network to network-related activities, then five major categories of organisation can be described:

  1. individual organisations;
  2. clusters of like organisations;
  3. cross-sectoral collections of geographically based entities;
  4. civic networks; and
  5. service and application provider organisations.

These categories are not mutually exclusive, and other groupings might be possible. On the basis of discussion and reflection, however, the categories provide a relatively good fit to the observed reality of the Australian community ICT scene. Creating a matrix with this primary classification against other dimensions--such as organisational aspirations and available main service/applications--optimises some illuminating patterns.

For example, a recent review of community informatics evaluations in a number of countries identified five key dimensions that also bear some relationship to intensive work that has been done concerning the meaning of community-building and capacity in the health field, including health informatics. These dimensions are: 1) enhancing strong democracy; 2) increasing social capital; 3) empowering individuals; 4) revitalising a sense of community; and 5) providing economic development opportunities. (Goodman, Speers, McLeroy and Fawcett 1998; Parker, Eng, Schulz and Israel 1999; O'Neil 2001) While there will certainly be variations, mapping the expressed aspirations of the clusters of organisations listed above against those key areas would look something like Figure 1.

Figure 1: Explicit aspirations for ICT engagement

Individual community organisations Cluster of like organisations intrasectional Collectives of neighbourhood stakeholders-- cross-sectoral Civic networks--Government-agency led Service and application provider organisations
Enhancing strong democracy Generally LOW Generally LOW HIGH HIGH HIGH
Increasing social capital Generally LOW Generally LOW HIGH HIGH HIGH
Empowering individuals Medium to High Medium to High Medium to High HIGH HIGH
Rerevitalising sense of community Generally LOW Generally LOW HIGH HIGH HIGH
Providing economic development opportunities Generally LOW Generally LOW HIGH HIGH HIGH

The entities represented in the first two columns have quite different aspirations to those in the other three columns--they are motivated by concerns of increasing efficiency and efficacy of operations to achieve organisational goals. They are consumers of networked ICT for particular ends. The organisations represented in the third, fourth, and fifth columns have wider aspirations which match O'Neill's key activity areas closely, and they are often providers of services and applications utilised by the entities in the first two columns. More detailed characteristics of each type are provided below.

Individual community organisations

Many organisational entities that deploy ICT to communicate and transact with their members, clients and the rest of the world have been described as 'community networks'. Large charities and not-for-profit organisations deploy extensive network infrastructure for their staff and have highly developed informational and interactive environments for their membership--they may have physical infrastructure, IT support and sometimes sophisticated applications development. At the other end of the scale, small organisations also use the many-to-many potentiality of the Internet to network amongst their membership.

Major characteristics of both types of individual community organisations include utilisation of ICT services and applications to meet organisation-specific goals and utilisation of ICT to enhance business organisational processes. Examples are almost too numerous to mention: they include thousands upon thousands of small community organisations from Alcoholics Anonymous Victoria to Znet (an activist site with materials on East Timor), and many large ones such as Red Cross Australia or the Smith Family.

Clusters of like organisations

These sorts of networks are exemplified by the 'virtual networks' that Infoxchange (www.infoxchange.net.au) helps to develop-that is, entities which are themselves peak body organisations bringing together groups within a specific activity area. One example is the Queensland Mental Health Association. Major characteristics include the utilisation of ICT services and applications to meet inter-organisation goals. Clusters of this type are often utilised by environmental, activist or service-provider organisations where there are strong needs for inter-organisational communication and interaction. These entities develop sites which are in effect extranets and information and communication platforms or tool kits for their constituent groups, and which often provide a limited window for outsiders interested or concerned with the sarea of activity in question. These sorts of 'networks' are primarily aimed at providing information and communication applications for staff, members and clients of the constituent groups.

Collectives of neighbourhood stakeholders

This is the classic 'community network' manifestation, the sort of activities articulated in the work of Doug Schuler, the Blacksburg Electronic Village, and the old freenet movement. (Schuler 1996; Cohill 1997) Major characteristics of this collectivity network include that they often draw on a community development model of social organisation, provide a wide range of functionalities, and are place-based. Amy Borgstrom, past president of the US Association for Community Networking, had this sort of entity in mind when she proposed the following definition:

Community networking to me comes out of a sense of place. Community networking to me is what happens when a group of people in a physical geographical community get together to solve problems and respond to opportunities. This can happen in a church basement, a local council office, or a meeting like this. Community networks are the electronic public spaces, the communication and information tools that we can use to facilitate the work we do to make our communities a better place to live. (Borgstrom 1999)

In Australia there have been various attempts to develop the classic community model on this broadly inclusive basis-examples include Virtual Moreland and Melbourne Freenet--but in general these have not flourished. Research about factors which have led to the failure or demise of this model in Australia demands future study, particularly since a number of local initiatives have begun with enthusiasm and technical skill.

Civic networks

In some areas, government aspirations for developing ICT capacity within a constituency have led to the development of entities which provide a range of supporting functionalities and services to organisations that encourage ICT involvement. A very visible manifestation of these activities is a portal bringing together communication and information activities of disparate organisations within one constituency.

Major characteristics of the civic network may include the following:

Examples include Ourbrisbane.com.au, Tasmaniaonline, Communitybuilders.nsw.gov.au, VICNET, Wellington City Council (NZ) and many local government sites in Australia.

Service and application provider organisations

A number of organisations have evolved in the Australian context to assist community organisations and the general community in their region to develop ICT capacity. These are often small collectives of ICT-skilled volunteers who offer services and applications to community organisations or groups of organisations.

Major characteristics include high levels of ICT expertise, a client provider model, and frequently, a high level of social entrepreneurialism. Examples include Infoxchange itself, Green.net, CASE, NetC, community technology centres and telecentres.

It is important to note that the model schema we have proposed here could be developed further. Our intention is to provoke fresh discussion rather than provide a definitive taxonomy. The purpose is not to exclude nor to establish some sort of "more-networking-than-thou" hierarchy, but instead, provide confidence in future assessments of where to invest resources. For example, the schema might assist policy-makers to be more focused and aware in their policy development and program implementation and to have more realistic expectations of different sorts of projects. In addition, it may enable researchers to ask more relevant questions, and help to more accurately contextualise practices within the community networking landscape.

Part 3: parsimonious evaluation for community networking

Thusfar, we have characterised internal and external features of community networks, moving from community and identity to a classification of community networks focused on functions and activities. In this final section of the paper we address issues surrounding the evaluation of the phenomena that we have identified.

The concept of 'parsimony' in evaluating electronic community networking was first advanced by Scott Patterson in recognition of the need for an inclusive yet comprehensive concept map of the Blacksburg Electronic Village. (Patterson 2000) His intention was to create an inclusive network model that could be used across disciplines. However, while Patterson outlined a "network theory" for a conceptual model of the human and technical components of a network, his paper did not provide details of what constituted a parsimonious evaluation methodology. This lapse, we argue, is characteristic of the general absence of knowledge in the community networking field about what evaluation actually constitutes as a form of knowledge enquiry.

By a parsimonious use of evaluation methodology, we mean the careful use of recognised evaluation concepts, stages, methods and techniques matched to the very practical limitations of time, human resources and capital, and technical skills. We hope that a better use of recognised evaluation methodologies in electronic community networking will lead to improved understandings and outcomes, and capacity-building for evaluation amongst all stakeholders. Evaluation processes are integral to the character of community networking, as they are to any form of strategic planning or assessment. (Compton, Baizerman and Stockdill 2002)

How do we measure the efficiency and effectiveness of outcomes of ICT and community-strengthening frameworks? What do we define as effects and outcomes online as distinct from offline? How are evaluation priorities set? These questions continue to bedevil discussions in community-networking literature and forums such as the community-informatics listserv, hosted on Vancouver CommunityNet. Such discussions have been taking place since the mid-1990s, without resolution, and continue to trouble policy-makers. (Gygi 1995; Odasz 1995; O'Neil 2001)

Why? The seemingly hypnotic attraction of ICT has tended to privilege questions of technology over questions of the dynamics and capacities of people, their communities and organisations. As Wellman and Gulia have put it, "consistent with the present-oriented ethos of computer users, pundits write as if people had never worried about community before the internet arose". (Wellman, Salaff and Dimitrova 1996: 2)

How to evaluate parsimoniously

Effective evaluation offers one pathway to address the above issues. However, the matching of recognised methods and practices of evaluation to the needs of community-networking projects has not yet been clearly articulated by any party. The outline below offers some preliminary steps in this regard. Effective use of such methodologies (and, of course, others can be proposed) should result in parsimonious use of human and technical resources, matched to the capacity of organisations to undertake a particular type of evaluative process.

Clarity about the type of evaluation being used is important, as this will determine the sorts of questions, forms of information (e.g., qualitative-quantitative), kinds of answers, and reporting methods. (Patton 1997; Owen and Owen with Rogers 1999) Questions about the merits of different sorts of data are understood as part of the mix and match of questioning methods, rather than the core of evaluation itself, a source of frequent confusion. (Datta 2001) It is not a question of surveys or samples versus case studies: rather, what is appropriate to the purposes, context, skills and resources available? This issue is highlighted by Frank Odasz's realisation that evaluation of electronic networking was more than logfiles of hits. It included people, but he was unsure as to how to take the next step. (Odasz 1995)

Types of evaluation

Owen et al. (1999) have outlined a number of types of characteristic methods of evaluation, and their work has been used to shape the conceptual framework below .

Evaluation for development

Evaluation for development takes place before a project or program is actually implemented. Other terms for it include 'up front evaluation'. Characteristic questions include: Is there a need for an evaluation study? What is recognised best practice or informed opinion? We often use needs assessments in this process on a just-in-time basis. This technique could be used fruitfully in the planning of many networking projects.

An example in the community networking environment would be a process of inquiry with stakeholders and experts as to the best model for developing a network in a particular community. Some of the group inquiry methodologies outlined at www.makingthenetwork.org could be used in this process to determine priorities. Furthermore, a proposed longitudinal survey series, the Monash Community ICT Index (CICT), of the Centre for Community Networking Research, could act as a planning tool for benchmarking of activity.

Design evaluation

This type of clarifying evaluation looks at the rationale and assumptions of a program, including what is sometimes referred to as conceptual logic or heuristic theory: "If we do x, how do we expect y & z to happen?" It can be used to bring out the implied theory, assumptions and causal chain of a project from amongst scattered opinions and documentation. (McClintock 1987; Chen 1994) Patterson's attempt to outline a conceptual theory for his network (Patterson 2000) falls into this category of evaluation. An example of the method would be a concept-mapping workshop to draw out understandings by different stakeholders of the use of a particular online technical application and the causation chain that leads to better adult literacy in rural communities.

Process evaluation

Process evaluation is often of interest to people concerned about the outcomes of community development, that is, what is the process that is bringing about (or not bringing about) a change? The final outcomes (e.g., obtaining a training certificate) may be of less interest than the actual effect that the project has had on its participants. One of the best-known methods used is called action research or empowerment evaluation, using community development techniques and the active participation of people subject to a particular intervention in evaluating its effect on them. Process evaluation can be combined with feminist or other perspectives that take account of diversity issues. There are a number of studies of networking which take these issues into account. ( Milio 1996; Stoecker 1996a; Stoecker 1996b)

A potential use of this type of evaluation, recently suggested by one of these authors, would be to set up a nationwide process amongst community groups-for example, iwi, hapu, and whanau in New Zealand--to record multiple snapshots and case studies of ICT usage to provide an overall picture of usage of ICT. The multiple-sources, depth and richness of reporting would demonstrate the rigour of the research, and the process itself should educate and empower the community.

Evaluation in program management

Evaluation in program management according to the following method is probably most familiar to workers in funded organisations: an external consultant evaluates the impact of a project according to the performance indicators in the project deed. This can happen on an ongoing basis. In the community networking field, an example would be yearly reviews of a three-year implementation program in community organisations by a contracting body.

Impact evaluation

Impact evaluation is focused around final outcomes of projects. Key questions include: What did this program finally achieve at its end-point? Did it meet the terms of the grant deed, and what were its intended and unintended impacts? What were the outcomes (intended and unintended) of a project? In the USA, for example, the United Way charitable network (http://national.unitedway.org/outcomes/) has developed tools to assist participating organisations in evaluating project outcomes, and some of these would be of great interest to people engaged in evaluating electronic impacts with a social edge.

Evaluation is multi-dimensional discipline: it offers different pathways to inquiry by providing guidance for "how to" and "what" questions, data methods to use, and the reporting of findings. With choice and care, parsimonious evaluation leads to a better appreciation of what actually occurs in the electronic community networking domain, from beginning to end. In this way it also ties in with clarification of the perceived value of community identity and acceptable methods for categorising existing networks.

Conclusions

The character of community networks of itself requires their de facto evaluation, because proper acceptance of the value of community demands adequate scope for continuous critique by any participants. Regardless of this inherent need, however, there are many other reasons why parsimonious evaluation methodologies assist community networks. Their adoption helps to highlight constructive aims, improve process, and enhance appreciation of achievements. The methodologies undermine the complacent hegemony of an omnipresent ICT in its own right, privileged over socially determined values and activities, and they help share understandings among stakeholders more equitably, usefully and democratically. In the field of community networks, integrated evaluation can provide many benefits over and above hard-and-fast measures of technical efficiency alone.

This paper has tried to find some answers to these important questions: Who is at the core of a community, and how is their identity expressed? What are the shared benefits of communities as promised by their stated aspirations? And how do we know if a community is effective?

We have argued that social interactions can be viewed as the expression of individual and group identities, whether communities are offline or online. The scope for interaction is greater on computer networks on occasion than it is in physically-bound communities. However, Australia has tended to create networks more within the sphere of formalised organisational activity and direction than informal collectivity, and why this is so needs further investigation. We are convinced that all of the categories of networks enhance democracy, increase social capital, empower individuals, and improve opportunities for all sorts of development. Connections posited in our matrix are impossible to prove precisely, of course, but in our experience probable relationships are reinforced by repeated observations of current practice. Finally, we suggest that when evaluation is integrated as a valued part of a specific community networking activity, whatever its scale or proportions, the clarity of its character and its sustainable effects are enhanced in many advantageous ways.

References

Agre, P.E. (1999). "Rethinking Networks and Communities in a Wired Society." (Accessed: 1 May 2002) [http://dlis.gseis.ucla.edu/people/pagre/asis.html]

Bakker, P. (1999). "Reinventing Roots; new media and national identity." (Accessed: 6 May 2002) [http://home.pscw.uva.nl/bakker/roots.html]

Black, A. and P. Hughes (2002). "The identification and analysis of indicators of community strength and outcomes." Occasional Paper no. 3. (Accessed: 6 March 2002) [http://www.facs.gov.au/internet/facsinternet.nsf/vIA/occasional_papers/$file/No.3.rtf]

Chen, H.-T. (1994). "Theory-Driven Evaluations: Need, Difficulties, and Options." Evaluation Practice 15(1): 79-86.

Cohill, A.M. and A.L. Kavanaugh, eds (2000). Community Networks. Lessons from Blacksburg, Artech House, Boston, Virginia.

Compton, D.W., Baizerman, M. and S.H. Stockdill, eds (2002). The Art, Craft, and Science of Evaluation Capacity Building. New Directions for Evaluation. Jossey-Bass, San Francisco, CA.

Datta, L.-E. (2001). "The Wheelbarrow, the mosaic, and the double helix." Evaluation Journal of Australia (New Series), 1(2): 33-40.

Denning, S., Pommier, M. and L. Shneier (2002). "Are There Laws of Knowledge Management?" (Accessed: 1 May 2002) [http://www.educationau.edu.au/globalsummit/papers/denning.htm]

Goodman, R.M., Speers, M.A., McLeroy, K., Fawcett, S. et al. (1998).
"Identifying and defining the dimensions of community capacity to provide a basis for measurement." Health Education and Behavior 25(3): 258-78.

Gygi, K. (1995). Developing an Evaluation Framework for Community Computer Networks. Community and Regional Planning Program, University of New Mexico.

Kitchin, R. (2000). Cyberspace: the world in the wires. J. Wiley, New York.

Jordan, T. (1999). Cyberpower: the culture and politics of cyberspace and the Internet. Routledge, New York.

McClintock, C. (1987). "Conceptual and Action Heuristics: Tools for the Evaluator", in Using Program Theory in Evaluation, ed. L. Bickman. Jossey Bass, San Francisco, CA. 43-57.

Milio, N. (1996). Engines of Empowerment: Using Information Technology to Create Healthy Communities and Challenge Public Policy. Health Administration Press, Chicago, Ill.

Morley, D. and K. Robins (1995). Spaces of identity: global media, electronic landscapes, and cultural boundaries. Routledge, London.

Odasz, F. (1994). "The need for rigorous evaluation of community networks", in Ties that Bind: Converging Communities, ed. S. Cisler. Apple Computer Corp. Library, Cupertino, CA.

O'Neil, D. (2001). Merging Theory with Practice: Towards an Evaluation Framework for Community Informatics. Georgia Institute of Public Policy, Atlanta, Ga.

Owen, J.M. and P.J. Owen with Rogers (1999). Program Evaluation, Forms and Approaches.

Parker, E.A., Eng, E., Schulz, A.J. and B.A. Israel (1999). "Evaluating Community-Based Health Programs That Seeks to Increase Community Capacity." New Directions for Evaluation (Fall), 83: 37-54.

Patterson, S.J. (2000). "Evaluating the Blacksburg Electronic Village", in Community Networks. Lessons from Blacksburg Virginia, eds A.M. Cohill and A.L. Kavanaugh. Artech House, Boston.

Patton, M.Q. (1997). Utilization-focussed Evaluation. Sage, Thousand Oaks.

Stoecker, R. (1996a). Limited Access: The Information Superhighway and Ohio-based Organizations. The Urban University and Neighbourhood Network, University of Toledo, OH.

Stoecker, R. (1996b). Putting Neighbourhoods on Line; Putting Academics in Touch. The Urban University and Neighbourhood Network, University of Toledo, OH.

Wallace, P.M. (1999). The psychology of the Internet. Cambridge University Press, Cambridge.

Wellman, B. (2001). "Computer Networks as Social Networks." (Accessed: 14 September 2001) [http://www.chass.utoronto.ca/~wellman/publications/index.html]

Wellman, B., Salaff, J. and D. Dimitrova (1996). "Computer Networks as Social Networks: Collaborative Work, Telework, and the Virtual Community." Annual Review of Sociology 22: 213-38.